When running PHoeNIx v2.3.2 with the --create_ncbi_sheet flag, I get an error message describing a missing colon in the collated_versions.yml file. I tried running without the --create_ncbi_sheet flag and PHoeNIx completed successfully. I also tried adding the colon to the collated_versions.yml file manually and resuming the pipeline, but PHoeNIx ended with the same error as before, because it created a new yml file still missing a colon.
$ nextflow run /path_to_phoenix/phoenix/main.nf -profile docker --mode PHOENIX --input samplesheet.csv --kraken2db /path_to_kraken --create_ncbi_sheet
......
[77/f028d5] PHO…OENIX_EXTERNAL:GRIPHIN (1) | 1 of 1 ✔
[a7/449059] PHO…EATE_NCBI_UPLOAD_SHEET (1) | 1 of 1 ✔
[96/ce05bf] PHO…M_DUMPSOFTWAREVERSIONS (1) | 0 of 1 ✘
[- ] PHO…X:PHOENIX_EXTERNAL:MULTIQC | 0 of 1
Execution cancelled -- Finishing pending tasks before exit
-[cdcgov/phoenix] Pipeline completed with errors-
ERROR ~ Error executing process > 'PHOENIX:PHOENIX_EXTERNAL:CUSTOM_DUMPSOFTWAREVERSIONS (1)'
Caused by:
Process `PHOENIX:PHOENIX_EXTERNAL:CUSTOM_DUMPSOFTWAREVERSIONS (1)` terminated with an error exit status (1)
Command executed [/home/kmiller/Applications/phoenix/modules/nf-core/modules/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py]:
#!/usr/bin/env python
import yaml
import platform
from textwrap import dedent
def _make_versions_html(versions):
html = [
dedent(
"""\
<style>
#nf-core-versions tbody:nth-child(even) {
background-color: #f2f2f2;
}
</style>
<table class="table" style="width:100%" id="nf-core-versions">
<thead>
<tr>
<th> Process Name </th>
<th> Software </th>
<th> Version </th>
</tr>
</thead>
"""
)
]
for process, tmp_versions in sorted(versions.items()):
html.append("<tbody>")
for i, (tool, version) in enumerate(sorted(tmp_versions.items())):
html.append(
dedent(
f"""\
<tr>
<td><samp>{process if (i == 0) else ''}</samp></td>
<td><samp>{tool}</samp></td>
<td><samp>{version}</samp></td>
</tr>
"""
)
)
html.append("</tbody>")
html.append("</table>")
return "\n".join(html)
versions_this_module = {}
versions_this_module["PHOENIX:PHOENIX_EXTERNAL:CUSTOM_DUMPSOFTWAREVERSIONS"] = {
"python": platform.python_version(),
"yaml": yaml.__version__,
}
with open("collated_versions.yml") as f:
#versions_by_process = yaml.load(f, Loader=yaml.BaseLoader) | versions_this_module
versions_by_process = yaml.load(f, Loader=yaml.BaseLoader)
versions_by_process= {**versions_by_process, **versions_this_module}
# aggregate versions by the module name (derived from fully-qualified process name)
versions_by_module = {}
for process, process_versions in versions_by_process.items():
module = process.split(":")[-1]
try:
assert versions_by_module[module] == process_versions, (
"We assume that software versions are the same between all modules. "
"If you see this error-message it means you discovered an edge-case "
"and should open an issue in nf-core/tools. "
)
except KeyError:
versions_by_module[module] = process_versions
versions_by_module["Workflow"] = {
"Nextflow": "25.10.2",
"cdcgov/phoenix": "2.3.2",
"Mode": "PHOENIX",
}
versions_mqc = {
"id": "software_versions",
"section_name": "cdcgov/phoenix Software Versions",
"section_href": "https://github.com/cdcgov/phoenix",
"plot_type": "html",
"description": "are collected at run time from the software output.",
"data": _make_versions_html(versions_by_module),
}
with open("software_versions.yml", "w") as f:
yaml.dump(versions_by_module, f, default_flow_style=False)
with open("software_versions_mqc.yml", "w") as f:
yaml.dump(versions_mqc, f, default_flow_style=False)
with open("versions.yml", "w") as f:
yaml.dump(versions_this_module, f, default_flow_style=False)
Command exit status:
1
Command output:
(empty)
Command error:
Traceback (most recent call last):
File ".command.sh", line 55, in <module>
versions_by_process = yaml.load(f, Loader=yaml.BaseLoader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/__init__.py", line 81, in load
return loader.get_single_data()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/constructor.py", line 49, in get_single_data
node = self.get_single_node()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/composer.py", line 127, in compose_mapping_node
while not self.check_event(MappingEndEvent):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key
if self.check_token(KeyToken):
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/scanner.py", line 115, in check_token
while self.need_more_tokens():
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yaml/scanner.py", line 152, in need_more_tokens
self.stale_possible_simple_keys()
File "/usr/local/lib/python3.11/site-packages/yaml/scanner.py", line 291, in stale_possible_simple_keys
raise ScannerError("while scanning a simple key", key.mark,
yaml.scanner.ScannerError: while scanning a simple key
in "collated_versions.yml", line 3116, column 5
could not find expected ':'
in "collated_versions.yml", line 3117, column 5
Work dir:
/wgs_analysis/KSM_dev/phoenix2.3.2_MVV/NFGC/test2/dataset1/phx_output2.3.2/work/96/ce05bfd97a7d28a6d593de80e3c115
Container:
quay.io/biocontainers/multiqc:1.14--pyhdfd78af_0
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
-- Check '.nextflow.log' file for details
Screenshot of collated_versions.yml file where colon missing (hopefully) in file attachments.
Description of the bug
When running PHoeNIx v2.3.2 with the --create_ncbi_sheet flag, I get an error message describing a missing colon in the collated_versions.yml file. I tried running without the --create_ncbi_sheet flag and PHoeNIx completed successfully. I also tried adding the colon to the collated_versions.yml file manually and resuming the pipeline, but PHoeNIx ended with the same error as before, because it created a new yml file still missing a colon.
Command used and terminal output
Relevant files
System information
Nextflow Version: 25.10.2
PHoeNIx Version: 2.3.2
Container Engine: Docker