Repository navigation
fix(star): star_delete_fastq_after_alignment = false keeps the reads (v0.1.19) - #37
Merged
Merged
Conversation
The STAR and ruSTAR subworkflows passed `params.star_delete_fastq_after_alignment ?: true`, and Groovy's ?: turns false into true, so the second pass always deleted its input reads. Pass the value as set. That flag also decides for the bqtools decode/encode step just before the aligner (BQTOOLS_DECODE, BQTOOLS_ENCODE_ALIGN), which deleted deacon's output unconditionally. Each step deletes its input, the output of an earlier task, so -resume after a failure could not reuse anything from trimming on: resuming the failed HLdesRot15A spleen run reran 19,153 of 19,177 tasks. Every other such deletion already has a keep switch. With all of them set (deacon_keep_fastp_fastq, star_delete_fastq_after_alignment = false, beaver_keep_aletsch_gtf, bigtools_keep_bedgraph, wigtools_keep_bigwig, wigtobigwig_keep_wig), a resumed test-bqc-star run reuses all 61 tasks. The defaults still delete. Bump to 0.1.19; usage (what each step deletes, what -resume needs) and Changelog.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
star_delete_fastq_after_alignment = falsehad no effect. The STAR and ruSTAR subworkflows passedparams.star_delete_fastq_after_alignment ?: true, and Groovy's?:turnsfalseintotrue, so the second pass always deleted its input reads.This matters beyond disk. Each read step deletes its input, which is the output of an earlier task. After a failure,
-resumefinds those outputs gone and reruns everything from trimming on. On HLdesRot15A spleen (9 samples), xORF's BLAST step failed on anorfpanic, and the resume reran 19,153 of 19,177 tasks (1 h 37 min). The panic itself is fixed in hillerlab/xorf#30;modules/xorfmoves to that release separately.Changes
subworkflows/star_alignment,subworkflows/rustar_alignment: passparams.star_delete_fastq_after_alignmentas set. The default staystrueinnextflow.config.subworkflows/metassembly:BQTOOLS_DECODE(STAR with CBQ reads) andBQTOOLS_ENCODE_ALIGN(bqtools_encode_before_alignment) deleted deacon's output unconditionally. They now follow the same flag, so withfalsethe reads that reach the aligner are kept in every format.assets/docs/usage.md(4.6): what each step deletes, and the switches-resumeneeds. Every other deletion already had a working keep switch:deacon_keep_fastp_fastq,beaver_keep_aletsch_gtf,bigtools_keep_bedgraph,wigtools_keep_bigwigandwigtobigwig_keep_wig.nextflow.config: version 0.1.19.Changelog.md: 0.1.19 entry.With the defaults, the same files are deleted as before.
Testing
nextflow config -flatfortest,test-sb,test-tm,test-polish,test-rustarandtest-bqc-star.-profile test-bqc-star,docker, which covers bqc, STAR and the CBQ decode step. Each run used Nextflow 25.10.4 on a clean export that includes the xorf submodule:verify.pygolden numberswork/-resume.cbq; the bqc, deacon and decode outputs are deleted, as beforedeacon_keep_fastp_fastq = true,star_delete_fastq_after_alignment = false.cbq, decoded FASTQ pairsRENAME_GTFon (beaver deleted its GTFs); the 4 coverage tasks skippedThe middle row is why the usage section lists every switch. With
bigtools_keep_bedgraphleft out, the cached STAR task has lost its bedGraph, and the resumed run skips coverage without an error.