diff --git a/assets/changelog/Changelog.md b/assets/changelog/Changelog.md index 2221ad1..d567c85 100644 --- a/assets/changelog/Changelog.md +++ b/assets/changelog/Changelog.md @@ -61,6 +61,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- +## [0.1.19] - 2026-10-25 + +### Fixed + +- `star_delete_fastq_after_alignment = false` had no effect. The STAR and ruSTAR subworkflows passed `params.star_delete_fastq_after_alignment ?: true`, and Groovy's `?:` turns `false` into `true`, so the second pass always deleted its input reads. The value is now passed as set. + +### Changed + +- With `star_delete_fastq_after_alignment = false`, the conversion just before the aligner also keeps its input, deacon's output: `BQTOOLS_DECODE` (STAR with CBQ reads) and `BQTOOLS_ENCODE_ALIGN` (`bqtools_encode_before_alignment`). Before, `-resume` after a failure could not reuse read preprocessing or alignment, because each step deletes its input, the output of an earlier task: resuming a failed 9-sample run (HLdesRot15A spleen) reran 19,153 of 19,177 tasks. With every keep switch on (usage, 4.6), a resumed `test-bqc-star` run reused all 61 tasks. The defaults still delete. +- Usage: what each step deletes, and the switches `-resume` needs. +- Version bumped to `0.1.19` in the pipeline manifest. + +--- + ## [0.1.18] - 2026-10-23 ### Fixed diff --git a/assets/docs/usage.md b/assets/docs/usage.md index ba3a76b..914b767 100644 --- a/assets/docs/usage.md +++ b/assets/docs/usage.md @@ -259,10 +259,30 @@ greatly improves detection of novel splice sites. | `star_extra_align_args` | – | Any extra STAR arguments you want appended to the alignment command. | | `star_save_unaligned` | `false` | Reserved; declared but not consumed by any step. | | `star_keep_first_pass_bam` | `false` | Keep the BAM of the first alignment pass (extra disk; normally deleted). | -| `star_delete_fastq_after_alignment` | `true` | Delete the aligned fastqs after the second pass to save disk. | +| `star_delete_fastq_after_alignment` | `true` | Delete the aligner's input reads after the second pass to save disk, and the CBQ/FASTQ file they were converted from just before the aligner (`bqtools decode` for STAR, `bqtools_encode_before_alignment`). `false` keeps them. | | `star_twopass_junctions_file` | – | Path to a pre-curated junction file (STAR sjdb format, 4 columns). Providing one **skips the first pass** entirely and aligns once with your junctions. | | `star_make_coverage` | `true` | Produce coverage tracks (per-sample BigWig → merged BigWig in `05_coverage`). Disable to save time/disk if you do not need them. | +**Disk or `-resume`.** To save disk, steps delete their inputs once their +output is written: deacon deletes the trimmed reads, the conversion just before +the aligner deletes deacon's output, the aligner deletes its input reads after +the second pass, beaver deletes the per-sample GTFs, and the coverage steps +delete the bedGraphs, the per-sample BigWigs and the merged `.wig`. These +inputs are outputs of earlier tasks. After a failure, `nextflow run ... -resume` +finds them gone and reruns those tasks and everything after them, i.e. from +trimming on. To keep `-resume` working, keep them all: + +- `deacon_keep_fastp_fastq = true` +- `star_delete_fastq_after_alignment = false` +- `beaver_keep_aletsch_gtf = true` +- `bigtools_keep_bedgraph = true`, `wigtools_keep_bigwig = true` and `wigtobigwig_keep_wig = true` +- with `star_make_coverage = false`, also `aletsch_keep_bam = true` + +Every sample's trimmed and decontaminated reads then stay in the work directory +until you remove it. Leaving out `bigtools_keep_bedgraph` is worse than a full +rerun: the cached alignment has lost its bedGraph, so the resumed run silently +skips coverage for that sample. + ### 4.6a CBQ reads (bqtools + bqc) CBQ is the columnar BINSEQ read format. A paired sample collapses into a @@ -291,7 +311,7 @@ in one `input_dir` is supported — each sample is routed by its own extension. | Parameter | Default | What it does | |-----------|---------|--------------| | `bqtools_encode_fastqs` | `false` | Encode FASTQ inputs to `.cbq` before QC. Your original files in `input_dir` are never deleted. | -| `bqtools_encode_before_alignment` | `false` | Encode to `.cbq` after decontamination, just for the aligner (**ruSTAR only**). The intermediate deacon FASTQs **are** deleted (they are work-directory files, and nothing downstream removes them otherwise). | +| `bqtools_encode_before_alignment` | `false` | Encode to `.cbq` after decontamination, just for the aligner (**ruSTAR only**). The intermediate deacon FASTQs **are** deleted (they are work-directory files, and nothing downstream removes them otherwise), unless `star_delete_fastq_after_alignment = false`. | | `bqc_min_length` | `15` | `--min-length`: reject reads shorter than this. Always sent — `bqc` refuses to run with no operation configured. | | `bqc_quality_tail` | `15` | `--quality-tail`: trim 3' bases until the trailing window reaches this Phred score. | | `bqc_poly_g` | `true` | `--poly-g`: trim G-rich 3' tails (two-colour chemistry). | diff --git a/nextflow.config b/nextflow.config index fafd10d..6584faf 100644 --- a/nextflow.config +++ b/nextflow.config @@ -41,7 +41,7 @@ manifest { mainScript = 'main.nf' defaultBranch = 'master' nextflowVersion = '!>=24.10.5' - version = '0.1.18' + version = '0.1.19' } validation { @@ -158,7 +158,7 @@ params { star_extra_align_args = null // extra arguments for STAR star_save_unaligned = false // saves unaligned reads star_keep_first_pass_bam = false // keep first pass bam - star_delete_fastq_after_alignment = true // delete fastq after second pass + star_delete_fastq_after_alignment = true // delete the aligner's input reads after the second pass [-resume needs false, see usage 4.6] star_twopass_junctions_file = null // path to pre-curated junction file [STAR sjdb 4-col] -> skips first pass star_make_coverage = true // make coverage [bedgraph -> bigwig -> wig] diff --git a/subworkflows/metassembly/main.nf b/subworkflows/metassembly/main.nf index cd5d0ff..fecba2c 100644 --- a/subworkflows/metassembly/main.nf +++ b/subworkflows/metassembly/main.nf @@ -176,13 +176,14 @@ workflow METASSEMBLE { .map { meta, reads, _ -> [meta, reads] } // Encode just before alignment: fastp and deacon still run on FASTQ and only the - // aligner input becomes CBQ. delete_input is true because the deacon FASTQs are - // work-dir intermediates that nothing downstream removes any more -- without it - // this path would raise peak disk instead of lowering it. + // aligner input becomes CBQ. The deacon FASTQs are work-dir intermediates that + // nothing downstream removes any more, so they are deleted (otherwise this path + // would raise peak disk instead of lowering it), unless the aligner keeps its input + // reads too (star_delete_fastq_after_alignment = false, for -resume). if (params.bqtools_encode_before_alignment) { BQTOOLS_ENCODE_ALIGN( ch_processed_reads.processed_reads.filter { _meta, reads -> !isCbq(reads) }, - true + params.star_delete_fastq_after_alignment ) ch_versions = ch_versions.mix(BQTOOLS_ENCODE_ALIGN.out.versions.first()) @@ -195,11 +196,12 @@ workflow METASSEMBLE { // STAR cannot read CBQ. Decode after deacon so bqc + deacon stay on CBQ // and only the aligner input becomes gzipped FASTQ. - // delete_input is true because the deacon .cbq is a work-dir intermediate. + // The deacon .cbq is a work-dir intermediate: deleted unless the aligner keeps + // its input reads too (star_delete_fastq_after_alignment = false, for -resume). if (params.aligner == 'STAR') { BQTOOLS_DECODE( ch_aligner_reads.filter { _meta, reads -> isCbq(reads) }, - true + params.star_delete_fastq_after_alignment ) ch_versions = ch_versions.mix(BQTOOLS_DECODE.out.versions) diff --git a/subworkflows/rustar_alignment/main.nf b/subworkflows/rustar_alignment/main.nf index b5f863d..1a9de1f 100644 --- a/subworkflows/rustar_alignment/main.nf +++ b/subworkflows/rustar_alignment/main.nf @@ -108,7 +108,7 @@ workflow RUSTAR_ALIGNMENT { params.star_seq_library ?: '', params.star_machine_type ?: '', true, // keep bam -> we need it for Aletsch - params.star_delete_fastq_after_alignment ?: true + params.star_delete_fastq_after_alignment ) } else { ch_rustar_second_pass_out = RUSTAR_ALIGN_2PASS( @@ -121,7 +121,7 @@ workflow RUSTAR_ALIGNMENT { params.star_seq_library ?: '', params.star_machine_type ?: '', true, // keep bam -> we need it for Aletsch - params.star_delete_fastq_after_alignment ?: true + params.star_delete_fastq_after_alignment ) } diff --git a/subworkflows/star_alignment/main.nf b/subworkflows/star_alignment/main.nf index 160ce49..ae3f43a 100644 --- a/subworkflows/star_alignment/main.nf +++ b/subworkflows/star_alignment/main.nf @@ -107,7 +107,7 @@ workflow STAR_ALIGNMENT { params.star_seq_library ?: '', params.star_machine_type ?: '', true, // keep bam -> we need it for Aletsch - params.star_delete_fastq_after_alignment ?: true + params.star_delete_fastq_after_alignment ) } else { ch_star_second_pass_out = STAR_ALIGN_2PASS( @@ -120,7 +120,7 @@ workflow STAR_ALIGNMENT { params.star_seq_library ?: '', params.star_machine_type ?: '', true, // keep bam -> we need it for Aletsch - params.star_delete_fastq_after_alignment ?: true + params.star_delete_fastq_after_alignment ) }