Repository navigation
Add refilter_profile.py: apply stricter SNV thresholds without re-profiling - #230
Merged
Merged
Conversation
Re-applies higher --min_cov / --min_freq to an existing inStrain profile using the data it already stores: sites in raw_snp_table are re-called from their A/C/T/G counts at each mm level (stricter thresholds can only remove calls), clonT is masked to positions with the new min_cov using covT, and the scaffold / genome / gene tables are rebuilt with inStrain's own functions. Linkage needs the reads, so it's left out. The test profiles the test data at the defaults and at -c 10 -f 0.1, and checks that re-filtering the default profile gives identical SNVs, scaffold_info, genome_info and gene_info tables (except rarefied diversity, which is a random subsample in every run). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X4KRcccF9fQPJkzpN9YaXw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
auxiliary_scripts/refilter_profile.pyfor #229. It re-applies stricter--min_cov/--min_freqto an existing inStrain profile without re-runninginStrain profile, which for large studies can take months.How it works
Raising the thresholds can only remove calls, and a profile already stores everything needed to recalculate them:
raw_snp_tablehas every site called at the original thresholds, with its A/C/T/G counts at each mismatch level. Each site is re-called with the new thresholds, following the same per-mm logic asupdate_snp_table, including "cryptic" SNVs.covThas per-position coverage at each mismatch level. It's used forbreadth_minCov, and to tell "not called because of low coverage" apart from "not called because only the reference base was present".clonThas per-position nucleotide diversity for positions at ≥ the originalmin_cov. Nucleotide diversity doesn't depend onmin_freq, so positions below the newmin_covare simply masked out.The scaffold, genome and (with
-g) gene tables are then rebuilt with inStrain's own functions (make_coverage_table,genomeLevel_from_IS,profile_genes). The original profile isn't modified; a new one is written to-o.Limitations:
--current_snv_thresholdsoverrides this.Testing
test_refilter_profileprofiles the test data at the defaults and at-c 10 -f 0.1, then re-filters the default profile to 10×/10%. The re-filteredSNVs.tsv,scaffold_info,genome_infoandgene_infotables are identical to the fresh 10×/10% run. The only exclusions are linkage and rarefied nucleotide diversity, which is a random subsample in every run and doesn't depend on these thresholds.This doesn't change the inStrain package itself, so merging won't publish a new version.
Generated by Claude Code