You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
~64K of the ~80K currently-unclassified files are non-primary-data, and a large chunk are companions of a real file — .md5 checksums (14,179), plus indexes (.bai/.fai/.gzi). Today they're silent gaps. They should be explicitly resolved, like classify_index_files.py already does for .tbi/.crai.
Idea (from the field, 2026-07-31)
"Classify the checksums as checksums and then call everything else NA, or link it to the thing it is a checksum of, like we do with index."
Propagate from parent — x.vcf.gz.md5 inherits x.vcf.gz's classification (as .tbi/.crai already do). Better when the parent is classified; keeps the companion consistent with what it accompanies.
Likely: propagate when the parent is present/classified, else tag by type.
Scope
Checksums:.md5 (~14K) — strip the trailing .md5 to find the parent; propagate, else tag checksum.
Standalone docs (.txt/.log/.pdf/README, ~46K) are NOT companions — out of scope here; they stay honestly unclassified (or a later "documentation" tag with genomic dims not_applicable).
Why it matters
Turns silent gaps into positive not_applicable/propagated answers → the residual "unclassified" count then means "real gap," not "auxiliary file."
Consistent with the existing index-propagation pattern and the accuracy-over-coverage principle (a checksum having no reference is a fact, not a failure).
~64K of the ~80K currently-unclassified files are non-primary-data, and a large chunk are companions of a real file —
.md5checksums (14,179), plus indexes (.bai/.fai/.gzi). Today they're silent gaps. They should be explicitly resolved, likeclassify_index_files.pyalready does for.tbi/.crai.Idea (from the field, 2026-07-31)
Two acceptable resolutions per companion file:
.md5→ data_typechecksum(or similar), genomic dimensions (reference/modality/assay/platform)not_applicable. A positive determination, not an unknown (cf. Distinguish 'unreadable' (fetch failed) from 'read but no expected content' — VCF/FASTQ mislabel a clean read as content_unreadable #299 unreadable-vs-not-this-type).x.vcf.gz.md5inheritsx.vcf.gz's classification (as.tbi/.craialready do). Better when the parent is classified; keeps the companion consistent with what it accompanies.Likely: propagate when the parent is present/classified, else tag by type.
Scope
.md5(~14K) — strip the trailing.md5to find the parent; propagate, else tagchecksum..tbi/.craipropagation to.bai/.fai/.gzi(~2.5K). (This supersedes the index-propagation item in Add classification rules for the unhandled data-format tail (~16K files: bigWig, GFF3, indexes, chains, h5ad, PAF…) #311; leave Add classification rules for the unhandled data-format tail (~16K files: bigWig, GFF3, indexes, chains, h5ad, PAF…) #311 for actual data formats — bigWig/GFF3/etc.).txt/.log/.pdf/README, ~46K) are NOT companions — out of scope here; they stay honestly unclassified (or a later "documentation" tag with genomic dimsnot_applicable).Why it matters
not_applicable/propagated answers → the residual "unclassified" count then means "real gap," not "auxiliary file."DoD
.md5resolved (propagate-from-parent, elsechecksum+ genomic dimsnot_applicable)..bai/.fai/.gzipropagate from parent like.tbi/.crai.classify_index_files.pyrather than a new one-off.Relates to #311 (data-format tail), #299 (resolved vs unknown), #301 (producing method).