Add license-safe real-world test fixtures with offline verification - #35
Merged
Conversation
Vendors 11 small, sliced excerpts of publicly redistributable genomics data (DeepVariant, FreeBayes, GRIDSS, GIAB HG002, 1000 Genomes, GENCODE, UCSC) so BioFmt's detection, validators, and indexed-query path are exercised against real caller/portal output rather than only hand-made samples. - test/fixtures/real/sources.json: manifest (source URL, license, extraction command, pinned sha256, assertions) as the source of truth. - scripts/fetch-real-fixtures.ts: dev-only fetch/slice/reindex/pin. Network is used only here, never by the test suite (npm run fetch:fixtures). - test/unit/real-fixtures.test.ts: offline per-fixture checks — presence, checksum, format support, the real validator runs without throwing, minimum row count, and an @gmod/tabix region query for indexed VCFs. - provenance/*.json and ATTRIBUTION.md record source, checksum, and license. Only redistribution-safe sources are vendored; API-resolved, terms-encumbered, and restrictively-licensed sources (Manta, 10x, controlled-access TCGA) are intentionally excluded. Total footprint ~544 KB.
Adds a manifest-completeness assertion to the real-fixtures suite: any entry with a pinned sha256 must have its committed file (and .tbi index, for indexed entries) present. The per-entry checks skip absent files by design, so without this guard a deleted fixture would silently stop being tested; dropping one now requires a visible edit to sources.json.
This was referenced Jul 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Vendors a small, license-vetted set of real-world genomics files so BioFmt's detection, validators, and indexed-query path are exercised against actual caller/portal output — not just hand-made samples. Adapted from a fixture-sourcing research plan, but re-cast into this repo's toolchain: no Python/pytest/shell-
tabixat test time, JSON manifest (no new dep), and a hard offline/network wall.What's here
test/fixtures/real/sources.json— manifest and source of truth: per fixture, the source URL, license, extraction command, pinnedsha256, andexpectassertions.scripts/fetch-real-fixtures.ts(npm run fetch:fixtures) — dev-only, the only thing that touches the network. Downloads → slices (tabix/bgzipregion extraction, or header+N-row slices) → reindexes → writes the fixture +provenance/<id>.json→ pinssha256. Never run by the test suite.test/unit/real-fixtures.test.ts— fully offline Mocha suite. Per fixture: presence → checksum → format is supported → the real validator runs without throwing (robustness) → minimum row count → for indexed VCFs, a region query through@gmod/tabix(the lib BioFmt ships, not shelltabix). Entries without a committed fileskip.provenance/*.json+ATTRIBUTION.md— source, checksum, and per-source license.Fixtures (11, ~544 KB total)
Robustness result: every real file validated without throwing — the validators hold up against real DeepVariant/GRIDSS/GENCODE/1000G output. Both indexed VCFs answer
@gmod/tabixregion queries offline against the committed.gz+.tbi.Deliberately excluded
API-resolved / non-deterministic or terms-encumbered sources: ENCODE (API pin), TCGA MAF/GCT (attribution + controlled API), 10x MTX (site redistribution terms), minimap2 PAF (tool absent), PRIDE mzTab/MGF (unpinned), Manta (Polyform Strict — never vendor), CNVkit
.seg/.cns(not a BioFmt format). The manifest is shaped so these can be pinned in follow-up commits.Testing
Offline suite green (46 assertions across the 11 fixtures); full unit suite 848 pass; lint, type-check, and build green.
Follow-ups (not in this PR)
c8coverage to include this suite; use a larger slice as a perf-gate input.