Fix downstream reference registry contracts

[iskandr]

Summary

• make cancer classification eligibility require both reviewed registry approval and a complete returnable reference
• align SCLC subtype source_cohort values with the physical U.Cologne source used by reference filters
• define the BTC, SGC, and NSCLC memberships consumed by downstream reference loaders
• require closed source-scope unions to be complete, preventing CHOL-only data from masquerading as pan-BTC evidence
• correct GSE328026/PEComa provenance to PMID 42331846 and add a structured source-PMID consistency gate
• preserve the complete legacy positional constructor order of the public ExpressionSource dataclass
• serialize first-use bundle staging across processes and verify the complete bundle before CI workers start

Root causes

The public registry previously regenerated is_classification_target entirely from reference availability, which allowed new comparison data to override reviewed policy. Three source-scope codes advertised member_union without rows in the canonical aggregate table. SCLC subtype rows used a derivation label where public loaders require a physical cohort identity. The PEComa registry citation pointed to an unrelated lymphoma publication.

The first revision also inserted source_pmid into the middle of ExpressionSource and allowed incomplete source-scope unions to return their available members. The final revision appends the dataclass field and treats these closed diagnostic unions as all-or-nothing.

Parallel first-use bundle access could also start two downloads into the same cache. One worker could remove a shard directory while another was reading it. Bundle writes now use a cross-process lock, and a waiting ensure_local() rechecks and reuses the completed cache.

Classification policy migration

The owner registry now explicitly marks CRC_MSI, RCC, THYM_EPITHELIAL, NEN, NET, NEC, and NEC_LUNG as reviewed classification targets. This makes the previous public behavior explicit data policy rather than recreating it implicitly from reference availability. A reviewed flag is still only necessary, not sufficient: the code must also have a complete direct or member-union reference.

CMN remains explicitly non-classifying despite its validation-only microarray proxy. BTC retains its reviewed eligibility flag but is effectively non-classifying with reference_source="none" until both declared members, CHOL and GBC, are reference-backed.

Consumer impact

Pirlygenes can use the public classification flag directly without a CMN workaround, and SCLC registry source identifiers round-trip through availability filters. Trufflepig can safely consume complete SGC and NSCLC member unions. BTC remains auditable as CHOL ∪ GBC, but its availability and expression accessors return no BTC reference while GBC is missing; no BTC pooled pan-cancer column is emitted.

Validation

• ./test.sh: 1,048 passed, 0 skipped, 86% coverage
• focused registry, expression, bundle, and workflow suite: 288 passed
• concurrent bundle regression: two simultaneous first-use calls, one completed fetch
• GitHub Actions: minimal, lint, and Python 3.9–3.12 all passed
• Ruff lint and formatting checks passed
• CSV schema, duplicate-row, stale-identifier, and public contract audits passed

Closes #468.
Closes #472.
Closes #479.
Closes #480.