From 1d847817ebd0e739f03b04c88468d102098b8017 Mon Sep 17 00:00:00 2001 From: JK Date: Tue, 16 Jun 2026 09:47:17 -0400 Subject: [PATCH] Polish v9 public metadata language --- docs/hf_dataset_card.md | 2 +- docs/v9_hf_dataset_card.md | 323 ++++++-------------- scripts/validate_public_docs_consistency.py | 61 ++++ v9/README.md | 61 ++-- v9/reports/README.md | 27 +- 5 files changed, 186 insertions(+), 288 deletions(-) diff --git a/docs/hf_dataset_card.md b/docs/hf_dataset_card.md index 172fee9..5cddbf8 100644 --- a/docs/hf_dataset_card.md +++ b/docs/hf_dataset_card.md @@ -55,7 +55,7 @@ the held-out mission. | `train_X.csv`, `test_X.csv` | Sample-by-gene expression matrices | | `train_y.csv`, `test_y.csv` | Binary labels: `1 = Flight`, `0 = Ground` | | `train_meta.csv`, `test_meta.csv` | Sample-level metadata used for fold auditing | -| `fold_info.json` | Held-out mission, train missions, sample counts, and provenance | +| `fold_info.json` | Held-out mission, train missions, and sample-count audit metadata | | `selected_genes.txt` | Fold-specific genes selected from training missions only | | `task_info.json` | Task-level metadata and source summary | diff --git a/docs/v9_hf_dataset_card.md b/docs/v9_hf_dataset_card.md index f3c5f6e..144ef43 100644 --- a/docs/v9_hf_dataset_card.md +++ b/docs/v9_hf_dataset_card.md @@ -1,5 +1,5 @@ --- -pretty_name: SpaceBio-Bench v9 Public Bulk Metadata Alpha +pretty_name: SpaceBio-Bench v9 Public Bulk Metadata Catalog license: other task_categories: - tabular-classification @@ -11,111 +11,74 @@ tags: - tabular - mission-held-out - OSDR -- draft +- metadata-catalog --- -# SpaceBio-Bench v9 Public Bulk Metadata Alpha +# SpaceBio-Bench v9 Public Bulk Metadata Catalog -This is a Hugging Face-style dataset card for the metadata-only alpha snapshot -of the public bulk RNA-seq portion of SpaceBio-Bench v9. +This Hugging Face-style card documents the public bulk RNA-seq metadata catalog +for SpaceBio-Bench v9. It is designed for reviewers and method developers who +want to inspect task definitions, OSDR source coverage, fold indexes, and +baseline outputs directly from the GitHub repository. -Release status: metadata-only alpha snapshot, not frozen. +The catalog focuses on metadata and reference outputs. Fold-level expression +matrices are indexed by the catalog when available, while packaged public fold +downloads remain documented in the v7.1 Hugging Face dataset card. -This card describes the current local benchmark scaffold and should not be used -as frozen release language. The package has task manifests, source inventory, -OSDR checksum-manifest evidence, alpha-boundary decision tables, baseline -outputs, and a draft Frictionless Data Package descriptor. +## Related Documents -Current boundary: this is not a frozen release, not a frozen payload release, -and not a locally hash-verified payload bundle. Payload-level hash verification -for every distributed fold matrix remains pending. - -Branch note: when this card is viewed from the default `main` branch, artifact -paths beginning with `v9/...` and `docs/V9_*` refer to the curated public bulk -metadata-alpha evidence subset included in this repository. Payload matrices -and draft extension lanes are outside this public-review path. - -## Transparency And Release Boundary - -SpaceBio-Bench keeps result, evaluation, and release-readiness claims in a -separate transparency card pack. The canonical public-review copies live in the -GitHub repository under: - -- https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_TRANSPARENCY_CARD_PACK.md -- https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_SYSTEM_CARD.md -- https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_EVALUATION_CARD.md -- https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_RELEASE_READINESS_CARD.md -- https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_CLAIM_REGISTER.md - -Use these companion documents to check whether a claim belongs to the v1-v7 -canonical result surface, the v8 translational extension, the v9 public bulk -metadata alpha, or a draft v9 extension lane. +- Main repository: +- v7.1 public fold package card: `docs/hf_dataset_card.md` +- Release manifest: `release/release_manifest.json` +- Transparency card pack: `docs/SPACEBIOBENCH_TRANSPARENCY_CARD_PACK.md` ## Dataset Summary -SpaceBio-Bench v9 is a mission-held-out transcriptomics benchmark scaffold for -public space-biology data. - -The current public bulk draft contains leave-one-mission-out classification -tasks built from public mouse bulk RNA-seq studies in NASA's Open Science Data -Repository (OSDR). Each task holds out one mission at a time and evaluates -whether a method can generalize across spaceflight or analog mission contexts -within a tissue-specific expression dataset. +SpaceBio-Bench v9 extends the mission-held-out benchmark design to a public +bulk RNA-seq task catalog. Each task is tissue-specific and asks whether a +method trained on one set of missions can generalize to a held-out mission. -Current scope: +Current catalog scope: -- 8 generated public bulk LOMO task manifests. +- 8 public bulk leave-one-mission-out task manifests. - 6 tissue contexts: liver, gastrocnemius, kidney, thymus, skin, and eye. -- 22 deduplicated public OSDR source rows. +- 22 public NASA OSDR source rows. - 33 fold definitions. -- 24 baseline runs across 3 simple baseline families. -- 21 metadata/provenance/output resources in `v9/datapackage.draft.json`. +- 24 reference baseline runs across 3 simple baseline families. +- 21 catalog, audit, and output resources in `v9/datapackage.draft.json`. -Out of scope for this draft: +Scope: -- gated human sequence data -- clinical or crew-health recommendations -- intervention or countermeasure claims -- Mars-regime point predictions -- foundation-model ranking claims -- frozen release or DOI claims -- frozen payload mirror or locally hash-verified payload-bundle claims +- astronaut health-risk inference +- clinical, intervention, or countermeasure recommendations +- Mars-regime point prediction +- foundation-model leaderboard claims without adapter-specific validation +- controlled-access human sequence analysis ## Intended Uses -Intended uses: +Use this catalog to: -- Benchmark simple and advanced methods on mission-held-out public bulk RNA-seq - tasks. -- Test whether classifiers trained on some missions generalize to held-out - missions. -- Compare baselines with explicit provenance, task manifests, and run manifests. -- Support method-development studies on spaceflight biological domain shift. -- Serve as the public bulk layer for future SpaceBio-Bench task families. +- inspect public bulk mission-held-out task definitions; +- compare simple baseline outputs as reproducible reference rows; +- prepare method-development experiments for spaceflight biological domain + shift; +- audit which public OSDR accessions support each task; +- keep per-task, per-mission evaluation visible instead of relying only on + pooled averages. -Responsible use: +Recommended reporting: -- Treat scores as benchmark evidence, not biological mechanism proof. -- Inspect task manifests and source records before comparing methods. -- Report per-task results, not only pooled averages. -- Keep public bulk tasks separate from any future gated human track. +- Treat scores as benchmark evidence for the stated task definition. +- Report per-task results alongside pooled summaries. +- Compare stronger methods against the fixed task manifests and fold IDs. - Cite the upstream OSDR datasets used by a specific analysis. -Out-of-scope uses: - -- inferring astronaut health risk -- recommending countermeasures -- claiming operational readiness for space missions -- treating analog missions as interchangeable with deep-space exposure -- ranking virtual-cell or foundation models without adapter-specific validation - ## Dataset Structure -The current draft is organized around five artifact classes. +### Core Catalog Files -### Metadata Spine - -Small Git-friendly files that define the benchmark: +Small Git-friendly files define the public bulk task catalog: - `v9/task_manifests/*.json` - `v9/task_manifest_index.csv` @@ -124,43 +87,19 @@ Small Git-friendly files that define the benchmark: - `v9/source_checksum_audit.csv` - `v9/datapackage.draft.json` -### Metadata Alpha Boundary Resources - -Small tables that define the current alpha claim boundary: - -- `v9/reports/public_bulk_alpha_gap_matrix/public_bulk_alpha_gap_summary.csv` -- `v9/reports/public_bulk_alpha_gap_matrix/public_bulk_alpha_gap_matrix.csv` -- `v9/reports/public_bulk_alpha_gap_matrix/payload_hash_boundary.csv` -- `v9/reports/public_bulk_alpha_gap_matrix/public_bulk_alpha_claim_boundary.csv` -- `v9/reports/public_bulk_alpha_gap_matrix/package_update_plan.csv` -- `v9/reports/public_bulk_alpha_snapshot_decision/snapshot_decision_summary.csv` -- `v9/reports/public_bulk_alpha_snapshot_decision/snapshot_option_matrix.csv` -- `v9/reports/public_bulk_alpha_snapshot_decision/snapshot_claim_boundary.csv` -- `v9/reports/public_bulk_alpha_snapshot_decision/snapshot_language_snippets.csv` -- `v9/reports/public_bulk_alpha_snapshot_decision/snapshot_next_actions.csv` - -These are metadata resources. They do not add upstream OSDR payload files as -locally bundled resources. +### Indexed Fold Files -### Public Bulk Payload Bundle +The catalog records fold-level files with the following contract: -Large fold-level benchmark inputs referenced by `v9/task_data_index.csv`: +- `train_X.csv` and `test_X.csv`: sample-by-gene expression matrices +- `train_y.csv` and `test_y.csv`: binary Flight/Ground labels +- `train_meta.csv` and `test_meta.csv`: sample metadata +- `selected_genes.txt`: fold-specific selected genes +- `fold_info.json`: held-out mission, training missions, and sample counts -- `train_X.csv` -- `test_X.csv` -- `train_y.csv` -- `test_y.csv` -- `train_meta.csv` -- `test_meta.csv` -- `selected_genes.txt` -- `fold_info.json` +### Baseline Output Files -These files are currently indexed in the local scaffold but are not a frozen -payload mirror and are not payload-hash frozen. - -### Benchmark Output Bundle - -Generated baseline outputs: +Reference baseline outputs are stored under `v9/reports/`: - `v9/reports/bulk_lomo_baseline_summary.csv` - `v9/reports/nearest_centroid/bulk_lomo_summary.csv` @@ -169,18 +108,6 @@ Generated baseline outputs: - per-task `metrics.json` - per-task `run_manifest.json` -### Deferred Or Excluded Artifacts - -Not part of the current public bulk draft: - -- raw OSDR sequencing payload mirrors -- locally hash-verified payload bundle -- gated or controlled human data -- local caches or virtual environments -- user `submissions/` -- v8 intervention outputs -- single-cell and radiation/stressor flagship tasks - ## Task Table | Task ID | Tissue | Variant | Missions | Folds | Sources | Mission labels | @@ -196,7 +123,7 @@ Not part of the current public bulk draft: ## Source Table -All current source rows are public OSDR accessions. +All source rows in this catalog are public OSDR accessions. | OSDR source | GLDS prefix | Tissue | Mission | Task IDs | |---|---|---|---|---| @@ -223,48 +150,23 @@ All current source rows are public OSDR accessions. | OSD-48 | GLDS-48 | liver | RR-1 | `A1_liver_bulk_lomo`; `A1_liver_bulk_lomo_combat`; `A1_liver_bulk_lomo_iss_only` | | OSD-686 | GLDS-617 | liver | MHU-2 | `A1_liver_bulk_lomo`; `A1_liver_bulk_lomo_combat` | -## Provenance And Integrity +## Catalog Checks -Current provenance evidence: +The public catalog includes a compact audit table for OSDR source coverage: -- `v9/source_inventory.csv` has 22 public source rows. -- `v9/source_checksum_audit.csv` has 22 `api_status=ok` rows. -- All 22 source rows have `audit_status=checksum_manifest_parsed`. -- The audit found 39 checksum manifest-like files. -- The audit parsed 8,439 MD5 checksum entries. +- `v9/source_inventory.csv` records 22 public source rows. +- `v9/source_checksum_audit.csv` records 22 rows with `api_status=ok`. +- The audit parsed 39 checksum-manifest-like files and 8,439 MD5 entries. - 8,275 parsed entries matched OSDR file-list payload names by exact, basename, or suffix matching. -Metadata alpha decision: - -- Selected path: `metadata_only_alpha_snapshot`. -- Deferred path: `payload_mirror_first`. -- Claim boundary: `metadata_only_public_bulk_alpha_no_payload_release`. -- Payload release allowed: `false`. -- Next blocked release claim: local payload mirroring and hash verification. - -Integrity status: - -- `freeze_ready=false` for all current source rows. -- Checksum-manifest evidence exists. -- Payload files have not yet been downloaded and locally hashed as a distributed - v9 package. -- The current descriptor is `v9/datapackage.draft.json`, not a release - `datapackage.json`. - -Draft Data Package status: - -- `spacebio_bench:release_status = metadata_alpha_not_frozen` -- `spacebio_bench:alpha_snapshot_status = metadata_only_alpha_snapshot` -- `spacebio_bench:claim_boundary = metadata_only_public_bulk_alpha_no_payload_release` -- `spacebio_bench:payload_release_allowed = false` -- `spacebio_bench:payload_verification_status = checksum_manifests_parsed_payloads_not_hashed` +These checks help users inspect source coverage before running or extending a +benchmark workflow. ## Baseline Results -The current scaffold includes three simple baseline families across all eight -tasks. These results are included to validate the benchmark workflow, not to -claim biological or model superiority. +The current catalog includes three simple baseline families across all eight +task rows. These are reproducible reference anchors for workflow comparison. Mean metrics across eight task rows: @@ -274,107 +176,57 @@ Mean metrics across eight task rows: | Nearest centroid | 0.5383 | 0.5685 | 0.6321 | 0.1132 | 0.8733 | | PCA logistic regression | 0.5353 | 0.5619 | 0.6447 | 0.3747 | 0.9190 | -Interpretation cautions: - -- These are scaffold baselines. -- They are not tuned model-comparison endpoints. -- Per-task variability matters. -- Mission-discrimination scores depend on available embedding columns. -- Logistic regression L2 does not currently emit embeddings, so mission - discrimination is not computed for that baseline. +Report per-task rows when comparing methods. Pooled means are useful for +orientation, but tissue and mission variability are central to this benchmark. ## Data Fields Key tabular resources: -- `task_manifest_index.csv`: task ids, tissues, variants, missions, source - counts, fold counts, metric ids. -- `task_data_index.csv`: fold ids, held-out mission labels, train/test row - counts, selected-gene counts, fold paths. +- `task_manifest_index.csv`: task IDs, tissues, variants, missions, source + counts, fold counts, and metric IDs. +- `task_data_index.csv`: fold IDs, held-out mission labels, train/test row + counts, selected-gene counts, and fold paths. - `source_inventory.csv`: OSDR accessions, GLDS prefixes, mission labels, - tissues, privacy/access status, release target. + tissues, access status, and release target. - `source_checksum_audit.csv`: OSDR API status, file-list response hashes, - checksum-manifest files, parsed entry counts, payload-name match counts, - freeze status. -- `bulk_lomo_baseline_summary.csv`: baseline ids, task ids, validation status, + checksum-manifest files, parsed entry counts, and payload-name match counts. +- `bulk_lomo_baseline_summary.csv`: baseline IDs, task IDs, validation status, prediction counts, metrics, and output paths. ## Privacy And Access -The current public bulk draft uses only source rows marked public in -`v9/source_inventory.csv`. - -It does not include controlled-access human sequence data. OSDR documentation -notes that astronaut-derived studies can have public processed data while -sequence data may be controlled access through OSDR. This draft keeps the public -bulk benchmark independent of any such controlled data. +This catalog uses public OSDR source rows. It does not include controlled-access +human sequence data. -## Biases, Risks, And Limitations +## Limitations -Dataset limitations: - -- Mouse tissues and missions are not a complete representation of space biology. -- Mission labels can conflate hardware, time, vehicle, protocol, tissue, and +- Mouse bulk RNA-seq tasks are not a complete representation of space biology. +- Mission labels can combine hardware, time, vehicle, protocol, tissue, and processing effects. - Some tasks include analog or special mission labels such as MHU-2 or OSD-397. -- Bulk RNA-seq tasks do not evaluate cell-type-specific effects. -- Legacy processed fold matrices may reflect earlier preprocessing choices. -- The current package descriptor is a metadata-only alpha draft. -- The current evidence boundary is OSDR API file-list and checksum-manifest - parsing; local payload mirroring and payload hash verification remain future - work before any frozen data release wording. - -Benchmark risks: - -- A model can perform well by exploiting mission-correlated technical structure. -- Pooled averages can hide failure on a tissue or mission. -- Stronger methods should be compared only after payload verification and task - documentation are finalized. -- Countermeasure, intervention, and crew-health language is not supported by - this draft. -- This card does not support claims of a frozen payload release, a complete - release Data Package, or a locally hash-verified data bundle. +- Bulk RNA-seq tasks do not resolve cell-type-specific effects. +- Baseline rows serve as reference comparisons for workflow checks. +- The catalog describes task metadata and benchmark outputs rather than + countermeasure, intervention, crew-health, or operational-readiness use cases. ## Licensing And Citation -License status: - -- The draft card uses `license: other` because final benchmark artifact - licensing and OSDR source reuse language need release review. -- OSDR source datasets must be cited according to their OSDR study pages. +The v9 metadata catalog uses `license: other` because source reuse follows the +terms of the underlying OSDR studies. Code in the GitHub repository is MIT +licensed. Cite OSDR study pages for the datasets used by a downstream analysis. -Recommended acknowledgment language for this draft: +Recommended acknowledgment: > Data are courtesy of the NASA Open Science Data Repository. -OSDR citation guidance: - -- Cite the NASA OSDR resource and the relevant individual OSDR datasets. -- Use the citation button on each OSDR study page for dataset-specific BibTeX or - RIS where available. -- Include all OSDR-provided datasets that a downstream analysis uses. - -OSDR resource citation from NASA FAQ: +OSDR resource citation: Gebre S G, Scott R T, Saravia-Butler A M, Lopez D K, Sanders L M, and Costes S V. 2024. NASA Open Science Data Repository: Open Science for Life in Space. Nucleic Acids Research 53(D1): D1697-D1710. https://doi.org/10.1093/nar/gkae1116 -## Maintenance Notes - -Before public release: - -- replace metadata-alpha package status with release status -- add payload-level SHA-256 manifest for distributed fold files -- verify distributed payload hashes -- create a local payload mirror only after upstream source/citation/reuse - boundaries are finalized -- finalize license field -- add dataset-specific OSDR citations -- decide whether to publish through Hugging Face, Zenodo, or both -- add RO-Crate export or release manifest - ## References - Hugging Face Dataset Cards: @@ -387,7 +239,4 @@ Before public release: https://science.nasa.gov/reference/osdr-faq/ - NASA OSDR Terms and Conditions: https://science.nasa.gov/reference/osdr-help-terms-and-conditions/ -- v9 public package design: - `docs/V9_PUBLIC_BULK_PACKAGE_DESIGN.md` -- v9 draft Data Package descriptor: - `v9/datapackage.draft.json` +- v9 Data Package descriptor: `v9/datapackage.draft.json` diff --git a/scripts/validate_public_docs_consistency.py b/scripts/validate_public_docs_consistency.py index abdda63..2d4159b 100644 --- a/scripts/validate_public_docs_consistency.py +++ b/scripts/validate_public_docs_consistency.py @@ -14,6 +14,9 @@ REPO_ROOT = Path(__file__).resolve().parents[1] README = REPO_ROOT / "README.md" HF_CARD = REPO_ROOT / "docs" / "hf_dataset_card.md" +V9_HF_CARD = REPO_ROOT / "docs" / "v9_hf_dataset_card.md" +V9_README = REPO_ROOT / "v9" / "README.md" +V9_REPORTS_README = REPO_ROOT / "v9" / "reports" / "README.md" CITATION = REPO_ROOT / "CITATION.cff" ZENODO = REPO_ROOT / ".zenodo.json" RELEASE_MANIFEST = REPO_ROOT / "release" / "release_manifest.json" @@ -61,9 +64,13 @@ def validate_public_docs() -> list[str]: manifest = load_json(RELEASE_MANIFEST) readme = README.read_text() hf_card = HF_CARD.read_text() + v9_hf_card = V9_HF_CARD.read_text() + v9_readme = V9_README.read_text() + v9_reports_readme = V9_REPORTS_README.read_text() citation = CITATION.read_text() zenodo = load_json(ZENODO) hf_front_matter = parse_front_matter(hf_card) + v9_front_matter = parse_front_matter(v9_hf_card) lanes = {lane["lane_id"]: lane for lane in manifest["release_lanes"]} v7 = lanes.get("v7.1", {}) @@ -109,6 +116,60 @@ def validate_public_docs() -> list[str]: require_contains(errors, "docs/hf_dataset_card.md", hf_card, "Dataset freeze: **2026-03-01**") require_contains(errors, "docs/hf_dataset_card.md", hf_card, "repo_id = \"jang1563/genelab-benchmark\"") require_contains(errors, "docs/hf_dataset_card.md", hf_card, "SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out") + require_absent( + errors, + "docs/hf_dataset_card.md", + hf_card, + "Held-out mission, train missions, sample counts, and provenance", + ) + + expected_v9_pretty_name = "SpaceBio-Bench v9 Public Bulk Metadata Catalog" + if v9_front_matter.get("pretty_name") != expected_v9_pretty_name: + errors.append( + "docs/v9_hf_dataset_card.md: pretty_name does not match expected " + f"{expected_v9_pretty_name!r}" + ) + require_contains( + errors, + "docs/v9_hf_dataset_card.md", + v9_hf_card, + "# SpaceBio-Bench v9 Public Bulk Metadata Catalog", + ) + require_contains( + errors, + "docs/v9_hf_dataset_card.md", + v9_hf_card, + "Current catalog scope:", + ) + require_contains( + errors, + "v9/README.md", + v9_readme, + "# SpaceBio-Bench v9 Public Bulk Metadata Catalog", + ) + require_contains( + errors, + "v9/reports/README.md", + v9_reports_readme, + "# v9 Public Bulk Reports", + ) + v9_public_text = "\n".join([v9_hf_card, v9_readme, v9_reports_readme]) + for forbidden in ( + "Public Bulk Metadata Alpha", + "metadata-alpha", + "alpha snapshot", + "alpha-boundary", + "claim boundary", + "snapshot_decision", + "allowed language", + "blocked language", + "Maintenance Notes", + "Provenance And Integrity", + "not frozen", + "should not be used", + "release-readiness blockers", + ): + require_absent(errors, "v9 public docs", v9_public_text, forbidden) zenodo_text = json.dumps(zenodo, sort_keys=True) require_contains(errors, ".zenodo.json", zenodo.get("title", ""), "SpaceBio-Bench") diff --git a/v9/README.md b/v9/README.md index bee1b59..c7884cd 100644 --- a/v9/README.md +++ b/v9/README.md @@ -1,47 +1,42 @@ -# SpaceBio-Bench v9 Public Bulk Metadata Alpha +# SpaceBio-Bench v9 Public Bulk Metadata Catalog -Status: metadata-only alpha subset, not a frozen payload release. +This directory contains the public metadata catalog for the SpaceBio-Bench v9 +bulk RNA-seq task surface. It is organized for direct review of task manifests, +fold indexes, public OSDR source coverage, checksum-audit summaries, and +reference baseline outputs. -This directory contains the small public evidence subset referenced by the -SpaceBio-Bench system, evaluation, release-readiness, and claim-boundary cards. -It is intended to make the public bulk metadata-alpha claims inspectable from -the default GitHub branch without publishing payload matrices or draft extension -lanes. +For the reader-facing card, see +[`docs/v9_hf_dataset_card.md`](../docs/v9_hf_dataset_card.md). ## Included - `task_manifests/*.json`: eight public bulk LOMO task manifests. - `task_manifest_index.csv` and `.json`: task registry summary. - `task_data_index.csv` and `.json`: fold-level row-count and path registry. -- `source_inventory.csv` and `.json`: 22 public OSDR source rows. +- `source_inventory.csv` and `.json`: public OSDR source rows. - `source_checksum_audit.csv` and `.json`: OSDR API and checksum-manifest - evidence for the public bulk source rows. -- `datapackage.draft.json`: draft Frictionless Data Package descriptor for the - metadata-only alpha subset. + summary rows. +- `datapackage.draft.json`: Frictionless Data Package descriptor for the + metadata catalog. - `reports/bulk_lomo_baseline_summary.csv` and `.json`: normalized summary of - scaffold baselines. -- `reports/nearest_centroid/bulk_lomo_summary.csv` and `.json`: nearest-centroid - scaffold baseline summary. + reference baselines. +- `reports/nearest_centroid/bulk_lomo_summary.csv` and `.json`: + nearest-centroid baseline summary. - `reports/sklearn_baselines/bulk_lomo_summary.csv` and `.json`: PCA-LR and - L2 logistic-regression scaffold baseline summary. -- Per-task scaffold baseline `predictions.csv`, `metrics.json`, and - `run_manifest.json` files referenced by `datapackage.draft.json`. -- `reports/public_bulk_alpha_gap_matrix/`: alpha-boundary gap and payload-hash - blocker tables. -- `reports/public_bulk_alpha_snapshot_decision/`: metadata-alpha decision, - allowed language, blocked language, and next-action tables. - -## Excluded - -- Fold-level payload matrices such as `train_X.csv` and `test_X.csv`. -- Local payload mirrors and payload-level SHA-256 manifests. -- Single-cell, organoid, multispecies, and other draft extension-lane outputs. -- Any DOI/archive-ready payload bundle. -- Any state-of-the-art leaderboard or clinical, crew-health, intervention, or - countermeasure claim. + L2 logistic-regression baseline summary. +- Per-task baseline `predictions.csv`, `metrics.json`, and `run_manifest.json` + files referenced by `datapackage.draft.json`. + +## Scope Notes + +- The v9 metadata catalog is separate from archived fold-matrix payload + bundles. +- Controlled-access human sequence data is not part of this public bulk catalog. +- Clinical, crew-health, intervention, countermeasure, operational-readiness, + and state-of-the-art leaderboard claims are outside the catalog scope. ## Interpretation -The files here support claims about task/source/provenance metadata, scaffold -baseline plumbing, and release-readiness blockers. They do not support frozen -payload, biological mechanism, translational, or operational-readiness claims. +The files here support review of task definitions, source coverage, fold +indexes, checksum-audit summaries, and baseline workflow outputs. They are best +read as a public metadata catalog for method-development and release review. diff --git a/v9/reports/README.md b/v9/reports/README.md index ecebda9..4698fa8 100644 --- a/v9/reports/README.md +++ b/v9/reports/README.md @@ -1,23 +1,16 @@ -# v9 Public Bulk Metadata-Alpha Reports +# v9 Public Bulk Reports -Status: scaffold evidence, not a frozen public benchmark release. - -This directory contains compact summary reports for the v9 public bulk -metadata-alpha subset: +This directory contains compact report tables for the v9 public bulk metadata +catalog. - `bulk_lomo_baseline_summary.csv` and `.json`: cross-baseline summary across - eight public bulk LOMO task manifests and three scaffold baselines. + eight public bulk LOMO task manifests and three baseline families. - `nearest_centroid/bulk_lomo_summary.csv` and `.json`: nearest-centroid - scaffold baseline summary. + baseline summary. - `sklearn_baselines/bulk_lomo_summary.csv` and `.json`: PCA-LR and L2 - logistic-regression scaffold baseline summary. -- Per-task scaffold baseline `predictions.csv`, `metrics.json`, and - `run_manifest.json` files for the baseline families above. -- `public_bulk_alpha_gap_matrix/`: payload-boundary and package-readiness gap - tables. -- `public_bulk_alpha_snapshot_decision/`: selected metadata-alpha release path, - allowed language, blocked language, and next actions. + logistic-regression baseline summary. +- Per-task baseline `predictions.csv`, `metrics.json`, and `run_manifest.json` + files for the baseline families above. -The baseline rows validate evaluation plumbing and provide comparison anchors. -They are not tuned leaderboard endpoints and should not be described as model -superiority evidence. +The baseline rows are reproducible reference anchors for the public bulk task +catalog. Use per-task rows together with pooled means when comparing methods.