Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/regression-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: "3.11"
cache: pip
Expand Down
10 changes: 5 additions & 5 deletions .zenodo.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"title": "GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models",
"upload_type": "software",
"version": "7.1.0",
"description": "GeneLab Benchmark provides standardized mission-held-out benchmark tasks for evaluating machine learning and foundation-model behavior on public NASA OSDR mouse spaceflight transcriptomics data. This archive metadata describes the v7.1 public documentation and release-candidate surface. v7.1 is a documentation and public metadata consistency patch and does not introduce new benchmark result generation.",
"version": "7.1.1",
"description": "GeneLab Benchmark provides standardized mission-held-out benchmark tasks for evaluating machine learning and foundation-model behavior on public NASA OSDR mouse spaceflight transcriptomics data. This archive metadata describes the current public documentation, card-pack, and release-candidate surface. v7.1.1 is a documentation, public-card, and metadata consistency patch and does not introduce new benchmark result generation.",
"creators": [
{
"name": "Kang, Jaeyoung",
"affiliation": "Cornell University"
"name": "Kim, Jihoon",
"affiliation": "Weill Cornell Medicine"
}
],
"access_right": "open",
Expand Down Expand Up @@ -40,5 +40,5 @@
"scheme": "url"
}
],
"notes": "Source biological data are derived from NASA OSDR public datasets and should be cited at the individual OSDR study level where applicable. This repository archive is not a clinical, crew-health, countermeasure, or intervention recommendation artifact. Final DOI publication should confirm author order, manuscript title, release tag, and upstream dataset citation metadata before deposition."
"notes": "Source biological data are derived from NASA OSDR public datasets and should be cited at the individual OSDR study level where applicable. This repository archive is not a clinical, crew-health, countermeasure, or intervention recommendation artifact. Final DOI publication should confirm the final manuscript author list, manuscript title, release tag, archive checksum, and upstream dataset citation metadata before deposition."
}
20 changes: 10 additions & 10 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ message: "If you use this benchmark in your research, please cite as below."
type: software

title: "GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models"
version: "7.1.0"
date-released: "2026-05-09"
version: "7.1.1"
date-released: "2026-06-05"

abstract: >
GeneLab Benchmark is a comprehensive benchmark for evaluating AI and machine learning
Expand All @@ -17,14 +17,14 @@ abstract: >
(mouse, Drosophila), spatial transcriptomics (Visium), and single-cell RNA-seq
(RRRM-1, RRRM-2). The v7 release also includes unified scPRINT2 and graph
neural network benchmarking, plus cross-method synthesis across classical,
foundation-model, and graph-based approaches. The v7.1 release is a
documentation and public metadata consistency patch; it does not introduce
new benchmark result generation.
foundation-model, and graph-based approaches. The v7.1.1 release is a
documentation, public-card, and metadata consistency patch; it does not
introduce new benchmark result generation.

authors:
- family-names: "Kim"
given-names: "JangKeun"
affiliation: "Cornell University"
given-names: "Jihoon"
affiliation: "Weill Cornell Medicine"

repository-code: "https://github.com/jang1563/GeneLab_benchmark"
url: "https://github.com/jang1563/GeneLab_benchmark"
Expand Down Expand Up @@ -68,8 +68,8 @@ preferred-citation:
title: "GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models"
authors:
- family-names: "Kim"
given-names: "JangKeun"
affiliation: "Cornell University"
given-names: "Jihoon"
affiliation: "Weill Cornell Medicine"
status: "in-preparation"
year: 2026
notes: "Manuscript in preparation; v7.1 documentation consistency patch."
notes: "Manuscript in preparation; v7.1.1 documentation, public-card, and metadata consistency patch."
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2026 Jaeyoung Kang
Copyright (c) 2026 Jihoon Kim

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ defines paper-archive scope. [`docs/RELEASE_ARCHIVE_MANIFEST.md`](docs/RELEASE_A
and [`docs/RELEASE_ARCHIVE_CHECKLIST.md`](docs/RELEASE_ARCHIVE_CHECKLIST.md)
track Zenodo metadata readiness and final DOI/tag gates.

Archive note: the existing `v7.1` GitHub release predates this public card and
archive-metadata polish. Use `v3` for the current detailed evidence surface and
`main` for the portfolio-facing entry point until the next DOI-oriented patch
tag is minted.

Branch note: `v3` is the canonical evidence branch for detailed v9
metadata-alpha artifacts. The default `main` branch mirrors the portfolio-facing
card surface for first-time readers.
Expand Down Expand Up @@ -415,10 +420,10 @@ Key methodological choices underpinning this benchmark:
```bibtex
@dataset{kim2026genelab,
title = {GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models},
author = {Kim, JangKeun},
author = {Kim, Jihoon},
year = {2026},
url = {https://huggingface.co/datasets/jang1563/genelab-benchmark},
note = {v7.0 with v7.1 documentation consistency patch; data freeze 2026-03-01}
note = {v7.0 with v7.1.1 documentation, public-card, and metadata consistency patch; data freeze 2026-03-01}
}
```

Expand Down
20 changes: 11 additions & 9 deletions docs/RELEASE_ARCHIVE_CARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: GeneLab Benchmark Release Archive Card
page_type: release_archive_card
status: release_candidate_metadata_ready
last_reviewed: 2026-06-04
last_reviewed: 2026-06-05
claim_boundary: archive_metadata_no_new_result_or_payload_claim
---

Expand All @@ -20,13 +20,13 @@ It does not create a DOI, approve a new result release, or freeze v9 payloads.

| Field | Current value |
|---|---|
| Release candidate | v7.1 public documentation and card-pack surface |
| Release candidate | Current `main` public documentation and card-pack surface |
| Repository branch | `main` as public entry point; `v3` as detailed v9 evidence branch |
| Current result boundary | v1-v7 canonical historical benchmark surface |
| Documentation patch | v7.1 consistency patch, no new result generation |
| Documentation patch | v7.1.1 metadata/card consistency patch, no new result generation |
| Dataset payload boundary | HF feature-matrix package plus GitHub task/evaluation metadata |
| DOI status | Not minted in this card |
| Archive status | Release-candidate metadata ready; final tag/DOI still pending |
| Archive status | Release-candidate metadata ready; next DOI-oriented tag, checksum, and DOI still pending |

## Archive Contents

Expand Down Expand Up @@ -57,9 +57,11 @@ The archive candidate should not claim to include:

Before a DOI-oriented release:

- Confirm author order, affiliations, and manuscript title in `CITATION.cff`
and `.zenodo.json`.
- Create an annotated GitHub release tag from the final commit.
- Confirm the final manuscript author list, affiliations, and manuscript title
in `CITATION.cff` and `.zenodo.json`.
- Create an annotated GitHub release tag from a final commit that includes the
public card and release-archive metadata polish. The existing `v7.1` tag
predates these files.
- Generate and store a checksum for the release source archive.
- Confirm the Hugging Face dataset card links to the intended canonical branch
and release tag.
Expand All @@ -75,10 +77,10 @@ Before a DOI-oriented release:
| Gate | Status | Note |
|---|---|---|
| Public README entry point | Pass | Links card pack and portfolio brief |
| Citation metadata | Partial pass | `CITATION.cff` exists; author/title review still needed |
| Citation metadata | Pass for current public identity | `CITATION.cff` exists; final manuscript author list/title review still needed |
| Zenodo metadata | Candidate ready | `.zenodo.json` added for DOI deposition |
| License | Pass for code | MIT license present |
| HF dataset card | Pass | Links transparency cards on canonical `v3` |
| HF dataset card | Pass | Links transparency cards on current public `main` |
| Claim-boundary cards | Pass | System, evaluation, readiness, and claim cards are public-review ready |
| Release archive manifest | Pass | `docs/RELEASE_ARCHIVE_MANIFEST.md` added |
| Source archive checksum | Pending final tag | Generate after final GitHub release tag |
Expand Down
15 changes: 9 additions & 6 deletions docs/RELEASE_ARCHIVE_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: GeneLab Benchmark Release Archive Checklist
page_type: release_archive_checklist
status: release_candidate_checklist
last_reviewed: 2026-06-04
last_reviewed: 2026-06-05
claim_boundary: checklist_no_new_release_or_doi_claim
---

Expand All @@ -17,8 +17,10 @@ must wait until the final manuscript, tag, and DOI are available.

- Root `README.md` presents the project, canonical result doc, transparency
card pack, portfolio brief, and HF dataset link.
- `CITATION.cff` exists and is machine-readable citation metadata.
- `.zenodo.json` exists as candidate Zenodo metadata.
- `CITATION.cff` exists and is machine-readable citation metadata using the
current public repository identity.
- `.zenodo.json` exists as candidate Zenodo metadata using the current public
repository identity.
- `LICENSE` is present and declares MIT for code.
- `docs/hf_dataset_card.md` is synchronized with the public HF card surface.
- SpaceBio-Bench system, evaluation, readiness, claim, and transparency cards
Expand All @@ -30,16 +32,17 @@ must wait until the final manuscript, tag, and DOI are available.

## Needs Final Author Or Manuscript Review

- Confirm author order and affiliations in `CITATION.cff`.
- Confirm creator metadata in `.zenodo.json`.
- Confirm final manuscript author list and affiliations in `CITATION.cff`.
- Confirm final creator metadata in `.zenodo.json`.
- Confirm final paper title and whether the archive title should match exactly.
- Confirm final version string and release date.
- Add DOI after Zenodo or the selected archive mints it.
- Add dataset-specific OSDR citations for the manuscript subset.

## Needs Final Release Action

- Create annotated Git tag.
- Create annotated Git tag from the final public-card/archive-metadata commit
(recommended next patch tag: `v7.1.1`).
- Create GitHub release from the tag.
- Verify Zenodo deposition metadata.
- Download source archive and record SHA-256 checksum.
Expand Down
19 changes: 12 additions & 7 deletions docs/RELEASE_ARCHIVE_MANIFEST.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: GeneLab Benchmark Release Archive Manifest
page_type: release_archive_manifest
status: release_candidate_manifest
last_reviewed: 2026-06-04
last_reviewed: 2026-06-05
claim_boundary: manifest_no_new_release_or_payload_claim
---

Expand Down Expand Up @@ -60,10 +60,12 @@ complete.

## Suggested Final Archive Procedure

1. Confirm final manuscript title, author order, affiliations, and release
1. Confirm final manuscript title, final author list, affiliations, and release
version.
2. Update `CITATION.cff` and `.zenodo.json` with final metadata.
3. Create an annotated Git tag, for example `v7.1.0`.
3. Create an annotated Git tag from the final public-card/archive-metadata
commit, for example `v7.1.1`. The existing `v7.1` GitHub release predates
the card-pack and archive-metadata files.
4. Create a GitHub release from that tag.
5. Let Zenodo mint the DOI from the GitHub release, or deposit manually with
`.zenodo.json` metadata.
Expand All @@ -77,9 +79,9 @@ complete.
Use these commands after the final release tag exists:

```bash
git archive --format=tar.gz --prefix=GeneLab_benchmark-v7.1.0/ \
-o GeneLab_benchmark-v7.1.0.tar.gz v7.1.0
shasum -a 256 GeneLab_benchmark-v7.1.0.tar.gz
git archive --format=tar.gz --prefix=GeneLab_benchmark-v7.1.1/ \
-o GeneLab_benchmark-v7.1.1.tar.gz v7.1.1
shasum -a 256 GeneLab_benchmark-v7.1.1.tar.gz
```

For a source checkout:
Expand All @@ -94,7 +96,10 @@ du -sh .
## Known Archive Caveats

- The paper archive DOI has not yet been minted.
- Author metadata should be reviewed before final deposition.
- The existing `v7.1` GitHub release predates the current card-pack and
archive-metadata polish; cite a successor tag once minted.
- Final manuscript author-list metadata should be reviewed before final
deposition.
- NASA OSDR source datasets need dataset-specific citation review for the
manuscript subset.
- v9 metadata-alpha evidence should remain separate from v1-v7 final result
Expand Down
19 changes: 11 additions & 8 deletions docs/hf_dataset_card.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,18 @@ system scope, evaluation protocol, release-readiness gates, and claim boundaries
can be reviewed together:

- Transparency card pack:
https://github.com/jang1563/GeneLab_benchmark/blob/v3/docs/SPACEBIOBENCH_TRANSPARENCY_CARD_PACK.md
https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_TRANSPARENCY_CARD_PACK.md
- System card:
https://github.com/jang1563/GeneLab_benchmark/blob/v3/docs/SPACEBIOBENCH_SYSTEM_CARD.md
https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_SYSTEM_CARD.md
- Evaluation card:
https://github.com/jang1563/GeneLab_benchmark/blob/v3/docs/SPACEBIOBENCH_EVALUATION_CARD.md
https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_EVALUATION_CARD.md
- Release readiness card:
https://github.com/jang1563/GeneLab_benchmark/blob/v3/docs/SPACEBIOBENCH_RELEASE_READINESS_CARD.md
https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_RELEASE_READINESS_CARD.md
- Claim register:
https://github.com/jang1563/GeneLab_benchmark/blob/v3/docs/SPACEBIOBENCH_CLAIM_REGISTER.md
https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/SPACEBIOBENCH_CLAIM_REGISTER.md

For DOI-oriented citation, use the final GitHub release tag and archive DOI
recorded in the release archive manifest once minted.

Current release wording: v1-v7 are the canonical historical benchmark and
reviewer-facing feature-matrix surface; v8 and v9 are documented as extension or
Expand Down Expand Up @@ -291,13 +294,13 @@ Lung and Colon additionally include Basal Control samples treated as ground cont
*(Manuscript in preparation)*

```bibtex
@dataset{kang2026genelab,
@dataset{kim2026genelab,
title = {GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics
Benchmark for AI/ML Models},
author = {Kang, Jaeyoung},
author = {Kim, Jihoon},
year = {2026},
url = {https://huggingface.co/datasets/jang1563/genelab-benchmark},
note = {v7.0 with v7.1 documentation consistency patch; data freeze 2026-03-01}
note = {v7.0 with v7.1.1 documentation, public-card, and metadata consistency patch; data freeze 2026-03-01}
}
```

Expand Down
19 changes: 13 additions & 6 deletions tests/test_review_fixes.py
Original file line number Diff line number Diff line change
Expand Up @@ -312,13 +312,20 @@ def test_public_release_metadata_uses_v7_consistently(self):

self.assertIn("Version: v7.0 (2026-04-12) | Dataset freeze: 2026-03-01", readme)
self.assertIn("Canonical v7.1 documentation source:", readme)
self.assertIn("note = {v7.0 with v7.1 documentation consistency patch; data freeze 2026-03-01}", readme)
self.assertIn("note = {v7.0 with v7.1.1 documentation, public-card, and metadata consistency patch; data freeze 2026-03-01}", readme)
self.assertIn("Version: v7.0 with v7.1 documentation consistency patch | Dataset freeze: 2026-03-01", hf_card)
self.assertIn("note = {v7.0 with v7.1 documentation consistency patch; data freeze 2026-03-01}", hf_card)
self.assertIn('version: "7.1.0"', citation)
self.assertIn('date-released: "2026-05-09"', citation)
self.assertIn('notes: "Manuscript in preparation; v7.1 documentation consistency patch."', citation)
self.assertIn("documentation and public metadata consistency patch", citation)
self.assertIn("note = {v7.0 with v7.1.1 documentation, public-card, and metadata consistency patch; data freeze 2026-03-01}", hf_card)
self.assertIn('version: "7.1.1"', citation)
self.assertIn('date-released: "2026-06-05"', citation)
self.assertIn('notes: "Manuscript in preparation; v7.1.1 documentation, public-card, and metadata consistency patch."', citation)
self.assertIn("documentation, public-card, and metadata consistency patch", citation)
self.assertIn('family-names: "Kim"', citation)
self.assertIn('given-names: "Jihoon"', citation)
self.assertIn('affiliation: "Weill Cornell Medicine"', citation)
self.assertNotIn("Kang", citation)
self.assertNotIn("Jaeyoung", citation)
self.assertNotIn("JangKeun", readme + hf_card + citation)
self.assertNotIn("blob/v3/docs/SPACEBIOBENCH", hf_card)
self.assertNotIn('version: "5.0.0"', citation)
self.assertNotIn("Target journal:", citation)

Expand Down