Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 16 additions & 14 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,24 @@ cff-version: 1.2.0
message: "If you use this benchmark in your research, please cite as below."
type: software

title: "GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models"
title: "SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark"
version: "7.1.2"
date-released: "2026-06-05"

abstract: >
GeneLab Benchmark is a comprehensive benchmark for evaluating AI and machine learning
models on NASA OSDR (Open Science Data Repository) mouse spaceflight transcriptomics
data. It provides standardized Leave-One-Mission-Out (LOMO) cross-validation splits,
multi-method evaluation (8 classifiers x 4 feature types), foundation model comparisons
(Geneformer, scGPT, UCE, scFoundation), and an evaluation framework for multi-tissue
spaceflight detection across 8 tissues (liver, gastrocnemius, kidney, thymus, skin, eye,
lung, colon), cross-mission transfer, cross-tissue transfer, multi-species analysis
(mouse, Drosophila), spatial transcriptomics (Visium), and single-cell RNA-seq
(RRRM-1, RRRM-2). The v7 release also includes unified scPRINT2 and graph
neural network benchmarking, plus cross-method synthesis across classical,
foundation-model, and graph-based approaches. The v7.1.2 release is a
documentation, public-card, and metadata patch; it does
SpaceBio-Bench / GeneLab Benchmark is a comprehensive benchmark for evaluating
AI and machine learning models on NASA OSDR (Open Science Data Repository)
mouse spaceflight transcriptomics data. It provides standardized
Leave-One-Mission-Out (LOMO) cross-validation splits, multi-method evaluation
(8 classifiers x 4 feature types), foundation model comparisons (Geneformer,
scGPT, UCE, scFoundation), and an evaluation framework for multi-tissue
spaceflight detection across 8 tissues (liver, gastrocnemius, kidney, thymus,
skin, eye, lung, colon), cross-mission transfer, cross-tissue transfer,
multi-species analysis (mouse, Drosophila), spatial transcriptomics (Visium),
and single-cell RNA-seq (RRRM-1, RRRM-2). The v7 release also includes
unified scPRINT2 and graph neural network benchmarking, plus cross-method
synthesis across classical, foundation-model, and graph-based approaches. The
v7.1.2 release is a documentation, public-card, and metadata patch; it does
not introduce new benchmark result generation.

authors:
Expand All @@ -37,6 +38,7 @@ keywords:
- machine learning
- NASA OSDR
- GeneLab
- SpaceBio-Bench
- LOMO
- cross-mission transfer
- foundation model
Expand Down Expand Up @@ -65,7 +67,7 @@ references:

preferred-citation:
type: article
title: "GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models"
title: "SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark"
authors:
- family-names: "Kim"
given-names: "JangKeun"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ same citation metadata in the repository citation panel.

```bibtex
@dataset{kim2026genelab,
title = {GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models},
title = {SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark},
author = {Kim, JangKeun},
year = {2026},
url = {https://huggingface.co/datasets/jang1563/genelab-benchmark},
Expand Down
2 changes: 1 addition & 1 deletion docs/hf_dataset_card.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ Please cite the software and benchmark using the GitHub `CITATION.cff` metadata.

```bibtex
@dataset{kim2026genelab,
title = {GeneLab Benchmark: A Multi-Tissue Spaceflight Transcriptomics Benchmark for AI/ML Models},
title = {SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark},
author = {Kim, JangKeun},
year = {2026},
url = {https://huggingface.co/datasets/jang1563/genelab-benchmark},
Expand Down
5 changes: 5 additions & 0 deletions scripts/validate_public_docs_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ def validate_public_docs() -> list[str]:
f"CITATION.cff: version {actual_version!r} does not match "
f"release manifest citation_version {citation_version!r}"
)
citation_title = parse_cff_scalar(citation, "title")
if citation_title and "SpaceBio-Bench / GeneLab Benchmark" not in citation_title:
errors.append("CITATION.cff: title should include SpaceBio-Bench / GeneLab Benchmark")

require_contains(errors, "README.md", readme, "# SpaceBio-Bench")
require_contains(errors, "README.md", readme, "Former public name: **GeneLab Benchmark**")
Expand All @@ -86,6 +89,7 @@ def validate_public_docs() -> list[str]:
require_contains(errors, "README.md", readme, "docs/assets/hf_benchmark_summary.png")
require_contains(errors, "README.md", readme, "CONTRIBUTING.md")
require_contains(errors, "README.md", readme, "docs/submission_format.md")
require_contains(errors, "README.md", readme, "SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out")
require_absent(errors, "README.md", readme, "Version: v7.0 (2026-04-12)")
require_absent(errors, "README.md", readme, "Status: **v1–v7 Complete**")

Expand All @@ -104,6 +108,7 @@ def validate_public_docs() -> list[str]:
require_contains(errors, "docs/hf_dataset_card.md", hf_card, "Public status: **v7.1.2 public-card/metadata patch")
require_contains(errors, "docs/hf_dataset_card.md", hf_card, "Dataset freeze: **2026-03-01**")
require_contains(errors, "docs/hf_dataset_card.md", hf_card, "repo_id = \"jang1563/genelab-benchmark\"")
require_contains(errors, "docs/hf_dataset_card.md", hf_card, "SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out")

zenodo_text = json.dumps(zenodo, sort_keys=True)
require_contains(errors, ".zenodo.json", zenodo.get("title", ""), "SpaceBio-Bench")
Expand Down
12 changes: 12 additions & 0 deletions tests/test_review_fixes.py
Original file line number Diff line number Diff line change
Expand Up @@ -326,11 +326,23 @@ def test_public_release_metadata_uses_v7_consistently(self):
self.assertIn("data freeze 2026-03-01", hf_card)
self.assertIn('version: "7.1.2"', citation)
self.assertIn('date-released: "2026-06-05"', citation)
self.assertIn(
'title: "SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark"',
citation,
)
self.assertIn(
'notes: "Manuscript in preparation; v7.1.2 documentation, public-card, and metadata patch."',
citation,
)
self.assertIn("documentation, public-card, and metadata patch", citation)
self.assertIn(
"title = {SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark}",
readme,
)
self.assertIn(
"title = {SpaceBio-Bench / GeneLab Benchmark: Mission-Held-Out Spaceflight Transcriptomics Benchmark}",
hf_card,
)
self.assertNotIn('version: "5.0.0"', citation)
self.assertNotIn("Target journal:", citation)

Expand Down