Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .github/ISSUE_TEMPLATE/benchmark_submission.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Benchmark submission discussion
description: Propose or discuss a model submission for the public benchmark folds.
title: "[Submission]: "
labels:
- benchmark
body:
- type: input
id: task_id
attributes:
label: Task ID or fold package
placeholder: A5_skin_lomo / fold_RR-7_test
validations:
required: true
- type: input
id: model_name
attributes:
label: Model name
placeholder: PCA-LR, Geneformer adapter, custom baseline
validations:
required: true
- type: textarea
id: method
attributes:
label: Method summary
description: Describe preprocessing, feature representation, training split, and prediction output.
validations:
required: true
- type: textarea
id: validation
attributes:
label: Validation command
description: Include the command used to validate or evaluate the JSON submission.
render: bash
validations:
required: false
- type: checkboxes
id: checks
attributes:
label: Checks
options:
- label: I followed docs/submission_format.md.
required: true
- label: I did not use held-out test labels during model training or feature selection.
required: true
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Hugging Face Dataset
url: https://huggingface.co/datasets/jang1563/genelab-benchmark
about: Download the public fold package and dataset card.
- name: Submission Format
url: https://github.com/jang1563/GeneLab_benchmark/blob/main/docs/submission_format.md
about: Read the public prediction-submission JSON contract.
36 changes: 36 additions & 0 deletions .github/ISSUE_TEMPLATE/data_or_docs_issue.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Data or documentation issue
description: Report a broken path, unclear public documentation, or release-surface inconsistency.
title: "[Data/docs]: "
labels:
- documentation
body:
- type: textarea
id: location
attributes:
label: Location
description: Link to the file, section, dataset path, or release asset.
placeholder: README.md, docs/hf_dataset_card.md, A5_skin_lomo/fold_RR-7_test/train_X.csv
validations:
required: true
- type: textarea
id: issue
attributes:
label: What is wrong or unclear?
description: Describe the observed issue and what you expected instead.
validations:
required: true
- type: textarea
id: reproduction
attributes:
label: Reproduction command or access path
description: Include a command, URL, or short sequence that reproduces the issue.
render: bash
validations:
required: false
- type: textarea
id: environment
attributes:
label: Environment
description: Include OS, Python version, package manager, or browser if relevant.
validations:
required: false
36 changes: 36 additions & 0 deletions .github/ISSUE_TEMPLATE/reproducibility_issue.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Reproducibility issue
description: Report a command, dependency, or release QA problem.
title: "[Repro]: "
labels:
- reproducibility
body:
- type: textarea
id: command
attributes:
label: Command
description: Paste the command or workflow step that failed.
render: bash
validations:
required: true
- type: textarea
id: output
attributes:
label: Output
description: Include the relevant error or unexpected output.
render: text
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
description: Describe what you expected the command to produce.
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
description: Include OS, Python version, package versions, and whether data came from GitHub or Hugging Face.
validations:
required: false
15 changes: 15 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## Summary

- Briefly describe the change.

## Verification

- [ ] `make release-qa`
- [ ] `make hpc-public-qa`
- [ ] Relevant unit tests or scripts:

## Public Surface Check

- [ ] README/HF/CITATION/release labels remain consistent
- [ ] No stale internal wording, placeholder paths, or obsolete release labels
- [ ] New public artifacts are linked from the appropriate card or manifest
50 changes: 50 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Contributing To SpaceBio-Bench

Thank you for helping improve SpaceBio-Bench / GeneLab Benchmark. The most
useful contributions are small, reproducible, and tied to the public release
surface.

## Good First Contributions

- Report a broken download path, stale link, or unclear dataset-card wording.
- Reproduce a public fold or release QA command and share the exact command and
output.
- Improve documentation around task layout, submission format, or citation.
- Submit benchmark predictions using the public JSON format.

## Benchmark Submissions

Prediction submissions should follow
[docs/submission_format.md](docs/submission_format.md). Include:

- task ID and fold IDs;
- model name and short model description;
- exact preprocessing or feature representation;
- the command used to validate or evaluate the submission.

Use a GitHub issue for discussion before opening a pull request with benchmark
results or new public artifacts.

## Pull Requests

Keep pull requests focused. For documentation, metadata, or public-card changes,
run:

```bash
make release-qa
make hpc-public-qa
```

For code changes that touch benchmark behavior, also run the relevant unit tests
or scripts and describe what changed in the PR body.

## Public Release Boundaries

Use the release labels in [README.md](README.md) and
[release/release_manifest.json](release/release_manifest.json). In short:

- v7.1 is the canonical historical result surface.
- v7.1.2 is a public-card and metadata patch over v7.1 results.
- v9 public bulk is a metadata catalog and baseline-summary surface.

Please avoid mixing future-lane claims into the v7.1 result surface.
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ scripts/ Data, evaluation, upload, validation, and figure scripts
| Release readiness | [docs/SPACEBIOBENCH_RELEASE_READINESS_CARD.md](docs/SPACEBIOBENCH_RELEASE_READINESS_CARD.md) |
| Hugging Face dataset card source | [docs/hf_dataset_card.md](docs/hf_dataset_card.md) |
| v9 metadata catalog card source | [docs/v9_hf_dataset_card.md](docs/v9_hf_dataset_card.md) |
| Contributing and submissions | [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/submission_format.md](docs/submission_format.md) |
| Machine-readable release state | [release/release_manifest.json](release/release_manifest.json) |

## Data And Release Notes
Expand All @@ -179,6 +180,12 @@ Release labels are intentionally separated:
- **v7.1**: canonical historical result surface and citation target.
- **v9 public bulk**: metadata catalog and baseline-summary surface.

## Contributing

Use [CONTRIBUTING.md](CONTRIBUTING.md) for documentation fixes, data-access
reports, reproducibility issues, and public benchmark submissions. Prediction
submissions should follow [docs/submission_format.md](docs/submission_format.md).

## Citation

Please cite the software using [CITATION.cff](CITATION.cff). GitHub renders the
Expand Down
2 changes: 2 additions & 0 deletions scripts/validate_public_docs_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ def validate_public_docs() -> list[str]:
require_contains(errors, "README.md", readme, "release/release_manifest.json")
require_contains(errors, "README.md", readme, "https://huggingface.co/datasets/jang1563/genelab-benchmark")
require_contains(errors, "README.md", readme, "docs/assets/hf_benchmark_summary.png")
require_contains(errors, "README.md", readme, "CONTRIBUTING.md")
require_contains(errors, "README.md", readme, "docs/submission_format.md")
require_absent(errors, "README.md", readme, "Version: v7.0 (2026-04-12)")
require_absent(errors, "README.md", readme, "Status: **v1–v7 Complete**")

Expand Down