Make "I want to add an example" a documented path instead of a reverse-engineering exercise

CONTRIBUTING opened with "this guide will walk you through the process of implementing a new
oracle", and Current Priorities listed three kinds of contribution, all of them models. Read
literally, someone with a variant they care about and no intention of porting a neural network is
told in sentence one that they are in the wrong document — and adding a worked example is the
cheapest useful contribution chorus can receive.

It was also genuinely undocumented. Four regeneration scripts own different example types, and the
matrix mapping example -> script -> valid --oracle -> conda env lived in **CLAUDE.md** (a file for
Claude sessions), the CHANGELOG, and a May audit report. None of those is where a contributor looks.
That matrix exists because getting it wrong is the single most common way a regeneration silently
does nothing:

* `regenerate_multioracle.py` accepts chrombpnet / cherimoya / legnet / alphagenome and pointedly
  **not** enformer, whose report comes from `regenerate_examples.py` instead;
* it also has no `--gpu`, while the other two do — passing one is an argparse error that scrolls
  past in a tailed log;
* the wrong conda env does not fail fast. A missing oracle package logs `Failed to load <track>`
  once per track and carries on, so a run can spend an hour loading nothing.

So: a routing table in the first screen (oracle / example / bug fix, with sizes), a
"Contributing an example or walkthrough" section carrying the verified matrix, the real entry shape
copied from `ENFORMER_EXAMPLES`, the four traps, and the six guard tests to expect — all of which
collect (138 tests), unlike the browser recipe that shipped collecting 0. Priorities now name
examples, doc corrections and tests alongside models. `examples/*/README.md` point at the one
canonical section rather than restating it.

Guard: 8 tests pinning the table to the scripts' argparse definitions rather than to a proofread —
every documented `--oracle` value must be a real choice (negations excluded and covered separately),
the no-enformer exception must remain true in both directions, the `--gpu` asymmetry must stay
correctly described, the notebook generator must stay argument-free, and every test file the section
tells a contributor to run must exist.

Fast suite 1,993 passed / 30 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: lucapinello

CONTRIBUTING.md

@@ -1,6 +1,19 @@
 # Contributing to Chorus
 
-Thank you for your interest in contributing to Chorus! This guide will walk you through the process of implementing a new oracle (genomic sequence prediction model) step by step.
+Thank you for your interest in contributing to Chorus! Most of this guide is a step-by-step for
+implementing a new **oracle** (a genomic sequence prediction model), because that is the largest
+kind of contribution — but it is not the only one we want, and the smaller ones are not lesser.
+
+**Start here — what are you contributing?**
+
+| you want to add | go to | rough size |
+|---|---|---|
+| a new model / oracle | [Implementing a new oracle](#step-by-step-guide-to-implementing-a-new-oracle) below | a day or more; 9 registration sites, an env, a background |
+| a **worked example or walkthrough** | [Contributing an example](#contributing-an-example-or-walkthrough) | an hour; one list entry and one script run |
+| a bug fix, a doc fix, a test | [Running the tests](#running-the-tests) then open the PR | minutes |
+
+You do not need a HuggingFace account, our GPU, or our HuggingFace dataset for any of these — see
+[You do not need our infrastructure](#you-do-not-need-our-infrastructure-to-develop-or-to-open-the-pr).
 
 ## Overview
 
@@ -432,6 +445,87 @@ BORZOI_ENV_CONFIG = {
 
 6. **Documentation**: Include docstrings for all public methods
 
+## Contributing an example or walkthrough
+
+The fastest useful contribution. If there is a variant, locus or question you care about and chorus
+already ships an oracle that can answer it, adding it as a worked example is roughly one list entry
+plus one script run.
+
+Chorus commits two different kinds of example:
+
+* **Walkthroughs** — `examples/walkthroughs/<category>/<name>/`. A real variant or locus, pre-run,
+  committed together with its HTML report, JSON and TSV, so a reader sees the answer with no GPU and
+  no install. These are **declarative**: you add a dict, a script produces everything else.
+* **Notebooks** — `examples/notebooks/*.ipynb`, hand-written library tutorials. Note the
+  `notebook.ipynb` *inside* each walkthrough is code-generated by
+  `scripts/generate_walkthrough_notebooks.py` and must not be hand-edited.
+
+### 1. Add the entry
+
+For a variant walkthrough, append to the matching list in `scripts/regenerate_examples.py`
+(`ALPHAGENOME_EXAMPLES`, `ENFORMER_EXAMPLES` or `CHROMBPNET_EXAMPLES`). An entry is just:
+
+```python
+{
+    "name": "SORT1 rs12740374 (Enformer)",
+    "dir": f"{BASE}/variant_analysis/SORT1_enformer",
+    "type": "discovery",
+    "position": "chr1:109274968",
+    "ref": "G", "alt": "T",
+    "gene": "SORT1",
+    "html_name": "rs12740374_SORT1_enformer_report.html",
+}
+```
+
+`ref` **must** match the reference genome at that position — `strict_ref=True` is the default, so a
+wrong ref raises `ReferenceAlleleMismatchError` rather than quietly substituting.
+
+### 2. Run the right script, in the right env
+
+This table is the part that costs people an afternoon. Getting it wrong usually does **not** fail
+loudly — see the traps below.
+
+| what you added | script | valid `--oracle` | conda env |
+|---|---|---|---|
+| a variant walkthrough | `scripts/regenerate_examples.py` | `alphagenome`, `enformer`, `chrombpnet`, `all` | `chorus` (these use `use_environment=True`, so they spawn the per-oracle env themselves) |
+| a per-oracle multioracle report | `scripts/regenerate_multioracle.py --oracle X` | `chrombpnet`, `cherimoya`, `legnet`, `alphagenome` — **no `enformer`** | `chorus-X` |
+| the unified IGV panel | `scripts/regenerate_multioracle.py --consolidate` | — | `chorus` |
+| discovery / causal / region_swap / integration / batch / TERT | `scripts/regenerate_remaining_examples.py --only all` | — | `chorus-alphagenome` |
+| the walkthrough's `notebook.ipynb` | `scripts/generate_walkthrough_notebooks.py` | — (codegen only, no GPU) | `chorus` |
+
+Enformer has no `--oracle enformer` in the multioracle script; its single-oracle report comes from
+`regenerate_examples.py`.
+
+### 3. Traps worth knowing before you start
+
+* **The wrong env does not fail fast.** A missing oracle package logs `Failed to load <track>` once
+  per track and carries on, so a long run can spend an hour loading nothing and then die at the end.
+* **`regenerate_multioracle.py` has no `--gpu` flag** (the other two do). Passing one is an argparse
+  error that scrolls past in a tailed log.
+* **Do not write `mamba run -n X --no-capture-output ...`** — a flag after `-n` makes the wrapper die
+  with `exec: --: invalid option` before your script starts, which reads like a completed step.
+* **Finish editing before you start regenerating.** Python reads source at process start, so a change
+  made mid-run reaches only the examples generated after it, leaving a committed set that is
+  internally inconsistent in a way no test compares.
+* `conda run` buffers stdout, so a long build's log stays empty until it exits. Use
+  `conda run --no-capture-output ... python -u` when you want to watch it.
+
+### 4. What will go red
+
+Committed examples are guarded, deliberately — an example whose HTML no longer matches its JSON is
+worse than no example. Expect to run:
+
+```bash
+pytest tests/test_committed_examples.py tests/test_json_tsv_parity.py \
+       tests/test_walkthrough_readmes_match_artefacts.py \
+       tests/test_summary_tables_match_their_artefact_by_label.py \
+       tests/test_batch_rows_reconcile_with_headline.py \
+       tests/test_rerender_refuses_to_degrade.py
+```
+
+If a guard fails, it is usually telling you a regeneration step was skipped or run in the wrong env,
+not that the guard is wrong.
+
 ## Running the tests
 
 From the `chorus` base env, at the repo root. This is the command CI runs, and a guard test enforces
@@ -509,8 +603,19 @@ chorus/
 ## Current Priorities
 
 All eight core oracles (Enformer, Borzoi, ChromBPNet/BPNet, Sei, LegNet, AlphaGenome, Cherimoya/CATv1, EPInformer-seq) are implemented — nine registered names, since AlphaGenome ships both a JAX and a PyTorch backend. We're interested in contributions for:
+
+**Models**
 1. **Custom fine-tuned models** — models trained on specific tissues or conditions
 2. **Species-specific oracles** — mouse, drosophila, etc.
 3. **New architectures** — HyenaDNA, Evo, Nucleotide Transformer, etc.
 
+**Everything else — genuinely wanted, and much smaller**
+4. **Worked examples** — a variant, locus or question you care about, run through an oracle we
+   already ship. This is the most useful contribution per hour of your time: it costs one entry in a
+   declarative list plus one script run, and it is how most people discover what chorus does. See
+   [Contributing an example](#contributing-an-example-or-walkthrough).
+5. **Documentation that was wrong or missing when you read it** — including "I followed this and it
+   failed". A copy-pasteable repro of where you got stuck is a useful issue even with no patch.
+6. **Bug fixes and tests**, especially a test that pins something you found the hard way.
+
 Thank you for contributing to Chorus! Your implementation will help make genomic deep learning models more accessible to the research community.

examples/notebooks/README.md

@@ -81,3 +81,10 @@ kernel from the Kernel menu.
   ~30 s.
 - **Notebook cells show `<Figure ... >` but no image** — check your
   matplotlib backend; `%matplotlib inline` should be in cell 1.
+
+## Want to add one?
+
+See [CONTRIBUTING.md § Contributing an example or walkthrough](../../CONTRIBUTING.md#contributing-an-example-or-walkthrough).
+Note the `notebook.ipynb` inside each `examples/walkthroughs/*/` directory is **code-generated** by
+`scripts/generate_walkthrough_notebooks.py` — the hand-written tutorials are the ones in this
+directory.

examples/walkthroughs/README.md

@@ -157,3 +157,10 @@ Every analysis tool produces outputs in four formats:
 | JSON | `report.to_dict()` | Programmatic analysis, pipelines, notebooks |
 | TSV | `report.to_tsv(path)` or `report.to_dataframe()` | Excel, R, pandas |
 | HTML | `report.to_html(path)` | Visual review with embedded IGV genome browser |
+
+## Want to add one?
+
+Worked examples are the smallest useful contribution to chorus — one entry in a declarative list
+plus one script run. The canonical step-by-step, including which regeneration script owns which
+kind of example and which conda env it needs, is
+[CONTRIBUTING.md § Contributing an example or walkthrough](../../CONTRIBUTING.md#contributing-an-example-or-walkthrough).

tests/test_contributing_example_path_matches_the_scripts.py

@@ -0,0 +1,160 @@
+"""CONTRIBUTING's "which script, which oracle, which env" table must match the scripts.
+
+Adding a worked example is the smallest useful contribution to chorus, and for a long time it was
+documented nowhere a contributor would look: the script/oracle/env matrix lived in `CLAUDE.md` (for
+Claude sessions) and in an audit report, while `CONTRIBUTING.md` opened by saying it was a guide to
+implementing an oracle. Someone who wanted to add a variant walkthrough had to reverse-engineer four
+regeneration scripts to find which one owned it.
+
+The matrix is now in CONTRIBUTING, which makes it a drift risk: `regenerate_multioracle.py` accepts
+four oracles and pointedly **not** `enformer`, `regenerate_examples.py` accepts a different three
+plus `all`, and only two of the three take `--gpu`. Every one of those asymmetries has already cost
+a debugging session, and a table that silently goes stale costs the next one too — so it is pinned
+to the argparse definitions rather than proofread.
+"""
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+
+import pytest
+
+REPO = Path(__file__).resolve().parent.parent
+CONTRIBUTING = REPO / "CONTRIBUTING.md"
+SCRIPTS = REPO / "scripts"
+
+
+def _arg_choices(script: Path, flag: str) -> list[str] | None:
+    """The `choices=[...]` of `flag` in the script's argparse setup, or None if the flag is absent."""
+    tree = ast.parse(script.read_text())
+    for node in ast.walk(tree):
+        if not isinstance(node, ast.Call):
+            continue
+        fn = node.func
+        if getattr(fn, "attr", None) != "add_argument":
+            continue
+        if not any(isinstance(a, ast.Constant) and a.value == flag for a in node.args):
+            continue
+        for kw in node.keywords:
+            if kw.arg == "choices" and isinstance(kw.value, (ast.List, ast.Tuple)):
+                return [e.value for e in kw.value.elts if isinstance(e, ast.Constant)]
+        return []  # flag exists but takes no choices
+    return None
+
+
+def _table() -> str:
+    """The example-contribution section, where the matrix lives."""
+    text = CONTRIBUTING.read_text()
+    i = text.index("## Contributing an example or walkthrough")
+    return text[i:text.index("\n## ", i + 10)]
+
+
+@pytest.mark.parametrize("script,flag", [
+    ("regenerate_examples.py", "--oracle"),
+    ("regenerate_multioracle.py", "--oracle"),
+    ("regenerate_remaining_examples.py", "--only"),
+])
+def test_every_documented_choice_is_a_real_choice(script: str, flag: str):
+    """No value in the table that argparse would reject."""
+    choices = _arg_choices(SCRIPTS / script, flag)
+    assert choices is not None, f"{script} no longer defines {flag}; the CONTRIBUTING table is stale"
+    table = _table()
+    row = next((ln for ln in table.splitlines() if script in ln), None)
+    assert row is not None, f"the CONTRIBUTING matrix has no row for {script}"
+
+    # anything rendered as `code` in that row which looks like an oracle name must be valid —
+    # except names the row explicitly calls out as *unsupported* ("no `enformer`"), which are
+    # covered by test_the_enformer_exception_is_still_true instead
+    import re
+    positive = re.sub(r"no\s+`[a-z_]+`", "", row)
+    cited = {m for m in re.findall(r"`([a-z_]+)`", positive)}
+    known_oracles = {"alphagenome", "enformer", "chrombpnet", "cherimoya", "legnet", "sei",
+                     "borzoi", "epinformerseq", "alphagenome_pt"}
+    for name in cited & known_oracles:
+        assert name in choices, (
+            f"CONTRIBUTING's row for {script} cites `{name}`, but its {flag} choices are {choices}. "
+            f"argparse would reject it — an invalid --oracle is an error that scrolls past in a "
+            f"tailed log, which is exactly how a regeneration silently does nothing."
+        )
+
+
+def test_the_enformer_exception_is_still_true():
+    """`regenerate_multioracle.py` deliberately has no `enformer`, and the docs say so.
+
+    This is the asymmetry most likely to be "helpfully" corrected by someone who assumes the table
+    is a typo. If enformer is ever added, the sentence has to go.
+    """
+    choices = _arg_choices(SCRIPTS / "regenerate_multioracle.py", "--oracle") or []
+    table = _table()
+    stated = "no `enformer`" in table or "no enformer" in table
+
+    if "enformer" in choices:
+        assert not stated, (
+            "regenerate_multioracle.py now accepts --oracle enformer, so CONTRIBUTING's "
+            "'no enformer' note is wrong and must be removed"
+        )
+    else:
+        assert stated, (
+            "regenerate_multioracle.py rejects --oracle enformer and CONTRIBUTING no longer says so; "
+            "a contributor will try it and read the argparse error as a broken script"
+        )
+
+
+def test_the_gpu_flag_asymmetry_is_documented_correctly():
+    """Two of the three regeneration scripts take `--gpu`; the multioracle one does not."""
+    has_gpu = {
+        s: _arg_choices(SCRIPTS / s, "--gpu") is not None
+        for s in ("regenerate_examples.py", "regenerate_multioracle.py",
+                  "regenerate_remaining_examples.py")
+    }
+    table = _table()
+    if not has_gpu["regenerate_multioracle.py"]:
+        assert "no `--gpu` flag" in table, (
+            "regenerate_multioracle.py still has no --gpu, but CONTRIBUTING does not warn about it; "
+            f"passing one is an argparse error. (--gpu presence: {has_gpu})"
+        )
+    else:
+        pytest.fail(
+            "regenerate_multioracle.py gained a --gpu flag — remove the warning from CONTRIBUTING"
+        )
+
+
+def test_the_notebook_generator_is_still_codegen_only():
+    """Documented as taking no arguments and needing no GPU."""
+    script = SCRIPTS / "generate_walkthrough_notebooks.py"
+    tree = ast.parse(script.read_text())
+    adds = [n for n in ast.walk(tree)
+            if isinstance(n, ast.Call) and getattr(n.func, "attr", None) == "add_argument"]
+    assert not adds, (
+        f"generate_walkthrough_notebooks.py now takes {len(adds)} argument(s); CONTRIBUTING "
+        f"describes it as codegen-only with no flags"
+    )
+
+
+def test_the_guard_files_named_in_contributing_exist():
+    """The section tells a contributor which tests to run. All of them must be real files."""
+    import re
+
+    table = _table()
+    named = re.findall(r"tests/(test_[a-z0-9_]+)\.py", table)
+    assert named, "the example section no longer names any guard tests to run"
+    missing = [n for n in named if not (REPO / "tests" / f"{n}.py").is_file()]
+    assert not missing, (
+        f"CONTRIBUTING tells contributors to run tests that do not exist: {missing}. A documented "
+        f"command that errors is worse than no command."
+    )
+
+
+def test_the_routing_table_covers_the_non_oracle_paths():
+    """The doc used to open by saying it was about implementing an oracle, and nothing else.
+
+    A contributor with an example to add read sentence one and concluded they were in the wrong
+    place. The routing table exists so that does not happen.
+    """
+    head = CONTRIBUTING.read_text()[:2000].lower()
+    for want, why in (
+        ("example", "an example contributor must see themselves in the first screen"),
+        ("bug fix", "small fixes are wanted and were never mentioned"),
+        ("oracle", "the oracle path is still the main one and must stay findable"),
+    ):
+        assert want in head, f"CONTRIBUTING's opening does not mention {want!r} — {why}"