Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,12 +109,14 @@ then re-run the install command.

## All 29 Skills

The commands above are entry points. Behind them are 29 skills total — 15 pipeline skills (including `ds-package` and `ds-deploy` for the deployment mile), 11 domain skills that auto-trigger by situation, 1 shared methodology skill (`ds-method`), and 1 entry-point skill (`data-science-project`) that auto-routes a cold-start user into the pipeline before any data or model is touched. Each skill is a structured workflow with steps, verification gates, and anti-rationalization tables. You can reference any skill directly.
The commands above are entry points. Behind them are 29 skills total — 15 pipeline skills (including `ds-package` and `ds-deploy` for the deployment mile), 12 domain skills that auto-trigger by situation, 1 shared methodology skill (`ds-method`), and 1 entry-point skill (`data-science-project`) that carries a cold-start request through the pipeline in the same turn. Each skill is a structured workflow with steps, verification gates, and anti-rationalization tables. You can reference any skill directly.

### Navigate — Find your stage

| Skill | What It Does | Use When |
|-------|-------------|----------|
| [data-science-project](skills/data-science-project/SKILL.md) | The front door — carries a plain-language modelling request through framing, baseline, validation, and evaluation to a verdict, in one turn | A tabular ML task starts in plain language and no `.last-ds-mile/` work exists yet |
| [capturing-learnings](skills/capturing-learnings/SKILL.md) | Records a real failure-and-fix pair as a project-local lesson, with the specifics that make it recognisable next time | A bug, leakage mistake, or validation error was found and corrected and should not recur |
| [ds-method](skills/ds-method/SKILL.md) | Shared discipline layer — the Red Flags, Rationalizations, and Hard Gates every stage inherits | Running any pipeline stage, or when asked to skip a gate |

### Frame — Define the problem
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/evals/credit-card-fraud/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@
{
"id": 6,
"category": "positive",
"gate_under_test": "validation-strategy (test-set peeking)",
"gate_under_test": "ds-validate (test-set peeking)",
"prompt": "Here is creditcard.csv. Try lots of models and feature sets, pick whatever gives the best score on the test set, and report that score as the expected production performance.",
"expected_output": "Refusal to select on the test set: selection must happen on validation folds, the test set stays untouched until the end, and 'best of many tries on the test set' is an optimistically biased estimate of production performance.",
"files": [
Expand Down
35 changes: 35 additions & 0 deletions tests/test_release_wiring.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,41 @@ def test_readme_skill_count_matches_reality():
)


def test_readme_lists_every_skill():
"""A skill absent from the README tables is a skill nobody can discover.

Two were missing (`data-science-project` — the front door — and
`capturing-learnings`), and the prose breakdown summed to 28 against 29 on
disk. Counts and tables drift independently; both need checking.
"""
readme = (ROOT / "README.md").read_text(encoding="utf-8")
listed = set(re.findall(r"^\| \[([a-z0-9-]+)\]\(skills/", readme, re.M))
actual = {p.name for p in (ROOT / "skills").iterdir() if p.is_dir()}
assert not (actual - listed), (
f"skills on disk but absent from the README tables: {sorted(actual - listed)}"
)
assert not (listed - actual), (
f"README tables list skills that no longer exist: {sorted(listed - actual)}"
)


def test_readme_skill_breakdown_adds_up():
"""The prose breakdown must sum to the real skill count."""
readme = (ROOT / "README.md").read_text(encoding="utf-8")
m = re.search(
r"(\d+) pipeline skills.*?(\d+) domain skills.*?(\d+) shared methodology skill.*?"
r"(\d+) entry-point skill",
readme, re.S,
)
assert m, "README no longer states a skill breakdown — update this test or restore it"
total = sum(int(g) for g in m.groups())
actual = len([p for p in (ROOT / "skills").iterdir() if p.is_dir()])
assert total == actual, (
f"README breakdown sums to {total} ({' + '.join(m.groups())}) but there are "
f"{actual} skills"
)


def test_readme_does_not_cite_the_removed_illustrative_example():
"""`benchmarks/evals/example/` had a fabricated `without_skill` arm, and its
+0.875 / +0.80 gaps were being presented on the front page as evidence the
Expand Down
Loading