Ship a GitHub Action and pre-commit hook that drill committed synthetic fixtures

[ChelseaKR]

What

A reusable GitHub Action (ChelseaKR/exitdrill/action@<sha>) and a pre-commit hook definition that run exitdrill validate and exitdrill drill over committed synthetic fixtures in another repository, write the receipt as a workflow artifact, and can compare it to a committed reference receipt with --fail-on-loss-signal-increase. Results are also emitted as JUnit XML (one test case per dimension, status as the outcome, limitations as system-out) so any CI reads them, plus a summary in the job's step summary. Data-mode is hard-wired to synthetic: the action refuses to run unless the fixture directory carries the data_mode: synthetic_only marker validate-exercise already understands.

Why it matters

ExitDrill is a verifier with exit codes and replayable receipts — the shape CI wants — but today only its own repo runs it. A team maintaining a normalizer for their own export contract (outside this repo, as ADR 0001 intends) has no way to gate a regression in structural loss on every commit. The action is packaging over existing commands: no connector, no evidence family, no data category, so it is inside the freeze; the synthetic-only marker keeps the data-governance gate intact by construction.

Scope

• action.yml (composite; pinned uv; inputs: baseline, export, attachment-root, reference-receipt, fail-on-loss-signal-increase), documented in docs/CI-USAGE.md.
• --format junit on drill and compare (and --out for it), with a committed schema/fixture.
• .pre-commit-hooks.yaml exposing exitdrill-validate.
• The action's own CI job runs it against examples/synthetic-crm* and asserts the documented exit codes.

Out of scope

• Running any normalizer or canary lab; SARIF (there is no location-based finding to map); hosted dashboards.

Done when

• A workflow using the action on the clean fixture is green; on the lossy fixture with a clean reference it fails with exit 3 and a JUnit file marking the changed dimensions failed.
• Removing the synthetic-only marker makes the action stop before invoking drill, with the reason in the summary.
• JUnit output contains no record values (disclosure gate) and validates against the JUnit schema.
• The pre-commit hook rejects a malformed baseline in under a second.

Pointers

• src/exitdrill/cli.py (exit codes 0/2/3), src/exitdrill/exercise.py (data_mode marker), Makefile (demo-compare-policy), .github/workflows/ci.yml
• docs/decisions/0002-synthetic-exercise-preflight.md, docs/DATA-GOVERNANCE.md

Proposed with AI assistance.

[ChelseaKR]

Measured against origin/main at 86e8773: the acceptance criterion names a mechanism that does not exist, and the two readings of it are worth different things. Reporting rather than picking one.

The criterion is "Removing the synthetic-only marker makes the action stop before invoking drill", and the Scope paragraph calls it "the data_mode: synthetic_only marker validate-exercise already understands". There is no such marker.

data_mode is a field of the exercise-plan document, and nothing else:

• schemas/exercise-plan-v0.1.schema.json lists it in required and pins it to "const": "synthetic_only".
• src/exitdrill/exercise.py:167-168 enforces it against the parsed plan JSON: if value.get("data_mode") != "synthetic_only": raise ExercisePlanError(...).
• The only committed instance is a JSON key, examples/synthetic-exercise/plan.json:4.
• git grep -ni '\bmarker\b' -- src/ docs/ README.md scripts/ tests/ returns zero hits. "Marker" is not a concept in this codebase.
• The fixture directories the criterion says would carry it hold five files between them and no sentinel: examples/synthetic-crm/{baseline.json,export.json,export-files/attachments/intake.txt} and examples/synthetic-crm-lossy/{export.json,export-files/attachments/intake.txt}.

There is also no path from a fixture directory to the plan validator. validate-exercise takes a plan file (Makefile:47, README.md:181-185), while drill takes a baseline + export pair — and neither the baseline schema nor the export schema has a data_mode field at all. Only exercise-plan-v0.1 does.

So the acceptance criterion admits two readings, and they are not worth the same thing:

1. Add a directory sentinel (say examples/synthetic-crm/.data-mode) that the action reads before invoking drill. This is buildable, but the green check it produces means "a file the action itself wrote the convention for is present" — a gate whose subject is its own convention. Nothing else in the repository reads it, and drill's own inputs remain undeclared.
2. Require an exercise plan alongside the fixture and run validate-exercise on it first. This reuses the enforcement that already exists and the green check means what validate-exercise means, but it is a larger change: the two synthetic fixtures have no plan document today, and writing one for a fixture that is not a target exercise stretches what exercise-plan-v0.1 describes.

Reading 1 is close to the "gate that cannot fail" shape this repository is otherwise careful about; reading 2 changes what an exercise plan is for. That is a call for the owner, so nothing has been built here.

The rest of the issue (a GitHub Action and a pre-commit hook that run drill over committed synthetic fixtures) is buildable as written once the gating condition is settled.

Prepared with AI assistance; reviewed before submission.