Gauntlet

Make the code prove itself.

The problem

Green tests and high coverage don't mean the code is safe to change.

Coverage tells you a line ran, not that a test would notice if it broke. Complex code with thin tests is where regressions hide, and copy-pasted logic quietly drifts apart over time. Each of these problems has a good tool, but the tools run separately, report in different shapes, and look at the whole codebase instead of the code you just touched.

Coding agents make this worse. They produce a lot of code and a lot of passing tests very quickly, but nothing checks whether those tests assert anything meaningful. "Tests pass" becomes a weak signal, and human review becomes the bottleneck.

What Gauntlet does

Gauntlet runs three complementary checks on only the code you changed and turns the results into deterministic, actionable findings: in your terminal, in CI, or fed straight back to a coding agent that can fix them.

It's built on three analyzers by Robert C. Martin (Uncle Bob). Gauntlet orchestrates them, scopes them to your diff, and merges their output into one report:

Analyzer	Question it answers	Why it matters
Crapper	Is this code too complex for its test coverage?	Complexity without tests is where regressions live.
Mutator	Would the tests notice a plausible defect?	Proves tests assert behavior, not just execute lines.
Dryer	Is similar code appearing in multiple places?	Catches duplication before the copies diverge.

Gauntlet produces evidence. You or your coding agent investigate and repair the findings. It runs entirely on your machine: no LLM calls and no source uploads.

How it works

git diff ─► coverage (generated or reused) ─► Crapper ─► Mutator ─► Dryer
                                                                      │
                                        terminal findings + .gauntlet/results.json

gauntlet check runs this pipeline on your changed source files. A failed prerequisite stops analysis early. Coverage is generated once or reused. Findings appear in the terminal and are saved to .gauntlet/results.json.

Get started

You need:

• Python 3.12+ and Git
• The Crapper, Mutator, and Dryer executables on your PATH. Install these separately; gauntlet doctor will tell you what's missing.

Install Gauntlet from this checkout:

uv tool install .
# Alternative: pipx install .

Then, inside the project you want to check:

gauntlet init
# Set your test and coverage commands in gauntlet.toml when needed.
gauntlet doctor
gauntlet check

Gauntlet detects common project commands, and explicit configuration overrides them. For example, a Python project using pytest and coverage.py can add:

[commands]
test = "uv run pytest"
coverage = "uv run coverage run -m pytest && uv run coverage lcov -o target/coverage/python/lcov.info"

Configure only the commands your project uses. doctor checks readiness without running tests. For a complete walkthrough, follow the sample project tutorial.

Everyday commands

Command	Use it to…
gauntlet or gauntlet check	Check staged, unstaged, and untracked source changes
gauntlet check --all	Check the entire repository
gauntlet check --base main	Check committed changes from the merge base with main to the checked-out HEAD
gauntlet check src/domain	Check a specific file or directory
gauntlet scan	Inspect hotspots without tests, coverage, or mutation execution
gauntlet check --json	Get JSON-only output for an agent or CI
gauntlet explain FINDING_ID	Inspect a finding from the saved report

An empty source change passes. Use --verbose for command diagnostics or --quiet to suppress terminal output. Run gauntlet check --help for all options.

Act on findings

Finding	Default	What to do
Surviving mutant	Blocks	Determine whether behavior changed. If it did, add the smallest meaningful test.
High CRAP on changed code	Blocks	Simplify branching or improve meaningful tests while preserving behavior.
Structural duplication	Review	Decide whether the similarity is accidental, intentional, or coincidental before refactoring.

A survivor requires investigation. Don't change production behavior just to kill a mutant, or deduplicate code just to remove a finding. Known findings can be accepted with an ID and a reason in .gauntlet/accept.toml.

A passing check exits with 0; blocking findings exit with 3. Review findings alone pass. See the reference for error exit codes.

Use with a coding agent or CI

Give your agent a gate it can't talk its way past. Add this rule to your project's AGENTS.md:

Before finishing a coding task, run `gauntlet check`.
Investigate findings, preserve specified behavior, and rerun after repairs.
Do not bypass or weaken the checks.

For machine-readable results, run gauntlet check --json. It prints only JSON to stdout and also saves .gauntlet/results.json. The same command works in CI, where a non-zero exit code fails the build on blocking findings.

On a clean CI checkout, use gauntlet check --base BASE_SHA --json; working-tree --changed does not select committed PR changes. Fetch base/head history first. --base requires a clean tracked working tree, excludes untracked/deleted files, and preserves committed line ranges for CRAP policy. JSON records the resolved base, head, and merge-base SHAs. It can also be used with scan and explicit paths.

For explicit installation of Gauntlet and compatible pinned analyzers, run bash scripts/setup-tools.sh /path/to/tool-environment from this checkout, then activate that environment. Setup uses the network; analysis does not invoke setup. The setup action runs the same installer in GitHub Actions:

- uses: actions/checkout@v4
  with:
    fetch-depth: 0
    persist-credentials: false
- uses: matheuseabra/gauntlet-cli@<reviewed-commit-sha>
- run: gauntlet check --base "$BASE_SHA" --json
  env:
    BASE_SHA: ${{ github.event.pull_request.base.sha }}

Run this example in a secretless pull_request job with read-only permissions. Install project runtimes and locked dependencies separately. Local/cloud agents must run setup inside their own environment; GitHub runner installation does not provision a remote agent.

More details

• Tutorial: build a sample project and repair a test gap.
• Reference: configuration, coverage, accepted findings, CI, and exit codes.
• Architecture: modules, adapter boundaries, and execution flow.

Contributing

uv sync --extra dev
uv run python -m unittest discover -s tests -v
uv run ruff check src tests
uv run ruff format --check src tests
uv run gauntlet --help

License

MIT