gh-whyfail

Your PR gets a red ❌. You run gh whyfail. Ten seconds later you know why it failed and what to change — without opening the Actions tab or scrolling 4,000 lines of logs.

Real output against whyfail-demo's intentionally broken build:

$ gh whyfail
Finding latest failed run for PatrickDamouni/whyfail-demo@main…
Asking claude-opus-4-8 (via api)…
✗ CI (push) — failed job(s): test

Root cause
──────────
The test test_math.py::test_add failed because add(2, 3) returned -1 instead of
the expected 5, indicating the add() function is implemented with subtraction
(or otherwise incorrect) rather than addition.

Suggested fix
─────────────
Fix the add() function (likely in the module imported by test_math.py) so it
returns the sum of its arguments rather than a difference (change `a - b` to
`a + b`).

→ category: test · confidence: medium

Install

gh extension install PatrickDamouni/gh-whyfail

Requirements:

• GitHub CLI (gh auth login)
• Claude access, either of:
  • an Anthropic API key: export ANTHROPIC_API_KEY=sk-ant-... (get one), or
  • Claude Code installed and logged in — whyfail automatically falls back to the claude CLI, which runs on your Claude subscription (no API key needed)

Usage

gh whyfail                 # explain the latest failed run on the current branch
gh whyfail --run 123456    # explain a specific run
gh whyfail --patch         # also print a suggested unified diff
gh whyfail --json          # machine-readable output
gh whyfail --model claude-sonnet-5   # use a different Claude model

How it works

A six-stage pipeline: find the latest failed run and its failed jobs (GitHub API) → download the job logs → distill them (error-pattern scanning, tail-weighting, and dedup shrink a multi-thousand-line log to the ~250 lines that matter, so the model sees signal instead of noise) → attach the PR diff so the model can judge whether your change actually caused the failure → one Claude call returning a structured diagnosis → render.

The interesting engineering is in internal/distill — dumping raw CI logs at a model is expensive and inaccurate; distillation is what makes the diagnosis cheap and specific.

Roadmap

• BrokenBuilds benchmark — a dataset of real failed workflow runs from public open-source repos, labeled with ground-truth root causes (the fix commit that made CI green again is the label), and a harness that scores frontier models on real broken builds.
• Findings writeup — what frontier models get wrong about broken builds (e.g. how often they blame your diff for infra flakes).
• --fix: apply the suggested patch, commit, and push — gated until the benchmark shows patch accuracy is high enough to trust.

Deliberately out of scope for v1: non-GitHub CI systems, caching, config files, multiple LLM providers.

Development

go test ./...
go build -o gh-whyfail . && gh extension install .

MIT licensed.