Skip to content
This repository was archived by the owner on Oct 6, 2026. It is now read-only.
prodeltPublic archive

About

Frozen at v5. Quality gate for Claude Code: project tests first, then one LLM judge, then a commit with proof.

Topics

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

578 Commits

Folders and files

Repository files navigation

ELT
a harness for Claude Code that keeps unverified work out of main

Frozen at v5. I stopped developing ELT in September 2026. Newer Claude models got good enough that stricter gates began to cost more quality than they protected, so I now work with a lighter setup of skills, safety hooks and CI test gates. The code, the benchmark and the write-ups stay here as they were. The repository is archived, and issues and pull requests are not monitored.

118 of 118 test files green node 18+ zero dependencies MIT license

Install · What it does · Proof · Usage · Limits


The problem

Your coding agent finished. It says the change is done. It is confident, it is articulate, and you have no idea whether it is right.

So you read the diff yourself — and you do that again on the next task, and the one after. The agent got faster; the bottleneck moved to you. And on the days you don't read carefully, something broken lands in main.

What it does

ELT sits between the agent and your commit and refuses to let unverified work through.

🔬 Tests run before the model does A model is never asked about code whose own tests are red. Mechanical checks are free; model calls are not.
⚖️ Exactly one judge, no second round An independent model reads the diff against the task. block means fix the cause — not argue with the reviewer.
🔍 Five review lenses in parallel Bugs, project rules, stale comments, reversed past decisions, repeat feedback — each reads the diff blind to the others' scores, then a cutoff at 80 drops the noise.
📓 A ledger of its own misses When a verdict turns out wrong, it gets recorded. Five hits on one rule raise that rule for review — once.
🔒 Commits carry proof Every commit under the harness leaves a row binding it to the test run and the verdict that let it through. Anything else is visibly ungated.

Install it as a plugin, type /elt, and it works out of what your project already has: your test command, your git history, your CLAUDE.md. There is nothing to configure first and no service to sign up for.

Does it actually work?

Measured on a third-party dataset, with the preregistration frozen before the first result row: 30 SWE-bench Verified instances, each with a correct patch and a deliberately broken one, 60 live judge calls. The judge in this run was Gemini 3.7 Flash (gemini-3.7-flash-high), not Claude Sonnet, which is the plugin's default judge.

xychart-beta
  title "Correct ship/stop decisions, 60 cells (SWE-bench Verified)"
  x-axis ["no gate", "ELT gate"]
  y-axis "accuracy, %" 0 --> 100
  bar [50, 85]
Loading
no gate with ELT
correct ship/stop decisions, Gemini 3.7 Flash judge 50%, analytic: everything ships 85.0% [73.9%, 91.9%]
broken patches caught 0 of 30 21 of 30
correct SWE-bench reference patches wrongly rejected 0 of 30 0/30

A gate that stops good work gets switched off within a week, so false blocks matter more than catches. The 0 of 30 holds for the SWE-bench reference patches. On real slices in my own projects the gate did block some correct work.

And the part that is unflattering — read this before quoting the numbers
  • 9/30 broken patches got through. All nine are cases where the truncated patch stayed coherent and reads like finished work. That is the harness's known architectural blind spot: the judge checks the diff against the task, not against external reality (a live DB schema, another API's actual semantics).
  • The harness does not make the writer smarter. A separate 30-pair experiment on Aider-AI/polyglot-benchmark found 100.0% with the harness and 100.0% without it — no difference, because the model already solves those tasks on the first try. A negative result is reported here as negative.
  • Resolve rate was not measured at all. No SWE-bench test was executed in either arm. This measures the gate's discriminating power — the share of issues solved is a different claim and this page does not make it.
  • Single-hunk instances were excluded from the sampling frame; the conclusion applies to multi-hunk patches.

Full write-up, raw rows and preregistrations: benchmarks/gemini-3.7-flash-high/.

Install

claude plugin marketplace add prodelt/elt-harness
claude plugin install elt@elt

Then, in any project:

/elt

That is the whole setup. Node 18+ and Git are the only requirements — no npm install, no lockfile, no dependencies to audit. ELT creates its own config on first run, and a missing config is an INFO, never a failure.

→ Install, update, rollback · Daily usage and troubleshooting

Four commands

command what it does
/elt a slice under the gate: plan if needed, tests, one judge, commit
/elt-verify review the current diff with five lenses in parallel
/elt-defects record a divergence between a verdict and reality
/elt-doctor check the install and the project's harness config

Two hooks work without being invoked: SessionStart prints a project summary, and Stop refuses to end a session that left uncommitted work behind.

How it works

Everything in blue is mechanical and costs no model call.

flowchart TD
  subgraph PLAN["1 · Plan — only for a large goal"]
    direction TB
    G["a goal"] --> S["specs/NNN/spec.md<br/>+ tasks.md"]
    S --> A["elt spec approve<br/><i>signature lives in git trailers</i>"]
  end

  A --> SL

  subgraph SLICE["2 · Slice — one open task at a time"]
    direction TB
    SL["elt slice next"] --> W["writer produces code<br/>+ the smallest regression"]
  end

  W --> L0

  subgraph GATE["3 · Gate"]
    direction TB
    L0{"L0 — mechanics<br/>no model call"}
    L0 -->|low risk| CM
    L0 -->|risk possible| OR
    OR{"tests<br/>the project's own command"}
    OR -->|red| FIX["STOP · fix the cause<br/><i>max 2 narrow attempts</i>"]
    OR -->|green| JD{"exactly one judge"}
    JD -->|block| FIX
    JD -->|inconclusive| RQ["review queue"]
    JD -->|pass| CM
    RQ --> CM
  end

  FIX -.->|re-run the whole chain| L0
  CM["elt commit<br/>branch · [X] · commit"]

  CM --> RL["run-log<br/><i>proof the commit was gated</i>"]
  CM --> BG["5 review lenses in parallel<br/>+ confidence cutoff 80"]
  BG --> LG["ledger<br/><i>verdict vs reality</i>"]
  LG -->|5 hits on one rule| ISS["raised for review — once"]

  classDef mech fill:#e8f0fe,stroke:#4285f4,color:#111;
  classDef stop fill:#fce8e6,stroke:#d93025,color:#111;
  classDef done fill:#e6f4ea,stroke:#137333,color:#111;
  class L0,OR,RL mech;
  class FIX stop;
  class CM done;
Loading

Done means four things at once — tests green, smoke green (if configured), one judge returned pass or inconclusive, and elt commit created both the commit and its proof row. Three out of four is not done.

A third verdict, inconclusive, exists so that "I am not sure" cannot silently become a soft block: it commits, and it lands in the review queue.

→ Architecture and design decisions

What ELT is not

Being wrong about this costs more than not using it at all.

  • Not a replacement for your tests. The judge is the second layer. If a project has no meaningful test command, the strongest half of the gate is missing and the model is guessing.
  • Not a check against external reality. It does not validate a production schema, another service's semantics, or whether users actually wanted the feature. It checks the diff against the task.
  • Not a compliance control. There is no independent human authorization step and no tamper-evident ledger. For a regulated pipeline, run ELT alongside required CI and protected branches, not instead of them.
  • Not autonomous. The review queue needs a person. Nothing here merges on its own.

The harness keeps its own defect registry public, including what is still open: docs/DEFECTS.md.

Evidence

Every claim on this page is reproducible from the command that produced it and locked to a versioned snapshot by a regression test — this page cannot drift from the measurements silently.

→ Measurements, methods and their limits

Contributing

The repository is frozen and archived, so issues and pull requests are not monitored. To build on ELT, fork it under the MIT license.

License

MIT — see LICENSE.

About

Frozen at v5. Quality gate for Claude Code: project tests first, then one LLM judge, then a commit with proof.

Topics

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages