[aoreshkov]

Summary

• Every verdict grades the finding. The verifier's severity is now required. The optional correctedSeverity it replaces was set by none of 78 verdicts in a real run, so every reported grade was the reviewer's own. A confirmed finding is ranked by the verified grade and keeps the reviewer's as claimedSeverity where they differ.
• Votes. New argument votes: independent verifiers per finding, by claimed severity (default 3 for critical and high, 1 for medium and low; votes: 1 restores one verifier).
  • Each vote takes its own refutation angle: reachability, handled, deliberate.
  • The majority of the votes that ruled decides. A tie or fewer than 2 rulings leaves the finding unverified (verifier-failed, or the new votes-split), so next and previous can re-verify it.
  • The worst case is maxJobs * (1 + maxVerifyPerJob * V) + 2 agents: 86 at the defaults, 38 with votes: 1.
• Per-pair yield. coverage.yield lists every reviewed pair with the findings it raised, confirmed and refuted. A continuation carries it forward, and the report ends with a per-lens table.
• Seeded-defects eval measures precision.
  • Four decoys: guarded code that looks like one of the defects.
  • Four no-decoy-* graders with match: not_contains, anchored on the package path so a finding that cites a decoy as the right way to do it does not count.
  • Three runs, and verification of up to 8 findings per reviewer.
  • The real-run script applies match and gains --model.

Eval (2.1.311, three real runs per setting)

Setting	Defects confirmed	Decoy checks confirmed	Per run
Default models	15/15	2/12	96-143 s, $1.65-1.94
model: "opus"	15/15	0/12	100-120 s, $1.94-2.34

Neither decoy confirmed on the default models was the defect it imitates:

• One was a test gap the verifier itself called "not a bug".
• The other was real rounding-down in the latency decoy, which the scaffold now documents as intended.

The figures come from re-grading the saved reports with the final graders. No live run has used the updated latency decoy yet.

Test plan

• node --test (203 pass)
• claude plugin validate --strict for ., ./review-kit, ./token-audit, ./token-meter
• claude plugin test ./token-meter (40 pass)
• Seeded-defects real runs as above

After merge: tag review-kit-v0.4.0 with notes from the changelog.

🤖 Generated with Claude Code