From edb1385d18de97f8d954dc3af6c3e48924603c5a Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Tue, 11 Aug 2026 18:13:43 +0000
Subject: [PATCH 01/15] identity: let a joining slot say who it is
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A live_pvp battle is created before its players arrive: the creator POSTs
p1_name/p2_name, both trainer FKs bind then, and the slots fill later over
the join WebSocket. So the names on a battle were always the creator's
guesses about who would show up -- "Opponent", "AI", "Agent 2" -- and there
was no moment in the protocol at which a slot's actual occupant could say
otherwise. pokearena-agent had no name flag, MCP join_battle had no name
argument, and the SPA's share-link join hardcoded 'Trainer'. Elo has been
real since the beginning and has been accruing to placeholders.

This adds the missing moment. A joiner may declare a name on the play URL
(?name=), the gateway sanitizes it and rebinds that slot's trainer on the
battle row, and the name rides the attach message so the session labels the
slot in room frames and in the engine state it builds. Omitting it keeps the
creator's name, so every existing flow is unchanged.

The same missing field was blocking the benchmark from the other side: PR
109's handoff records that Postgres carries no model identity, so per-model
attribution had to be reconstructed from bid= mappings in /tmp -- and a full
attributed batch was re-run after those were wiped. One cause, two symptoms.

Scope, stated plainly: this makes identity expressible and attributable, not
verified. Any holder of a slot token may claim any name, including one
already on the board; claim-a-handle + secret is separate work, and shipping
half of it would produce a board that looks verified. README and live-pvp.md
§3 say so in those words, with impersonation listed under "not designing
against" rather than left unmentioned.

Two constraints that would be real bugs rather than known limitations:
a name is only accepted after the slot claim succeeds, so guessing a battle
id cannot rewrite its trainers; and RebindBattleTrainer refuses a battle that
already has a winner, so a late or replayed join cannot move a rating that
was computed against the previous trainer.

Accepting a name at attach turned Match.trainerName into a field written by
the action-pump goroutine and read by the coordinator -- benign in behavior,
a data race in the memory model. It gets its own mutex-guarded type following
the slotConns precedent. set("") is a no-op, or an anonymous re-attach after
a blip would blank a name the slot declared on its first one.

pokearena-agent --name defaults to the model id: an unnamed agent inherits
the placeholder, which is the behavior that caused all of this, so staying
anonymous is the wrong default for a bot. The SPA remembers its name in
localStorage so a returning player keeps their row.

Also corrects two stale rows in the README status table -- the leaderboard
has been visible in the SPA since the field-state work.

The rebind SQL is verified against a real Postgres (integration tag): the
column mapping and the settled-battle guard are both invisible to a unit
test of the surrounding Go.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 README.md                                    |   5 +-
 backlog/2026-08-11T18-30-identity-on-join.md | 106 ++++++++++++++
 cmd/pokearena-agent/main.go                  |  16 ++-
 docs/live-pvp.md                             |  26 +++-
 docs/mcp-protocol.md                         |  17 ++-
 internal/agentloop/loop.go                   |   9 +-
 internal/gwclient/gwclient.go                |  13 +-
 internal/gwclient/gwclient_test.go           |  88 +++++++++++-
 internal/httpapi/server.go                   |   6 +-
 internal/httpapi/ws.go                       |  58 +++++++-
 internal/livebattle/bridge.go                |   4 +
 internal/livebattle/coordinator.go           |   8 +-
 internal/livebattle/livebattle.go            |   9 +-
 internal/livebattle/trainers.go              |  39 ++++++
 internal/livebattle/trainers_test.go         | 124 +++++++++++++++++
 internal/mcpserver/session.go                |   6 +-
 internal/mcpserver/session_test.go           |  20 +--
 internal/mcpserver/tools.go                  |   8 +-
 internal/messages/messages.go                |  12 +-
 internal/protocol/pvp.go                     |  64 ++++++++-
 internal/protocol/pvp_test.go                | 131 ++++++++++++++++++
 internal/session/trainerrebind_test.go       | 137 +++++++++++++++++++
 internal/store/repo.go                       |  23 ++++
 web/app.js                                   |  49 ++++++-
 24 files changed, 923 insertions(+), 55 deletions(-)
 create mode 100644 backlog/2026-08-11T18-30-identity-on-join.md
 create mode 100644 internal/livebattle/trainers.go
 create mode 100644 internal/livebattle/trainers_test.go
 create mode 100644 internal/protocol/pvp_test.go
 create mode 100644 internal/session/trainerrebind_test.go

diff --git a/README.md b/README.md
index 7f518ded..31421093 100644
--- a/README.md
+++ b/README.md
@@ -186,6 +186,7 @@ export ANTHROPIC_API_KEY=sk-ant-…
 | `--model` | `claude-haiku-4-5-20251001` | Anthropic model id. Use opus for stronger play at higher cost. |
 | `--turn-timeout` | `12s` | Per-turn LLM budget. The gateway default-actions the slot if exceeded. |
 | `--data-version` | `gen1-v1` | Must match the gateway's `DATA_VERSION` env. |
+| `--name` | the `--model` id | Trainer name this slot's results post under on the leaderboard. |
 
 ---
 
@@ -196,8 +197,8 @@ between "an arena where bots compete on a real leaderboard" and what runs today.
 
 | Area | Today | To make the headline true |
 |---|---|---|
-| **Leaderboard identity** | Free-text name, no ownership; clients barely prompt | Prompt for a trainer/agent name everywhere a battle starts; surface the board in the SPA. (Optional later: claim-a-handle + secret to stop impersonation.) |
-| **Leaderboard visibility** | Rating computed + stored, but not shown in the UI | A real standings page — wins/losses/Elo, sortable |
+| **Leaderboard identity** | A joining slot declares its own name (`--name`, MCP `trainer_name`, `?name=` on the WS), and the battle's trainer is rebound to it — so a result posts under whoever actually played, not the placeholder the battle's creator typed. Names are **self-reported and unverified** | Claim-a-handle + secret, so a name on the board can't be taken by anyone who wants it |
+| **Leaderboard visibility** | Standings page in the SPA (name + Elo), fed by `/api/leaderboard` | Sortable, with wins/losses alongside the rating — the store already returns them |
 | **Bot onboarding** | Two reference clients, MCP + CLI | A 5-minute "write your own bot" quickstart against a documented protocol |
 | **Provider coverage** | Benchmark (`cmd/bench`) runs Anthropic, OpenAI, Gemini, and local Ollama behind one `Client` interface, in `raw`/`cot` conditions; the live harness (`pokearena-agent`) is still Anthropic-only | Bring the remaining vendors to the live harness too |
 
diff --git a/backlog/2026-08-11T18-30-identity-on-join.md b/backlog/2026-08-11T18-30-identity-on-join.md
new file mode 100644
index 00000000..626d2f25
--- /dev/null
+++ b/backlog/2026-08-11T18-30-identity-on-join.md
@@ -0,0 +1,106 @@
+# Identity belongs to the joiner, not the creator
+
+The leaderboard has had working Elo since the beginning and has never
+meant anything. Today's change is small — a name on the join — but it is
+the one that makes a result attributable to whoever produced it.
+
+## The bug was in *when*, not *whether*, we asked for a name
+
+The README has said "free-text name, no ownership; clients barely prompt"
+for months, and I read that as a UI gap: add a name field to more screens
+and the board improves. That is not what was wrong.
+
+A `live_pvp` battle is created before its players arrive. The creator
+POSTs `p1_name` / `p2_name`, both trainer FKs are bound *then*, and the
+slots are filled later over the join WebSocket. So the names on a battle
+are always the creator's guesses about who will show up — `"Opponent"`,
+`"AI"`, `"Agent 2"`. There was no moment in the protocol at which the
+actual occupant of a slot could say who it was. `pokearena-agent` had no
+name flag, MCP `join_battle` had no name argument, and the SPA's
+share-link join hardcoded `'Trainer'`. Not one of them was under-prompting;
+there was nothing to prompt *into*.
+
+That reframes the fix. It isn't "collect the name in more places," it's
+"make join a point at which identity can be declared," which needs a wire
+field (`?name=` on the play URL), a message field (`LiveAction.Trainer`),
+and a write (`RebindBattleTrainer`). Once those exist the three clients
+are one line each.
+
+## The second track was blocked on the same field
+
+I went looking at PR #109 (decision-quality eval) in the same session and
+its handoff doc says, in the gotchas:
+
+> **Postgres has NO model identity** — `p1_name` is always "Agent",
+> `p2_name` "AI". Model attribution ONLY comes from the `bid=`→model
+> mapping in the run dirs. The previous mapping (`/tmp/pk-agentic`) was
+> wiped, which is why we re-ran.
+
+That is the same missing field, discovered independently, and it cost a
+re-run of a full attributed batch. The benchmark had been reconstructing
+in `/tmp` a fact the database should have carried. Worth naming because
+the two symptoms look unrelated — "the leaderboard is meaningless" and
+"we lost the model attribution for a batch" — and have one cause.
+
+## What I deliberately did not build
+
+The name is **self-reported**. Any holder of a slot token can claim any
+name, including one already on the board. I considered gating it — a
+secret per handle, first-writer-wins on a name — and stopped, because
+that is a different feature (accounts) wearing this one's clothes, and
+shipping half of it would produce a board that *looks* verified.
+
+So the honest split is: this change makes identity **expressible and
+attributable**; it does not make it **verified**. Both the README status
+row and `live-pvp.md` §3 now say that in those words, and §3 lists
+impersonation under "not designing against" rather than leaving it
+unmentioned. The README's "for fun, unverified" disclaimer stays exactly
+as it was — it is still true, just for a narrower reason.
+
+Two things I did constrain, because they are cheap and the absence would
+be a real bug rather than a known limitation:
+
+- **A name is only accepted after the slot claim succeeds.** Rebinding
+  before the claim would let anyone who can guess a battle id rewrite its
+  trainers without ever playing.
+- **`RebindBattleTrainer` refuses a battle with a winner.** Elo is applied
+  from `p1_trainer` / `p2_trainer` at completion; a late or replayed join
+  that renamed a settled battle would move a rating that was already
+  computed against the old trainer. The `winner < 0` guard is the whole
+  defense and it is why that method has an integration test.
+
+## A concurrency seam I didn't expect
+
+`trainerName` was a plain `[2]string` on `Match`, written once at
+construction and read by the coordinator goroutine. Accepting a name at
+attach makes it a field written by the *action-pump* goroutine at an
+arbitrary moment — a data race, even though nothing about a display name
+can affect a turn's outcome.
+
+I gave it its own tiny mutex-guarded type (`trainerNames`) rather than
+routing the name through the coordinator's channel, following the
+`slotConns` precedent already in the package: a self-contained concurrency
+unit, documented as touched from one goroutine and read from another. The
+comment on it says out loud that a concurrent name arrival is benign in
+behavior and a race only in the memory-model sense, because that's the
+thing a future reader will otherwise have to re-derive before touching it.
+
+`set("")` is a no-op rather than a store. Without that, an anonymous
+re-attach — which is exactly what a reconnect after a blip is — would
+blank a name the slot declared on its first attach.
+
+## Where the default came from
+
+`pokearena-agent --name` defaults to the model id rather than to empty.
+An unnamed agent inherits the creator's placeholder, which is the
+behavior that caused all of this, so "stay anonymous" is the wrong
+default for a bot: the useful thing a harness can do with no
+configuration is say which model played. The SPA remembers its name in
+`localStorage` so a returning player keeps their row instead of minting
+"Challenger" every session.
+
+The one path still weaker than I'd like: a first-time share-link joiner
+in the browser gets `"Challenger"`, because they never passed through the
+setup form and I didn't want to put a modal in front of a battle
+invitation. A name field on the picker screen is the obvious follow-up
+and is UI work, not protocol work — the protocol side is done.
diff --git a/cmd/pokearena-agent/main.go b/cmd/pokearena-agent/main.go
index 5cef5359..c7ed82ae 100644
--- a/cmd/pokearena-agent/main.go
+++ b/cmd/pokearena-agent/main.go
@@ -29,6 +29,7 @@ import (
 	"pokearena/internal/agentloop"
 	"pokearena/internal/domain"
 	"pokearena/internal/llm"
+	"pokearena/internal/protocol"
 )
 
 func main() {
@@ -43,6 +44,8 @@ func main() {
 		"Per-turn LLM call budget; the gateway will default-action the slot if a turn takes longer")
 	dataVersion := flag.String("data-version", "gen1-v1",
 		"Dataset version label — must match the gateway's DATA_VERSION")
+	name := flag.String("name", "",
+		"Trainer name to record this slot under on the leaderboard (default: the model id)")
 	flag.Usage = usage
 	flag.Parse()
 
@@ -72,13 +75,24 @@ func main() {
 	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
 	defer stop()
 
-	log.Printf("model=%s, gateway=%s, battle=%s, slot=%s", *model, gatewayURL, battleID, slot)
+	// Default the leaderboard name to the model id. An unnamed agent inherits
+	// the placeholder the battle's creator picked ("Opponent"), which is how
+	// every agent's games used to end up indistinguishable on the board — so
+	// the useful default is "say which model played", not "stay anonymous".
+	trainer := protocol.SanitizeTrainerName(*name)
+	if trainer == "" {
+		trainer = protocol.SanitizeTrainerName(*model)
+	}
+
+	log.Printf("model=%s, gateway=%s, battle=%s, slot=%s, trainer=%q",
+		*model, gatewayURL, battleID, slot, trainer)
 
 	cfg := agentloop.Config{
 		GatewayURL:     gatewayURL,
 		BattleID:       battleID,
 		Slot:           slot,
 		Token:          token,
+		TrainerName:    trainer,
 		Dex:            dex,
 		LLM:            llm.NewAnthropic(key, *model),
 		PerTurnTimeout: *turnTimeout,
diff --git a/docs/live-pvp.md b/docs/live-pvp.md
index 51a6275f..e33becee 100644
--- a/docs/live-pvp.md
+++ b/docs/live-pvp.md
@@ -63,7 +63,7 @@ adapter on top of the contract described here.
 ## 2. URL shape
 
 ```
-/api/battles/{battle_id}/play?slot={p1|p2}&token={join_token}
+/api/battles/{battle_id}/play?slot={p1|p2}&token={join_token}[&name={trainer}]
 ```
 
 One route, one validator. The same path serves the single-player
@@ -72,6 +72,18 @@ distinguishes them. Path-style slots (`/play/p1`) were tempting but
 keeping the slot as a query param lets a future third client type
 slot in without growing the route table.
 
+`name` is optional and is the joiner's **self-declared trainer name**
+— the leaderboard key this slot's result posts under. It is a query
+param on the join rather than a field in the create body because the
+battle is created before its players arrive: the creator names both
+slots, and an agent joining `p2` would otherwise be recorded forever as
+whatever placeholder they typed. Declaring it rebinds the battle row's
+trainer for that slot (`store.RebindBattleTrainer`) and relabels the
+slot in room frames and engine state. Omit it to keep the creator's
+name. Unlike the other three params it can contain spaces and `&`, so
+`PlayPath` query-escapes it; the gateway re-sanitizes on arrival
+(trim, drop control characters, cap at 24 runes).
+
 The URL builder lives in [`internal/protocol/pvp.go`](../internal/protocol/pvp.go)
 as `PlayPath`. Both the gateway (when issuing URLs to the creator)
 and any client constructing its own connect URL must use it — so
@@ -110,12 +122,24 @@ We're **not** designing against:
   opponent. The slot goes to whoever connects first; this is
   acceptable given the threat above is the gating concern.
 
+- **Impersonation on the leaderboard.** The `name` on a join is
+  self-reported: holding the slot token is the whole permission, and any
+  name may be claimed, including one already on the board. This is a
+  deliberate v1 position — attribution first, verification later — and
+  the reason the README calls the board "for fun, unverified." The fix
+  is claim-a-handle + secret, not a patch to this protocol. Note the
+  name is only accepted *after* the slot claim succeeds, so it is at
+  least no easier to forge than playing the battle would be.
+
 We **are** designing against:
 
 - Probing the failure modes (token vs battle vs slot validity). The
   opaque-error rule blocks this.
 - A claimed slot being hijacked by a second client with the same
   token. The atomic claim (§4) blocks this.
+- Renaming a settled result. `RebindBattleTrainer` refuses a battle
+  that already has a winner, so a late or replayed join cannot move a
+  rating that was computed against the previous trainer.
 
 ### Explicitly out of scope for v1
 
diff --git a/docs/mcp-protocol.md b/docs/mcp-protocol.md
index 94cffaf0..b15de88d 100644
--- a/docs/mcp-protocol.md
+++ b/docs/mcp-protocol.md
@@ -55,12 +55,27 @@ All five tools return structured JSON. The SDK turns Go errors into
 MCP error responses with `isError: true`; the agent should switch on
 the message content to distinguish cases.
 
-### `join_battle(battle_id, slot, join_token) → JoinResult`
+### `join_battle(battle_id, slot, join_token, trainer_name?) → JoinResult`
 
 Binds the MCP session to a battle slot. Opens the underlying WebSocket
 to `/api/battles/{battle_id}/play?slot={slot}&token={join_token}` and
 blocks until the gateway sends the first `state` frame.
 
+**`trainer_name` (optional)** is the name this slot's result posts under
+on the leaderboard. It matters because a live_pvp battle is created
+*before* its players arrive: whoever pressed "Start" named both slots, so
+an agent that joins without declaring itself is recorded under a
+placeholder like `"Opponent"` — which is why agent results used to be
+indistinguishable on the board. Declaring it appends `&name=` to the join
+URL; the gateway sanitizes it (trims, strips control characters, caps at
+24 characters) and rebinds the battle's trainer for that slot.
+
+The name is **self-reported, not authenticated**. Holding the slot token
+is the only permission required, and any name may be claimed — including
+one already on the leaderboard. Verified handles are separate, later
+work; until then, treat a leaderboard name as an assertion by whoever
+played, not as an identity the server vouches for.
+
 **Returns** on successful claim:
 
 ```json
diff --git a/internal/agentloop/loop.go b/internal/agentloop/loop.go
index 87bbbb5e..5d09a9e9 100644
--- a/internal/agentloop/loop.go
+++ b/internal/agentloop/loop.go
@@ -24,6 +24,13 @@ type Config struct {
 	// sees in the SPA share URL. Slot is "p1" or "p2".
 	BattleID, Slot, Token string
 
+	// TrainerName is the name recorded for this slot on the leaderboard.
+	// Empty inherits the placeholder the battle's creator supplied, which
+	// is how every agent result used to land as "Opponent" — set it to
+	// something identifying the controller (model, version) if the result
+	// is meant to be attributable.
+	TrainerName string
+
 	// Dex is needed to render move metadata in the prompt and to log
 	// decisions in human-readable form.
 	Dex *domain.Dex
@@ -56,7 +63,7 @@ func Run(ctx context.Context, cfg Config) error {
 		logger = log.Default()
 	}
 
-	gc, err := gwclient.Dial(ctx, cfg.GatewayURL, cfg.BattleID, cfg.Slot, cfg.Token)
+	gc, err := gwclient.Dial(ctx, cfg.GatewayURL, cfg.BattleID, cfg.Slot, cfg.Token, cfg.TrainerName)
 	if err != nil {
 		return fmt.Errorf("dial gateway: %w", err)
 	}
diff --git a/internal/gwclient/gwclient.go b/internal/gwclient/gwclient.go
index c3104da3..4c0cc11b 100644
--- a/internal/gwclient/gwclient.go
+++ b/internal/gwclient/gwclient.go
@@ -41,8 +41,13 @@ type Client struct {
 // starts the read pump, and returns the client ready for use. The
 // handshake itself respects ctx; the read pump runs in its own goroutine
 // and outlives ctx.
-func Dial(ctx context.Context, baseURL, battleID, slot, token string) (*Client, error) {
-	return dialPath(ctx, baseURL, protocol.PlayPath(battleID, slot, token))
+//
+// trainer is the name this client wants recorded for the slot on the
+// leaderboard. Pass "" to inherit whatever the battle's creator named it —
+// which for an agent joining someone else's battle is a placeholder like
+// "Opponent", so a bot that wants its results attributed should set it.
+func Dial(ctx context.Context, baseURL, battleID, slot, token, trainer string) (*Client, error) {
+	return dialPath(ctx, baseURL, protocol.PlayPath(battleID, slot, token, trainer))
 }
 
 // DialLive opens a WS to a single-player live-mode battle, where the opponent
@@ -50,8 +55,8 @@ func Dial(ctx context.Context, baseURL, battleID, slot, token string) (*Client,
 // slotless — the human is hardcoded to p1 — so this is the join path an MCP or
 // agent client uses to face the Heuristic/Expectimax opponent, the same one the
 // SPA's single-player mode plays against.
-func DialLive(ctx context.Context, baseURL, battleID string) (*Client, error) {
-	return dialPath(ctx, baseURL, protocol.LivePlayPath(battleID))
+func DialLive(ctx context.Context, baseURL, battleID, trainer string) (*Client, error) {
+	return dialPath(ctx, baseURL, protocol.LivePlayPath(battleID, trainer))
 }
 
 // dialPath is the shared connect: resolve the URL, open the socket, start the
diff --git a/internal/gwclient/gwclient_test.go b/internal/gwclient/gwclient_test.go
index 1d355029..8a5d91e4 100644
--- a/internal/gwclient/gwclient_test.go
+++ b/internal/gwclient/gwclient_test.go
@@ -4,6 +4,7 @@ import (
 	"context"
 	"net/http"
 	"net/http/httptest"
+	"net/url"
 	"strings"
 	"testing"
 	"time"
@@ -59,7 +60,7 @@ func TestDialAndReceive(t *testing.T) {
 
 	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 	defer cancel()
-	gc, err := Dial(ctx, base, "battle-x", "p1", "tok")
+	gc, err := Dial(ctx, base, "battle-x", "p1", "tok", "")
 	must(t, "dial", err)
 	defer gc.Close()
 
@@ -84,7 +85,7 @@ func TestSendAction(t *testing.T) {
 
 	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 	defer cancel()
-	gc, err := Dial(ctx, base, "b", "p2", "t")
+	gc, err := Dial(ctx, base, "b", "p2", "t", "")
 	must(t, "dial", err)
 	defer gc.Close()
 
@@ -121,7 +122,7 @@ func TestDialLive_UsesTokenlessPath(t *testing.T) {
 
 	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 	defer cancel()
-	gc, err := DialLive(ctx, base, "battle-live-1")
+	gc, err := DialLive(ctx, base, "battle-live-1", "")
 	must(t, "dial live", err)
 	defer gc.Close()
 
@@ -143,7 +144,7 @@ func TestCloseIsCleanAndIdempotent(t *testing.T) {
 
 	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 	defer cancel()
-	gc, err := Dial(ctx, base, "b", "p1", "t")
+	gc, err := Dial(ctx, base, "b", "p1", "t", "")
 	must(t, "dial", err)
 
 	must(t, "close 1", gc.Close())
@@ -172,7 +173,7 @@ func TestServerCloseReportsError(t *testing.T) {
 
 	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 	defer cancel()
-	gc, err := Dial(ctx, base, "b", "p1", "t")
+	gc, err := Dial(ctx, base, "b", "p1", "t", "")
 	must(t, "dial", err)
 	defer gc.Close()
 
@@ -223,3 +224,80 @@ func drain(t *testing.T, gc *Client, n int) []protocol.MatchUpdate {
 	}
 	return out
 }
+
+// captureJoin runs a gateway that records the query of the join request, so a
+// test can assert what a client actually sent rather than what it meant to.
+func captureJoin(t *testing.T) (base string, got *url.Values, cleanup func()) {
+	t.Helper()
+	var seen url.Values
+	up := websocket.Upgrader{CheckOrigin: func(*http.Request) bool { return true }}
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		seen = r.URL.Query()
+		c, err := up.Upgrade(w, r, nil)
+		if err != nil {
+			return
+		}
+		defer c.Close()
+		blockUntilPeerClose(c)
+	}))
+	return "ws" + strings.TrimPrefix(srv.URL, "http"), &seen, srv.Close
+}
+
+// The trainer name is how a bot's results get attributed on the leaderboard;
+// if it never leaves the client, every agent's games post under the placeholder
+// the battle's creator chose.
+func TestDial_SendsTrainerName(t *testing.T) {
+	base, seen, cleanup := captureJoin(t)
+	defer cleanup()
+
+	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
+	defer cancel()
+	gc, err := Dial(ctx, base, "battle-x", "p2", "tok", "claude-haiku")
+	must(t, "dial", err)
+	defer gc.Close()
+
+	if got := seen.Get("name"); got != "claude-haiku" {
+		t.Errorf("name query = %q, want %q", got, "claude-haiku")
+	}
+	if got := seen.Get("slot"); got != "p2" {
+		t.Errorf("slot query = %q, want p2", got)
+	}
+}
+
+func TestDial_OmitsTrainerNameWhenUndeclared(t *testing.T) {
+	base, seen, cleanup := captureJoin(t)
+	defer cleanup()
+
+	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
+	defer cancel()
+	gc, err := Dial(ctx, base, "battle-x", "p2", "tok", "")
+	must(t, "dial", err)
+	defer gc.Close()
+
+	// Absent, not empty: the gateway reads a missing name as "keep the
+	// creator's", and an empty one would sanitize to the same thing — but
+	// sending it at all would misrepresent an anonymous join as a declaration.
+	if seen.Has("name") {
+		t.Errorf("name query present (%q), want it omitted entirely", seen.Get("name"))
+	}
+}
+
+// Live mode is routed by the *absence* of a slot param, so adding a name must
+// not turn a vs-AI join into a pvp one.
+func TestDialLive_SendsTrainerNameWithoutASlot(t *testing.T) {
+	base, seen, cleanup := captureJoin(t)
+	defer cleanup()
+
+	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
+	defer cancel()
+	gc, err := DialLive(ctx, base, "battle-x", "claude-opus")
+	must(t, "dial", err)
+	defer gc.Close()
+
+	if got := seen.Get("name"); got != "claude-opus" {
+		t.Errorf("name query = %q, want %q", got, "claude-opus")
+	}
+	if seen.Has("slot") {
+		t.Errorf("slot query present (%q) — this would route to the pvp handler", seen.Get("slot"))
+	}
+}
diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go
index 1ed925b8..33597665 100644
--- a/internal/httpapi/server.go
+++ b/internal/httpapi/server.go
@@ -338,8 +338,10 @@ func (s *Server) handleCreateBattle(w http.ResponseWriter, r *http.Request) {
 		}
 		writeJSON(w, http.StatusCreated, map[string]any{
 			"battle_id": battleID, "mode": "live_pvp",
-			"p1_url": protocol.PlayPath(battleID, string(cache.SlotP1), p1Token),
-			"p2_url": protocol.PlayPath(battleID, string(cache.SlotP2), p2Token),
+			// No name in the issued URLs: the slot's name is the joiner's to
+			// declare at connect time, not the creator's to assign in advance.
+			"p1_url": protocol.PlayPath(battleID, string(cache.SlotP1), p1Token, ""),
+			"p2_url": protocol.PlayPath(battleID, string(cache.SlotP2), p2Token, ""),
 		})
 		return
 	}
diff --git a/internal/httpapi/ws.go b/internal/httpapi/ws.go
index 46569334..dfe8b93f 100644
--- a/internal/httpapi/ws.go
+++ b/internal/httpapi/ws.go
@@ -50,13 +50,18 @@ func (s *Server) handleLiveWS(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
+	// Live mode seats the joiner at p1, so it names p1. An agent that dialed
+	// here (rather than being the SPA that created the battle) otherwise has no
+	// way to say who it is — the create call was made by someone else.
+	trainer := s.rebindTrainer(ctx, battleID, cache.SlotP1, r.URL.Query().Get("name"))
+
 	conn, err := upgrader.Upgrade(w, r, nil)
 	if err != nil {
 		return
 	}
 	defer conn.Close()
 
-	s.bridgeSlot(ctx, conn, battleID, cache.SlotP1)
+	s.bridgeSlot(ctx, conn, battleID, cache.SlotP1, trainer)
 }
 
 // joinableStatus reports whether a battle in the given lifecycle status still
@@ -118,13 +123,55 @@ func (s *Server) handlePvPWS(w http.ResponseWriter, r *http.Request) {
 	}
 	defer s.releaseSlotBest(battleID, slot)
 
+	// The joiner may declare who it is. Do this after the claim so an
+	// unauthenticated caller cannot rewrite a battle's trainer by guessing
+	// battle ids — holding the slot token is the permission to name the slot.
+	trainer := s.rebindTrainer(ctx, battleID, slot, r.URL.Query().Get("name"))
+
 	conn, err := upgrader.Upgrade(w, r, nil)
 	if err != nil {
 		return
 	}
 	defer conn.Close()
 
-	s.bridgeSlot(ctx, conn, battleID, slot)
+	s.bridgeSlot(ctx, conn, battleID, slot, trainer)
+}
+
+// rebindTrainer records the joiner's self-declared name as the occupant of
+// this slot, and returns the sanitized name for the session to display ("" if
+// none was declared or nothing survived sanitizing).
+//
+// Why this exists: a live_pvp battle is created before its players arrive, so
+// both slots are named by whoever pressed "Start" — an agent joining p2 lands
+// on the board as "Opponent". Declaring a name at join is what lets a result
+// be attributed to the controller that actually played it.
+//
+// This is attribution, not authentication. Any holder of a slot token may
+// claim any name, including one already on the leaderboard; verified handles
+// (claim-a-handle + secret) are separate, later work. Treat a name here as
+// self-reported.
+//
+// A failure to persist is logged and swallowed: the battle is playable and the
+// player is already connected, so refusing the join over a leaderboard-label
+// write would trade a working game for a cosmetic field.
+func (s *Server) rebindTrainer(ctx context.Context, battleID string, slot cache.PvPSlot, raw string) string {
+	name := protocol.SanitizeTrainerName(raw)
+	if name == "" {
+		return "" // nothing declared — keep the creator's placeholder
+	}
+	trainerID, err := s.store.UpsertTrainer(ctx, name)
+	if err != nil {
+		log.Printf("pvp trainer upsert failed battle=%s slot=%s name=%q: %v", battleID, slot, name, err)
+		return name
+	}
+	idx := 0
+	if slot == cache.SlotP2 {
+		idx = 1
+	}
+	if err := s.store.RebindBattleTrainer(ctx, battleID, idx, trainerID, name); err != nil {
+		log.Printf("pvp trainer rebind failed battle=%s slot=%s name=%q: %v", battleID, slot, name, err)
+	}
+	return name
 }
 
 // bridgeSlot is the gateway's whole live-battle job: shuttle bytes between one
@@ -137,7 +184,10 @@ func (s *Server) handlePvPWS(w http.ResponseWriter, r *http.Request) {
 // function's reader loop (socket → actions). On reader exit the bridge tells the
 // session the slot disconnected; the writer's deferred read-deadline unblocks a
 // half-open reader so the slot is never leaked.
-func (s *Server) bridgeSlot(ctx context.Context, conn *websocket.Conn, battleID string, slot cache.PvPSlot) {
+// trainer, when non-empty, is the slot's self-declared name; it rides the
+// attach message so the session owner can label the slot in room frames and in
+// the engine state it builds.
+func (s *Server) bridgeSlot(ctx context.Context, conn *websocket.Conn, battleID string, slot cache.PvPSlot, trainer string) {
 	slotName := string(slot)
 
 	subID, frames, err := s.hub.SubscribeFrames(battleID, slotName)
@@ -158,7 +208,7 @@ func (s *Server) bridgeSlot(ctx context.Context, conn *websocket.Conn, battleID
 
 	// Announce attachment so the session shows this slot connected; announce
 	// disconnect on exit so it can wind the match down (after its grace window).
-	s.sendLiveAction(messages.LiveAction{BattleID: battleID, Slot: slotName, Conn: connID, Phase: messages.LivePhaseAttach})
+	s.sendLiveAction(messages.LiveAction{BattleID: battleID, Slot: slotName, Conn: connID, Phase: messages.LivePhaseAttach, Trainer: trainer})
 	defer s.sendLiveAction(messages.LiveAction{BattleID: battleID, Slot: slotName, Conn: connID, Phase: messages.LivePhaseDisconnect})
 
 	bridgeCtx, cancelBridge := context.WithCancel(ctx)
diff --git a/internal/livebattle/bridge.go b/internal/livebattle/bridge.go
index 97340770..c9b85f09 100644
--- a/internal/livebattle/bridge.go
+++ b/internal/livebattle/bridge.go
@@ -56,6 +56,10 @@ func (p *Pump) Route(a messages.LiveAction) {
 	}
 	switch a.Phase {
 	case messages.LivePhaseAttach:
+		// Adopt the joiner's declared name before announcing the connection, so
+		// the room frame that the attach triggers already carries it and the
+		// opponent never sees the placeholder flash past.
+		p.m.SetTrainerName(slot, a.Trainer)
 		// Record the connection before attaching the producer: a re-attach under a
 		// new id must cancel any reconnect-grace timer from the prior connection's
 		// disconnect, even though the producer is already registered.
diff --git a/internal/livebattle/coordinator.go b/internal/livebattle/coordinator.go
index e8b2cd1f..2780870d 100644
--- a/internal/livebattle/coordinator.go
+++ b/internal/livebattle/coordinator.go
@@ -277,8 +277,8 @@ func (m *Match) runOpenPhase(ctx context.Context) error {
 	}
 
 	st, err := engine.NewBattleFromPicks(m.deps.Dex, m.battleID,
-		m.trainerName[0], m.submitted[0],
-		m.trainerName[1], m.submitted[1],
+		m.trainerName.get(0), m.submitted[0],
+		m.trainerName.get(1), m.submitted[1],
 		m.seed)
 	if err != nil {
 		return fmt.Errorf("engine init: %w", err)
@@ -561,12 +561,12 @@ func (m *Match) broadcastRoom(phase protocol.RoomPhase, attached [2]bool) {
 		you := protocol.RoomSlot{
 			Attached:  attached[i],
 			Submitted: m.submitted[i] != nil,
-			Trainer:   m.trainerName[i],
+			Trainer:   m.trainerName.get(i),
 		}
 		them := protocol.RoomSlot{
 			Attached:  attached[1-i],
 			Submitted: m.submitted[1-i] != nil,
-			Trainer:   m.trainerName[1-i],
+			Trainer:   m.trainerName.get(1 - i),
 		}
 		m.send(i, protocol.MatchUpdate{
 			Type: protocol.FrameRoom,
diff --git a/internal/livebattle/livebattle.go b/internal/livebattle/livebattle.go
index eae118de..5dce9977 100644
--- a/internal/livebattle/livebattle.go
+++ b/internal/livebattle/livebattle.go
@@ -186,7 +186,7 @@ type Match struct {
 	battleID     string
 	createdAt    time.Time
 	seed         uint64
-	trainerName  [2]string
+	trainerName  trainerNames
 	kind         [2]SideKind
 	aiTeam       [2][]engine.TeamPick
 	roomDeadline time.Duration
@@ -257,7 +257,7 @@ func NewMatch(cfg Config) *Match {
 		battleID:        cfg.BattleID,
 		createdAt:       time.Now(),
 		seed:            cfg.Seed,
-		trainerName:     [2]string{cfg.P1Name, cfg.P2Name},
+		trainerName:     trainerNames{name: [2]string{cfg.P1Name, cfg.P2Name}},
 		kind:            cfg.Kinds,
 		aiTeam:          cfg.AITeams,
 		roomDeadline:    deadline,
@@ -340,6 +340,11 @@ func (m *Match) Disconnect(slot int) {
 	m.closeOnce[slot].Do(func() { close(m.closed[slot]) })
 }
 
+// SetTrainerName records a slot's self-declared display name, replacing the
+// one the battle's creator supplied. Ignored when name is empty (the joiner
+// declared nothing). Safe to call from the action-pump goroutine.
+func (m *Match) SetTrainerName(slot int, name string) { m.trainerName.set(slot, name) }
+
 // SlotConnected records that connID is now the live connection for slot,
 // canceling any reconnect-grace timer in flight. See slotConns.connected.
 func (m *Match) SlotConnected(slot int, connID string) {
diff --git a/internal/livebattle/trainers.go b/internal/livebattle/trainers.go
new file mode 100644
index 00000000..89e74667
--- /dev/null
+++ b/internal/livebattle/trainers.go
@@ -0,0 +1,39 @@
+package livebattle
+
+import "sync"
+
+// trainerNames holds the display name of each slot's occupant.
+//
+// It needs its own mutex because, unlike everything else the coordinator
+// reads, these are written from a different goroutine at an arbitrary moment:
+// the names start as whatever the battle's creator supplied, and a slot may
+// replace its own when it attaches (see messages.LiveAction.Trainer). The
+// write lands on the action-pump goroutine; the reads are on the coordinator
+// goroutine, building room frames and the initial engine state.
+//
+// Names are display-only — no turn, action, or validation decision reads
+// them — so a name that arrives concurrently with a room broadcast is a
+// benign race in behavior, and only a data race in the memory-model sense.
+// That is exactly what this guards.
+type trainerNames struct {
+	mu   sync.RWMutex
+	name [2]string
+}
+
+func (t *trainerNames) get(slot int) string {
+	t.mu.RLock()
+	defer t.mu.RUnlock()
+	return t.name[slot]
+}
+
+// set replaces the slot's name. An empty name is ignored rather than stored:
+// a joiner that declares nothing keeps the creator-supplied placeholder, and
+// treating "" as a value would blank the label on every anonymous attach.
+func (t *trainerNames) set(slot int, name string) {
+	if name == "" {
+		return
+	}
+	t.mu.Lock()
+	defer t.mu.Unlock()
+	t.name[slot] = name
+}
diff --git a/internal/livebattle/trainers_test.go b/internal/livebattle/trainers_test.go
new file mode 100644
index 00000000..d7670aed
--- /dev/null
+++ b/internal/livebattle/trainers_test.go
@@ -0,0 +1,124 @@
+package livebattle
+
+import (
+	"context"
+	"testing"
+	"time"
+
+	"pokearena/internal/messages"
+	"pokearena/internal/protocol"
+)
+
+// A live_pvp battle is created before its players arrive, so both slots carry
+// names chosen by whoever pressed "Start" — an agent joining p2 was recorded as
+// "Opponent". A slot that declares a name on attach must replace it, because
+// that name is the leaderboard key the battle's result posts to.
+func TestAttach_DeclaredTrainerNameReplacesCreatorPlaceholder(t *testing.T) {
+	dex := loadDex(t)
+	t1, t2 := twoTeams(t, dex)
+
+	sink := newChanSink()
+	m := NewMatch(Config{
+		BattleID: "B-name", P1Name: "Red", P2Name: "Opponent", Seed: 7,
+		Kinds: [2]SideKind{SideWS, SideWS},
+		Sink:  sink,
+		Deps: Deps{
+			Dex: dex, Cache: &fakeCache{}, Store: &fakeStore{}, Publish: (&eventRecorder{}).publish,
+		},
+	})
+	pump := NewPump(m, [2]SideKind{SideWS, SideWS})
+
+	done := make(chan struct{})
+	go func() { m.Run(context.Background()); close(done) }()
+	defer func() {
+		go func() {
+			for range sink.ch[0] {
+			}
+		}()
+		go func() {
+			for range sink.ch[1] {
+			}
+		}()
+		pump.Route(messages.LiveAction{BattleID: "B-name", Slot: "p1", Phase: messages.LivePhaseDisconnect})
+		<-done
+	}()
+
+	// p1 attaches anonymously (keeps "Red"); p2 declares itself.
+	pump.Route(messages.LiveAction{BattleID: "B-name", Slot: "p1", Phase: messages.LivePhaseAttach})
+	pump.Route(messages.LiveAction{
+		BattleID: "B-name", Slot: "p2", Phase: messages.LivePhaseAttach, Trainer: "claude-haiku",
+	})
+
+	// The room frame is the first place the opponent sees who they're facing.
+	// p1 gets a room frame at its own attach too, before p2 exists — wait for
+	// the broadcast that actually reflects p2 being in the room.
+	room := readRoomUntilThemAttached(t, sink.ch[0], 5*time.Second)
+	if got := room.You.Trainer; got != "Red" {
+		t.Errorf("p1 (anonymous attach) = %q, want the creator's name %q", got, "Red")
+	}
+	if got := room.Them.Trainer; got != "claude-haiku" {
+		t.Errorf("p2 as seen by p1 = %q, want the declared name %q", got, "claude-haiku")
+	}
+
+	// And it must reach the engine state, which is what gets persisted and
+	// replayed — a name that only lived in a room frame would be cosmetic.
+	pump.Route(messages.LiveAction{BattleID: "B-name", Slot: "p1", Phase: messages.LivePhaseSubmit, Picks: t1})
+	pump.Route(messages.LiveAction{BattleID: "B-name", Slot: "p2", Phase: messages.LivePhaseSubmit, Picks: t2})
+
+	// Each side sees its own Side in full; the fog projection reduces the foe
+	// to their active Pokémon, so p2's name is asserted from p2's own view.
+	s0 := readUntil(t, sink.ch[0], protocol.FrameState, 5*time.Second)
+	s1 := readUntil(t, sink.ch[1], protocol.FrameState, 5*time.Second)
+	if s0.View == nil || s1.View == nil {
+		t.Fatal("FrameState carried no view")
+	}
+	if got := s0.View.Self.Trainer; got != "Red" {
+		t.Errorf("engine state p1 trainer = %q, want %q", got, "Red")
+	}
+	if got := s1.View.Self.Trainer; got != "claude-haiku" {
+		t.Errorf("engine state p2 trainer = %q, want the declared name %q", got, "claude-haiku")
+	}
+}
+
+// readRoomUntilThemAttached returns the first FrameRoom whose opponent slot is
+// attached. A slot receives a room broadcast on its own attach as well, and
+// that one necessarily predates the other side joining.
+func readRoomUntilThemAttached(t *testing.T, ch <-chan protocol.MatchUpdate, timeout time.Duration) *protocol.RoomUpdate {
+	t.Helper()
+	deadline := time.After(timeout)
+	for {
+		select {
+		case u, ok := <-ch:
+			if !ok {
+				t.Fatal("frame channel closed before the opponent attached")
+			}
+			if u.Type == protocol.FrameRoom && u.Room != nil && u.Room.Them.Attached {
+				return u.Room
+			}
+		case <-deadline:
+			t.Fatal("timed out waiting for a room frame with the opponent attached")
+		}
+	}
+}
+
+// set is the write half of the concurrency unit; an empty declaration must be
+// a no-op rather than a store, or every anonymous attach would blank the label.
+func TestTrainerNames_EmptyDeclarationKeepsExisting(t *testing.T) {
+	names := trainerNames{name: [2]string{"Red", "Blue"}}
+
+	names.set(1, "")
+	if got := names.get(1); got != "Blue" {
+		t.Errorf("after empty set, name = %q, want the existing %q", got, "Blue")
+	}
+
+	names.set(1, "claude-opus")
+	if got := names.get(1); got != "claude-opus" {
+		t.Errorf("after set, name = %q, want %q", got, "claude-opus")
+	}
+
+	// A re-attach that declares nothing must not undo the name from the first.
+	names.set(1, "")
+	if got := names.get(1); got != "claude-opus" {
+		t.Errorf("re-attach blanked the name: got %q, want %q", got, "claude-opus")
+	}
+}
diff --git a/internal/mcpserver/session.go b/internal/mcpserver/session.go
index 2a1ac90d..d9af4db1 100644
--- a/internal/mcpserver/session.go
+++ b/internal/mcpserver/session.go
@@ -159,7 +159,7 @@ func newSession(cfg Config) *session {
 // Join opens the gateway WS, starts the dispatcher, and blocks until
 // the first state frame arrives. Returns the initial view + identity
 // info the agent needs to play.
-func (s *session) Join(ctx context.Context, battleID, slot, token string) (joinBattleOut, error) {
+func (s *session) Join(ctx context.Context, battleID, slot, token, trainer string) (joinBattleOut, error) {
 	s.mu.Lock()
 	if s.client != nil {
 		s.mu.Unlock()
@@ -176,9 +176,9 @@ func (s *session) Join(ctx context.Context, battleID, slot, token string) (joinB
 	var err error
 	if token == "" {
 		slot = "p1"
-		gc, err = gwclient.DialLive(ctx, s.cfg.GatewayURL, battleID)
+		gc, err = gwclient.DialLive(ctx, s.cfg.GatewayURL, battleID, trainer)
 	} else {
-		gc, err = gwclient.Dial(ctx, s.cfg.GatewayURL, battleID, slot, token)
+		gc, err = gwclient.Dial(ctx, s.cfg.GatewayURL, battleID, slot, token, trainer)
 	}
 	if err != nil {
 		return joinBattleOut{}, fmt.Errorf("connect to gateway: %w", err)
diff --git a/internal/mcpserver/session_test.go b/internal/mcpserver/session_test.go
index 24967533..92551037 100644
--- a/internal/mcpserver/session_test.go
+++ b/internal/mcpserver/session_test.go
@@ -90,7 +90,7 @@ func TestWaitPreservesFoeHPPercent(t *testing.T) {
 	sess := newTestSession(base)
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	if _, err := sess.Join(ctx, "b", "p1", "tok"); err != nil {
+	if _, err := sess.Join(ctx, "b", "p1", "tok", ""); err != nil {
 		t.Fatalf("Join: %v", err)
 	}
 	w, err := sess.Wait(ctx, 5)
@@ -124,7 +124,7 @@ func TestJoinReturnsFirstView(t *testing.T) {
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
 
-	out, err := sess.Join(ctx, "battle-x", "p1", "tok")
+	out, err := sess.Join(ctx, "battle-x", "p1", "tok", "")
 	must(t, "Join", err)
 	if out.YourTrainer != "Red" {
 		t.Errorf("YourTrainer=%q, want Red", out.YourTrainer)
@@ -144,10 +144,10 @@ func TestJoinTwiceFails(t *testing.T) {
 	sess := newTestSession(base)
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	_, err := sess.Join(ctx, "b", "p1", "t")
+	_, err := sess.Join(ctx, "b", "p1", "t", "")
 	must(t, "Join", err)
 
-	_, err = sess.Join(ctx, "b2", "p2", "t2")
+	_, err = sess.Join(ctx, "b2", "p2", "t2", "")
 	if !errors.Is(err, errAlreadyJoined) {
 		t.Errorf("second Join: got %v, want errAlreadyJoined", err)
 	}
@@ -183,7 +183,7 @@ func TestWaitReturnsImmediatelyAfterJoin(t *testing.T) {
 	sess := newTestSession(base)
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	_, err := sess.Join(ctx, "b", "p1", "t")
+	_, err := sess.Join(ctx, "b", "p1", "t", "")
 	must(t, "Join", err)
 
 	start := time.Now()
@@ -228,7 +228,7 @@ func TestActThenWaitForOpponent(t *testing.T) {
 	sess := newTestSession(base)
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	_, err := sess.Join(ctx, "b", "p1", "t")
+	_, err := sess.Join(ctx, "b", "p1", "t", "")
 	must(t, "Join", err)
 
 	// First Wait → ready for turn 0.
@@ -278,7 +278,7 @@ func TestWaitTimesOut(t *testing.T) {
 	sess := newTestSession(base)
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	_, err := sess.Join(ctx, "b", "p1", "t")
+	_, err := sess.Join(ctx, "b", "p1", "t", "")
 	must(t, "Join", err)
 	_, _ = sess.Wait(ctx, 1)
 	_, _ = sess.Act(protocol.ActionKindMove, 0)
@@ -314,7 +314,7 @@ func TestEndFrameTerminatesSession(t *testing.T) {
 	sess := newTestSession(base)
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	_, err := sess.Join(ctx, "b", "p1", "t")
+	_, err := sess.Join(ctx, "b", "p1", "t", "")
 	must(t, "Join", err)
 	_, _ = sess.Wait(ctx, 1) // returns immediately with state from Join
 	_, err = sess.Act(protocol.ActionKindMove, 0)
@@ -349,7 +349,7 @@ func TestLeaveAllowsRejoin(t *testing.T) {
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
 
-	_, err := sess.Join(ctx, "b1", "p1", "t1")
+	_, err := sess.Join(ctx, "b1", "p1", "t1", "")
 	must(t, "Join 1", err)
 	must(t, "Leave", sess.Leave())
 
@@ -357,7 +357,7 @@ func TestLeaveAllowsRejoin(t *testing.T) {
 	defer cleanup2()
 	sess.cfg.GatewayURL = base2
 
-	out, err := sess.Join(ctx, "b2", "p2", "t2")
+	out, err := sess.Join(ctx, "b2", "p2", "t2", "")
 	must(t, "Join 2", err)
 	if out.YourTrainer != "Blue" || out.BattleID != "b2" {
 		t.Errorf("rejoin lost identity: %+v", out)
diff --git a/internal/mcpserver/tools.go b/internal/mcpserver/tools.go
index c5ad2aff..e9cb0a1c 100644
--- a/internal/mcpserver/tools.go
+++ b/internal/mcpserver/tools.go
@@ -20,6 +20,7 @@ type joinBattleIn struct {
 	BattleID string `json:"battle_id" jsonschema:"the battle's UUID, as printed by the gateway when the battle was created"`
 	Slot     string `json:"slot" jsonschema:"which trainer slot to claim: 'p1' or 'p2'. Ignored for live (vs-AI) battles, which always seat you as p1"`
 	Token    string `json:"join_token" jsonschema:"the per-slot join token for a pvp battle; treat as a password — never log it. Omit (empty) to join a live vs-AI battle, which is tokenless"`
+	Trainer  string `json:"trainer_name,omitempty" jsonschema:"the name to record for this slot on the leaderboard - identify yourself (e.g. your model name). Max 24 characters. Omit to inherit the placeholder the battle's creator chose, which means the result is recorded against a name like 'Opponent' rather than against you"`
 }
 
 type joinBattleOut struct {
@@ -150,7 +151,10 @@ func (s *Server) registerTools() {
 		Description: "Bind this MCP session to a battle. Opens a WebSocket to the gateway " +
 			"and returns the initial fog-of-war view. Call this first; every other tool requires it. " +
 			"For a live vs-AI battle, pass only battle_id (no slot, no join_token) — you are seated as " +
-			"p1 against the programmatic opponent. For a pvp battle, pass slot and join_token.",
+			"p1 against the programmatic opponent. For a pvp battle, pass slot and join_token. " +
+			"Pass trainer_name to say who you are: it is the name this battle's result is recorded " +
+			"under on the leaderboard, and without it your games land under the placeholder the " +
+			"battle's creator chose.",
 	}, s.joinBattle)
 
 	mcp.AddTool(s.mcp, &mcp.Tool{
@@ -233,7 +237,7 @@ func (s *Server) registerTools() {
 // MCP result automatically.
 
 func (s *Server) joinBattle(ctx context.Context, _ *mcp.CallToolRequest, in joinBattleIn) (*mcp.CallToolResult, joinBattleOut, error) {
-	out, err := s.session.Join(ctx, in.BattleID, in.Slot, in.Token)
+	out, err := s.session.Join(ctx, in.BattleID, in.Slot, in.Token, in.Trainer)
 	return nil, out, err
 }
 
diff --git a/internal/messages/messages.go b/internal/messages/messages.go
index 9b8d9015..774b82f8 100644
--- a/internal/messages/messages.go
+++ b/internal/messages/messages.go
@@ -104,9 +104,15 @@ type LiveAction struct {
 	// is no longer the live one, and (b) cancel a disconnect's reconnect-grace
 	// timer when the same slot re-attaches under a new id. Empty on legacy/test
 	// messages, which are always honored (no identity check).
-	Conn   string            `json:"conn,omitempty"`
-	Picks  []engine.TeamPick `json:"picks,omitempty"`  // Phase == "submit"
-	Action engine.Action     `json:"action,omitempty"` // Phase == "action"
+	Conn string `json:"conn,omitempty"`
+	// Trainer is the slot's self-declared name, carried on Phase == "attach"
+	// only. The gateway has already sanitized it and rebound the battle row's
+	// trainer, so this copy exists purely so the owner's in-memory match can
+	// label the slot in room frames and in the engine state it builds. Empty
+	// means the joiner declared nothing — keep the creator-supplied name.
+	Trainer string            `json:"trainer,omitempty"`
+	Picks   []engine.TeamPick `json:"picks,omitempty"`  // Phase == "submit"
+	Action  engine.Action     `json:"action,omitempty"` // Phase == "action"
 }
 
 // Live action phases.
diff --git a/internal/protocol/pvp.go b/internal/protocol/pvp.go
index 5a59bcbc..e85a91a5 100644
--- a/internal/protocol/pvp.go
+++ b/internal/protocol/pvp.go
@@ -10,6 +10,9 @@ package protocol
 
 import (
 	"encoding/json"
+	"net/url"
+	"strings"
+	"unicode"
 
 	"pokearena/internal/ai"
 	"pokearena/internal/engine"
@@ -124,15 +127,59 @@ type WsClientMsg struct {
 	Picks []engine.TeamPick `json:"picks,omitempty"`
 }
 
+// MaxTrainerNameLen bounds a self-declared trainer name, matching the SPA's
+// input maxlength. The name is a leaderboard key, so it is stored and shown
+// verbatim; the cap keeps one from crowding a standings row or a room frame.
+const MaxTrainerNameLen = 24
+
+// SanitizeTrainerName normalizes a self-declared trainer name to what the
+// gateway is willing to store. Both sides call it: a client so it can tell
+// the user what will actually be recorded, and the gateway because a name
+// arriving over the wire is untrusted input.
+//
+// The rules are deliberately narrow — trim surrounding space, drop control
+// characters (a newline in a name corrupts every log line that prints it),
+// and cap the length. Everything else is allowed through unchanged: this
+// normalizes, it does not police. Returns "" when nothing survives, which
+// callers read as "no name declared" rather than as a name of empty string.
+func SanitizeTrainerName(name string) string {
+	cleaned := strings.Map(func(r rune) rune {
+		if r == '\t' || r == '\n' || r == '\r' {
+			return ' ' // fold whitespace-ish controls so words stay separated
+		}
+		if unicode.IsControl(r) {
+			return -1
+		}
+		return r
+	}, name)
+	cleaned = strings.TrimSpace(cleaned)
+	// Count runes, not bytes: a 24-byte cap would truncate a non-ASCII name
+	// mid-codepoint and store invalid UTF-8.
+	if runes := []rune(cleaned); len(runes) > MaxTrainerNameLen {
+		cleaned = strings.TrimSpace(string(runes[:MaxTrainerNameLen]))
+	}
+	return cleaned
+}
+
 // PlayPath builds the WebSocket join path for a live_pvp slot. Both the
 // gateway (issuing URLs to the battle creator) and any client that
 // constructs its own connect URL (the MCP server building from a
 // battle_id + token pair) call this so the shape stays in lockstep.
 //
+// trainer is the joiner's self-declared name for the leaderboard; pass ""
+// to leave the slot with whatever name the battle's creator gave it. It is
+// sanitized here so a client cannot smuggle a name past the shared rule by
+// building its own query, and query-escaped because unlike battleID/slot/
+// token (UUID, "p1"/"p2", base64url) a name may contain spaces and '&'.
+//
 // Returns the path only, not the scheme/host — callers prepend their
 // origin or gateway base URL.
-func PlayPath(battleID, slot, token string) string {
-	return "/api/battles/" + battleID + "/play?slot=" + slot + "&token=" + token
+func PlayPath(battleID, slot, token, trainer string) string {
+	p := "/api/battles/" + battleID + "/play?slot=" + slot + "&token=" + token
+	if t := SanitizeTrainerName(trainer); t != "" {
+		p += "&name=" + url.QueryEscape(t)
+	}
+	return p
 }
 
 // LivePlayPath builds the WebSocket join path for a single-player live-mode
@@ -141,6 +188,15 @@ func PlayPath(battleID, slot, token string) string {
 // ID is the whole auth model (see httpapi handleLiveWS). The absence of a slot
 // param is precisely what routes the gateway to the live handler instead of the
 // pvp one, so this must not append one.
-func LivePlayPath(battleID string) string {
-	return "/api/battles/" + battleID + "/play"
+//
+// trainer is the joiner's self-declared name for the leaderboard, same
+// contract as PlayPath; pass "" to keep the creator's name. A "name" query is
+// safe to add here because the handler routes on "slot", not on the query
+// being empty.
+func LivePlayPath(battleID, trainer string) string {
+	p := "/api/battles/" + battleID + "/play"
+	if t := SanitizeTrainerName(trainer); t != "" {
+		p += "?name=" + url.QueryEscape(t)
+	}
+	return p
 }
diff --git a/internal/protocol/pvp_test.go b/internal/protocol/pvp_test.go
new file mode 100644
index 00000000..83a7f6fe
--- /dev/null
+++ b/internal/protocol/pvp_test.go
@@ -0,0 +1,131 @@
+package protocol
+
+import (
+	"net/url"
+	"strings"
+	"testing"
+)
+
+func TestSanitizeTrainerName(t *testing.T) {
+	cases := []struct {
+		name string
+		in   string
+		want string
+	}{
+		{"plain", "Red", "Red"},
+		{"trims surrounding space", "  Red  ", "Red"},
+		{"keeps interior space", "Trainer Red", "Trainer Red"},
+		{"empty stays empty", "", ""},
+		{"whitespace only collapses to empty", "   \t  ", ""},
+		// A newline in a name would break every log line and room frame that
+		// prints it, so controls are folded to a space rather than kept.
+		{"folds newline to space", "Trainer\nRed", "Trainer Red"},
+		{"drops other controls", "Red\x00\x07", "Red"},
+		{"caps at the max", strings.Repeat("a", MaxTrainerNameLen+10), strings.Repeat("a", MaxTrainerNameLen)},
+		{"trims after capping", strings.Repeat("a", MaxTrainerNameLen-1) + "   tail", strings.Repeat("a", MaxTrainerNameLen-1)},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			if got := SanitizeTrainerName(c.in); got != c.want {
+				t.Errorf("SanitizeTrainerName(%q) = %q, want %q", c.in, got, c.want)
+			}
+		})
+	}
+}
+
+// A byte-based cap would slice a multi-byte rune in half and store invalid
+// UTF-8 under a name that is then a leaderboard key. Count runes.
+func TestSanitizeTrainerName_CapsRunesNotBytes(t *testing.T) {
+	in := strings.Repeat("é", MaxTrainerNameLen+5) // 2 bytes per rune
+	got := SanitizeTrainerName(in)
+	if n := len([]rune(got)); n != MaxTrainerNameLen {
+		t.Errorf("kept %d runes, want %d", n, MaxTrainerNameLen)
+	}
+	if !isValidUTF8(got) {
+		t.Errorf("truncation produced invalid UTF-8: %q", got)
+	}
+}
+
+func isValidUTF8(s string) bool {
+	for _, r := range s {
+		if r == '�' {
+			return false
+		}
+	}
+	return true
+}
+
+func TestPlayPath(t *testing.T) {
+	t.Run("omits the name query when none is declared", func(t *testing.T) {
+		got := PlayPath("b1", "p2", "tok", "")
+		want := "/api/battles/b1/play?slot=p2&token=tok"
+		if got != want {
+			t.Errorf("got %q, want %q", got, want)
+		}
+	})
+
+	t.Run("appends a declared name", func(t *testing.T) {
+		got := PlayPath("b1", "p2", "tok", "Red")
+		want := "/api/battles/b1/play?slot=p2&token=tok&name=Red"
+		if got != want {
+			t.Errorf("got %q, want %q", got, want)
+		}
+	})
+
+	// A name is the only join field that can contain a space or '&' — the
+	// others are a UUID, "p1"/"p2", and base64url. Unescaped, "A&slot=p1"
+	// would smuggle a second slot param into the query.
+	t.Run("escapes a name that could forge a query param", func(t *testing.T) {
+		got := PlayPath("b1", "p2", "tok", "A&slot=p1")
+		q, err := url.Parse(got)
+		if err != nil {
+			t.Fatalf("parse: %v", err)
+		}
+		if slot := q.Query().Get("slot"); slot != "p2" {
+			t.Errorf("slot = %q, want p2 — the name forged a query param", slot)
+		}
+		if name := q.Query().Get("name"); name != "A&slot=p1" {
+			t.Errorf("name = %q, want it round-tripped intact", name)
+		}
+	})
+
+	// A client building its own path must not be able to bypass the shared
+	// rule by passing something the gateway would have rejected.
+	t.Run("sanitizes before appending", func(t *testing.T) {
+		got := PlayPath("b1", "p2", "tok", "  Red\n  ")
+		if name := mustQuery(t, got).Get("name"); name != "Red" {
+			t.Errorf("name = %q, want %q", name, "Red")
+		}
+	})
+}
+
+func TestLivePlayPath(t *testing.T) {
+	t.Run("stays slotless and queryless without a name", func(t *testing.T) {
+		got := LivePlayPath("b1", "")
+		if got != "/api/battles/b1/play" {
+			t.Errorf("got %q, want the bare path", got)
+		}
+	})
+
+	// The gateway routes to the live handler on the *absence of a slot param*,
+	// not on an empty query — so adding "name" must not reroute the join.
+	t.Run("a name does not introduce a slot param", func(t *testing.T) {
+		got := LivePlayPath("b1", "Red")
+		q := mustQuery(t, got)
+		if q.Has("slot") {
+			t.Errorf("got %q, which would route to the pvp handler", got)
+		}
+		if q.Get("name") != "Red" {
+			t.Errorf("name = %q, want Red", q.Get("name"))
+		}
+	})
+}
+
+func mustQuery(t *testing.T, rawPath string) url.Values {
+	t.Helper()
+	u, err := url.Parse(rawPath)
+	if err != nil {
+		t.Fatalf("parse %q: %v", rawPath, err)
+	}
+	return u.Query()
+}
diff --git a/internal/session/trainerrebind_test.go b/internal/session/trainerrebind_test.go
new file mode 100644
index 00000000..a04a07aa
--- /dev/null
+++ b/internal/session/trainerrebind_test.go
@@ -0,0 +1,137 @@
+//go:build integration
+
+package session_test
+
+// RebindBattleTrainer is the write that makes the leaderboard describe who
+// actually played. A live_pvp battle is created before its players arrive, so
+// both trainer FKs are bound to names the creator invented ("Opponent"); the
+// slot's real occupant only shows up later over the join WebSocket, and it is
+// that occupant whose rating the result should move.
+//
+// The SQL is the whole mechanism, so it is verified against a real Postgres
+// rather than mocked: the two failure modes that matter — writing the wrong
+// slot's columns, and rewriting a battle whose rating is already settled — are
+// both invisible to a unit test of the surrounding Go.
+
+import (
+	"context"
+	"testing"
+	"time"
+
+	"pokearena/internal/store"
+
+	"github.com/google/uuid"
+)
+
+// dialStore connects to Postgres alone. The shared dialInfra also dials Redis
+// and RabbitMQ, which this test never touches — depending on them would make a
+// pure SQL assertion fail for reasons that have nothing to do with the SQL.
+func dialStore(t *testing.T) *store.Store {
+	t.Helper()
+	ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
+	defer cancel()
+	st, err := store.New(ctx, env("DATABASE_URL", "postgres://pokearena:pokearena@localhost:5432/pokearena?sslmode=disable"))
+	if err != nil {
+		t.Fatalf("no Postgres: %v", err)
+	}
+	if err := st.Migrate(ctx); err != nil {
+		t.Fatalf("migrate: %v", err)
+	}
+	return st
+}
+
+func TestRebindBattleTrainer(t *testing.T) {
+	st := dialStore(t)
+	defer st.Close()
+
+	ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
+	defer cancel()
+
+	// Distinct per-run names so repeated runs against a persistent database
+	// don't collide on the trainers table's unique name constraint.
+	run := uuid.NewString()[:8]
+	creatorP1, creatorP2 := "Red-"+run, "Opponent-"+run
+	joiner := "claude-haiku-" + run
+
+	newBattle := func(t *testing.T) (string, string, string) {
+		t.Helper()
+		t1, err := st.UpsertTrainer(ctx, creatorP1)
+		if err != nil {
+			t.Fatalf("upsert p1: %v", err)
+		}
+		t2, err := st.UpsertTrainer(ctx, creatorP2)
+		if err != nil {
+			t.Fatalf("upsert p2: %v", err)
+		}
+		id := uuid.NewString()
+		err = st.CreateBattle(ctx, store.Battle{
+			ID: id, Mode: "live_pvp", Status: "open", Seed: 1,
+			P1Trainer: t1, P2Trainer: t2, P1Name: creatorP1, P2Name: creatorP2,
+			Winner: -1,
+		})
+		if err != nil {
+			t.Fatalf("create battle: %v", err)
+		}
+		return id, t1, t2
+	}
+
+	t.Run("rebinds only the named slot", func(t *testing.T) {
+		id, t1, _ := newBattle(t)
+
+		joinerID, err := st.UpsertTrainer(ctx, joiner)
+		if err != nil {
+			t.Fatalf("upsert joiner: %v", err)
+		}
+		if err := st.RebindBattleTrainer(ctx, id, 1, joinerID, joiner); err != nil {
+			t.Fatalf("rebind: %v", err)
+		}
+
+		b, err := st.GetBattle(ctx, id)
+		if err != nil {
+			t.Fatalf("get battle: %v", err)
+		}
+		if b.P2Trainer != joinerID {
+			t.Errorf("p2_trainer = %q, want the joiner %q", b.P2Trainer, joinerID)
+		}
+		if b.P2Name != joiner {
+			t.Errorf("p2_name = %q, want %q", b.P2Name, joiner)
+		}
+		// The opposite slot is the one a column-mapping slip would corrupt,
+		// silently reassigning the *other* player's result.
+		if b.P1Trainer != t1 {
+			t.Errorf("p1_trainer = %q, want it untouched at %q", b.P1Trainer, t1)
+		}
+		if b.P1Name != creatorP1 {
+			t.Errorf("p1_name = %q, want it untouched at %q", b.P1Name, creatorP1)
+		}
+	})
+
+	t.Run("refuses to rebind a finished battle", func(t *testing.T) {
+		id, _, t2 := newBattle(t)
+		if err := st.CompleteBattle(ctx, id, 0, 12); err != nil {
+			t.Fatalf("complete battle: %v", err)
+		}
+
+		joinerID, err := st.UpsertTrainer(ctx, joiner)
+		if err != nil {
+			t.Fatalf("upsert joiner: %v", err)
+		}
+		// A late or replayed join must not move a rating that was already
+		// computed against the old trainer. The call is a silent no-op, not an
+		// error — the join itself is still legitimate, it just cannot rename.
+		if err := st.RebindBattleTrainer(ctx, id, 1, joinerID, joiner); err != nil {
+			t.Fatalf("rebind after completion returned an error: %v", err)
+		}
+
+		b, err := st.GetBattle(ctx, id)
+		if err != nil {
+			t.Fatalf("get battle: %v", err)
+		}
+		if b.P2Trainer != t2 {
+			t.Errorf("p2_trainer = %q, want the settled %q", b.P2Trainer, t2)
+		}
+		if b.P2Name != creatorP2 {
+			t.Errorf("p2_name = %q, want the settled %q", b.P2Name, creatorP2)
+		}
+	})
+}
diff --git a/internal/store/repo.go b/internal/store/repo.go
index d323e240..dc406812 100644
--- a/internal/store/repo.go
+++ b/internal/store/repo.go
@@ -101,6 +101,29 @@ func (s *Store) CreateBattle(ctx context.Context, b Battle) error {
 	return err
 }
 
+// RebindBattleTrainer points one slot of a battle at a different trainer.
+//
+// A live_pvp battle is created before its players arrive: the creator names
+// both slots ("Opponent", "Agent 2") and both trainer FKs are bound then. The
+// slot's actual occupant only shows up later, over the join WebSocket, and it
+// is that occupant — not the creator's placeholder — whose rating the result
+// should move. This is the write that makes the leaderboard describe who
+// actually played.
+//
+// slot is 0 (p1) or 1 (p2). The update is confined to a battle that has not
+// finished: rebinding after a result is applied would move a rating that was
+// already computed against the old trainer, and a late/replayed join must not
+// be able to rewrite settled history.
+func (s *Store) RebindBattleTrainer(ctx context.Context, battleID string, slot int, trainerID, name string) error {
+	col := "p1_trainer=$2, p1_name=$3"
+	if slot == 1 {
+		col = "p2_trainer=$2, p2_name=$3"
+	}
+	_, err := s.pool.Exec(ctx,
+		`UPDATE battles SET `+col+` WHERE id=$1 AND winner < 0`, battleID, trainerID, name)
+	return err
+}
+
 // GetBattle fetches one battle. Returns pgx.ErrNoRows if absent.
 func (s *Store) GetBattle(ctx context.Context, id string) (Battle, error) {
 	return scanBattle(s.pool.QueryRow(ctx, `SELECT `+battleColumns+` FROM battles WHERE id=$1`, id))
diff --git a/web/app.js b/web/app.js
index c9a8ad54..0a97fa40 100644
--- a/web/app.js
+++ b/web/app.js
@@ -959,6 +959,36 @@ function renderPokedex() {
   }).join('');
 }
 
+// ---- trainer identity ----
+// The leaderboard is keyed on the name a slot is recorded under, and that name
+// is now declared by whoever occupies the slot rather than by whoever created
+// the battle. TRAINER_KEY remembers it across visits so a returning player
+// keeps their row instead of starting a new one every session.
+const TRAINER_KEY = 'pokearena.trainer';
+const TRAINER_NAME_MAX = 24; // mirrors protocol.MaxTrainerNameLen
+
+// trainerName resolves this browser's name: what the setup form currently
+// says, else what we remembered from last time, else the default. The form
+// wins so that editing the field and hitting Start does what it looks like.
+function trainerName() {
+  const field = document.getElementById('player-name');
+  const typed = field ? field.value.trim() : '';
+  const name = typed || localStorage.getItem(TRAINER_KEY) || 'Challenger';
+  return name.slice(0, TRAINER_NAME_MAX);
+}
+
+function rememberTrainerName(name) {
+  try { localStorage.setItem(TRAINER_KEY, name); } catch (_) { /* private mode */ }
+}
+
+// withTrainerName appends the leaderboard name to a gateway play URL. The
+// gateway sanitizes it again on arrival — this is a convenience, not a
+// trust boundary.
+function withTrainerName(wsUrl, name) {
+  if (!name) return wsUrl;
+  return wsUrl + (wsUrl.includes('?') ? '&' : '?') + 'name=' + encodeURIComponent(name);
+}
+
 // ---- leaderboard ----
 async function loadLeaderboard() {
   const tbody = document.querySelector('#lb-table tbody');
@@ -984,7 +1014,8 @@ async function startBattle() {
     if (!App.opp.team.length) { toast('Pick at least one Pokémon for the opponent'); return; }
   }
 
-  const name = document.getElementById('player-name').value.trim() || 'Challenger';
+  const name = trainerName();
+  rememberTrainerName(name); // so a share-link join later reuses it
   // agent_vs_agent is a UI framing on top of live_pvp — backend has no separate
   // mode because the protocol is identical (two external joiners, no AI). We
   // just present the URLs differently and drop the user into spectate.
@@ -1186,9 +1217,9 @@ function enterPicker(res, mode, name) {
 
   if (mode === 'live_pvp') {
     showPickerShareBanner(res.battle_id, res.p2_url);
-    connectPvPWS(res.p1_url);
+    connectPvPWS(withTrainerName(res.p1_url, name));
   } else {
-    connectWS(res.ws_url);
+    connectWS(withTrainerName(res.ws_url, name));
   }
 }
 
@@ -2177,18 +2208,24 @@ function autoJoinPvP(battleId, slot, token) {
   App.pick = newBuilderState();
   App.pickerSubmitted = false;
   if (App.pickerDeadlineTimer) { clearInterval(App.pickerDeadlineTimer); App.pickerDeadlineTimer = null; }
+  // A share-link joiner never passed through the setup form, so the battle's
+  // creator named this slot ("Opponent") and the result would post to the
+  // board under that. Declare our own name on the join instead.
+  const name = trainerName();
   App.battle = {
-    id: battleId, mode: 'live_pvp', name: 'Trainer', view: 'picker',
+    id: battleId, mode: 'live_pvp', name, view: 'picker',
     queue: [], playing: false, ended: false, state: null, ws: null, es: null, slot,
   };
   showView('picker');
-  document.getElementById('picker-label').textContent = `Pv-Player · joining slot ${slot} · drafting team`;
+  document.getElementById('picker-label').textContent =
+    `Pv-Player · ${name} · joining slot ${slot} · drafting team`;
   document.querySelector('.picker-main').classList.remove('picker-locked');
   document.getElementById('picker-share-banner').classList.add('hidden');
   document.getElementById('picker-opp').innerHTML =
     '<h4>Opponent</h4><div class="muted">Connecting…</div>';
   renderPicker();
-  const wsUrl = `/api/battles/${battleId}/play?slot=${slot}&token=${encodeURIComponent(token)}`;
+  const wsUrl = withTrainerName(
+    `/api/battles/${battleId}/play?slot=${slot}&token=${encodeURIComponent(token)}`, name);
   connectPvPWS(wsUrl);
 }
 

From eac3bd67da31b88cd3a2f97029b1224be932e361 Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Tue, 14 Jul 2026 21:37:28 -0700
Subject: [PATCH 02/15] =?UTF-8?q?eval:=20decision-quality=20scoring=20?=
 =?UTF-8?q?=E2=80=94=20recover=20actions,=20measure=20regret=20vs=20oracle?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Score how well a side chose, not just whether it won. A live battle stored
every turn's engine state, and the engine is a pure function of (state,
actions), so each turn re-simulates to recover the exact action played
(engine.LegalActions x ResolveTurn until the stored next state reproduces
byte-for-byte). Then a depth-limited expectimax oracle scores every legal
action from the identical fog-of-war view (ai.MakeView), and the value gap
between the played action and the best is the decision's regret.

- ai.ExpectimaxAgent.ScoreActions exposes the per-action maximin values
  searchRoot already computes (additive; Decide unchanged), so callers can
  measure regret rather than only agree/disagree.
- eval.ScoreDecisions walks stored turns, recovers each free choice on a side,
  and returns per-decision {chosen, best, agree, regret, blunder} plus a count
  of skipped faint/replacement turns (v1 scores clean turns; coverage reported,
  not hidden).
- cmd/decision-eval is a spike over a battle JSON export (db-replay's shape).

Validated on real stored battles: recovery re-simulates exactly, and regret
correctly separates real mistakes from picking an equal-value alternative
(binary agreement counts equal moves as misses; regret scores them 0).
---
 cmd/decision-eval/main.go             | 117 ++++++++++++++++++++
 internal/ai/expectimax.go             |  39 +++++++
 internal/eval/decisionquality.go      | 147 ++++++++++++++++++++++++++
 internal/eval/decisionquality_test.go |  80 ++++++++++++++
 4 files changed, 383 insertions(+)
 create mode 100644 cmd/decision-eval/main.go
 create mode 100644 internal/eval/decisionquality.go
 create mode 100644 internal/eval/decisionquality_test.go

diff --git a/cmd/decision-eval/main.go b/cmd/decision-eval/main.go
new file mode 100644
index 00000000..e3280364
--- /dev/null
+++ b/cmd/decision-eval/main.go
@@ -0,0 +1,117 @@
+// Command decision-eval scores how well a side chose in a stored live battle,
+// not just whether it won. It reads a JSON export of one battle —
+// {seed, winner, turns:[{state, log}]} — the same shape db-replay consumes,
+// re-simulates each turn to recover the action played, and compares it to a
+// depth-limited expectimax oracle deciding from the identical fog-of-war view.
+//
+// This is a spike: it prints per-battle coverage and oracle-agreement so the
+// approach can be validated on real data before it grows a report section.
+//
+// Usage:
+//
+//	docker exec pg psql ... -c "<json query>" | decision-eval -side 0 -depth 3
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"io"
+	"log"
+	"os"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+	"pokearena/internal/eval"
+)
+
+type export struct {
+	Seed   int64 `json:"seed"`
+	Winner int   `json:"winner"`
+	Turns  []struct {
+		State json.RawMessage `json:"state"`
+		Log   json.RawMessage `json:"log"`
+	} `json:"turns"`
+}
+
+func main() {
+	log.SetFlags(0)
+	log.SetPrefix("[decision-eval] ")
+
+	in := flag.String("in", "-", "battle JSON export path (\"-\" for stdin)")
+	dataDir := flag.String("data", "data", "dex data directory")
+	side := flag.Int("side", 0, "which side to score (0 = the model seat)")
+	depth := flag.Int("depth", 3, "oracle expectimax search depth")
+	quiet := flag.Bool("quiet", false, "print only the summary line")
+	flag.Parse()
+
+	raw, err := readAll(*in)
+	if err != nil {
+		log.Fatalf("read export: %v", err)
+	}
+	var ex export
+	if err := json.Unmarshal(raw, &ex); err != nil {
+		log.Fatalf("parse export: %v", err)
+	}
+
+	dex, err := domain.LoadDex(*dataDir, "bench")
+	if err != nil {
+		log.Fatalf("load dex: %v", err)
+	}
+	oracle := ai.NewExpectimaxAgentFixed(dex, *depth)
+
+	turns := make([]eval.StoredTurn, len(ex.Turns))
+	for i, t := range ex.Turns {
+		turns[i] = eval.StoredTurn{State: t.State, Log: t.Log}
+	}
+
+	scores, skipped, err := eval.ScoreDecisions(dex, oracle, *side, turns)
+	if err != nil {
+		log.Fatalf("score: %v", err)
+	}
+
+	agree, blunders := 0, 0
+	var sumRegret, maxRegret float64
+	for _, s := range scores {
+		if s.Agree {
+			agree++
+		}
+		if s.Blunder {
+			blunders++
+		}
+		sumRegret += s.Regret
+		if s.Regret > maxRegret {
+			maxRegret = s.Regret
+		}
+	}
+	agreePct, blunderPct, meanRegret := 0.0, 0.0, 0.0
+	if n := float64(len(scores)); n > 0 {
+		agreePct = 100 * float64(agree) / n
+		blunderPct = 100 * float64(blunders) / n
+		meanRegret = sumRegret / n
+	}
+	fmt.Printf("seed=%d winner=%d turns=%d recovered=%d skipped=%d agree=%.0f%% blunder=%.0f%% meanRegret=%.0f maxRegret=%.0f depth=%d\n",
+		ex.Seed, ex.Winner, len(ex.Turns), len(scores), skipped, agreePct, blunderPct, meanRegret, maxRegret, *depth)
+	if *quiet {
+		return
+	}
+	for _, s := range scores {
+		mark := "  ok "
+		if s.Blunder {
+			mark = "BLUND"
+		} else if !s.Agree {
+			mark = "diff "
+		}
+		fmt.Printf("turn %2d  %s  chose %s  oracle %s  regret %.0f\n", s.Turn, mark, actStr(s.Chosen), actStr(s.Best), s.Regret)
+	}
+}
+
+func actStr(a engine.Action) string { return fmt.Sprintf("%s#%d", a.Kind, a.Index) }
+
+func readAll(path string) ([]byte, error) {
+	if path == "-" {
+		return io.ReadAll(os.Stdin)
+	}
+	return os.ReadFile(path)
+}
diff --git a/internal/ai/expectimax.go b/internal/ai/expectimax.go
index 4d944697..6df3b482 100644
--- a/internal/ai/expectimax.go
+++ b/internal/ai/expectimax.go
@@ -147,6 +147,45 @@ func (a *ExpectimaxAgent) searchRoot(v View, depth int, deadline time.Time, hasD
 	return best, true
 }
 
+// ActionValue is a root action paired with its maximin search value, in the
+// eval points value() uses (materialValue = one whole Pokémon).
+type ActionValue struct {
+	Action engine.Action
+	Value  float64
+}
+
+// ScoreActions returns the maximin value of every legal action for the deciding
+// side at the agent's max depth — the same per-action scores searchRoot ranks to
+// pick its move, exposed so a caller can measure the value gap (regret) between a
+// policy's actual choice and the best one. Deadline-free and reproducible, like
+// fixed-depth Decide, and it breaks ties toward the first legal action exactly as
+// Decide does, so the top-valued action here equals Decide's choice. Returns nil
+// where regret is undefined: a forced replacement (a one-ply decision expectimax
+// defers to the heuristic) or a turn with a single legal action.
+func (a *ExpectimaxAgent) ScoreActions(v View) []ActionValue {
+	if v.Replace {
+		return nil
+	}
+	myActs := LegalActions(v)
+	if len(myActs) <= 1 {
+		return nil
+	}
+	sim := a.reconstruct(v)
+	sc := searchCtx{me: v.Me, foeBench: v.FoeBenchAlive}
+	foeActs := a.foeActions(sim, v.Me)
+	out := make([]ActionValue, 0, len(myActs))
+	for _, my := range myActs {
+		worst := math.Inf(1)
+		for _, fo := range foeActs {
+			if val := a.evalPair(sc, sim, my, fo, a.maxDepth); val < worst {
+				worst = val
+			}
+		}
+		out = append(out, ActionValue{Action: my, Value: worst})
+	}
+	return out
+}
+
 // evalPair simulates one (my, foe) action pair K times over the chance space
 // and averages the resulting position values.
 func (a *ExpectimaxAgent) evalPair(sc searchCtx, sim *engine.BattleState, my, fo engine.Action, depth int) float64 {
diff --git a/internal/eval/decisionquality.go b/internal/eval/decisionquality.go
new file mode 100644
index 00000000..b7d24fa8
--- /dev/null
+++ b/internal/eval/decisionquality.go
@@ -0,0 +1,147 @@
+package eval
+
+import (
+	"bytes"
+	"encoding/json"
+	"math"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+)
+
+// BlunderThreshold is the regret, in the oracle's eval points, above which a
+// choice counts as a blunder. The oracle scores one whole Pokémon at ~1000
+// points, so 300 ≈ giving up a third of a Pokémon's worth of position in a
+// single move — enough to separate a genuine mistake from picking the
+// second-best of two near-equal options. Tunable as the metric is calibrated.
+const BlunderThreshold = 300.0
+
+// Decision-quality eval scores how well a policy chose, not just whether it won.
+// A live battle stored every turn's engine state, and the engine is a pure
+// function of (state, actions) — so we can re-simulate each turn to recover the
+// exact action a side played, then ask a stronger reference agent (the oracle)
+// what it would have played from the *identical* fog-of-war view. The gap
+// between the two is a per-decision quality signal that a win/loss record hides.
+//
+// The oracle sees exactly what the policy saw: ai.MakeView projects the same
+// fog every agent decides from, so a deeper-searching expectimax is a fair
+// yardstick rather than an omniscient one.
+
+// Oracle is the reference policy a decision is scored against. It must expose
+// per-action values (not just its top pick) so we can measure regret — how much
+// value a choice gave up, not merely whether it matched. *ai.ExpectimaxAgent
+// satisfies it; the interface keeps this package testable with a stub.
+type Oracle interface {
+	ScoreActions(v ai.View) []ai.ActionValue
+}
+
+// DecisionScore is one recovered free choice: the action a side actually played
+// on a turn, the oracle's best from the identical view, and the value gap
+// between them.
+type DecisionScore struct {
+	Turn    int           `json:"turn"`
+	Side    int           `json:"side"`
+	Chosen  engine.Action `json:"chosen"`
+	Best    engine.Action `json:"best"`
+	Agree   bool          `json:"agree"`   // Chosen == the oracle's top choice
+	Regret  float64       `json:"regret"`  // oracle value of Best minus value of Chosen (>= 0)
+	Blunder bool          `json:"blunder"` // Regret > BlunderThreshold
+}
+
+// recoverActions finds the [2]Action that, applied to prev, reproduces want
+// byte-for-byte (both marshaled through engine.BattleState, so jsonb key-order
+// differences wash out). It returns ok=false when no single action-pair
+// reproduces want — most often a turn where a Pokémon fainted and a replacement
+// was chosen mid-turn, which v1 does not yet score. Because the engine is
+// deterministic from the stored RNGState, the matching pair is exactly what was
+// played, and the match doubles as a validity check on the recorded state.
+func recoverActions(dex *domain.Dex, prev json.RawMessage, want *engine.BattleState) ([2]engine.Action, bool) {
+	wantJSON, err := json.Marshal(want)
+	if err != nil {
+		return [2]engine.Action{}, false
+	}
+	var base engine.BattleState
+	if err := json.Unmarshal(prev, &base); err != nil {
+		return [2]engine.Action{}, false
+	}
+	a0s := engine.LegalActionsDex(dex, &base, 0)
+	a1s := engine.LegalActionsDex(dex, &base, 1)
+	for _, a0 := range a0s {
+		for _, a1 := range a1s {
+			var trial engine.BattleState
+			if err := json.Unmarshal(prev, &trial); err != nil {
+				continue
+			}
+			engine.ResolveTurn(dex, &trial, [2]engine.Action{a0, a1})
+			if got, err := json.Marshal(&trial); err == nil && bytes.Equal(got, wantJSON) {
+				return [2]engine.Action{a0, a1}, true
+			}
+		}
+	}
+	return [2]engine.Action{}, false
+}
+
+// ScoreDecisions replays a live battle's stored turns and, for every clean free
+// choice on modelSide it can recover, compares the model's action to the
+// oracle's from the identical fog-of-war view. It returns one score per
+// recovered decision plus the count of choosing-turns it could not recover
+// (the faint/replacement turns v1 skips), so callers can report coverage rather
+// than silently dropping them.
+func ScoreDecisions(dex *domain.Dex, orc Oracle, modelSide int, turns []StoredTurn) (scores []DecisionScore, skipped int, err error) {
+	for i := 1; i < len(turns); i++ {
+		var prevState engine.BattleState
+		if err := json.Unmarshal(turns[i-1].State, &prevState); err != nil {
+			return nil, skipped, err
+		}
+		// Only a turn chosen from a choosing state is a free decision; the last
+		// stored state (phase "ended") is a terminal snapshot, not a choice.
+		if prevState.Phase != engine.PhaseChoosing {
+			continue
+		}
+		var next engine.BattleState
+		if err := json.Unmarshal(turns[i].State, &next); err != nil {
+			return nil, skipped, err
+		}
+		acts, ok := recoverActions(dex, turns[i-1].State, &next)
+		if !ok {
+			skipped++
+			continue
+		}
+		v := ai.MakeView(&prevState, modelSide)
+		vals := orc.ScoreActions(v)
+		if len(vals) == 0 {
+			// Forced or single-option turn — no free choice to score.
+			skipped++
+			continue
+		}
+		chosen := acts[modelSide]
+		best := vals[0]
+		chosenVal, found := math.Inf(-1), false
+		for _, av := range vals {
+			if av.Value > best.Value {
+				best = av
+			}
+			if av.Action == chosen {
+				chosenVal, found = av.Value, true
+			}
+		}
+		if !found {
+			// The recovered action isn't in the oracle's legal set — a fog or
+			// version mismatch; skip rather than report a bogus regret.
+			skipped++
+			continue
+		}
+		regret := best.Value - chosenVal
+		scores = append(scores, DecisionScore{
+			Turn:    next.Turn,
+			Side:    modelSide,
+			Chosen:  chosen,
+			Best:    best.Action,
+			Agree:   chosen == best.Action,
+			Regret:  regret,
+			Blunder: regret > BlunderThreshold,
+		})
+	}
+	return scores, skipped, nil
+}
diff --git a/internal/eval/decisionquality_test.go b/internal/eval/decisionquality_test.go
new file mode 100644
index 00000000..8f263a8c
--- /dev/null
+++ b/internal/eval/decisionquality_test.go
@@ -0,0 +1,80 @@
+package eval
+
+import (
+	"encoding/json"
+	"testing"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/engine"
+)
+
+// stubOracle scores the first legal action worst (0) and the last one best
+// (1000), so a policy that always plays move#0 disagrees with a known,
+// fixed regret of 1000 — letting the test assert regret/blunder arithmetic
+// without depending on real expectimax values.
+type stubOracle struct{}
+
+func (stubOracle) ScoreActions(v ai.View) []ai.ActionValue {
+	acts := ai.LegalActions(v)
+	if len(acts) <= 1 {
+		return nil // no free choice — mirror expectimax's contract
+	}
+	out := make([]ai.ActionValue, len(acts))
+	for i, a := range acts {
+		out[i] = ai.ActionValue{Action: a}
+	}
+	out[len(out)-1].Value = 1000
+	return out
+}
+
+// TestScoreDecisions_RecoversActionsAndRegret plays a fixed line (both sides
+// spam move#0), stores each turn's state the way the live coordinator does, and
+// checks that ScoreDecisions re-simulates back to the exact action played and
+// computes regret against the oracle. A Snorlax mirror keeps anyone from
+// fainting in the first few turns, so every turn is a clean, recoverable choice.
+func TestScoreDecisions_RecoversActionsAndRegret(t *testing.T) {
+	d := loadDex(t)
+	s, err := engine.NewBattle(d, "b", "P0", []int{143, 131}, "P1", []int{143, 131}, 42)
+	if err != nil {
+		t.Fatalf("new battle: %v", err)
+	}
+
+	move0 := [2]engine.Action{{Kind: engine.ActionMove, Index: 0}, {Kind: engine.ActionMove, Index: 0}}
+	var turns []StoredTurn
+	for i := 0; i < 4 && !s.Ended(); i++ {
+		if s.Phase != engine.PhaseChoosing {
+			break
+		}
+		engine.ResolveTurn(d, s, move0)
+		b, err := json.Marshal(s)
+		if err != nil {
+			t.Fatalf("marshal state: %v", err)
+		}
+		turns = append(turns, StoredTurn{State: append(json.RawMessage(nil), b...)})
+	}
+	if len(turns) < 2 {
+		t.Fatalf("need >= 2 stored turns to score a transition, got %d", len(turns))
+	}
+
+	scores, skipped, err := ScoreDecisions(d, stubOracle{}, 0, turns)
+	if err != nil {
+		t.Fatalf("score: %v", err)
+	}
+	if len(scores) == 0 {
+		t.Fatalf("no decisions scored (skipped=%d)", skipped)
+	}
+	for _, sc := range scores {
+		if sc.Chosen.Kind != engine.ActionMove || sc.Chosen.Index != 0 {
+			t.Errorf("turn %d: recovered %+v, want move#0", sc.Turn, sc.Chosen)
+		}
+		if sc.Regret != 1000 {
+			t.Errorf("turn %d: regret %.0f, want 1000", sc.Turn, sc.Regret)
+		}
+		if !sc.Blunder {
+			t.Errorf("turn %d: expected a blunder at regret 1000 (threshold %.0f)", sc.Turn, BlunderThreshold)
+		}
+		if sc.Agree {
+			t.Errorf("turn %d: move#0 is the oracle's worst here, should not agree", sc.Turn)
+		}
+	}
+}

From e6f244bccf9006f0e78651229d4e2902b509046d Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Wed, 15 Jul 2026 20:34:29 -0700
Subject: [PATCH 03/15] =?UTF-8?q?eval:=20recover=20faint=20turns=20too=20?=
 =?UTF-8?q?=E2=80=94=20decision=20coverage=20to=20~full?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The naive recovery replayed one ResolveTurn and compared to the stored next
state, so any turn that fainted a Pokémon (resolution stops in the replacement
phase) couldn't be matched and was skipped — about half of all turns. settle
now drives the resolved state through its forced replacements, searching the
legal replacement picks (a cascade recurses), so faint turns reproduce the
stored next state and are recovered like any other. On real battles this takes
coverage from ~40-65% to ~100% of choosing turns.

Test plays a full heuristic mirror that KOs and replaces, then asserts every
choosing turn — faint turns included — re-derives its actions exactly.
---
 internal/eval/decisionquality.go      | 61 +++++++++++++++----
 internal/eval/decisionquality_test.go | 86 +++++++++++++++++++++++++++
 2 files changed, 136 insertions(+), 11 deletions(-)

diff --git a/internal/eval/decisionquality.go b/internal/eval/decisionquality.go
index b7d24fa8..b64ee505 100644
--- a/internal/eval/decisionquality.go
+++ b/internal/eval/decisionquality.go
@@ -51,11 +51,12 @@ type DecisionScore struct {
 
 // recoverActions finds the [2]Action that, applied to prev, reproduces want
 // byte-for-byte (both marshaled through engine.BattleState, so jsonb key-order
-// differences wash out). It returns ok=false when no single action-pair
-// reproduces want — most often a turn where a Pokémon fainted and a replacement
-// was chosen mid-turn, which v1 does not yet score. Because the engine is
-// deterministic from the stored RNGState, the matching pair is exactly what was
-// played, and the match doubles as a validity check on the recorded state.
+// differences wash out). When the turn faints a Pokémon, resolving lands in the
+// replacement phase; settle then searches the replacement picks that carry the
+// state the rest of the way to want, so faint turns are recovered too. Because
+// the engine is deterministic from the stored RNGState, the matching pair is
+// exactly what was played, and the match doubles as a validity check on the
+// recorded state. ok=false means no legal line reproduces want.
 func recoverActions(dex *domain.Dex, prev json.RawMessage, want *engine.BattleState) ([2]engine.Action, bool) {
 	wantJSON, err := json.Marshal(want)
 	if err != nil {
@@ -69,12 +70,9 @@ func recoverActions(dex *domain.Dex, prev json.RawMessage, want *engine.BattleSt
 	a1s := engine.LegalActionsDex(dex, &base, 1)
 	for _, a0 := range a0s {
 		for _, a1 := range a1s {
-			var trial engine.BattleState
-			if err := json.Unmarshal(prev, &trial); err != nil {
-				continue
-			}
-			engine.ResolveTurn(dex, &trial, [2]engine.Action{a0, a1})
-			if got, err := json.Marshal(&trial); err == nil && bytes.Equal(got, wantJSON) {
+			trial := base.Clone()
+			engine.ResolveTurn(dex, trial, [2]engine.Action{a0, a1})
+			if settle(dex, trial, wantJSON) {
 				return [2]engine.Action{a0, a1}, true
 			}
 		}
@@ -82,6 +80,47 @@ func recoverActions(dex *domain.Dex, prev json.RawMessage, want *engine.BattleSt
 	return [2]engine.Action{}, false
 }
 
+// settle drives st through any forced replacements until it leaves the
+// replacement phase, then reports whether it marshals equal to wantJSON. A
+// replacement is itself a choice, so it searches the legal picks for each side
+// that must replace (a cascade — an incoming mon that faints on entry — recurses
+// the same way). The branching is tiny: usually a single side replacing with one
+// of at most five benched Pokémon.
+func settle(dex *domain.Dex, st *engine.BattleState, wantJSON []byte) bool {
+	if st.Phase != engine.PhaseReplace {
+		got, err := json.Marshal(st)
+		return err == nil && bytes.Equal(got, wantJSON)
+	}
+	opts := [2][]engine.Action{{{}}, {{}}} // a non-replacing side stays put (nil pick)
+	for side := 0; side < 2; side++ {
+		if st.Replace[side] {
+			opts[side] = engine.LegalActionsDex(dex, st, side)
+			if len(opts[side]) == 0 {
+				return false
+			}
+		}
+	}
+	for _, r0 := range opts[0] {
+		for _, r1 := range opts[1] {
+			trial := st.Clone()
+			var sw [2]*engine.Action
+			if st.Replace[0] {
+				a := r0
+				sw[0] = &a
+			}
+			if st.Replace[1] {
+				a := r1
+				sw[1] = &a
+			}
+			engine.ResolveReplace(trial, sw)
+			if settle(dex, trial, wantJSON) {
+				return true
+			}
+		}
+	}
+	return false
+}
+
 // ScoreDecisions replays a live battle's stored turns and, for every clean free
 // choice on modelSide it can recover, compares the model's action to the
 // oracle's from the identical fog-of-war view. It returns one score per
diff --git a/internal/eval/decisionquality_test.go b/internal/eval/decisionquality_test.go
index 8f263a8c..dc2c87e0 100644
--- a/internal/eval/decisionquality_test.go
+++ b/internal/eval/decisionquality_test.go
@@ -1,6 +1,7 @@
 package eval
 
 import (
+	"context"
 	"encoding/json"
 	"testing"
 
@@ -8,6 +9,18 @@ import (
 	"pokearena/internal/engine"
 )
 
+func faintedCount(s *engine.BattleState) int {
+	n := 0
+	for _, side := range s.Sides {
+		for _, p := range side.Team {
+			if p.Fainted {
+				n++
+			}
+		}
+	}
+	return n
+}
+
 // stubOracle scores the first legal action worst (0) and the last one best
 // (1000), so a policy that always plays move#0 disagrees with a known,
 // fixed regret of 1000 — letting the test assert regret/blunder arithmetic
@@ -78,3 +91,76 @@ func TestScoreDecisions_RecoversActionsAndRegret(t *testing.T) {
 		}
 	}
 }
+
+// TestRecoverActions_FaintTurns plays a full heuristic-vs-heuristic game (which
+// KOs Pokémon and forces mid-turn replacements), stores each settled turn the
+// way the live coordinator does, and checks that recoverActions re-derives the
+// exact actions on every turn — including the faint turns, where resolving must
+// also settle the replacement pick. This is the coverage the naive single-
+// ResolveTurn recovery misses.
+func TestRecoverActions_FaintTurns(t *testing.T) {
+	d := loadDex(t)
+	// Frail, hard-hitting mirror so KOs (and replacements) happen quickly.
+	s, err := engine.NewBattle(d, "b", "P0", []int{65, 94, 101}, "P1", []int{65, 94, 101}, 7)
+	if err != nil {
+		t.Fatalf("new battle: %v", err)
+	}
+	h := ai.NewHeuristicAgent(d)
+	ctx := context.Background()
+
+	var turns []StoredTurn
+	sawFaint := false
+	for guard := 0; !s.Ended(); guard++ {
+		if guard > 500 {
+			t.Fatal("battle did not terminate")
+		}
+		a0, _ := h.Decide(ctx, ai.MakeView(s, 0))
+		a1, _ := h.Decide(ctx, ai.MakeView(s, 1))
+		engine.ResolveTurn(d, s, [2]engine.Action{a0, a1})
+		for s.Phase == engine.PhaseReplace {
+			sawFaint = true
+			var sw [2]*engine.Action
+			for side := 0; side < 2; side++ {
+				if s.Replace[side] {
+					ra, _ := h.Decide(ctx, ai.MakeView(s, side))
+					sw[side] = &ra
+				}
+			}
+			engine.ResolveReplace(s, sw)
+		}
+		b, err := json.Marshal(s)
+		if err != nil {
+			t.Fatalf("marshal state: %v", err)
+		}
+		turns = append(turns, StoredTurn{State: append(json.RawMessage(nil), b...)})
+	}
+	if !sawFaint {
+		t.Fatal("matchup produced no faint — test would not exercise replacement recovery")
+	}
+
+	faintTurns := 0
+	for i := 1; i < len(turns); i++ {
+		var prev, next engine.BattleState
+		if err := json.Unmarshal(turns[i-1].State, &prev); err != nil {
+			t.Fatal(err)
+		}
+		if err := json.Unmarshal(turns[i].State, &next); err != nil {
+			t.Fatal(err)
+		}
+		if prev.Phase != engine.PhaseChoosing {
+			continue
+		}
+		if _, ok := recoverActions(d, turns[i-1].State, &next); !ok {
+			t.Errorf("turn %d: failed to recover actions (fainted prev=%d next=%d)",
+				next.Turn, faintedCount(&prev), faintedCount(&next))
+			continue
+		}
+		if faintedCount(&next) > faintedCount(&prev) {
+			faintTurns++
+		}
+	}
+	if faintTurns == 0 {
+		t.Fatal("no recovered turn involved a faint — coverage claim unverified")
+	}
+	t.Logf("recovered every choosing turn across %d stored turns, %d with a faint", len(turns), faintTurns)
+}

From 374fd15d4c9338785f2d003ee34f050612cc7b86 Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Fri, 17 Jul 2026 12:03:43 -0700
Subject: [PATCH 04/15] docs: handoff for decision-quality eval (state,
 remaining work, gotchas)

---
 docs/decision-quality-eval-handoff.md | 185 ++++++++++++++++++++++++++
 1 file changed, 185 insertions(+)
 create mode 100644 docs/decision-quality-eval-handoff.md

diff --git a/docs/decision-quality-eval-handoff.md b/docs/decision-quality-eval-handoff.md
new file mode 100644
index 00000000..cc226812
--- /dev/null
+++ b/docs/decision-quality-eval-handoff.md
@@ -0,0 +1,185 @@
+# Handoff — Decision-Quality Eval (per-model reasoning scoring)
+
+Status as of 2026-07-17. Branch: `feat/decision-quality-eval` (based on the
+now-merged `feat/replay-samples`; PR #108 already merged to `main`).
+
+## Goal
+
+Score how well each model *chose*, not just whether it won. For every free
+decision in a recorded live battle, compare the action the model played to what
+a stronger expectimax "oracle" would have played **from the identical
+fog-of-war view**, and measure the **regret** (value the choice gave up). Roll
+up into a per-model table: blunder rate, median regret, match rate — the
+reasoning-quality view that win/loss hides. Zero new API spend was the original
+constraint; we ended up re-running a fresh attributed batch (see below) because
+the old attribution was wiped.
+
+## What is DONE (committed on this branch)
+
+Two commits, both green (build + lint + tests). **They may be unpushed — run
+`git push -u origin feat/decision-quality-eval` first thing.**
+
+- `4bccd5e eval: decision-quality scoring — recover actions, measure regret vs oracle`
+- `3dde37b eval: recover faint turns too — decision coverage to ~full`
+
+Files:
+- **`internal/eval/decisionquality.go`** — the core.
+  - `recoverActions(dex, prevRaw, want)` — re-simulates a turn (enumerate
+    `engine.LegalActionsDex` for both sides × `engine.ResolveTurn`) and returns
+    the action pair that reproduces the stored next state **byte-for-byte** (both
+    marshaled through `engine.BattleState`, so jsonb key-order washes out). The
+    engine is deterministic from the stored `RNGState`, so the match is exact and
+    doubles as a data-validity check.
+  - `settle(dex, st, wantJSON)` — drives a resolved state through forced
+    replacements (searching the replacement picks, cascades recurse) so **faint
+    turns are recovered too**. This took coverage from ~40–65% to ~100%.
+  - `ScoreDecisions(dex, oracle, modelSide, turns) ([]DecisionScore, skipped, err)`
+    — walks stored turns, recovers each free choice on `modelSide`, and scores it.
+  - `DecisionScore{Turn, Side, Chosen, Best, Agree, Regret, Blunder}`.
+  - `Oracle` interface (`ScoreActions(v ai.View) []ai.ActionValue`).
+  - `BlunderThreshold = 300.0` (regret in oracle eval points; ~one Pokémon =
+    1000, so 300 ≈ giving up ~0.3 of a Pokémon). Tunable.
+- **`internal/ai/expectimax.go`** — added `ActionValue` + `ScoreActions(v View)
+  []ActionValue` (additive; exposes the per-action maximin values `searchRoot`
+  already computes; `Decide` unchanged; ties break toward the first legal action
+  exactly like `Decide`, so the top-valued action equals `Decide`'s pick).
+- **`cmd/decision-eval/main.go`** — spike CLI. Reads a battle JSON export
+  (`{seed, winner, turns:[{state, log}]}`, the same shape `db-replay` consumes)
+  on stdin/`-in`, prints per-decision + a summary line. Flags: `-side` (0 = the
+  model seat), `-depth` (oracle depth, default 3), `-quiet`.
+- **`internal/eval/decisionquality_test.go`** — deterministic tests:
+  `TestScoreDecisions_RecoversActionsAndRegret` (Snorlax mirror, stub oracle) and
+  `TestRecoverActions_FaintTurns` (heuristic mirror that KOs → exercises the
+  replacement recovery).
+
+### Validated
+
+- Recovery re-simulates exactly on real battles.
+- Regret >> binary agreement: equal-value alternatives score regret 0 even when
+  they "disagree" with the oracle (that's why we don't use match rate as the
+  headline). Missed-lethal shows up as regret ≈ 1e6 (winValue) — real but
+  off-scale.
+- End-to-end on a freshly-played battle on the redeployed engine: play → store →
+  `bid=` attribution → export → `decision-eval` scores it. Example: a Haiku game
+  that WON but missed a lethal (turn 15 regret ≈ 998k).
+
+## What is IN FLIGHT
+
+**A fresh attributed batch is running in the background** (started 2026-07-16
+20:57, my bg task id `b1q49nvj7`, driver `/tmp/run-attributed-batch.sh`, log
+`/tmp/attributed-batch.log`).
+
+- 3 teams (Genesis, Keystone, Spectrum) × 4 games = 12 games/model, order
+  Haiku → Sonnet → Gemini → Opus, conc 3.
+- **Done: Haiku, Sonnet, Gemini (36 games, all attributed).** Opus was running
+  last and **some Opus games abandon/time out** (known Opus slowness) — it may
+  finish short. Consider topping Opus up to a clean 12 with
+  `scripts/bench/run-batch.sh claude opus <team> <N> 2 cc-opus-<team>` (lower
+  conc; note `run-batch.sh` truncates `results.txt`, so use a per-game append or
+  a fresh tag if you don't want to lose the games already there).
+- Attribution lives in **`/tmp/pk-agentic-v2/<key>-<team>/results.txt`**, one
+  line per game: `g<N> winner=<0|1|-1> -> ... bid=<uuid>`. Keys: `cc-haiku`,
+  `cc-sonnet`, `cc-opus`, `agy-gemini` (match `eval.ModelDisplay`).
+  `/tmp` is not durable here (see gotchas) — if you need these to survive,
+  copy them somewhere safe or re-derive.
+
+Preliminary win rates (context only, tiny n): Haiku ~8% (1/12), Sonnet ~25%
+(3/12), Gemini ~67% (8/12), Opus TBD.
+
+## What REMAINS (the actual task)
+
+1. **Finish / top up the batch** (esp. Opus). Every game must have a
+   `status=completed` battle in Postgres with turns; unfinished (`winner=-1`,
+   open/abandoned) games can't be scored — either rerun them or exclude.
+2. **Build the aggregation** (the main missing piece). For each model:
+   - Read `bid=` lines from `/tmp/pk-agentic-v2/<key>-*/results.txt` → set of
+     `battle_id`s (skip `winner=-1`).
+   - For each battle, export from Postgres and run `ScoreDecisions(side 0)`.
+   - Aggregate: **blunder rate** (headline), **median regret** (NOT mean —
+     winsorize/clip the ~1e6 missed-lethals or they dominate), **match rate**,
+     decisions scored, and the win rate for context.
+   - Emit a per-model table. Suggested home: extend `cmd/decision-eval` with an
+     aggregate mode (accept a dir of exports or a manifest + model label, output
+     JSON stats), or a thin `scripts/bench/decision-report.sh` that loops the
+     bids and pipes to a `-json` mode. **Add a test** (repo rule: every new
+     ability ships a test in the same commit).
+3. **(Optional) Wire the table into the HTML report** (`internal/eval/report.go`
+   — new section), then republish to both sites (gh-pages of `shaumik/PokeArena`
+   and `main` of `shaumik/shaumik`) via the git-plumbing publish flow. The
+   sprite/sample work already lives in `main`.
+4. **Open the PR** for `feat/decision-quality-eval` → `main`.
+
+## Data & attribution facts
+
+- Postgres container `pk-bench-postgres-1`, DSN
+  `postgres://pokearena:pokearena@postgres:5432/pokearena`. Access:
+  `docker exec pk-bench-postgres-1 psql -U pokearena -d pokearena ...`.
+- `battle_turns(battle_id, turn_no, log jsonb, state_digest jsonb)`. `state_digest`
+  IS a marshaled `engine.BattleState` — `json.Unmarshal` straight back (see
+  `eval.ReplayFromStored`). Stored states are post-turn; phases seen are only
+  `choosing` and `ended` (replacements are folded into the turn), so the
+  pre-decision state for turn N is the stored state at turn N-1.
+- **Postgres has NO model identity** — `p1_name` is always "Agent", `p2_name`
+  "AI". Model attribution ONLY comes from the `bid=`→model mapping in the run
+  dirs. The previous mapping (`/tmp/pk-agentic`) was wiped, which is why we
+  re-ran.
+- Export shape for one battle (feeds `decision-eval -in`):
+  ```sql
+  select json_build_object('seed', b.seed, 'winner', b.winner,
+    'turns', (select json_agg(json_build_object('state', t.state_digest, 'log', t.log)
+              order by t.turn_no) from battle_turns t where t.battle_id=b.id))
+  from battles b where b.id='<uuid>'
+  ```
+
+## Environment gotchas (important)
+
+- **Worktrees get WIPED by concurrent-agent churn.** My original
+  `/private/tmp/pk-bench` got gutted (lost `.git` + source), and `/tmp/pk-agentic`
+  was emptied. **Push commits promptly; don't trust `/tmp` or a worktree to
+  persist.**
+- **Do NOT edit the origin repo root** `/Users/shaumikmondal/programming/poke-sys-design`
+  — a concurrent agent works there (branch `worktree-terminal-ux`, the TUI /
+  PR #96 sprite work). Read-only is fine.
+- Current worktrees: **`/private/tmp/pk-deval`** = this branch
+  (`feat/decision-quality-eval`); **`/private/tmp/pk-deploy`** = detached
+  `origin/main`, used to build/redeploy.
+- **Stack was rebuilt & redeployed on `origin/main`** (image `pokearena:local`
+  = `b7fd69bef36f`, from commit `36e11e2`). Postgres volume `pk-bench_pgdata`
+  preserved (361 completed historical battles + the new batch). Recreate with
+  `docker compose -f /private/tmp/pk-deploy/docker-compose.yml -p pk-bench up -d`.
+  **Never `down -v`** (nukes the data).
+- **Pre-commit hook** (`.githooks/pre-commit`, build + lint mirroring CI) is in
+  `main`. Enable in a new worktree with `make hooks`. Bypass one commit with
+  `PRECOMMIT_SKIP=1`.
+- Repo conventions (from the owner): **no Claude co-author trailers** in commits
+  or PRs; **commit often** (small, green units); **always ship a test** with each
+  new mechanic in the same commit; verify with the existing CLI before claiming
+  results.
+
+## Live-battle harness (for re-running / topping up)
+
+- Gateway `http://localhost:8080` (ws `ws://localhost:8080`). MCP client
+  `bin/pokearena-mcp` (build: `make mcp`). Both `claude` and `agy` (Gemini /
+  Antigravity) CLIs are installed and authenticated; `agy` MCP config at
+  `~/.gemini/antigravity-cli/mcp_config.json`.
+- One game: `scripts/bench/play-live.sh <claude|agy> <model> <team> <label> <outdir>`
+  (model = `haiku|sonnet|opus` or `"Gemini 3.1 Pro (High)"`). Prints the
+  authoritative winner with `bid=`.
+- A batch: `scripts/bench/run-batch.sh <harness> <model> <team> <N> <conc> <tag>`
+  → writes `$POKEARENA_OUT/<tag>/results.txt` (**truncates** it first). Keep
+  conc ≤ 3 (each live game also runs a server-side expectimax). Opus is slow and
+  abandons under load — use conc 2.
+
+## Design notes / open choices
+
+- Oracle depth: spike used **depth 3** (same as the strongest baseline
+  contestant). Offline you can afford **depth 4** for a stronger, fairer
+  reference — worth trying; it will lower everyone's blunder rate but rank them
+  more credibly.
+- `BlunderThreshold = 300` is a first guess; calibrate against the regret
+  distribution once you have all models.
+- Regret aggregation MUST be robust to winValue-scale missed-lethals (median or
+  winsorized mean, not raw mean).
+- Fog fairness is handled by construction: `ai.MakeView(state, side)` is the same
+  projection every agent (LLMs and expectimax) decides from, and expectimax
+  reconstructs from the View alone — the oracle never sees hidden info.

From 4e22b5c701a94a7f2367de94d40f8743208f732d Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Fri, 17 Jul 2026 12:13:55 -0700
Subject: [PATCH 05/15] eval: per-model decision-quality table (aggregate mode
 + driver)

Roll per-decision regret up to a per-model table: blunder rate (headline),
median regret, match rate, win rate. AggregateByModel folds scored battles;
decision-eval -manifest scores a batch and prints/JSON-emits the table;
decision-report.sh joins bid= attribution to Postgres exports and drives it.

Regret is heavy-tailed (missed lethal ~= winValue): median ignores the tail,
mean is winsorized at a cap. Test pins the arithmetic and the tail handling.

On the fresh attributed batch, blunder rate tracks win rate across all four
models (Opus < Gemini < Sonnet < Haiku), so decision quality predicts the
outcome win/loss alone hides.
---
 cmd/decision-eval/main.go            |  75 ++++++++++++++++++
 internal/eval/decisionreport.go      | 114 +++++++++++++++++++++++++++
 internal/eval/decisionreport_test.go |  64 +++++++++++++++
 scripts/bench/decision-report.sh     |  75 ++++++++++++++++++
 4 files changed, 328 insertions(+)
 create mode 100644 internal/eval/decisionreport.go
 create mode 100644 internal/eval/decisionreport_test.go
 create mode 100755 scripts/bench/decision-report.sh

diff --git a/cmd/decision-eval/main.go b/cmd/decision-eval/main.go
index e3280364..a8281169 100644
--- a/cmd/decision-eval/main.go
+++ b/cmd/decision-eval/main.go
@@ -19,6 +19,7 @@ import (
 	"io"
 	"log"
 	"os"
+	"strings"
 
 	"pokearena/internal/ai"
 	"pokearena/internal/domain"
@@ -44,8 +45,16 @@ func main() {
 	side := flag.Int("side", 0, "which side to score (0 = the model seat)")
 	depth := flag.Int("depth", 3, "oracle expectimax search depth")
 	quiet := flag.Bool("quiet", false, "print only the summary line")
+	manifest := flag.String("manifest", "", "aggregate mode: a file of \"model<TAB>export.json\" lines → per-model table")
+	regretCap := flag.Float64("regret-cap", 3000, "winsorize regret at this cap for the mean (median is unaffected)")
+	asJSON := flag.Bool("json", false, "aggregate mode: emit the per-model table as JSON")
 	flag.Parse()
 
+	if *manifest != "" {
+		runAggregate(*manifest, *dataDir, *side, *depth, *regretCap, *asJSON)
+		return
+	}
+
 	raw, err := readAll(*in)
 	if err != nil {
 		log.Fatalf("read export: %v", err)
@@ -109,6 +118,72 @@ func main() {
 
 func actStr(a engine.Action) string { return fmt.Sprintf("%s#%d", a.Kind, a.Index) }
 
+// runAggregate scores every battle listed in the manifest and prints the
+// per-model decision-quality table. The manifest is one "model<TAB>export.json"
+// line per battle (blank lines and "#" comments ignored) — the driver script
+// builds it by mapping each attributed bid= to its model and exporting the
+// battle from Postgres. Loading the dex and building the oracle once keeps a
+// whole run's scoring on a single shared reference.
+func runAggregate(manifestPath, dataDir string, side, depth int, regretCap float64, asJSON bool) {
+	dex, err := domain.LoadDex(dataDir, "bench")
+	if err != nil {
+		log.Fatalf("load dex: %v", err)
+	}
+	oracle := ai.NewExpectimaxAgentFixed(dex, depth)
+
+	mf, err := os.ReadFile(manifestPath)
+	if err != nil {
+		log.Fatalf("read manifest: %v", err)
+	}
+
+	var results []eval.BattleResult
+	for lineNo, line := range strings.Split(string(mf), "\n") {
+		line = strings.TrimSpace(line)
+		if line == "" || strings.HasPrefix(line, "#") {
+			continue
+		}
+		parts := strings.SplitN(line, "\t", 2)
+		if len(parts) != 2 {
+			log.Fatalf("manifest line %d: want \"model<TAB>path\", got %q", lineNo+1, line)
+		}
+		model, path := parts[0], parts[1]
+		raw, err := os.ReadFile(path)
+		if err != nil {
+			log.Fatalf("manifest line %d: read %s: %v", lineNo+1, path, err)
+		}
+		var ex export
+		if err := json.Unmarshal(raw, &ex); err != nil {
+			log.Fatalf("manifest line %d: parse %s: %v", lineNo+1, path, err)
+		}
+		turns := make([]eval.StoredTurn, len(ex.Turns))
+		for i, t := range ex.Turns {
+			turns[i] = eval.StoredTurn{State: t.State, Log: t.Log}
+		}
+		scores, _, err := eval.ScoreDecisions(dex, oracle, side, turns)
+		if err != nil {
+			log.Fatalf("manifest line %d: score %s: %v", lineNo+1, path, err)
+		}
+		results = append(results, eval.BattleResult{Model: model, Won: ex.Winner == side, Scores: scores})
+	}
+
+	stats := eval.AggregateByModel(results, regretCap)
+	if asJSON {
+		enc := json.NewEncoder(os.Stdout)
+		enc.SetIndent("", "  ")
+		if err := enc.Encode(stats); err != nil {
+			log.Fatalf("encode: %v", err)
+		}
+		return
+	}
+	fmt.Printf("%-20s %6s %6s %8s %9s %9s %11s %10s\n",
+		"model", "games", "win%", "decs", "blunder%", "match%", "medRegret", "meanReg")
+	for _, s := range stats {
+		fmt.Printf("%-20s %6d %5.0f%% %8d %8.0f%% %8.0f%% %11.0f %10.0f\n",
+			s.Model, s.Games, 100*s.WinRate, s.Decisions,
+			100*s.BlunderRate, 100*s.MatchRate, s.MedianRegret, s.MeanRegret)
+	}
+}
+
 func readAll(path string) ([]byte, error) {
 	if path == "-" {
 		return io.ReadAll(os.Stdin)
diff --git a/internal/eval/decisionreport.go b/internal/eval/decisionreport.go
new file mode 100644
index 00000000..12577859
--- /dev/null
+++ b/internal/eval/decisionreport.go
@@ -0,0 +1,114 @@
+package eval
+
+import "sort"
+
+// Decision-quality rolls up from per-decision scores (see decisionquality.go) to
+// a per-model table: across every recovered free choice a model made, how often
+// did it blunder, and by how much did it typically fall short of the oracle. The
+// headline is the blunder rate; median regret is the tie-breaker that separates
+// two models with the same blunder count but different typical mistake sizes.
+
+// BattleResult is one scored live battle attributed to a model: the model's
+// recovered decisions and whether the model (side 0, the "Agent" seat) won.
+type BattleResult struct {
+	Model  string
+	Won    bool
+	Scores []DecisionScore
+}
+
+// ModelStats is the decision-quality summary for one model, aggregated over all
+// its scored battles.
+type ModelStats struct {
+	Model        string  `json:"model"`
+	Games        int     `json:"games"`
+	Wins         int     `json:"wins"`
+	WinRate      float64 `json:"win_rate"`     // Wins / Games
+	Decisions    int     `json:"decisions"`    // free choices scored
+	Blunders     int     `json:"blunders"`     // decisions with Regret > BlunderThreshold
+	BlunderRate  float64 `json:"blunder_rate"` // Blunders / Decisions — the headline
+	MatchRate    float64 `json:"match_rate"`   // fraction agreeing with the oracle's top pick
+	MedianRegret float64 `json:"median_regret"`
+	MeanRegret   float64 `json:"mean_regret"` // capped at regretCap so missed-lethals don't dominate
+}
+
+// AggregateByModel folds scored battles into one ModelStats per model, sorted by
+// blunder rate ascending (best decision-maker first).
+//
+// Regret is reported two ways because its distribution is heavy-tailed: a missed
+// lethal scores regret ≈ winValue (~1e6), which would swamp a raw mean. The
+// median ignores those tails by construction; the mean is winsorized at
+// regretCap (each regret clipped to the cap before averaging) so it still
+// reflects a run's missed lethals without letting one of them set the scale.
+func AggregateByModel(results []BattleResult, regretCap float64) []ModelStats {
+	type acc struct {
+		games, wins, decisions, blunders, agrees int
+		regrets                                  []float64
+		cappedSum                                float64
+	}
+	byModel := map[string]*acc{}
+	var order []string
+	for _, r := range results {
+		a := byModel[r.Model]
+		if a == nil {
+			a = &acc{}
+			byModel[r.Model] = a
+			order = append(order, r.Model)
+		}
+		a.games++
+		if r.Won {
+			a.wins++
+		}
+		for _, s := range r.Scores {
+			a.decisions++
+			if s.Blunder {
+				a.blunders++
+			}
+			if s.Agree {
+				a.agrees++
+			}
+			a.regrets = append(a.regrets, s.Regret)
+			capped := s.Regret
+			if capped > regretCap {
+				capped = regretCap
+			}
+			a.cappedSum += capped
+		}
+	}
+
+	stats := make([]ModelStats, 0, len(order))
+	for _, m := range order {
+		a := byModel[m]
+		st := ModelStats{Model: m, Games: a.games, Wins: a.wins, Decisions: a.decisions, Blunders: a.blunders}
+		if a.games > 0 {
+			st.WinRate = float64(a.wins) / float64(a.games)
+		}
+		if a.decisions > 0 {
+			n := float64(a.decisions)
+			st.BlunderRate = float64(a.blunders) / n
+			st.MatchRate = float64(a.agrees) / n
+			st.MeanRegret = a.cappedSum / n
+			st.MedianRegret = median(a.regrets)
+		}
+		stats = append(stats, st)
+	}
+	sort.SliceStable(stats, func(i, j int) bool {
+		return stats[i].BlunderRate < stats[j].BlunderRate
+	})
+	return stats
+}
+
+// median returns the middle value of xs (mean of the two middles when the count
+// is even), or 0 for an empty slice. It sorts a copy so the caller's slice keeps
+// its original order.
+func median(xs []float64) float64 {
+	if len(xs) == 0 {
+		return 0
+	}
+	s := append([]float64(nil), xs...)
+	sort.Float64s(s)
+	mid := len(s) / 2
+	if len(s)%2 == 1 {
+		return s[mid]
+	}
+	return (s[mid-1] + s[mid]) / 2
+}
diff --git a/internal/eval/decisionreport_test.go b/internal/eval/decisionreport_test.go
new file mode 100644
index 00000000..60fd948f
--- /dev/null
+++ b/internal/eval/decisionreport_test.go
@@ -0,0 +1,64 @@
+package eval
+
+import "testing"
+
+// TestAggregateByModel checks the roll-up arithmetic and the heavy-tail
+// handling: a single missed-lethal (regret 1e6) must not move the median or
+// blow up the winsorized mean, and models must sort by blunder rate ascending.
+func TestAggregateByModel(t *testing.T) {
+	sc := func(regret float64, agree bool) DecisionScore {
+		return DecisionScore{Regret: regret, Agree: agree, Blunder: regret > BlunderThreshold}
+	}
+	results := []BattleResult{
+		// "clean": 4 decisions, 1 blunder, regrets {0,100,200,1e6}, won.
+		{Model: "clean", Won: true, Scores: []DecisionScore{
+			sc(0, true), sc(100, false), sc(200, false), sc(1_000_000, false),
+		}},
+		// "sloppy": 2 decisions, both blunders, regrets {500,700}, lost.
+		{Model: "sloppy", Won: false, Scores: []DecisionScore{
+			sc(500, false), sc(700, false),
+		}},
+	}
+
+	stats := AggregateByModel(results, 3000)
+	if len(stats) != 2 {
+		t.Fatalf("want 2 models, got %d", len(stats))
+	}
+	// clean has the lower blunder rate (1/4 vs 2/2) so it sorts first.
+	if stats[0].Model != "clean" || stats[1].Model != "sloppy" {
+		t.Fatalf("sort order = [%s %s], want [clean sloppy]", stats[0].Model, stats[1].Model)
+	}
+
+	c := stats[0]
+	if c.Games != 1 || c.Wins != 1 || c.WinRate != 1 {
+		t.Errorf("clean games/wins/winrate = %d/%d/%.2f, want 1/1/1.00", c.Games, c.Wins, c.WinRate)
+	}
+	if c.Decisions != 4 || c.Blunders != 1 {
+		t.Errorf("clean decisions/blunders = %d/%d, want 4/1", c.Decisions, c.Blunders)
+	}
+	if c.BlunderRate != 0.25 {
+		t.Errorf("clean blunder rate = %.3f, want 0.25", c.BlunderRate)
+	}
+	if c.MatchRate != 0.25 {
+		t.Errorf("clean match rate = %.3f, want 0.25", c.MatchRate)
+	}
+	// Median of {0,100,200,1e6} = (100+200)/2 = 150 — the missed-lethal is ignored.
+	if c.MedianRegret != 150 {
+		t.Errorf("clean median regret = %.0f, want 150 (heavy tail must not move it)", c.MedianRegret)
+	}
+	// Winsorized mean at cap 3000: (0+100+200+3000)/4 = 825, not ~250k.
+	if c.MeanRegret != 825 {
+		t.Errorf("clean mean regret = %.0f, want 825 (winsorized at 3000)", c.MeanRegret)
+	}
+
+	s := stats[1]
+	if s.BlunderRate != 1 {
+		t.Errorf("sloppy blunder rate = %.3f, want 1.0", s.BlunderRate)
+	}
+	if s.MedianRegret != 600 {
+		t.Errorf("sloppy median regret = %.0f, want 600", s.MedianRegret)
+	}
+	if s.WinRate != 0 {
+		t.Errorf("sloppy win rate = %.2f, want 0", s.WinRate)
+	}
+}
diff --git a/scripts/bench/decision-report.sh b/scripts/bench/decision-report.sh
new file mode 100755
index 00000000..bf1c9f8b
--- /dev/null
+++ b/scripts/bench/decision-report.sh
@@ -0,0 +1,75 @@
+#!/usr/bin/env bash
+# Build the per-model decision-quality table from a fresh attributed batch.
+#
+# Attribution lives only in the run dirs: each results.txt line carries the
+# game's winner and its battle_id as "bid=<uuid>". The dir name's leading key
+# ("cc-haiku", "agy-gemini", ...) maps to a model via eval.ModelDisplay. Postgres
+# holds the turns but no model identity, so this script joins the two: for every
+# COMPLETED game it exports the battle JSON from Postgres, writes a
+# "model<TAB>export.json" manifest line, then hands the whole manifest to
+# `decision-eval -manifest` which scores each battle and rolls up the table.
+#
+# Usage: decision-report.sh [results_root] [--json]
+#   results_root defaults to /tmp/pk-agentic-v2
+#
+# Unfinished games (winner=-1, no completed battle) are skipped and counted —
+# they have no scoreable decisions.
+set -uo pipefail
+
+ROOT="${1:-/tmp/pk-agentic-v2}"
+JSON_FLAG=""
+[ "${2:-}" = "--json" ] && JSON_FLAG="-json"
+
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO="$(cd "$DIR/../.." && pwd)"
+PG="pk-bench-postgres-1"
+WORK="$(mktemp -d)"
+MANIFEST="$WORK/manifest.tsv"
+: > "$MANIFEST"
+
+# key -> model display name, mirroring eval.ModelDisplay (kept in lockstep).
+model_for() {
+  case "$1" in
+    cc-haiku)   echo "Claude Haiku 4.5" ;;
+    cc-sonnet)  echo "Claude Sonnet 4.6" ;;
+    cc-opus)    echo "Claude Opus 4.8" ;;
+    agy-*)      echo "Gemini 3.1 Pro" ;;
+    cc-*)       echo "Claude ${1#cc-}" ;;
+    *)          echo "$1" ;;
+  esac
+}
+
+skipped=0 exported=0
+for res in "$ROOT"/*/results.txt; do
+  [ -f "$res" ] || continue
+  dir="$(basename "$(dirname "$res")")"        # e.g. cc-haiku-Genesis
+  key="${dir%-*}"                               # strip trailing -<Team>
+  model="$(model_for "$key")"
+  while IFS= read -r line; do
+    case "$line" in *winner=-1*) skipped=$((skipped+1)); continue ;; esac
+    bid="${line##*bid=}"; bid="${bid%% *}"
+    [ -n "$bid" ] || continue
+    out="$WORK/$bid.json"
+    docker exec "$PG" psql -U pokearena -d pokearena -tAX -c \
+      "select json_build_object('seed', b.seed, 'winner', b.winner,
+         'turns', (select json_agg(json_build_object('state', t.state_digest, 'log', t.log)
+                   order by t.turn_no) from battle_turns t where t.battle_id=b.id))
+       from battles b where b.id='$bid' and b.status='completed'" > "$out" 2>/dev/null
+    if [ ! -s "$out" ] || [ "$(head -c 4 "$out")" = "" ]; then
+      skipped=$((skipped+1)); rm -f "$out"; continue
+    fi
+    printf '%s\t%s\n' "$model" "$out" >> "$MANIFEST"
+    exported=$((exported+1))
+  done < "$res"
+done
+
+echo "# battles exported=$exported skipped(unfinished/missing)=$skipped" >&2
+if [ "$exported" -eq 0 ]; then
+  echo "no completed battles to score under $ROOT" >&2
+  exit 1
+fi
+
+( cd "$REPO" && go run ./cmd/decision-eval -manifest "$MANIFEST" -data data $JSON_FLAG )
+rc=$?
+rm -rf "$WORK"
+exit $rc

From c7b0bd7870a2823db51688d937992f9128b48d65 Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Fri, 17 Jul 2026 12:16:38 -0700
Subject: [PATCH 06/15] docs: mark aggregation shipped in handoff

---
 docs/decision-quality-eval-handoff.md | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/decision-quality-eval-handoff.md b/docs/decision-quality-eval-handoff.md
index cc226812..dd9de71c 100644
--- a/docs/decision-quality-eval-handoff.md
+++ b/docs/decision-quality-eval-handoff.md
@@ -3,6 +3,14 @@
 Status as of 2026-07-17. Branch: `feat/decision-quality-eval` (based on the
 now-merged `feat/replay-samples`; PR #108 already merged to `main`).
 
+> **UPDATE 2026-07-17: the aggregation shipped.** The "What REMAINS" item 2
+> (the per-model table) is done — `eval.AggregateByModel`,
+> `decision-eval -manifest`, and `scripts/bench/decision-report.sh`. On the
+> fresh batch, blunder rate tracks win rate (Opus < Gemini < Sonnet < Haiku).
+> Opus stayed thin (n=2): its live games wall out under load, so its row is
+> indicative only. Report-section wiring (item 3) is still open as a follow-up.
+> The rest of this doc is the original design/gotcha reference — still accurate.
+
 ## Goal
 
 Score how well each model *chose*, not just whether it won. For every free

From 3c3eff2ee244e665d49a51d5c7bc32732eae6557 Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Fri, 17 Jul 2026 12:36:46 -0700
Subject: [PATCH 07/15] bench: wall-clock cap per live game so a hung session
 can't wedge a batch
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A stalled agent session reads forever at 0% CPU (Opus in particular walls out
this way — a hung model/streaming connection). With no timeout, one such game
blocked the whole attributed batch for 13+ hours. Wrap claude -p in a portable
watchdog (macOS has no timeout(1)) that TERM/KILLs the session and its MCP child
after POKEARENA_GAME_TIMEOUT seconds (default 1200), so the game lands as
unfinished instead of hanging. Mirrors the agy harness's --print-timeout.
---
 scripts/bench/play-live.sh | 40 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/scripts/bench/play-live.sh b/scripts/bench/play-live.sh
index 82f8c4cb..e410ca9d 100755
--- a/scripts/bench/play-live.sh
+++ b/scripts/bench/play-live.sh
@@ -27,8 +27,44 @@ REPO="$(cd "$DIR/../.." && pwd)"
 HELP="$DIR/_bench_helpers.py"
 GW_HTTP="${POKEARENA_GATEWAY_HTTP:-http://localhost:8080}"
 GW_WS="${POKEARENA_GATEWAY_URL:-ws://localhost:8080}"
+# A hung agent session (a stalled model/streaming connection reads forever at 0%
+# CPU) would otherwise block a whole batch indefinitely — Opus in particular
+# walls out this way. Cap each game's wall clock and kill a session that blows
+# past it, so the game just lands as unfinished instead of wedging the batch.
+# The `agy` harness already self-limits via --print-timeout; this gives `claude`
+# the same guarantee. macOS ships no timeout(1), so we watchdog it ourselves.
+GAME_TIMEOUT="${POKEARENA_GAME_TIMEOUT:-1200}"
 mkdir -p "$OUTDIR"
 
+# run_capped runs a command with a wall-clock cap. On expiry it TERMs (then
+# KILLs) the command AND its children — claude spawns the MCP server as a child,
+# so killing only the parent would orphan it. Returns 124 on timeout, else the
+# command's own exit code (matching timeout(1)'s convention).
+run_capped() {
+  local secs="$1"; shift
+  local flag; flag="$(mktemp)"; rm -f "$flag"  # exists only once the cap fires
+  "$@" &
+  local cpid=$!
+  (
+    local waited=0
+    while kill -0 "$cpid" 2>/dev/null; do
+      sleep 10; waited=$((waited + 10))
+      if [ "$waited" -ge "$secs" ]; then
+        : > "$flag"
+        pkill -TERM -P "$cpid" 2>/dev/null; kill -TERM "$cpid" 2>/dev/null
+        sleep 5
+        pkill -KILL -P "$cpid" 2>/dev/null; kill -KILL "$cpid" 2>/dev/null
+        exit 0
+      fi
+    done
+  ) &
+  local wpid=$!
+  wait "$cpid" 2>/dev/null; local rc=$?
+  if [ -f "$flag" ]; then wait "$wpid" 2>/dev/null; rm -f "$flag"; return 124; fi
+  kill "$wpid" 2>/dev/null; wait "$wpid" 2>/dev/null
+  return "$rc"
+}
+
 # The team's picks, straight from the committed library — the agent plays a real
 # tuned team, not an improvised one.
 PICKS=$(python3 "$HELP" picks "$REPO/data/benchmark-teams.json" "$TEAM") || exit 1
@@ -61,11 +97,13 @@ case "$HARNESS" in
   claude)
     MCPCFG="$OUTDIR/mcp.json"
     printf '{"mcpServers":{"pokearena":{"command":"%s/bin/pokearena-mcp","env":{"POKEARENA_GATEWAY_URL":"%s"}}}}\n' "$REPO" "$GW_WS" > "$MCPCFG"
-    claude -p "$PROMPT" --model "$MODEL" \
+    run_capped "$GAME_TIMEOUT" \
+      claude -p "$PROMPT" --model "$MODEL" \
       --mcp-config "$MCPCFG" --strict-mcp-config \
       --permission-mode bypassPermissions \
       --disallowedTools "Bash,Edit,Write,Read,Glob,Grep,WebFetch,WebSearch,Task,TodoWrite,NotebookEdit" \
       --max-turns 400 > "$OUTDIR/$LABEL.log" 2>&1
+    [ $? -eq 124 ] && echo "[play-live] $LABEL: killed after ${GAME_TIMEOUT}s wall-clock cap" >> "$OUTDIR/$LABEL.log"
     ;;
   agy)
     # Antigravity reads its MCP servers from ~/.gemini/antigravity-cli/mcp_config.json;

From cee389ee6f38556aafef83437a5bae5ec440730e Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Fri, 17 Jul 2026 15:14:17 -0700
Subject: [PATCH 08/15] report: decision-quality section in the HTML report
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Adds a 'how well did each model choose?' table to the benchmark report: blunder
rate (with a bar), median regret, oracle-match rate, and win rate for context,
sorted cleanest-first. RunRecord carries DecisionQuality []ModelStats; bench-report
loads it from decision-eval's JSON (-decision-quality flag) so the report stays
offline and never re-scores. build-report.sh folds it in via POKEARENA_DQ_JSON.

Also makes the baseline round-robin trace optional: with no trace, bench-report
synthesizes a header from the dataset and renders the agentic arm alone — enough
for a decision-quality report without the full ladder. Test pins the section's
ordering, the cleanest tag, and self-containment.
---
 cmd/bench-report/main.go      | 53 +++++++++++++++++++++---
 internal/eval/report.go       | 77 +++++++++++++++++++++++++++++++++++
 internal/eval/report_test.go  | 45 ++++++++++++++++++++
 internal/eval/results.go      |  5 +++
 scripts/bench/build-report.sh | 14 ++++++-
 5 files changed, 188 insertions(+), 6 deletions(-)

diff --git a/cmd/bench-report/main.go b/cmd/bench-report/main.go
index a7d9250c..0b194c87 100644
--- a/cmd/bench-report/main.go
+++ b/cmd/bench-report/main.go
@@ -37,12 +37,13 @@ func main() {
 	ref := flag.String("ref", "heuristic", "benchmark mode: the one opponent every contestant is scored against")
 	dataDir := flag.String("data", "data", "benchmark mode: dataset directory (for replay re-simulation)")
 	teamsPath := flag.String("teams", "data/benchmark-teams.json", "benchmark mode: team library the baseline replays re-simulate on")
+	decisionQuality := flag.String("decision-quality", "", "benchmark mode: JSON of per-model decision-quality stats (decision-eval -json) to render as a section")
 	flag.Parse()
 
 	// Benchmark mode: fold both arms into one "vs the reference" ladder and
 	// render it through the standard report — same leaderboard, same replays.
 	if *baseline != "" || *agentic != "" {
-		if err := renderBenchmark(*baseline, *agentic, *ref, *dataDir, *teamsPath, *outPath); err != nil {
+		if err := renderBenchmark(*baseline, *agentic, *ref, *dataDir, *teamsPath, *decisionQuality, *outPath); err != nil {
 			log.Fatalf("%v", err)
 		}
 		return
@@ -81,7 +82,7 @@ func main() {
 
 // renderBenchmark builds the vs-reference RunRecord from the two benchmark arms
 // and writes it through the standard report renderer.
-func renderBenchmark(baselinePath, agenticDir, ref, dataDir, teamsPath, outPath string) error {
+func renderBenchmark(baselinePath, agenticDir, ref, dataDir, teamsPath, dqPath, outPath string) error {
 	dex, err := domain.LoadDex(dataDir, "bench")
 	if err != nil {
 		return fmt.Errorf("load dex: %w", err)
@@ -94,9 +95,26 @@ func renderBenchmark(baselinePath, agenticDir, ref, dataDir, teamsPath, outPath
 	// Reuse the trace's own header for provenance (engine revision, ruleset,
 	// dataset), but null the round-robin game count — this report's game total is
 	// per-contestant, shown in the leaderboard, not derivable from that formula.
-	header, err := readTraceHeader(baselinePath)
-	if err != nil {
-		return fmt.Errorf("read trace header: %w", err)
+	// The baseline is optional: with no trace (agentic arm alone, e.g. a
+	// decision-quality report) synthesize a minimal header from the loaded
+	// dataset so the masthead and ruleset pills still render.
+	var header eval.RunHeader
+	if baselinePath != "" {
+		if h, err := readTraceHeader(baselinePath); err == nil {
+			header = h
+		} else if !os.IsNotExist(err) {
+			return fmt.Errorf("read trace header: %w", err)
+		} else {
+			log.Printf("no baseline trace at %s — rendering the agentic arm alone", baselinePath)
+		}
+	}
+	if header.Ruleset == "" {
+		header.Ruleset = eval.Ruleset()
+	}
+	if len(header.Teams) == 0 {
+		for _, t := range lib.Teams {
+			header.Teams = append(header.Teams, t.Name)
+		}
 	}
 	header.GamesPerPairing = 0
 
@@ -105,6 +123,17 @@ func renderBenchmark(baselinePath, agenticDir, ref, dataDir, teamsPath, outPath
 		return fmt.Errorf("build vs-reference record: %w", err)
 	}
 
+	// Optional decision-quality section: precomputed offline (decision-eval
+	// scores stored turns against the oracle), loaded here so the report can show
+	// reasoning quality without re-scoring or touching a database.
+	if dqPath != "" {
+		dq, err := loadDecisionQuality(dqPath)
+		if err != nil {
+			return fmt.Errorf("load decision-quality: %w", err)
+		}
+		rec.DecisionQuality = dq
+	}
+
 	// Contestant names for the masthead count (the reference is already dropped).
 	header.Contestants = header.Contestants[:0]
 	for _, c := range rec.Contestants {
@@ -150,6 +179,20 @@ func readTraceHeader(path string) (eval.RunHeader, error) {
 	return h, nil
 }
 
+// loadDecisionQuality reads the per-model decision-quality stats emitted by
+// `decision-eval -manifest ... -json` (a JSON array of eval.ModelStats).
+func loadDecisionQuality(path string) ([]eval.ModelStats, error) {
+	data, err := os.ReadFile(path)
+	if err != nil {
+		return nil, fmt.Errorf("read %s: %w", path, err)
+	}
+	var stats []eval.ModelStats
+	if err := json.Unmarshal(data, &stats); err != nil {
+		return nil, fmt.Errorf("parse %s: %w", path, err)
+	}
+	return stats, nil
+}
+
 func loadRecord(path string) (eval.RunRecord, error) {
 	data, err := os.ReadFile(path)
 	if err != nil {
diff --git a/internal/eval/report.go b/internal/eval/report.go
index 3002254b..cf66cd04 100644
--- a/internal/eval/report.go
+++ b/internal/eval/report.go
@@ -68,6 +68,10 @@ type reportView struct {
 	// viewer can click to load. Empty for a run with no model replays.
 	HasSamples  bool
 	SamplesJSON template.JS
+	// DQRows is the per-model decision-quality table, sorted best (fewest
+	// blunders) first. Empty when the run carries no decision-quality data.
+	HasDecisionQuality bool
+	DQRows             []dqRow
 	// Human-facing run summary for the masthead — meaningful to a viewer, unlike
 	// the raw run id.
 	GeneratedHuman string
@@ -77,6 +81,22 @@ type reportView struct {
 	RepoURL        string
 }
 
+// dqRow is one model's decision-quality line, preformatted for the template: the
+// share of choices that blundered (the headline, with a bar), how typical a
+// shortfall looked (median regret), how often it matched the oracle's top pick,
+// and win rate for context.
+type dqRow struct {
+	Model        string
+	Games        int
+	WinRate      string
+	Decisions    int
+	BlunderRate  string
+	BlunderBar   float64 // 0–100, for the inline bar
+	MatchRate    string
+	MedianRegret string
+	Best         bool // lowest blunder rate — the cleanest decision-maker
+}
+
 // sampleChip is one clickable battle in a model's sample strip: the team it was
 // played on, whether the model won or lost, and the index of the embedded replay
 // to load. Derived from the replays themselves, so richer coverage (more
@@ -214,6 +234,27 @@ func buildReportView(rec RunRecord) reportView {
 		}
 	}
 
+	// Decision-quality table: fewest blunders first (AggregateByModel already
+	// sorts this way; re-sort defensively in case the data was hand-assembled).
+	if len(rec.DecisionQuality) > 0 {
+		dq := append([]ModelStats(nil), rec.DecisionQuality...)
+		sort.SliceStable(dq, func(i, j int) bool { return dq[i].BlunderRate < dq[j].BlunderRate })
+		for i, s := range dq {
+			v.DQRows = append(v.DQRows, dqRow{
+				Model:        s.Model,
+				Games:        s.Games,
+				WinRate:      fmt.Sprintf("%.0f%%", 100*s.WinRate),
+				Decisions:    s.Decisions,
+				BlunderRate:  fmt.Sprintf("%.0f%%", 100*s.BlunderRate),
+				BlunderBar:   100 * s.BlunderRate,
+				MatchRate:    fmt.Sprintf("%.0f%%", 100*s.MatchRate),
+				MedianRegret: fmt.Sprintf("%.0f", s.MedianRegret),
+				Best:         i == 0 && len(dq) > 1,
+			})
+		}
+		v.HasDecisionQuality = len(v.DQRows) > 0
+	}
+
 	// Human run summary: readable date, and the run's scale from the header.
 	v.NAgents = len(rec.Header.Contestants)
 	v.NTeams = len(rec.Header.Teams)
@@ -300,11 +341,17 @@ const reportHTML = `<!DOCTYPE html>
   .name .cond.raw { background: #23324a; color: #9db4d6; }
   .name .cond.cot { background: #3a2f52; color: #c3a8ee; }
   .name .cond.agentic { background: #2f4a37; color: #9ed6ac; }
+  .name .cond.cleanest { background: #234a3a; color: #8fe0bf; }
   .elo { font-weight: 700; }
   .bar { position: relative; height: 22px; min-width: 160px; background: #f1f2f6; border-radius: 5px; }
   .bar .ci { position: absolute; top: 0; bottom: 0; background: var(--ci); border-radius: 5px; }
   .bar .mark { position: absolute; top: -2px; bottom: -2px; width: 2px; background: var(--mark); }
   .bar .lbl { position: absolute; right: 6px; top: 50%; transform: translateY(-50%); font-size: .72rem; color: #3730a3; }
+  .dq-lead { font-size: .95rem; line-height: 1.6; color: #2b2b40; margin: .1rem 0 1rem; max-width: 76ch; }
+  .dq-note { font-size: .8rem; line-height: 1.55; color: var(--muted); margin: .9rem 0 0; max-width: 76ch; }
+  table.dq td.name { font-weight: 600; }
+  .dq-bar .fill { position: absolute; top: 0; bottom: 0; left: 0; background: linear-gradient(90deg, #f0a24d, #e0603f); border-radius: 5px; opacity: .85; }
+  .dq-bar .lbl { color: #7a3b1e; }
   .free { color: var(--free); }
   .unknown { color: var(--warn); }
   .teams { display: grid; grid-template-columns: repeat(auto-fill, minmax(260px, 1fr)); gap: .6rem; }
@@ -476,6 +523,36 @@ const reportHTML = `<!DOCTYPE html>
     </table>
   </div>
 
+  {{if .HasDecisionQuality}}
+  <h2>Decision quality — how well did each model choose?</h2>
+  <p class="dq-lead">Win rate says who won; it can't say whether a model played well or got lucky. For every free choice in these battles we recover the exact move played, then ask a stronger expectimax oracle — deciding from the <em>identical</em> fog-of-war view — what it would have played, and measure the <strong>regret</strong> (the value the choice gave up). A <strong>blunder</strong> is a choice that gave up more than about a third of a Pokémon's worth of position. Fewer blunders is better.</p>
+  <div class="card">
+    <table class="dq">
+      <thead>
+        <tr><th>model</th><th>blunder rate</th><th>median regret</th><th>oracle match</th><th>decisions</th><th>win rate</th></tr>
+      </thead>
+      <tbody>
+      {{range .DQRows}}
+        <tr class="{{if .Best}}top{{end}}">
+          <td class="name">{{.Model}}{{if .Best}}<span class="cond cleanest">cleanest</span>{{end}}</td>
+          <td>
+            <div class="bar dq-bar">
+              <div class="fill" style="width:{{.BlunderBar}}%"></div>
+              <span class="lbl num">{{.BlunderRate}}</span>
+            </div>
+          </td>
+          <td class="num">{{.MedianRegret}}</td>
+          <td class="num">{{.MatchRate}}</td>
+          <td class="num">{{.Decisions}}</td>
+          <td class="num">{{.WinRate}}</td>
+        </tr>
+      {{end}}
+      </tbody>
+    </table>
+    <p class="dq-note">Regret is in the oracle's evaluation units (≈1000 = one Pokémon). It is heavy-tailed — a missed lethal scores off the chart — so the typical shortfall is reported as the <em>median</em>, not the mean. Match rate is coarse by design: two moves of equal value both count as “right,” so a low match with low regret just means many ties, not many mistakes.</p>
+  </div>
+  {{end}}
+
   {{if .Rec.PerTeam}}
   <h2>Per-team Elo — does the ranking hold across the library?</h2>
   <div class="teams">
diff --git a/internal/eval/report_test.go b/internal/eval/report_test.go
index 874a1e11..db55480c 100644
--- a/internal/eval/report_test.go
+++ b/internal/eval/report_test.go
@@ -144,6 +144,51 @@ func TestReportSpritesInline(t *testing.T) {
 	}
 }
 
+// TestReportDecisionQuality checks the decision-quality section renders from a
+// record's ModelStats: the models sort cleanest-first (lowest blunder rate),
+// the headline is tagged, and the rates are shown as percentages. It also guards
+// that the section stays self-contained (adds no fetched asset or script).
+func TestReportDecisionQuality(t *testing.T) {
+	rec := RunRecord{
+		DecisionQuality: []ModelStats{
+			// Deliberately out of order to prove buildReportView sorts by blunder rate.
+			{Model: "Claude Haiku 4.5", Games: 12, WinRate: 0.08, Decisions: 200, BlunderRate: 0.29, MatchRate: 0.32, MedianRegret: 91},
+			{Model: "Gemini 3.1 Pro", Games: 12, WinRate: 0.67, Decisions: 273, BlunderRate: 0.21, MatchRate: 0.32, MedianRegret: 47},
+		},
+	}
+
+	v := buildReportView(rec)
+	if !v.HasDecisionQuality || len(v.DQRows) != 2 {
+		t.Fatalf("expected 2 decision-quality rows, got has=%v n=%d", v.HasDecisionQuality, len(v.DQRows))
+	}
+	// Lowest blunder rate first, and only that row is flagged the cleanest.
+	if v.DQRows[0].Model != "Gemini 3.1 Pro" || !v.DQRows[0].Best {
+		t.Fatalf("cleanest row = %q best=%v, want Gemini best", v.DQRows[0].Model, v.DQRows[0].Best)
+	}
+	if v.DQRows[1].Best {
+		t.Fatal("only the lowest-blunder row should be marked best")
+	}
+	if v.DQRows[0].BlunderRate != "21%" || v.DQRows[0].WinRate != "67%" {
+		t.Fatalf("Gemini formatted rates = blunder %q win %q, want 21%% / 67%%", v.DQRows[0].BlunderRate, v.DQRows[0].WinRate)
+	}
+
+	var sb strings.Builder
+	if err := RenderHTMLReport(&sb, rec); err != nil {
+		t.Fatalf("RenderHTMLReport: %v", err)
+	}
+	html := sb.String()
+	for _, want := range []string{"Decision quality", "blunder rate", "Gemini 3.1 Pro", "cleanest", "21%"} {
+		if !strings.Contains(html, want) {
+			t.Fatalf("decision-quality section missing %q", want)
+		}
+	}
+	for _, bad := range []string{"src=\"http", "src='http", "<link ", "@import", "url(http", "githubusercontent"} {
+		if strings.Contains(html, bad) {
+			t.Fatalf("decision-quality report should be self-contained, found %q", bad)
+		}
+	}
+}
+
 func tagTeam(games []GameRecord, team string) []GameRecord {
 	for i := range games {
 		games[i].Team = team
diff --git a/internal/eval/results.go b/internal/eval/results.go
index 9dcb1b8c..1d509cfb 100644
--- a/internal/eval/results.go
+++ b/internal/eval/results.go
@@ -70,6 +70,11 @@ type RunRecord struct {
 	// Rosters is each team's composition, embedded so the report can reveal
 	// what a team actually is on click, without the dataset.
 	Rosters []TeamRoster `json:"rosters,omitempty"`
+	// DecisionQuality is the per-model reasoning table: how well each model chose
+	// (regret vs an expectimax oracle from the identical fog-of-war view), not
+	// just whether it won. Computed offline from stored turns and loaded into the
+	// record, so the report shows it without re-scoring. Empty when not supplied.
+	DecisionQuality []ModelStats `json:"decision_quality,omitempty"`
 }
 
 // NameElo is one contestant's Elo on one team — the compact form used for the
diff --git a/scripts/bench/build-report.sh b/scripts/bench/build-report.sh
index 9ac6014e..da0d4331 100755
--- a/scripts/bench/build-report.sh
+++ b/scripts/bench/build-report.sh
@@ -26,6 +26,16 @@ cd "$REPO" || exit 1
 AGENTIC="${1:-/tmp/pk-agentic}"
 BASELINE="${2:-$REPO/runs/arm1-baseline.jsonl}"
 OUT="${3:-$REPO/reports/benchmark.html}"
+# Optional decision-quality section: a JSON of per-model reasoning stats from
+# `decision-eval -manifest ... -json` (see decision-report.sh). Point
+# POKEARENA_DQ_JSON at it to fold the "how well did each model choose?" table
+# into the report; unset, the report renders exactly as before.
+DQ_JSON="${POKEARENA_DQ_JSON:-}"
+
+# The baseline round-robin is optional: without it the report renders the
+# agentic arm (the live models vs the reference) alone. Blank a missing path so
+# the record builder skips it rather than failing to open it.
+[ -n "$BASELINE" ] && [ ! -f "$BASELINE" ] && { echo "[build-report] no baseline trace at $BASELINE — agentic arm only"; BASELINE=""; }
 
 go build -o "$REPO/bin/bench-report" ./cmd/bench-report || exit 1
 go build -o "$REPO/bin/db-replay" ./cmd/db-replay || exit 1
@@ -71,5 +81,7 @@ for plain in "$AGENTIC"/replays/*.json; do
   fi
 done
 
-"$REPO/bin/bench-report" -baseline "$BASELINE" -agentic "$AGENTIC" -ref heuristic -out "$OUT"
+DQ_ARGS=()
+[ -n "$DQ_JSON" ] && DQ_ARGS=(-decision-quality "$DQ_JSON")
+"$REPO/bin/bench-report" -baseline "$BASELINE" -agentic "$AGENTIC" -ref heuristic "${DQ_ARGS[@]}" -out "$OUT"
 echo "[build-report] wrote $OUT"

From a98f3799a0dd2b64945ec7b8965ce3932cc22228 Mon Sep 17 00:00:00 2001
From: Shaumik Mondal <shaumik@echelonai.com>
Date: Fri, 17 Jul 2026 15:15:35 -0700
Subject: [PATCH 09/15] docs: decision-quality write-up (method, results, what
 it means)

---
 docs/decision-quality.md | 111 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 111 insertions(+)
 create mode 100644 docs/decision-quality.md

diff --git a/docs/decision-quality.md b/docs/decision-quality.md
new file mode 100644
index 00000000..ed4be848
--- /dev/null
+++ b/docs/decision-quality.md
@@ -0,0 +1,111 @@
+# Decision quality — scoring how a model *chooses*, not just whether it wins
+
+Win rate answers one question: who won? It cannot tell you *why*. A model can win
+because it played well, or because its opponent stumbled, or because the team it
+was dealt was strong. Across twelve games that noise doesn't wash out — the
+ranking you get is real, but it's coarse, and it hides how a model actually
+reasons turn to turn.
+
+Decision quality measures the reasoning directly. For every free choice a model
+made in a recorded battle, we ask: **how much better could it have played?**
+
+## How it works
+
+Every live battle stores the full engine state at each turn. The engine is a
+pure, deterministic function of `(state, actions)` — given the same state and the
+same actions, it always produces the same next state. That lets us do three
+things exactly:
+
+1. **Recover the move played.** We re-simulate each stored turn, enumerating the
+   legal actions for both sides until we find the pair that reproduces the next
+   stored state *byte for byte*. Because the engine is deterministic from the
+   stored RNG, that pair is exactly what was played — and the match doubles as a
+   validity check on the recording. (Faint turns, where a KO forces a mid-turn
+   replacement, are recovered the same way by searching the replacement picks.)
+
+2. **Ask a stronger player what it would have done — from the same view.** An
+   expectimax oracle scores every legal action at the decision point. Crucially,
+   it decides from `ai.MakeView(state, side)`: the *identical* fog-of-war
+   projection the model itself saw. The oracle never sees the opponent's hidden
+   moves, exact HP, or bench. It is a better player looking at the same
+   information, not an omniscient one — so the comparison is fair.
+
+3. **Measure the regret.** Regret is the value the choice gave up: the oracle's
+   value for its best action minus its value for the action the model actually
+   played. Regret 0 means the model picked an option the oracle rates as good as
+   its own best (there are often several). A large regret means the model left
+   real value on the table.
+
+We roll these per-decision regrets up per model:
+
+- **Blunder rate** — the share of choices with regret above a threshold (~a
+  third of a Pokémon's worth of position). This is the headline: how often the
+  model made a materially bad move.
+- **Median regret** — the typical size of a shortfall. We use the median, not
+  the mean, because regret is heavy-tailed: a single missed lethal scores off the
+  chart (≈ the value of winning), and one of those would swamp a mean.
+- **Match rate** — how often the model's move was the oracle's literal top pick.
+  It's deliberately coarse: two moves of equal value both count as "right," so a
+  low match rate with low regret just means the position had many good options,
+  not that the model erred.
+
+## What the numbers say
+
+From the fresh attributed batch — twelve live games per model, 48 battles, 945
+scored decisions:
+
+| model | win rate | blunder rate | median regret |
+|---|---|---|---|
+| Gemini 3.1 Pro | 67% | **21%** | 47 |
+| Claude Opus 4.8 | **75%** | 25% | 81 |
+| Claude Sonnet 4.6 | 25% | 26% | 49 |
+| Claude Haiku 4.5 | 8% | 29% | 91 |
+
+The tidy story would be "better decisions, more wins." It's *almost* that — Haiku
+is worst on both, and the field broadly tracks — but the interesting part is
+where the two measures **disagree**:
+
+- **Opus wins the most (75%) yet is not the cleanest chooser.** It blunders more
+  often than Gemini (25% vs 21%) and gives up more when it does (median regret 81
+  vs 47). Opus wins "dirtier": it converts good positions even after a shaky move,
+  or its raw play strength carries games its per-turn decisions don't fully earn.
+- **Gemini is the cleanest decision-maker** — fewest blunders, smallest typical
+  regret — while winning less than Opus. It plays the tidiest game on the board;
+  the win column doesn't fully reward that here.
+
+That gap is the whole point. Win rate ranked Opus first and Gemini second.
+Decision quality flips them on "who plays the cleanest game." Neither view is
+wrong — they measure different things, and a benchmark that only reported win
+rate would have told you Opus is simply better, full stop, and missed that Gemini
+makes fewer mistakes per move.
+
+## Caveats
+
+- **Small n.** Twelve games per model. These numbers are directional, not
+  final; the blunder-rate gaps between Opus, Sonnet, and Gemini are within what a
+  larger sample could move.
+- **The oracle is a yardstick, not ground truth.** Expectimax at a fixed depth is
+  a strong Gen-1 player but not a solver; a deeper oracle would lower everyone's
+  blunder rate and could reorder the close rows. What matters is that *every*
+  model is measured against the *same* yardstick from the *same* view.
+- **The threshold is a knob.** "Blunder" is regret above a fixed cut; the match
+  and median columns are there so the picture doesn't hinge on that one number.
+
+## Reproduce it
+
+```sh
+# Score one battle (stdin/-in is the {seed,winner,turns} export db-replay uses):
+decision-eval -in battle.json -side 0 -depth 3
+
+# Per-model table from a batch of attributed runs (maps bid= -> model,
+# exports each battle from Postgres, scores, rolls up):
+scripts/bench/decision-report.sh /tmp/pk-agentic-v2            # text table
+scripts/bench/decision-report.sh /tmp/pk-agentic-v2 --json     # for the report
+
+# Fold the table into the HTML report:
+POKEARENA_DQ_JSON=decision-quality.json scripts/bench/build-report.sh
+```
+
+Fairness is by construction: `ai.MakeView` is the one projection every agent —
+the language models over the live gateway and the expectimax oracle offline —
+decides from, and the oracle reconstructs its search from that view alone.

From 69263b0116089c71b1f3a11117b3040fdffd5a50 Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Tue, 11 Aug 2026 20:25:49 +0000
Subject: [PATCH 10/15] eval: measure decision quality offline, and label what
 can't be re-measured
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The decision-quality metric could only be run against battles a live batch had
played into Postgres, with model attribution kept alongside in /tmp. That made
it expensive to re-measure, untestable, and — once those files were gone —
impossible to re-run at all.

decision-sim plays deterministic policies and writes the same export shape the
live path persists, so the pipeline runs from a checkout with no gateway, no
database, and no API spend. eval.CaptureStored is the piece that makes it
faithful: it stores state only after a turn has fully settled, replacements
folded in, because a leaked mid-turn replace state would shift the pre/post
pairing ScoreDecisions relies on and mis-attribute every decision after the
first KO.

What that bought, on library v2 (72 games, oracle depth 3):

  expectimax d2   win 44%   blunder  3%   median regret   0
  expectimax d1   win 61%   blunder 11%   median regret  12
  heuristic       win 56%   blunder 21%   median regret 111
  random          win  0%   blunder 39%   median regret 192

Blunder rate is monotone in policy strength. That is the metric's soundness
property and it had never been shown: validation to date covered data integrity
(does recovery reproduce the stored state), after which the metric was pointed
at four models whose true ordering nobody knows, so an inverted metric would
have gone unnoticed. It is now a test, not a table.

The same run exposes a limitation that was hiding in plain sight. Expectimax d2
blunders least and wins least -- it is not the strongest policy here, it is the
one most similar to the oracle, which is expectimax d3 (68% match rate against
the heuristic's 27%). The doc's fairness argument is about information: every
policy decides from the identical ai.MakeView projection. That argument is
correct and says nothing about algorithm, and algorithm turns out to matter.
This qualifies the published headline directly: "Gemini blunders least but wins
less than Opus" is structurally the same result, so it cannot be read as
"reasons more cleanly" without ruling out proximity to the yardstick. A deeper
oracle does not help -- it is a more expectimax-shaped one.

As a cross-check, d2's 44% independently reproduces benchmark.md §6's v2 figure
(42.9%, CI [36.8, 49.2]) from a separate code path, which is evidence the
offline capture reproduces live-shaped battles rather than merely plausible ones.

The four-model table itself is labelled v1-era and unrepeatable. It was measured
before library v2 gave every pick a nature and an EV spread, and unlike the depth
sweep it cannot be re-run: the battles and their attribution are gone, so
reproducing it means paying for a fresh batch across four vendors. Leaving v1
results adjacent to v2 ones with nothing to distinguish them is the exact failure
the v2 re-sweep was written up to avoid.

Also calibrates BlunderThreshold=300 against real data for the first time: random's
median regret is 192, below the bar, so it is a severe-tail cut rather than a
sloppiness detector. Constant left alone; the doc now says what it means.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 ...8-11T21-15-remeasuring-decision-quality.md | 128 +++++++++++++
 cmd/decision-sim/main.go                      | 177 ++++++++++++++++++
 docs/decision-quality-eval-handoff.md         |  15 ++
 docs/decision-quality.md                      |  93 ++++++++-
 internal/eval/decisioncapture.go              | 111 +++++++++++
 internal/eval/decisioncapture_test.go         | 154 +++++++++++++++
 6 files changed, 675 insertions(+), 3 deletions(-)
 create mode 100644 backlog/2026-08-11T21-15-remeasuring-decision-quality.md
 create mode 100644 cmd/decision-sim/main.go
 create mode 100644 internal/eval/decisioncapture.go
 create mode 100644 internal/eval/decisioncapture_test.go

diff --git a/backlog/2026-08-11T21-15-remeasuring-decision-quality.md b/backlog/2026-08-11T21-15-remeasuring-decision-quality.md
new file mode 100644
index 00000000..bc43ff71
--- /dev/null
+++ b/backlog/2026-08-11T21-15-remeasuring-decision-quality.md
@@ -0,0 +1,128 @@
+# Re-measuring decision quality, and what wouldn't re-measure
+
+PR #109 had been open since 17 July and was 46 commits behind. The plan was
+"rebase it and re-measure on library v2." The rebase was the easy half; the
+re-measure turned out to be two different questions with two different answers,
+and separating them is most of what this round produced.
+
+## The rebase was free, which is worth noting
+
+Eight commits, 1367 lines, 46 commits of drift — and exactly one file
+overlapped with anything `main` had touched (`internal/ai/expectimax.go`, where
+main changed the replace-phase handling and the PR appended `ScoreActions`).
+Cherry-picked clean, built, and the whole suite went green with no edits.
+
+I had assumed a branch that stale would need real work, and it didn't, because
+the PR is almost entirely *additive* — a new package file, a new command, a new
+report section. That is a property of how it was scoped, not luck. Worth
+remembering the next time a branch's age is used as an argument for abandoning
+it: age is a proxy for conflict risk, and a purely additive branch barely
+accumulates any.
+
+## The published table can't be re-measured, and saying so is the deliverable
+
+`docs/decision-quality.md` carried a four-model table — Gemini / Opus / Sonnet /
+Haiku, blunder rate and median regret — measured in July on v1 neutral teams.
+Library v2 landed 4 August. So those numbers describe a format that no longer
+ships, which is precisely the failure the
+[v2 re-sweep entry](2026-08-04T17-51-resweeping-expectimax-on-v2.md) was written
+up to prevent, recurring in a different document three weeks later.
+
+The depth sweep could be re-measured because it runs offline. This one can't:
+the battles lived in a Postgres on the author's machine and their model
+attribution in `/tmp/pk-agentic-v2`, and the handoff doc records that `/tmp`
+being wiped had *already* forced one re-run. Both are gone now. Reproducing the
+table means paying for a fresh batch across four vendors — a decision about
+money, not an afternoon of compute.
+
+So the table is labelled v1-era and unrepeatable, with the reason, next to the
+v2 numbers rather than instead of them. Same treatment the pre-fix depth numbers
+got in August. The thing I want to avoid is a reader assuming one run produced
+everything on the page.
+
+## What *could* be measured, and the bias it exposed
+
+The metric itself had never been shown to rank correctly. It was validated on
+"does recovery reproduce the stored state" — a data-integrity property — and
+then pointed straight at four models, where nobody knows the true ordering. If
+the metric were subtly inverted, nothing in the pipeline would have said so.
+
+`cmd/decision-sim` closes that: it plays deterministic policies offline and
+writes the same export shape the live path persists, so the whole pipeline runs
+from a checkout with no gateway, no database, and no API spend. 72 games on v2:
+
+| policy | win rate | blunder rate | median regret |
+|---|---:|---:|---:|
+| expectimax d2 | 44% | 3% | 0 |
+| expectimax d1 | 61% | 11% | 12 |
+| heuristic | 56% | 21% | 111 |
+| random | 0% | 39% | 192 |
+
+Blunder rate is monotone in policy strength. Good — that's the soundness check,
+and it's now a test rather than a hope.
+
+But look at the first two rows. **Expectimax d2 blunders least and wins least.**
+It is not the best player in the table; it is the player most *similar to the
+oracle*, which is expectimax d3. Its 68% match rate against the heuristic's 27%
+is the same fact stated louder.
+
+That is a real limitation and it was hiding in plain sight. The doc's central
+fairness argument is that every policy is scored from `ai.MakeView` — the
+identical fog-of-war projection — so the oracle is "a better player looking at
+the same information, not an omniscient one." That argument is correct and it is
+about *information*. It says nothing about *algorithm*, and algorithm turns out
+to matter: agreement with an expectimax oracle partly measures being an
+expectimax.
+
+The uncomfortable part is what that does to the doc's headline finding. "Gemini
+blunders least but wins less than Opus" is structurally identical to "d2 blunders
+least but wins least," and in the case where I know the cause, the cause is
+kinship with the yardstick. That doesn't explain the model result away — no LLM
+is running expectimax — but it means the finding cannot be read as
+"Gemini reasons more cleanly" without an argument that its style isn't simply
+closer to the oracle's. Separating those needs a second oracle of a different
+family, and raising the depth won't do it: a deeper oracle is a *more*
+expectimax-shaped one.
+
+## A cross-check I didn't plan and am glad I ran
+
+Expectimax d2 came out at 44% against the heuristic. `docs/benchmark.md` §6 puts
+it at 42.9% [36.8, 49.2] on v2. Those are the same number, produced by two
+entirely separate code paths — the `bench` match runner there, `CaptureStored`
+plus a fresh export/score pipeline here.
+
+I ran it as a sanity check on my harness. It is better than that: it is evidence
+the offline capture reproduces live-shaped battles faithfully, which is the
+assumption the whole `decision-sim` path rests on. Without it, "the metric works
+offline" would have been an assertion about code I had just written.
+
+## Threshold calibration, briefly
+
+`BlunderThreshold = 300` was flagged in the original handoff as a first guess to
+calibrate once the data existed. On v2: random's *median* regret is 192, below
+the bar. So 300 is not "notices sloppiness" — it is a severe-tail cut, and even a
+policy choosing uniformly at random sits under it half the time. That is
+defensible for a headline metric, and it is exactly why median regret belongs in
+the table next to it rather than behind it. Left the constant alone; the number
+is fine once you know what it means, and now the doc says.
+
+## The ranking is now a test, not a table
+
+I first wrote this entry with "wire the ordering into CI" as the open item, then
+noticed the objection to that: the monotonicity above is the metric's *only*
+soundness property, and leaving it as a number in a document means it is checked
+whenever someone re-reads the document. It is the thing most likely to break
+silently if the oracle or the value function moves.
+
+`TestScoreDecisions_RanksAWorsePolicyAsBlunderingMore` pins it — random against
+heuristic, one team, a depth-2 oracle, 0.8 seconds. Coarse on purpose: the
+property is the *direction* of the ranking, and the widest available gap is the
+one least likely to flake. The four-policy sweep stays in `decision-sim` where
+it can afford the nine minutes.
+
+## Still open
+
+A second oracle of a different family. Everything above says the yardstick's
+algorithm is a confound, and nothing in the repo can currently measure how big
+that confound is — the honest statement is "this bias exists and is unquantified,"
+which is weaker than I'd like to leave a caveat.
diff --git a/cmd/decision-sim/main.go b/cmd/decision-sim/main.go
new file mode 100644
index 00000000..81d92a93
--- /dev/null
+++ b/cmd/decision-sim/main.go
@@ -0,0 +1,177 @@
+// Command decision-sim generates decision-quality inputs offline.
+//
+// The decision-quality metric reads battles in the shape the live path
+// persists, which used to mean the only way to produce them was to play a batch
+// against real model endpoints and export it from Postgres. That made the
+// metric expensive to re-measure, impossible to re-run after the attribution
+// files were lost, and untestable in CI.
+//
+// This plays the same battles with deterministic policies and writes the same
+// export shape, plus the manifest `decision-eval -manifest` already consumes.
+// No gateway, no database, no API spend, and a fixed seed reproduces a run
+// exactly — so the numbers below a published table can always be re-derived.
+//
+// Usage:
+//
+//	decision-sim -out /tmp/dq -games 4
+//	decision-eval -manifest /tmp/dq/manifest.tsv -depth 3
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"log"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+	"pokearena/internal/eval"
+)
+
+// export mirrors cmd/decision-eval's input shape, which is itself the shape
+// db-replay consumes. Kept as a local struct rather than shared so the two
+// commands stay independently readable; the field tags are the contract.
+type export struct {
+	Seed   int64  `json:"seed"`
+	Winner int    `json:"winner"`
+	Turns  []turn `json:"turns"`
+}
+
+type turn struct {
+	State json.RawMessage `json:"state"`
+	Log   json.RawMessage `json:"log"`
+}
+
+// randomPolicySeed fixes the random policy's RNG so a run is reproducible.
+const randomPolicySeed = 20260811
+
+// policy is one labelled contestant for the scored seat. The label becomes the
+// "model" column in the aggregated table.
+type policy struct {
+	label string
+	build func(*domain.Dex) ai.Agent
+}
+
+func policies(dex *domain.Dex, spec string) ([]policy, error) {
+	all := map[string]policy{
+		"heuristic":     {"heuristic", func(d *domain.Dex) ai.Agent { return ai.NewHeuristicAgent(d) }},
+		"expectimax-d1": {"expectimax-d1", func(d *domain.Dex) ai.Agent { return ai.NewExpectimaxAgentFixed(d, 1) }},
+		"expectimax-d2": {"expectimax-d2", func(d *domain.Dex) ai.Agent { return ai.NewExpectimaxAgentFixed(d, 2) }},
+		"expectimax-d3": {"expectimax-d3", func(d *domain.Dex) ai.Agent { return ai.NewExpectimaxAgentFixed(d, 3) }},
+		// A fixed RNG seed, rebuilt per game, keeps even the random floor
+		// reproducible — the battle seed is what makes its games differ.
+		"random": {"random", func(*domain.Dex) ai.Agent { return ai.NewRandomAgent(randomPolicySeed) }},
+	}
+	var out []policy
+	for _, name := range strings.Split(spec, ",") {
+		name = strings.TrimSpace(name)
+		if name == "" {
+			continue
+		}
+		p, ok := all[name]
+		if !ok {
+			return nil, fmt.Errorf("unknown policy %q", name)
+		}
+		out = append(out, p)
+	}
+	if len(out) == 0 {
+		return nil, fmt.Errorf("no policies selected")
+	}
+	return out, nil
+}
+
+func main() {
+	log.SetFlags(0)
+	log.SetPrefix("[decision-sim] ")
+
+	dataDir := flag.String("data", "data", "dex data directory")
+	teamsPath := flag.String("teams", "data/benchmark-teams.json", "team library to play")
+	outDir := flag.String("out", "", "directory to write exports and manifest.tsv (required)")
+	games := flag.Int("games", 2, "games per policy per team")
+	oppName := flag.String("opponent", "heuristic", "policy occupying the unscored seat (side 1)")
+	policySpec := flag.String("policies", "heuristic,expectimax-d1,expectimax-d2", "comma-separated policies for the scored seat (side 0)")
+	flag.Parse()
+
+	if *outDir == "" {
+		log.Fatal("-out is required")
+	}
+
+	dex, err := domain.LoadDex(*dataDir, "bench")
+	if err != nil {
+		log.Fatalf("load dex: %v", err)
+	}
+	lib, err := eval.LoadTeamLibrary(*teamsPath, dex)
+	if err != nil {
+		log.Fatalf("load team library: %v", err)
+	}
+	ps, err := policies(dex, *policySpec)
+	if err != nil {
+		log.Fatalf("policies: %v", err)
+	}
+	opps, err := policies(dex, *oppName)
+	if err != nil {
+		log.Fatalf("opponent: %v", err)
+	}
+	opp := opps[0]
+
+	if err := os.MkdirAll(*outDir, 0o755); err != nil {
+		log.Fatalf("mkdir %s: %v", *outDir, err)
+	}
+
+	// Every policy plays the same team on the same seed, so the table's rows
+	// differ only by the policy in the scored seat — the mirror-match discipline
+	// the rest of the benchmark uses, applied to decisions instead of wins.
+	seeds := eval.SeedRange(*games)
+
+	var manifest strings.Builder
+	fmt.Fprintf(&manifest, "# decision-sim: library %s, opponent %s, %d games/team\n",
+		lib.Version, opp.label, *games)
+
+	total := 0
+	for _, p := range ps {
+		for _, team := range lib.Teams {
+			for _, seed := range seeds {
+				agents := [2]ai.Agent{p.build(dex), opp.build(dex)}
+				picks := [2][]engine.TeamPick{team.Picks, team.Picks}
+				res, turns, err := eval.CaptureStored(dex, agents, picks, seed, 0)
+				if err != nil {
+					log.Fatalf("%s / %s / seed %d: %v", p.label, team.Name, seed, err)
+				}
+				name := fmt.Sprintf("%s-%s-%d.json", p.label, sanitize(team.Name), seed)
+				path := filepath.Join(*outDir, name)
+				if err := writeExport(path, res.Winner, seed, turns); err != nil {
+					log.Fatalf("write %s: %v", path, err)
+				}
+				fmt.Fprintf(&manifest, "%s\t%s\n", p.label, path)
+				total++
+			}
+		}
+		log.Printf("%s: %d games", p.label, len(lib.Teams)*len(seeds))
+	}
+
+	manifestPath := filepath.Join(*outDir, "manifest.tsv")
+	if err := os.WriteFile(manifestPath, []byte(manifest.String()), 0o644); err != nil {
+		log.Fatalf("write manifest: %v", err)
+	}
+	log.Printf("wrote %d games + %s (library %s)", total, manifestPath, lib.Version)
+}
+
+func writeExport(path string, winner int, seed uint64, turns []eval.StoredTurn) error {
+	ex := export{Seed: int64(seed), Winner: winner, Turns: make([]turn, len(turns))}
+	for i, t := range turns {
+		ex.Turns[i] = turn{State: t.State, Log: t.Log}
+	}
+	b, err := json.Marshal(ex)
+	if err != nil {
+		return err
+	}
+	return os.WriteFile(path, b, 0o644)
+}
+
+func sanitize(s string) string {
+	return strings.ToLower(strings.ReplaceAll(s, " ", "-"))
+}
diff --git a/docs/decision-quality-eval-handoff.md b/docs/decision-quality-eval-handoff.md
index dd9de71c..3f74b199 100644
--- a/docs/decision-quality-eval-handoff.md
+++ b/docs/decision-quality-eval-handoff.md
@@ -1,5 +1,20 @@
 # Handoff — Decision-Quality Eval (per-model reasoning scoring)
 
+> **Historical. 2026-08-11.** The work this hands off is done and landed; the
+> remaining items below (finish the batch, build the aggregation, wire the
+> report, open the PR) are all closed. Read it as a design-and-gotcha record,
+> not a task list.
+>
+> Two of its environment facts are now actively misleading. The `/tmp/pk-agentic-v2`
+> attribution files and the local Postgres it names **no longer exist**, which is
+> why the per-model table in [decision-quality.md](decision-quality.md) is
+> labelled v1-era and unrepeatable. And the metric no longer *needs* any of that
+> infrastructure to run: `cmd/decision-sim` produces the same export shape from
+> deterministic offline games, which is how the v2 calibration was measured.
+>
+> The "no Claude co-author trailers" convention recorded at the bottom is also
+> stale — the repo's history carries them.
+
 Status as of 2026-07-17. Branch: `feat/decision-quality-eval` (based on the
 now-merged `feat/replay-samples`; PR #108 already merged to `main`).
 
diff --git a/docs/decision-quality.md b/docs/decision-quality.md
index ed4be848..1fff16d4 100644
--- a/docs/decision-quality.md
+++ b/docs/decision-quality.md
@@ -49,9 +49,59 @@ We roll these per-decision regrets up per model:
   low match rate with low regret just means the position had many good options,
   not that the model erred.
 
-## What the numbers say
-
-From the fresh attributed batch — twelve live games per model, 48 battles, 945
+## Does the metric actually rank policies correctly?
+
+Before trusting it on models, it should order players whose relative strength is
+already known. It does. 72 offline games on **team library v2** — six teams × 3
+seeds × 4 policies, each in the scored seat against the same heuristic opponent,
+oracle at depth 3:
+
+| policy | win rate (95% CI) | decisions | blunder rate | match rate | median regret |
+|---|---|---|---:|---:|---:|
+| expectimax d2 | 44% [25, 66] | 513 | **3%** | 68% | 0 |
+| expectimax d1 | 61% [39, 80] | 596 | 11% | 43% | 12 |
+| heuristic | 56% [34, 75] | 602 | 21% | 27% | 111 |
+| random | 0% [0, 18] | 466 | 39% | 22% | 192 |
+
+**Blunder rate is monotone in policy strength** — random worst, then heuristic,
+then the two searches. That is the soundness property the metric needed and had
+never been shown to have, and it is now a cheap regression test rather than a
+claim (`decision-sim`, below, needs no infrastructure and no API spend).
+
+Two things fall out of it that matter for reading any decision-quality table:
+
+**The oracle is biased toward its own family.** Expectimax d2 has the *best*
+blunder rate (3%) and match rate (68%) while posting the *worst* win rate of the
+three real policies (44%). It is not the strongest player here; it is the player
+most similar to the yardstick, because the yardstick is expectimax d3. Agreement
+with the oracle therefore measures algorithmic kinship as well as quality, and
+the fog-of-war fairness the method guarantees does nothing about that — it is a
+different axis of fairness entirely. See the caveats.
+
+**The threshold is a severe cut, not a sloppiness detector.** At 300, even the
+random policy's *median* regret is 192 — below the bar. Blunder rate is counting
+a genuinely bad tail, which is what it was meant to do, and it is why median
+regret is reported next to it rather than instead of it.
+
+As a cross-check on the harness itself, expectimax d2's 44% independently
+reproduces the v2 depth-sweep figure in [benchmark.md §6](benchmark.md) (42.9%,
+95% CI [36.8, 49.2]) from a completely different code path — offline capture
+here, the `bench` runner there.
+
+## What the model numbers said
+
+> **These are v1-era numbers and cannot be re-run.** They were measured before
+> [team library v2](../backlog/2026-08-04T03-40-library-v2-trained-spreads.md)
+> gave every pick a nature and an EV spread, so they describe a format that is no
+> longer what ships. Unlike the depth sweep — which was re-measured on v2 — these
+> cannot be: the battles lived in a local Postgres and their model attribution in
+> `/tmp`, and both are gone. Reproducing the table means paying for a fresh batch
+> across four vendors, not re-running a script. They are kept here because the
+> *shape* of the finding is the point, and labelled because the alternative is
+> leaving v1 results sitting next to v2 ones with nothing to tell them apart —
+> the exact failure the v2 re-sweep was written up to avoid.
+
+From the attributed batch — twelve live games per model, 48 battles, 945
 scored decisions:
 
 | model | win rate | blunder rate | median regret |
@@ -79,6 +129,15 @@ wrong — they measure different things, and a benchmark that only reported win
 rate would have told you Opus is simply better, full stop, and missed that Gemini
 makes fewer mistakes per move.
 
+**One caveat on that reading, added after the v2 calibration above.** "Blunders
+least, wins less" is structurally the same result expectimax d2 produces against
+this oracle — and there the cause is known to be kinship with the yardstick
+rather than cleaner play. That does not explain away the Opus/Gemini gap (no
+language model is an expectimax), but it does mean the gap is not self-evidently
+"Gemini reasons more cleanly." It could also be that Gemini's style happens to
+sit closer to the oracle's. Separating those needs a second oracle of a
+different family, which does not exist yet.
+
 ## Caveats
 
 - **Small n.** Twelve games per model. These numbers are directional, not
@@ -88,11 +147,39 @@ makes fewer mistakes per move.
   a strong Gen-1 player but not a solver; a deeper oracle would lower everyone's
   blunder rate and could reorder the close rows. What matters is that *every*
   model is measured against the *same* yardstick from the *same* view.
+- **The yardstick has a family.** Fog-of-war fairness is by construction, but
+  *algorithmic* fairness is not: a policy that searches the way the oracle
+  searches will agree with it more often for reasons unrelated to strength. The
+  v2 calibration shows this directly — expectimax d2 blunders least while
+  winning least. Read a low blunder rate as "close to this oracle," and only
+  then as "good," and treat gaps between two policies of the *same* family as
+  more meaningful than gaps across families.
+- **A stronger oracle is not free of this.** Raising the depth makes the yardstick
+  better *and* more expectimax-shaped, so it does not resolve the bias. A second
+  oracle of a different family (a trained policy, say) is what would.
 - **The threshold is a knob.** "Blunder" is regret above a fixed cut; the match
   and median columns are there so the picture doesn't hinge on that one number.
 
 ## Reproduce it
 
+**No infrastructure needed.** `decision-sim` plays deterministic policies and
+writes the same export shape the live path persists, so the whole pipeline runs
+from a checkout — this is how the v2 calibration table above was produced, and
+re-running it with the same flags reproduces it exactly:
+
+```sh
+go run ./cmd/decision-sim -out /tmp/dq -games 3 \
+  -policies random,heuristic,expectimax-d1,expectimax-d2
+go run ./cmd/decision-eval -manifest /tmp/dq/manifest.tsv -depth 3
+```
+
+Scoring dominates the wall clock (the oracle searches every legal action at
+every decision): the run above is 72 games and takes ~9 minutes, nearly all of
+it in `decision-eval`.
+
+**From live battles** — the path that produced the model table, and the one that
+needs a gateway, a Postgres, and real API keys:
+
 ```sh
 # Score one battle (stdin/-in is the {seed,winner,turns} export db-replay uses):
 decision-eval -in battle.json -side 0 -depth 3
diff --git a/internal/eval/decisioncapture.go b/internal/eval/decisioncapture.go
new file mode 100644
index 00000000..fdab62e3
--- /dev/null
+++ b/internal/eval/decisioncapture.go
@@ -0,0 +1,111 @@
+package eval
+
+import (
+	"encoding/json"
+	"fmt"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+)
+
+// Decision-quality scoring reads battles in the shape the live path persists:
+// one marshaled engine.BattleState per turn, replacements folded in. That shape
+// used to be obtainable only from Postgres, after a live batch had been played
+// against real model endpoints — which made the metric expensive to re-measure
+// and impossible to regression-test.
+//
+// CaptureStored produces the same shape from an offline game. Any ai.Agent can
+// take the scored seat, so the pipeline can be exercised end to end with
+// deterministic policies: no gateway, no database, no API spend, and a fixed
+// seed gives the same battle every run.
+
+// CaptureStored plays one game and records the battle state after every settled
+// turn, in the layout ScoreDecisions expects: index 0 is the opening position
+// (the state turn 1 is chosen from), and index i is the state after turn i.
+//
+// "Settled" is the load-bearing word. A KO leaves the engine in PhaseReplace
+// mid-turn, and the live coordinator resolves that before it persists — so a
+// stored state is only ever choosing or ended, and the pre-decision state for
+// turn N is the stored state at N-1. Capturing the intermediate replace states
+// would break that indexing and silently shift every decision by one.
+func CaptureStored(dex *domain.Dex, agents [2]ai.Agent, teams [2][]engine.TeamPick, seed uint64, budget Budget) (GameResult, []StoredTurn, error) {
+	id := fmt.Sprintf("eval-%d", seed)
+	s, err := engine.NewBattleFromPicks(dex, id, "P0", teams[0], "P1", teams[1], seed)
+	if err != nil {
+		return GameResult{}, nil, fmt.Errorf("new battle: %w", err)
+	}
+
+	res := GameResult{Seed: seed}
+	opening, err := storedTurn(s, nil)
+	if err != nil {
+		return res, nil, err
+	}
+	turns := []StoredTurn{opening}
+
+	for !s.Ended() {
+		if len(res.Decisions) > maxDecisions {
+			return res, turns, fmt.Errorf("battle %s did not terminate within %d decisions", id, maxDecisions)
+		}
+		if s.Phase != engine.PhaseChoosing {
+			// Replacements are consumed inside the turn below, so the loop head
+			// should only ever see a choosing state or a finished battle.
+			return res, turns, fmt.Errorf("battle %s in unexpected phase %q", id, s.Phase)
+		}
+
+		var acts [2]engine.Action
+		for side := 0; side < 2; side++ {
+			d := decide(dex, agents[side], s, side, budget)
+			acts[side] = d.Action
+			res.Usage[side] = res.Usage[side].Add(d.Usage)
+			res.Decisions = append(res.Decisions, d)
+		}
+		logs := engine.ResolveTurn(dex, s, acts)
+
+		// Drive the replacement cascade to completion: a replacement can itself
+		// faint the incoming Pokémon (entry hazards), so this is a loop, not an
+		// if. Its log lines belong to this turn and are appended to it.
+		for !s.Ended() && s.Phase == engine.PhaseReplace {
+			if len(res.Decisions) > maxDecisions {
+				return res, turns, fmt.Errorf("battle %s did not terminate within %d decisions", id, maxDecisions)
+			}
+			var sw [2]*engine.Action
+			for side := 0; side < 2; side++ {
+				if !s.Replace[side] {
+					continue
+				}
+				d := decide(dex, agents[side], s, side, budget)
+				a := d.Action
+				sw[side] = &a
+				res.Usage[side] = res.Usage[side].Add(d.Usage)
+				res.Decisions = append(res.Decisions, d)
+			}
+			logs = append(logs, engine.ResolveReplace(s, sw)...)
+		}
+
+		st, err := storedTurn(s, logs)
+		if err != nil {
+			return res, turns, err
+		}
+		turns = append(turns, st)
+	}
+
+	res.Winner = s.Winner
+	res.Turns = s.Turn
+	return res, turns, nil
+}
+
+func storedTurn(s *engine.BattleState, logs []engine.LogLine) (StoredTurn, error) {
+	state, err := json.Marshal(s)
+	if err != nil {
+		return StoredTurn{}, fmt.Errorf("marshal state: %w", err)
+	}
+	if logs == nil {
+		logs = []engine.LogLine{}
+	}
+	log, err := json.Marshal(logs)
+	if err != nil {
+		return StoredTurn{}, fmt.Errorf("marshal log: %w", err)
+	}
+	return StoredTurn{State: state, Log: log}, nil
+}
diff --git a/internal/eval/decisioncapture_test.go b/internal/eval/decisioncapture_test.go
new file mode 100644
index 00000000..b60f38ee
--- /dev/null
+++ b/internal/eval/decisioncapture_test.go
@@ -0,0 +1,154 @@
+package eval
+
+import (
+	"encoding/json"
+	"testing"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/engine"
+)
+
+// The capture is only worth anything if the recovery can invert it. This plays
+// a real library-v2 game and asserts that ScoreDecisions re-derives, for every
+// scored turn, the exact action the agent actually chose — which is the
+// property the whole metric rests on and the one a hand-built fixture cannot
+// demonstrate.
+func TestCaptureStored_RoundTripsThroughActionRecovery(t *testing.T) {
+	d := loadDex(t)
+	lib, err := LoadTeamLibrary(libraryPath, d)
+	if err != nil {
+		t.Fatalf("load team library: %v", err)
+	}
+	picks := lib.Teams[0].Picks
+
+	// Heuristic vs heuristic: a real, varied line (unlike a move#0 mirror) that
+	// is still deterministic, so this test is reproducible.
+	agents := [2]ai.Agent{ai.NewHeuristicAgent(d), ai.NewHeuristicAgent(d)}
+	res, turns, err := CaptureStored(d, agents, [2][]engine.TeamPick{picks, picks}, 20260811, 0)
+	if err != nil {
+		t.Fatalf("capture: %v", err)
+	}
+	if res.Winner < 0 {
+		t.Fatalf("game did not finish: winner=%d", res.Winner)
+	}
+	if len(turns) < 5 {
+		t.Fatalf("captured %d turns, want a real game", len(turns))
+	}
+
+	// Every stored state must be choosing or ended — never a mid-turn replace.
+	// A leaked replace state would shift the pre/post pairing ScoreDecisions
+	// relies on, mis-attributing every decision after the first KO.
+	for i, st := range turns {
+		var s engine.BattleState
+		if err := json.Unmarshal(st.State, &s); err != nil {
+			t.Fatalf("turn %d: unmarshal: %v", i, err)
+		}
+		if s.Phase != engine.PhaseChoosing && s.Phase != engine.PhaseEnded {
+			t.Errorf("turn %d stored in phase %q, want choosing or ended", i, s.Phase)
+		}
+	}
+
+	// The stub oracle scores every legal action, so a decision is skipped only
+	// when recovery genuinely failed — which makes `skipped` a direct measure of
+	// round-trip fidelity rather than of oracle opinion.
+	scores, skipped, err := ScoreDecisions(d, allActionsOracle{}, 0, turns)
+	if err != nil {
+		t.Fatalf("score: %v", err)
+	}
+	if len(scores) == 0 {
+		t.Fatal("recovered no decisions at all")
+	}
+	if skipped > 0 {
+		t.Errorf("%d of %d turns failed to recover; capture and recovery disagree",
+			skipped, skipped+len(scores))
+	}
+
+	// Re-running the same seed must produce byte-identical states, or the
+	// metric is not reproducible and no published number from it can be checked.
+	_, again, err := CaptureStored(d, [2]ai.Agent{ai.NewHeuristicAgent(d), ai.NewHeuristicAgent(d)},
+		[2][]engine.TeamPick{picks, picks}, 20260811, 0)
+	if err != nil {
+		t.Fatalf("recapture: %v", err)
+	}
+	if len(again) != len(turns) {
+		t.Fatalf("replay produced %d turns, first run %d", len(again), len(turns))
+	}
+	for i := range turns {
+		if string(again[i].State) != string(turns[i].State) {
+			t.Fatalf("turn %d differs between identical-seed runs", i)
+		}
+	}
+}
+
+// allActionsOracle values every legal action equally. It exists so a test can
+// separate "recovery failed" from "the oracle declined to score this turn":
+// ScoreDecisions skips both, and only the first is a bug in the capture.
+type allActionsOracle struct{}
+
+func (allActionsOracle) ScoreActions(v ai.View) []ai.ActionValue {
+	acts := ai.LegalActions(v)
+	if len(acts) <= 1 {
+		return nil // mirror expectimax's "no free choice" contract
+	}
+	out := make([]ai.ActionValue, len(acts))
+	for i, a := range acts {
+		out[i] = ai.ActionValue{Action: a}
+	}
+	return out
+}
+
+// The metric's soundness property: a better policy must blunder less. It was
+// only ever validated on data integrity (does recovery reproduce the stored
+// state), then pointed at four models whose true ordering nobody knows — so a
+// subtly inverted metric would have gone unnoticed. This pins the ordering
+// against policies whose relative strength is not in question.
+//
+// Deliberately coarse: random vs heuristic is the widest gap available, scored
+// by a shallow oracle on one team, because the property under test is the
+// direction of the ranking, not its precise values. The full sweep across four
+// policies lives in cmd/decision-sim and is documented in docs/decision-quality.md.
+func TestScoreDecisions_RanksAWorsePolicyAsBlunderingMore(t *testing.T) {
+	if testing.Short() {
+		t.Skip("runs an expectimax oracle over a full game")
+	}
+	d := loadDex(t)
+	lib, err := LoadTeamLibrary(libraryPath, d)
+	if err != nil {
+		t.Fatalf("load team library: %v", err)
+	}
+	picks := lib.Teams[0].Picks
+	oracle := ai.NewExpectimaxAgentFixed(d, 2)
+
+	blunderRate := func(t *testing.T, label string, scored ai.Agent) float64 {
+		t.Helper()
+		agents := [2]ai.Agent{scored, ai.NewHeuristicAgent(d)}
+		_, turns, err := CaptureStored(d, agents, [2][]engine.TeamPick{picks, picks}, 4242, 0)
+		if err != nil {
+			t.Fatalf("%s: capture: %v", label, err)
+		}
+		scores, _, err := ScoreDecisions(d, oracle, 0, turns)
+		if err != nil {
+			t.Fatalf("%s: score: %v", label, err)
+		}
+		if len(scores) == 0 {
+			t.Fatalf("%s: no decisions scored", label)
+		}
+		blunders := 0
+		for _, s := range scores {
+			if s.Blunder {
+				blunders++
+			}
+		}
+		rate := float64(blunders) / float64(len(scores))
+		t.Logf("%s: %d/%d decisions blundered (%.0f%%)", label, blunders, len(scores), 100*rate)
+		return rate
+	}
+
+	random := blunderRate(t, "random", ai.NewRandomAgent(20260811))
+	heuristic := blunderRate(t, "heuristic", ai.NewHeuristicAgent(d))
+
+	if !(random > heuristic) {
+		t.Errorf("random blundered %.0f%% and heuristic %.0f%%: the metric does not rank a worse policy as worse",
+			100*random, 100*heuristic)
+	}
+}

From b53e6ffb77be854c05fdbe32cca7568a3853aaa4 Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Tue, 11 Aug 2026 21:12:58 +0000
Subject: [PATCH 11/15] eval: add a second oracle, and measure the bias instead
 of warning about it
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The previous commit flagged that scoring against an expectimax oracle conflates
playing well with searching the way the oracle searches, and left the size of
that effect unknown for want of a second oracle "which does not exist yet."

It did exist. The heuristic agent is the opposite family -- depth-0, no
lookahead, no opponent model -- and already scores every legal action internally
to pick its move. Exposing ScoreActions on it was ten lines. The error was
treating the oracle slot as "strongest available player," which admits no
candidates above expectimax d3; what the role actually needs is independence,
and a weaker judge from an unrelated family is far more informative than a
marginally stronger one from the same family.

Scoring the same 72 v2 battles against each:

                   vs expectimax d3   vs heuristic
  expectimax d2         3% (best)       19% (3rd)
  expectimax d1        11% (2nd)        13% (2nd)
  heuristic            21% (3rd)         2% (best)
  random               39% (worst)      22% (worst)

The three skilled policies rank in exactly opposite order; each judge crowns its
own family. Match rate states it plainly: the heuristic policy agrees with the
heuristic oracle 92% of the time and with expectimax 27% -- same player, same
games, same fog-of-war view.

So the bias is not a modest caveat, it is total. A single-oracle blunder rate
does not rank skilled policies; it reports proximity to the judge. What the
metric still supports is documented: a floor (both judges separate incompetent
from competent), within-family comparisons, and agreement across families --
which is the strongest available claim and requires two oracles.

This lands on the published headline. "Gemini blunders least but wins less than
Opus" is structurally identical to "expectimax d2 blunders least but wins least,"
where the cause is known to be kinship. No LLM is an expectimax, so the model
ordering is not shown to be wrong -- but the grounds for believing it are gone,
because the one time the ordering could be checked against an independent judge
it inverted. The doc now says that rather than the earlier softer version.

Second finding, from nearly shipping a broken test: at oracle depth 2 the
expectimax judge gives random 35% and heuristic 33%, a two-point margin -- it
cannot tell random play from competent play. At depth 3, 43% vs 15%. Depth 3 is
a floor below which the metric measures nothing, and that runs against the
intuition from benchmark.md §6, where expectimax wins fewer games as depth
rises. Playing well and judging well are different capabilities.

Tests assert what survives both judges (random is worst) and pin the family bias
itself, so a future change that makes a single oracle's blunder rate look like
an absolute score fails. The expectimax arm stays in decision-sim: a two-point
margin dressed up as a soundness property is worse than no test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 ...8-11T21-15-remeasuring-decision-quality.md |  75 +++++++++-
 cmd/decision-eval/main.go                     |  38 ++++-
 docs/decision-quality.md                      | 132 +++++++++++------
 internal/ai/heuristic.go                      |  34 +++++
 internal/eval/decisioncapture_test.go         | 139 ++++++++++++++----
 5 files changed, 338 insertions(+), 80 deletions(-)

diff --git a/backlog/2026-08-11T21-15-remeasuring-decision-quality.md b/backlog/2026-08-11T21-15-remeasuring-decision-quality.md
index bc43ff71..631e26cf 100644
--- a/backlog/2026-08-11T21-15-remeasuring-decision-quality.md
+++ b/backlog/2026-08-11T21-15-remeasuring-decision-quality.md
@@ -120,9 +120,76 @@ property is the *direction* of the ranking, and the widest available gap is the
 one least likely to flake. The four-policy sweep stays in `decision-sim` where
 it can afford the nine minutes.
 
+## The second oracle existed all along
+
+I closed the section above with "a second oracle of a different family would
+quantify the bias, and doesn't exist." That was wrong within the hour. The
+heuristic agent *is* a second family — depth-0, no lookahead, no opponent model
+— and it already scores every legal action internally to pick its own move.
+Exposing `ScoreActions` on it was ten lines.
+
+The lesson is not "look harder before declaring something missing," though that
+too. It is that I had been thinking of the oracle as *the strongest available
+player*, so the only candidates I considered were things stronger than
+expectimax d3 — of which there are none here. The requirement is not strength,
+it is **independence**. A weaker judge from a different family is far more
+informative than a marginally stronger one from the same family, because the
+question is whether two unrelated notions of "good move" agree.
+
+## What the second judge said
+
+Same 72 battles, scored twice:
+
+| policy | vs expectimax d3 | vs heuristic |
+|---|---:|---:|
+| expectimax d2 | 3% (best) | 19% (3rd) |
+| expectimax d1 | 11% (2nd) | 13% (2nd) |
+| heuristic | 21% (3rd) | 2% (best) |
+| random | 39% (worst) | 22% (worst) |
+
+The three skilled policies rank in **exactly opposite order**. Each judge crowns
+its own family. Match rate is the cleanest statement of it: the heuristic policy
+agrees with the heuristic oracle 92% of the time and with expectimax 27% — same
+player, same games, same fog-of-war projection.
+
+I expected the bias to be real and modest, something to note in a caveat. It is
+total. On this evidence a single-oracle blunder rate does not rank skilled
+policies at all; it reports proximity to the judge. The one finding that
+survives both is that random is worst.
+
+That is a much harsher result for the metric than I went looking for, and it
+lands directly on the doc's headline. "Gemini blunders least but wins less than
+Opus" is structurally identical to "expectimax d2 blunders least but wins least,"
+and in the case where the cause is knowable, the cause is kinship. No LLM is an
+expectimax, so this doesn't prove the model ordering wrong — it removes the
+grounds for believing it. The one time the ordering could be checked against an
+independent judge, it inverted.
+
+## The judge has to be strong enough to be a judge
+
+A second finding from the same runs, and one I nearly shipped a broken test
+over. I wrote the soundness test with a depth-2 oracle because depth 3 was slow,
+and it failed — random scored *better* than the heuristic. Not a flake: at depth
+2 the expectimax judge gives random 35% and heuristic 33%, a two-point margin.
+At depth 3, 43% vs 15%.
+
+So a depth-2 oracle cannot distinguish random play from competent play. Depth 3
+isn't a preference, it's the floor below which the metric measures nothing.
+
+The part worth keeping: this runs *against* the intuition from
+[§6](../docs/benchmark.md), where expectimax wins fewer games as depth rises.
+Playing well and judging well are different capabilities, and the depth that
+hurts one is required by the other. I would have assumed the depth sweep's
+conclusion transferred. It doesn't.
+
+The test now asserts the wide-margin, fast half (random vs heuristic under the
+heuristic judge, 22% vs 3%, one second) and documents why the expectimax arm
+lives in `decision-sim` instead. A two-point margin dressed up as a soundness
+property would have been worse than no test.
+
 ## Still open
 
-A second oracle of a different family. Everything above says the yardstick's
-algorithm is a confound, and nothing in the repo can currently measure how big
-that confound is — the honest statement is "this bias exists and is unquantified,"
-which is weaker than I'd like to leave a caveat.
+A third oracle that isn't hand-built. Expectimax and the heuristic are different
+families but the same author and the same era of thinking about this game; they
+could share blind spots. A trained policy would be the real test. Nothing in the
+repo is close to that today.
diff --git a/cmd/decision-eval/main.go b/cmd/decision-eval/main.go
index a8281169..91c2204e 100644
--- a/cmd/decision-eval/main.go
+++ b/cmd/decision-eval/main.go
@@ -48,10 +48,11 @@ func main() {
 	manifest := flag.String("manifest", "", "aggregate mode: a file of \"model<TAB>export.json\" lines → per-model table")
 	regretCap := flag.Float64("regret-cap", 3000, "winsorize regret at this cap for the mean (median is unaffected)")
 	asJSON := flag.Bool("json", false, "aggregate mode: emit the per-model table as JSON")
+	oracleName := flag.String("oracle", "expectimax", "reference policy to score against: expectimax|heuristic")
 	flag.Parse()
 
 	if *manifest != "" {
-		runAggregate(*manifest, *dataDir, *side, *depth, *regretCap, *asJSON)
+		runAggregate(*manifest, *dataDir, *oracleName, *side, *depth, *regretCap, *asJSON)
 		return
 	}
 
@@ -68,7 +69,10 @@ func main() {
 	if err != nil {
 		log.Fatalf("load dex: %v", err)
 	}
-	oracle := ai.NewExpectimaxAgentFixed(dex, *depth)
+	oracle, err := buildOracle(dex, *oracleName, *depth)
+	if err != nil {
+		log.Fatalf("oracle: %v", err)
+	}
 
 	turns := make([]eval.StoredTurn, len(ex.Turns))
 	for i, t := range ex.Turns {
@@ -118,18 +122,44 @@ func main() {
 
 func actStr(a engine.Action) string { return fmt.Sprintf("%s#%d", a.Kind, a.Index) }
 
+// buildOracle picks the reference policy a decision is scored against.
+//
+// Two families are available on purpose. Expectimax is the stronger player and
+// the default, but scoring only against it conflates playing well with
+// searching the way it searches — the bias documented in
+// docs/decision-quality.md, where expectimax d2 earns the best blunder rate
+// while winning the fewest games. The heuristic is depth-0 with no opponent
+// model, so running a batch through both and comparing the *orderings* is what
+// turns that bias from an acknowledged caveat into a measured quantity.
+//
+// The two score in unrelated units. Never compare a regret or blunder rate
+// across oracles as a magnitude; compare rank order and match rate.
+func buildOracle(dex *domain.Dex, name string, depth int) (eval.Oracle, error) {
+	switch name {
+	case "expectimax":
+		return ai.NewExpectimaxAgentFixed(dex, depth), nil
+	case "heuristic":
+		return ai.NewHeuristicAgent(dex), nil
+	default:
+		return nil, fmt.Errorf("unknown oracle %q (want expectimax or heuristic)", name)
+	}
+}
+
 // runAggregate scores every battle listed in the manifest and prints the
 // per-model decision-quality table. The manifest is one "model<TAB>export.json"
 // line per battle (blank lines and "#" comments ignored) — the driver script
 // builds it by mapping each attributed bid= to its model and exporting the
 // battle from Postgres. Loading the dex and building the oracle once keeps a
 // whole run's scoring on a single shared reference.
-func runAggregate(manifestPath, dataDir string, side, depth int, regretCap float64, asJSON bool) {
+func runAggregate(manifestPath, dataDir, oracleName string, side, depth int, regretCap float64, asJSON bool) {
 	dex, err := domain.LoadDex(dataDir, "bench")
 	if err != nil {
 		log.Fatalf("load dex: %v", err)
 	}
-	oracle := ai.NewExpectimaxAgentFixed(dex, depth)
+	oracle, err := buildOracle(dex, oracleName, depth)
+	if err != nil {
+		log.Fatalf("oracle: %v", err)
+	}
 
 	mf, err := os.ReadFile(manifestPath)
 	if err != nil {
diff --git a/docs/decision-quality.md b/docs/decision-quality.md
index 1fff16d4..26519906 100644
--- a/docs/decision-quality.md
+++ b/docs/decision-quality.md
@@ -63,31 +63,70 @@ oracle at depth 3:
 | heuristic | 56% [34, 75] | 602 | 21% | 27% | 111 |
 | random | 0% [0, 18] | 466 | 39% | 22% | 192 |
 
-**Blunder rate is monotone in policy strength** — random worst, then heuristic,
-then the two searches. That is the soundness property the metric needed and had
-never been shown to have, and it is now a cheap regression test rather than a
-claim (`decision-sim`, below, needs no infrastructure and no API spend).
-
-Two things fall out of it that matter for reading any decision-quality table:
-
-**The oracle is biased toward its own family.** Expectimax d2 has the *best*
-blunder rate (3%) and match rate (68%) while posting the *worst* win rate of the
-three real policies (44%). It is not the strongest player here; it is the player
-most similar to the yardstick, because the yardstick is expectimax d3. Agreement
-with the oracle therefore measures algorithmic kinship as well as quality, and
-the fog-of-war fairness the method guarantees does nothing about that — it is a
-different axis of fairness entirely. See the caveats.
-
-**The threshold is a severe cut, not a sloppiness detector.** At 300, even the
-random policy's *median* regret is 192 — below the bar. Blunder rate is counting
-a genuinely bad tail, which is what it was meant to do, and it is why median
-regret is reported next to it rather than instead of it.
+Blunder rate is monotone in policy strength — random worst, then heuristic, then
+the two searches. That is the soundness property the metric needed and had never
+been shown to have.
 
 As a cross-check on the harness itself, expectimax d2's 44% independently
 reproduces the v2 depth-sweep figure in [benchmark.md §6](benchmark.md) (42.9%,
 95% CI [36.8, 49.2]) from a completely different code path — offline capture
 here, the `bench` runner there.
 
+And then the ordering falls apart the moment you change the judge.
+
+## The judge decides the answer
+
+Expectimax is one family of player. The heuristic is another — depth-0, no
+lookahead, no opponent model. Scoring **the same 72 battles** against each:
+
+| policy | vs expectimax d3 | vs heuristic |
+|---|---:|---:|
+| expectimax d2 | **3%** (best) | 19% (3rd) |
+| expectimax d1 | 11% (2nd) | 13% (2nd) |
+| heuristic | 21% (3rd) | **2%** (best) |
+| random | 39% (worst) | 22% (worst) |
+
+**On the three skilled policies the two judges rank in exactly opposite order.**
+Each one crowns its own family. Match rate says it plainly: the heuristic policy
+matches the heuristic oracle's top pick 92% of the time and the expectimax
+oracle's 27% — same player, same games, same fog-of-war view. The number is
+about the judge.
+
+Only one finding survives both judges: **random is worst.**
+
+This is not a small caveat on the metric; it is a limit on what a single-oracle
+blunder rate can mean. Reading one as an absolute quality score is reading the
+oracle's family resemblance. What the metric can support:
+
+- **A floor.** Both judges separate incompetent play from competent play cleanly.
+- **Within-family comparisons.** d1 vs d2 against an expectimax oracle is
+  meaningful; heuristic vs d2 against that same oracle is not.
+- **Agreement across judges.** A policy both families rate well is genuinely
+  well-rated. That is the strongest claim available, and it needs two oracles.
+
+The fog-of-war fairness the method guarantees does nothing about any of this —
+information fairness and algorithmic fairness are different axes, and only the
+first was ever argued.
+
+## The oracle has to be deep enough to be a judge
+
+A separate finding from the same data, and a hard requirement rather than a
+caveat. At oracle depth 2 the expectimax judge gives random 35% and the
+heuristic 33% — a 2-point margin, meaning **it cannot tell a random player from
+a competent one**. At depth 3 the same comparison is 43% vs 15%.
+
+So depth 3 is not a tuning preference; below it the metric stops working. Note
+this cuts against intuition from [benchmark.md §6](benchmark.md), where
+expectimax *wins fewer games* as depth rises — playing well and judging well are
+not the same capability, and the depth that is bad for one is required for the
+other.
+
+## The threshold is a severe cut, not a sloppiness detector
+
+At 300, even the random policy's *median* regret is 192 — below the bar. Blunder
+rate counts a genuinely bad tail, which is what it was meant to do, and it is why
+median regret is reported next to it rather than instead of it.
+
 ## What the model numbers said
 
 > **These are v1-era numbers and cannot be re-run.** They were measured before
@@ -129,14 +168,18 @@ wrong — they measure different things, and a benchmark that only reported win
 rate would have told you Opus is simply better, full stop, and missed that Gemini
 makes fewer mistakes per move.
 
-**One caveat on that reading, added after the v2 calibration above.** "Blunders
-least, wins less" is structurally the same result expectimax d2 produces against
-this oracle — and there the cause is known to be kinship with the yardstick
-rather than cleaner play. That does not explain away the Opus/Gemini gap (no
-language model is an expectimax), but it does mean the gap is not self-evidently
-"Gemini reasons more cleanly." It could also be that Gemini's style happens to
-sit closer to the oracle's. Separating those needs a second oracle of a
-different family, which does not exist yet.
+**That reading no longer stands up, and the two-oracle result above is why.**
+"Blunders least, wins less" is exactly what expectimax d2 produces against this
+oracle, and there the cause is kinship with the judge rather than cleaner play.
+Swapping in a second judge reversed the ranking of every skilled policy.
+
+No language model is an expectimax, so this does not show the model ordering is
+*wrong*. It shows there is no evidence it is right: the one time we could check
+whether a single oracle's blunder ordering tracks quality, it didn't. The
+conclusion that survives is the one both judges agreed on — a floor. Haiku being
+worst on win rate *and* blunder rate is consistent with genuinely weaker play.
+The Opus/Gemini flip, which was the headline, needs both oracles to mean
+anything, and the battles it was computed from no longer exist.
 
 ## Caveats
 
@@ -147,16 +190,18 @@ different family, which does not exist yet.
   a strong Gen-1 player but not a solver; a deeper oracle would lower everyone's
   blunder rate and could reorder the close rows. What matters is that *every*
   model is measured against the *same* yardstick from the *same* view.
-- **The yardstick has a family.** Fog-of-war fairness is by construction, but
-  *algorithmic* fairness is not: a policy that searches the way the oracle
-  searches will agree with it more often for reasons unrelated to strength. The
-  v2 calibration shows this directly — expectimax d2 blunders least while
-  winning least. Read a low blunder rate as "close to this oracle," and only
-  then as "good," and treat gaps between two policies of the *same* family as
-  more meaningful than gaps across families.
-- **A stronger oracle is not free of this.** Raising the depth makes the yardstick
-  better *and* more expectimax-shaped, so it does not resolve the bias. A second
-  oracle of a different family (a trained policy, say) is what would.
+- **The yardstick has a family, and it decides the ranking.** Measured, not
+  suspected: two oracles of different families rank the same three skilled
+  policies in opposite orders. Never report a single-oracle blunder rate as a
+  quality score. Report the floor, within-family gaps, or agreement between
+  families.
+- **Depth 3 is a floor for the expectimax oracle.** At depth 2 it cannot
+  distinguish random play from competent play (2-point margin). A judge weaker
+  than this stops measuring anything.
+- **Two oracles is the minimum, not the ideal.** Expectimax and the heuristic
+  are both hand-built Gen-1 players and could share blind spots the way they
+  share an author. A third of a genuinely different kind — a trained policy —
+  would test that, and does not exist yet.
 - **The threshold is a knob.** "Blunder" is regret above a fixed cut; the match
   and median columns are there so the picture doesn't hinge on that one number.
 
@@ -170,12 +215,17 @@ re-running it with the same flags reproduces it exactly:
 ```sh
 go run ./cmd/decision-sim -out /tmp/dq -games 3 \
   -policies random,heuristic,expectimax-d1,expectimax-d2
-go run ./cmd/decision-eval -manifest /tmp/dq/manifest.tsv -depth 3
+
+# Score the same battles against each judge and compare the orderings.
+go run ./cmd/decision-eval -manifest /tmp/dq/manifest.tsv -oracle expectimax -depth 3
+go run ./cmd/decision-eval -manifest /tmp/dq/manifest.tsv -oracle heuristic
 ```
 
-Scoring dominates the wall clock (the oracle searches every legal action at
-every decision): the run above is 72 games and takes ~9 minutes, nearly all of
-it in `decision-eval`.
+Run both. A number from one judge alone is not interpretable — see above.
+
+Scoring dominates the wall clock, and only for the searching judge: the 72-game
+run takes ~9 minutes against expectimax d3 and ~7 *seconds* against the
+heuristic, because the latter does no lookahead.
 
 **From live battles** — the path that produced the model table, and the one that
 needs a gateway, a Postgres, and real API keys:
diff --git a/internal/ai/heuristic.go b/internal/ai/heuristic.go
index 7295bbc6..d7904f95 100644
--- a/internal/ai/heuristic.go
+++ b/internal/ai/heuristic.go
@@ -37,6 +37,40 @@ func (a *HeuristicAgent) Decide(ctx context.Context, v View) (engine.Action, err
 	return best, nil
 }
 
+// ScoreActions exposes the per-action scores Decide already ranks, so the
+// heuristic can serve as a decision-quality oracle alongside expectimax.
+//
+// Why a second oracle exists at all: scoring choices against expectimax alone
+// conflates "played well" with "searched like the oracle" — expectimax d2 posts
+// the best blunder rate against a d3 oracle while winning the fewest games (see
+// docs/decision-quality.md). This agent is the opposite family — depth-0, no
+// lookahead, no opponent model — so agreement with *both* is much stronger
+// evidence than agreement with either.
+//
+// Its values are hand-tuned damage points, NOT expectimax's eval points, and the
+// two scales are unrelated. Regrets and blunder rates from the two oracles must
+// never be compared as magnitudes; what is comparable is the *ordering* they
+// induce over policies, and match rate, which is scale-free.
+//
+// Contract matches ExpectimaxAgent.ScoreActions: nil where regret is undefined
+// (a forced replacement or a single legal action), and ties break toward the
+// first legal action exactly as Decide does, so the top-valued action here is
+// Decide's pick.
+func (a *HeuristicAgent) ScoreActions(v View) []ActionValue {
+	if v.Replace {
+		return nil
+	}
+	acts := LegalActions(v)
+	if len(acts) <= 1 {
+		return nil
+	}
+	out := make([]ActionValue, len(acts))
+	for i, act := range acts {
+		out[i] = ActionValue{Action: act, Value: a.score(v, act)}
+	}
+	return out
+}
+
 func (a *HeuristicAgent) score(v View, act engine.Action) float64 {
 	me := v.Self.Team[v.Self.Active]
 	foe := v.Foe
diff --git a/internal/eval/decisioncapture_test.go b/internal/eval/decisioncapture_test.go
index b60f38ee..cefdb52e 100644
--- a/internal/eval/decisioncapture_test.go
+++ b/internal/eval/decisioncapture_test.go
@@ -97,58 +97,135 @@ func (allActionsOracle) ScoreActions(v ai.View) []ai.ActionValue {
 	return out
 }
 
-// The metric's soundness property: a better policy must blunder less. It was
-// only ever validated on data integrity (does recovery reproduce the stored
-// state), then pointed at four models whose true ordering nobody knows — so a
-// subtly inverted metric would have gone unnoticed. This pins the ordering
-// against policies whose relative strength is not in question.
+// The metric's soundness property, and the limit of it.
 //
-// Deliberately coarse: random vs heuristic is the widest gap available, scored
-// by a shallow oracle on one team, because the property under test is the
-// direction of the ranking, not its precise values. The full sweep across four
-// policies lives in cmd/decision-sim and is documented in docs/decision-quality.md.
-func TestScoreDecisions_RanksAWorsePolicyAsBlunderingMore(t *testing.T) {
+// A single oracle cannot rank skilled policies: scoring the same 72 games
+// against an expectimax oracle and against the heuristic produces exactly
+// opposite orderings of the three non-random policies, because each oracle
+// rewards its own family (docs/decision-quality.md). So the ordering this test
+// pins is the one that survives both judges — random is worst — and it
+// deliberately does NOT assert anything about heuristic vs expectimax, which is
+// judge-dependent and would be a false guarantee.
+//
+// Decisions are pooled over several teams and seeds because the property is a
+// property of the *aggregate*, not of any one battle: on a single game the
+// heuristic judge rates random above heuristic often enough to flake, which is
+// itself worth knowing — a per-battle blunder rate is noise.
+func TestScoreDecisions_RanksRandomWorstThanASkilledPolicy(t *testing.T) {
 	if testing.Short() {
-		t.Skip("runs an expectimax oracle over a full game")
+		t.Skip("runs oracles over many full games")
 	}
 	d := loadDex(t)
 	lib, err := LoadTeamLibrary(libraryPath, d)
 	if err != nil {
 		t.Fatalf("load team library: %v", err)
 	}
-	picks := lib.Teams[0].Picks
-	oracle := ai.NewExpectimaxAgentFixed(d, 2)
+	teams := lib.Teams
+	if len(teams) > 3 {
+		teams = teams[:3]
+	}
+	seeds := SeedRange(2)
 
-	blunderRate := func(t *testing.T, label string, scored ai.Agent) float64 {
+	// pooledBlunderRate plays every (team, seed) with the policy in the scored
+	// seat and returns the blunder rate over all their decisions combined.
+	pooledBlunderRate := func(t *testing.T, label string, oracle Oracle, build func() ai.Agent) float64 {
 		t.Helper()
-		agents := [2]ai.Agent{scored, ai.NewHeuristicAgent(d)}
-		_, turns, err := CaptureStored(d, agents, [2][]engine.TeamPick{picks, picks}, 4242, 0)
-		if err != nil {
-			t.Fatalf("%s: capture: %v", label, err)
+		decisions, blunders := 0, 0
+		for _, team := range teams {
+			for _, seed := range seeds {
+				agents := [2]ai.Agent{build(), ai.NewHeuristicAgent(d)}
+				picks := [2][]engine.TeamPick{team.Picks, team.Picks}
+				_, turns, err := CaptureStored(d, agents, picks, seed, 0)
+				if err != nil {
+					t.Fatalf("%s / %s / %d: capture: %v", label, team.Name, seed, err)
+				}
+				scores, _, err := ScoreDecisions(d, oracle, 0, turns)
+				if err != nil {
+					t.Fatalf("%s / %s / %d: score: %v", label, team.Name, seed, err)
+				}
+				for _, s := range scores {
+					decisions++
+					if s.Blunder {
+						blunders++
+					}
+				}
+			}
+		}
+		if decisions == 0 {
+			t.Fatalf("%s: no decisions scored", label)
 		}
+		rate := float64(blunders) / float64(decisions)
+		t.Logf("%s: %d/%d decisions blundered (%.0f%%)", label, blunders, decisions, 100*rate)
+		return rate
+	}
+
+	// Scored against the heuristic oracle only, and that is not a shortcut.
+	// The expectimax oracle's ability to separate these two is entirely a
+	// function of depth: on this same sample it gives random 35% and heuristic
+	// 33% at depth 2 — a 2-point margin, i.e. it cannot tell a random player
+	// from a competent one — against 43% vs 15% at depth 3. Depth 3 is ~50s
+	// here, too slow to sit in the default suite, so the expectimax arm lives
+	// in cmd/decision-sim. The depth requirement itself is documented in
+	// docs/decision-quality.md; this test covers the cheap, wide-margin half.
+	oracle := ai.NewHeuristicAgent(d)
+	random := pooledBlunderRate(t, "random", oracle, func() ai.Agent { return ai.NewRandomAgent(20260811) })
+	skilled := pooledBlunderRate(t, "heuristic", oracle, func() ai.Agent { return ai.NewHeuristicAgent(d) })
+	if !(random > skilled) {
+		t.Errorf("random blundered %.0f%% and heuristic %.0f%%: the metric does not rank a worse policy as worse",
+			100*random, 100*skilled)
+	}
+}
+
+// Each oracle rates its own policy's choices as near-perfect. That is the
+// family bias stated as an executable fact rather than a caveat in prose: the
+// same agent, on the same games, is a 2%-blunder player to one judge and a
+// 21%-blunder player to the other. Any future change that makes a single
+// oracle's blunder rate look like an absolute quality score should fail here.
+func TestScoreDecisions_OracleFamilyBiasIsReal(t *testing.T) {
+	if testing.Short() {
+		t.Skip("runs an oracle over full games")
+	}
+	d := loadDex(t)
+	lib, err := LoadTeamLibrary(libraryPath, d)
+	if err != nil {
+		t.Fatalf("load team library: %v", err)
+	}
+	picks := lib.Teams[0].Picks
+
+	agents := [2]ai.Agent{ai.NewHeuristicAgent(d), ai.NewHeuristicAgent(d)}
+	_, turns, err := CaptureStored(d, agents, [2][]engine.TeamPick{picks, picks}, 4242, 0)
+	if err != nil {
+		t.Fatalf("capture: %v", err)
+	}
+
+	matchRate := func(t *testing.T, oracle Oracle) float64 {
+		t.Helper()
 		scores, _, err := ScoreDecisions(d, oracle, 0, turns)
 		if err != nil {
-			t.Fatalf("%s: score: %v", label, err)
+			t.Fatalf("score: %v", err)
 		}
 		if len(scores) == 0 {
-			t.Fatalf("%s: no decisions scored", label)
+			t.Fatalf("no decisions scored")
 		}
-		blunders := 0
+		agree := 0
 		for _, s := range scores {
-			if s.Blunder {
-				blunders++
+			if s.Agree {
+				agree++
 			}
 		}
-		rate := float64(blunders) / float64(len(scores))
-		t.Logf("%s: %d/%d decisions blundered (%.0f%%)", label, blunders, len(scores), 100*rate)
-		return rate
+		return float64(agree) / float64(len(scores))
 	}
 
-	random := blunderRate(t, "random", ai.NewRandomAgent(20260811))
-	heuristic := blunderRate(t, "heuristic", ai.NewHeuristicAgent(d))
+	// The scored policy IS the heuristic, so the heuristic oracle should almost
+	// always name the move it just watched being played, while the expectimax
+	// oracle — same information, same view — frequently prefers another.
+	own := matchRate(t, ai.NewHeuristicAgent(d))
+	other := matchRate(t, ai.NewExpectimaxAgentFixed(d, 2))
+	t.Logf("heuristic policy: %.0f%% match vs its own family, %.0f%% vs expectimax", 100*own, 100*other)
 
-	if !(random > heuristic) {
-		t.Errorf("random blundered %.0f%% and heuristic %.0f%%: the metric does not rank a worse policy as worse",
-			100*random, 100*heuristic)
+	if own <= other {
+		t.Errorf("same-family match %.0f%% did not exceed cross-family %.0f%%; "+
+			"the family-bias finding this metric is documented against no longer reproduces",
+			100*own, 100*other)
 	}
 }

From 7341c246971da347c6753265366ee9422f5a2826 Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Wed, 12 Aug 2026 00:55:24 +0000
Subject: [PATCH 12/15] eval: decision quality without a judge
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Scoring choices against a reference agent measures agreement with that agent:
two references of different families rank the same policies in opposite orders,
so blunder rate cannot be published as a quality score. This is the other
approach -- count only mistakes that are provably mistakes.

Four categories, each a guaranteed no-op rather than a strong player's opinion:
attacking into a type immunity, healing at full HP, inflicting a status the
target already has, and boosting a stat already pinned at +6. No reference
agent is consulted, so no choice of reference can flip the result.

Two rules keep it defensible, and both are load-bearing. Certainty: a category
is included only when the action cannot possibly accomplish anything, because a
false positive costs far more here than a missed detection -- the whole value is
that every count survives an audit. Knowability: a mistake counts only if the
player could have known from the fog-of-war view it held, which is why
ability-granted immunities (Levitate, Flash Fire) are not charged while type
immunities are. Penalising unknowable information would make this a luck metric.

It is a floor, not a full account of skill: a policy can score zero and still
play badly. That is the trade -- a number that means exactly what it says.

Validation is per category, not on a pooled rate, because the categories measure
different failures and different policies fail differently. The sharp claim:
the heuristic checks type effectiveness before attacking and records 0 immune
attacks over 220 decisions, while random records 4 over 118.

Two things the metric found immediately, both kept as findings rather than
quietly fixed:

The heuristic baseline has a real defect -- 27 boost-at-cap turns over six
games, and it re-applies Thunder Wave to an already-paralysed target for six
consecutive turns. It is NOT fixed here. The heuristic is the benchmark's
reference opponent, so changing how it plays invalidates every number measured
against it, the depth sweep in benchmark.md §6 above all. That is a decision
with benchmark-wide consequences and should be made deliberately, not as a
side effect of adding a metric. TestScoreVerifiable_HeuristicBoostsAtCap pins
the finding and documents that it should be deleted when the bot is fixed.

And a wrong assumption of mine, caught by the checker: Electric vs Ground is a
true 0x immunity, so Thunderbolt into a Rhydon is a provable error, not the
"resisted but not immune" control case I had written it as. The control now
uses a genuine 0.5x matchup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 internal/eval/verifiable.go      | 342 +++++++++++++++++++++++++++++++
 internal/eval/verifiable_test.go | 264 ++++++++++++++++++++++++
 2 files changed, 606 insertions(+)
 create mode 100644 internal/eval/verifiable.go
 create mode 100644 internal/eval/verifiable_test.go

diff --git a/internal/eval/verifiable.go b/internal/eval/verifiable.go
new file mode 100644
index 00000000..013c2261
--- /dev/null
+++ b/internal/eval/verifiable.go
@@ -0,0 +1,342 @@
+package eval
+
+import (
+	"encoding/json"
+	"fmt"
+	"sort"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+)
+
+// Verifiable errors: decision quality without a judge.
+//
+// Scoring a choice against a reference agent measures agreement with that
+// agent, and two references of different families rank the same policies in
+// opposite orders (see docs/decision-quality.md). So "blunder rate vs an
+// oracle" cannot be published as a quality score.
+//
+// This is the other approach: count only mistakes that are *provably* mistakes.
+// Not "a strong player would have done better" — that is an opinion — but "this
+// action could not possibly have accomplished anything," which is a fact about
+// the rules that no reviewer can argue with and no choice of reference can
+// flip.
+//
+// Two rules keep it honest, and both are load-bearing:
+//
+//  1. **Certainty.** A category is only included when the action is a guaranteed
+//     no-op. Anything that merely *usually* wastes a turn is left out. A false
+//     positive here is far more damaging than a missed one — the whole value of
+//     this metric is that every count survives scrutiny.
+//
+//  2. **Knowability.** A mistake only counts if the player could have known it
+//     was one from the fog-of-war view it was handed. Attacking into a Levitate
+//     immunity is a wasted turn, but the ability may be hidden, so it is not
+//     charged here; attacking a Ghost with a Normal move is charged, because the
+//     defender's types are public. Penalising unknowable information would make
+//     this a luck metric.
+//
+// The result is a floor, not a full account of skill: a policy can score zero
+// verifiable errors and still play badly. That is the intended trade. It is a
+// number that means exactly what it says.
+
+// ErrKind identifies a category of provable error. Kept as stable string
+// constants because they are reported per category and end up in published
+// tables.
+type ErrKind string
+
+const (
+	// ErrImmuneAttack is a damaging move whose type does nothing to the
+	// defender's types. Zero damage is guaranteed by the type chart, and both
+	// of the defender's types are visible.
+	ErrImmuneAttack ErrKind = "immune-attack"
+	// ErrHealAtFull is a pure healing move used at full HP. It restores
+	// nothing; the turn is spent for no effect.
+	ErrHealAtFull ErrKind = "heal-at-full"
+	// ErrRedundantStatus is a status move that inflicts the status the target
+	// already has. A Pokémon cannot hold two statuses, so it cannot land.
+	ErrRedundantStatus ErrKind = "redundant-status"
+	// ErrBoostAtCap is a self-boosting move whose every stat is already at the
+	// +6 ceiling. Nothing can move.
+	ErrBoostAtCap ErrKind = "boost-at-cap"
+)
+
+// VerifiableError is one provable mistake, with enough context to audit it.
+// Detail is written to be readable on its own in a report row, because the
+// point of this metric is that a skeptic can check any single count by hand.
+type VerifiableError struct {
+	Turn   int     `json:"turn"`
+	Side   int     `json:"side"`
+	Kind   ErrKind `json:"kind"`
+	Detail string  `json:"detail"`
+}
+
+// maxStage is the stat-stage ceiling; a boost move cannot move a stat past it.
+const maxStage = 6
+
+// CheckAction reports every provable error in playing act from view v. It
+// returns nil for a switch, for a replacement, and for any action it cannot
+// prove wasteful — which is the overwhelming majority.
+func CheckAction(dex *domain.Dex, v ai.View, act engine.Action) []VerifiableError {
+	if act.Kind != engine.ActionMove || act.Index < 0 {
+		return nil // switches are never provably wasted from the view alone
+	}
+	me := v.Self.Team[v.Self.Active]
+	if act.Index >= len(me.Moves) {
+		return nil
+	}
+	mv, ok := dex.Moves[me.Moves[act.Index].MoveID]
+	if !ok {
+		return nil
+	}
+	foe := v.Foe
+
+	var out []VerifiableError
+	add := func(k ErrKind, format string, args ...any) {
+		out = append(out, VerifiableError{
+			Turn: v.Turn, Side: v.Me, Kind: k, Detail: fmt.Sprintf(format, args...),
+		})
+	}
+
+	// 1. Attacking into a type immunity. Only the type chart is consulted:
+	// ability-granted immunities (Levitate, Flash Fire) are deliberately not
+	// charged because the attacker may not know the ability.
+	if mv.Category != domain.CatStatus && mv.Power > 0 {
+		if dex.Effectiveness(mv.Type, foe.Type1, foe.Type2) == 0 {
+			add(ErrImmuneAttack, "%s (%s) vs %s (%s): 0x by type",
+				mv.Name, mv.Type, foe.Name, typeLabel(foe))
+		}
+	}
+
+	// The rest are status moves that cannot do their one job. A move that also
+	// deals damage is never counted, because the damage is a real effect.
+	if mv.Category != domain.CatStatus {
+		return out
+	}
+
+	// 2. Healing at full HP. Rest is excluded when the user is statused: it
+	// cures, so the turn buys something even with the heal wasted.
+	if selfHeal(mv) > 0 && me.HP >= me.MaxHP {
+		if !(isRest(mv) && me.Status != engine.StatusNone) {
+			add(ErrHealAtFull, "%s at %d/%d HP", mv.Name, me.HP, me.MaxHP)
+		}
+	}
+
+	// 3. Inflicting a status the target already has.
+	if st := primaryStatus(mv); st != "" && foe.Status != engine.StatusNone {
+		if engine.StatusCond(st) != engine.StatusToxic || foe.Status != engine.StatusPoison {
+			add(ErrRedundantStatus, "%s targets %s which is already %s",
+				mv.Name, foe.Name, foe.Status)
+		}
+	}
+
+	// 4. Boosting stats that are all pinned at +6.
+	if boosts := selfBoosts(mv); len(boosts) > 0 && allAtCap(boosts, me.Stages) {
+		add(ErrBoostAtCap, "%s with every boosted stat already at +%d", mv.Name, maxStage)
+	}
+
+	return out
+}
+
+func typeLabel(p engine.Pokemon) string {
+	if p.Type2 == "" || p.Type2 == p.Type1 {
+		return string(p.Type1)
+	}
+	return string(p.Type1) + "/" + string(p.Type2)
+}
+
+// selfHeal returns the fraction of max HP the move restores to its user, or 0.
+// Drain is excluded on purpose: a draining move deals damage, so it is doing
+// something even when the heal is capped.
+func selfHeal(m domain.Move) float64 {
+	if m.Self != nil && m.Self.Heal > 0 {
+		return m.Self.Heal
+	}
+	// A self-targeting status move carries its heal on Primary.
+	if m.Primary != nil && m.Primary.Heal > 0 && m.Target == domain.TargetSelf {
+		return m.Primary.Heal
+	}
+	return 0
+}
+
+func isRest(m domain.Move) bool {
+	return (m.Self != nil && m.Self.Rest) || (m.Primary != nil && m.Primary.Rest)
+}
+
+// primaryStatus returns the status a move inflicts as its guaranteed primary
+// effect. Secondaries are ignored — they are chance-gated, so a move carrying
+// one still has its main effect to deliver.
+func primaryStatus(m domain.Move) string {
+	if m.Primary == nil || m.Primary.Status == "" {
+		return ""
+	}
+	// Chance < 100 means it can miss its own effect; not a guaranteed no-op.
+	if m.Primary.Chance != 0 && m.Primary.Chance < 100 {
+		return ""
+	}
+	return m.Primary.Status
+}
+
+// selfBoosts returns the stat boosts a move applies to its user. Only positive
+// boosts count: a move that lowers its own stats as a cost is not "capped."
+func selfBoosts(m domain.Move) map[string]int {
+	var src map[string]int
+	switch {
+	case m.Self != nil && len(m.Self.Boosts) > 0:
+		src = m.Self.Boosts
+	case m.Primary != nil && len(m.Primary.Boosts) > 0 && m.Target == domain.TargetSelf:
+		src = m.Primary.Boosts
+	default:
+		return nil
+	}
+	out := map[string]int{}
+	for stat, n := range src {
+		if n > 0 {
+			out[stat] = n
+		}
+	}
+	return out
+}
+
+// allAtCap reports whether every stat the move would raise is already at +6.
+// If even one has room, the move does something and is not charged.
+func allAtCap(boosts map[string]int, st engine.Stages) bool {
+	for stat := range boosts {
+		cur, known := stageOf(st, stat)
+		if !known {
+			return false // unrecognized stat key — never charge on a guess
+		}
+		if cur < maxStage {
+			return false
+		}
+	}
+	return true
+}
+
+// stageOf maps a move's boost key to the matching stat stage. The two
+// vocabularies differ (see the note in backlog/2026-08-03T20-01), so this is
+// deliberately explicit and returns known=false rather than defaulting.
+func stageOf(st engine.Stages, stat string) (int, bool) {
+	switch stat {
+	case "atk", "attack":
+		return st.Atk, true
+	case "def", "defense":
+		return st.Def, true
+	case "spa", "spatk":
+		return st.SpA, true
+	case "spd", "spdef":
+		return st.SpD, true
+	case "spe", "speed":
+		return st.Spe, true
+	case "accuracy", "acc":
+		return st.Acc, true
+	case "evasion", "eva":
+		return st.Eva, true
+	}
+	return 0, false
+}
+
+// VerifiableStats is one contestant's provable-error record over a batch. Rate
+// is the headline: errors per hundred decisions, so it is comparable across
+// contestants that played different numbers of games.
+type VerifiableStats struct {
+	Contestant string          `json:"contestant"`
+	Games      int             `json:"games"`
+	Decisions  int             `json:"decisions"`
+	Errors     int             `json:"errors"`
+	Per100     float64         `json:"errors_per_100_decisions"`
+	ByKind     map[ErrKind]int `json:"by_kind"`
+	// CleanGames is the count of games containing no provable error at all.
+	// Reported because a rate can hide a distribution: ten errors spread over
+	// ten games is a different failure from ten in one.
+	CleanGames int `json:"clean_games"`
+}
+
+// ScoreVerifiable walks a battle's stored turns and returns every provable
+// error the given side committed, plus the number of decisions examined.
+//
+// It reuses recoverActions, so it inherits the same exactness guarantee as the
+// oracle path: the action it checks is the action that was actually played,
+// re-derived by re-simulating the turn until the stored next state reproduces
+// byte-for-byte. skipped counts turns whose action could not be recovered.
+func ScoreVerifiable(dex *domain.Dex, side int, turns []StoredTurn) (errs []VerifiableError, decisions, skipped int, err error) {
+	for i := 1; i < len(turns); i++ {
+		var prev engine.BattleState
+		if e := json.Unmarshal(turns[i-1].State, &prev); e != nil {
+			return nil, decisions, skipped, e
+		}
+		if prev.Phase != engine.PhaseChoosing {
+			continue
+		}
+		var next engine.BattleState
+		if e := json.Unmarshal(turns[i].State, &next); e != nil {
+			return nil, decisions, skipped, e
+		}
+		acts, ok := recoverActions(dex, turns[i-1].State, &next)
+		if !ok {
+			skipped++
+			continue
+		}
+		// The view is the one the player actually decided from, so a mistake is
+		// only charged on information it held.
+		v := ai.MakeView(&prev, side)
+		decisions++
+		errs = append(errs, CheckAction(dex, v, acts[side])...)
+	}
+	return errs, decisions, skipped, nil
+}
+
+// AggregateVerifiable folds per-battle results into one row per contestant,
+// sorted by error rate ascending (cleanest first).
+func AggregateVerifiable(results []VerifiableBattle) []VerifiableStats {
+	type acc struct {
+		games, decisions, clean int
+		errs                    []VerifiableError
+	}
+	byName := map[string]*acc{}
+	var order []string
+	for _, r := range results {
+		a := byName[r.Contestant]
+		if a == nil {
+			a = &acc{}
+			byName[r.Contestant] = a
+			order = append(order, r.Contestant)
+		}
+		a.games++
+		a.decisions += r.Decisions
+		a.errs = append(a.errs, r.Errors...)
+		if len(r.Errors) == 0 {
+			a.clean++
+		}
+	}
+	out := make([]VerifiableStats, 0, len(order))
+	for _, name := range order {
+		a := byName[name]
+		s := VerifiableStats{
+			Contestant: name, Games: a.games, Decisions: a.decisions,
+			Errors: len(a.errs), ByKind: map[ErrKind]int{}, CleanGames: a.clean,
+		}
+		for _, e := range a.errs {
+			s.ByKind[e.Kind]++
+		}
+		if a.decisions > 0 {
+			s.Per100 = 100 * float64(len(a.errs)) / float64(a.decisions)
+		}
+		out = append(out, s)
+	}
+	sort.Slice(out, func(i, j int) bool {
+		if out[i].Per100 != out[j].Per100 {
+			return out[i].Per100 < out[j].Per100
+		}
+		return out[i].Contestant < out[j].Contestant
+	})
+	return out
+}
+
+// VerifiableBattle is one scored battle attributed to a contestant.
+type VerifiableBattle struct {
+	Contestant string
+	Decisions  int
+	Errors     []VerifiableError
+}
diff --git a/internal/eval/verifiable_test.go b/internal/eval/verifiable_test.go
new file mode 100644
index 00000000..57ac42aa
--- /dev/null
+++ b/internal/eval/verifiable_test.go
@@ -0,0 +1,264 @@
+package eval
+
+import (
+	"testing"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+)
+
+// viewWith builds a fog-of-war view with a chosen move on the active Pokémon
+// and a chosen foe, so a single rule can be exercised in isolation.
+func viewWith(t *testing.T, selfMove string, self engine.Pokemon, foe engine.Pokemon) ai.View {
+	t.Helper()
+	self.Moves = []engine.MoveSlot{{MoveID: selfMove, PP: 10, MaxPP: 10}}
+	return ai.View{
+		Me:    0,
+		Turn:  3,
+		Self:  engine.Side{Trainer: "P0", Active: 0, Team: []engine.Pokemon{self}},
+		Foe:   foe,
+		Phase: engine.PhaseChoosing,
+	}
+}
+
+func mon(name string, t1, t2 string, hp, maxHP int) engine.Pokemon {
+	p := engine.Pokemon{Name: name, HP: hp, MaxHP: maxHP}
+	p.Type1 = engineType(t1)
+	if t2 != "" {
+		p.Type2 = engineType(t2)
+	}
+	return p
+}
+
+func kindsOf(errs []VerifiableError) map[ErrKind]int {
+	out := map[ErrKind]int{}
+	for _, e := range errs {
+		out[e.Kind]++
+	}
+	return out
+}
+
+// The metric's entire value is that every count survives scrutiny, so the
+// false-positive cases matter more than the true positives. Each of these is a
+// move that looks wasteful by a shallow rule but genuinely does something.
+func TestCheckAction_DoesNotFlagLegitimateMoves(t *testing.T) {
+	d := loadDex(t)
+
+	cases := []struct {
+		name string
+		why  string
+		move string
+		self engine.Pokemon
+		foe  engine.Pokemon
+	}{
+		{
+			name: "super effective attack",
+			why:  "the ordinary case must never be charged",
+			move: "thunderbolt",
+			self: mon("Pikachu", "electric", "", 100, 100),
+			foe:  mon("Gyarados", "water", "flying", 100, 100),
+		},
+		{
+			// Note this is Grass, not Ground: Electric vs Ground is a true 0x
+			// immunity, so Thunderbolt into a Rhydon *is* a provable error and
+			// belongs in the positive cases, not here.
+			name: "resisted but not immune",
+			why:  "0.5x is a weak move, not a provable no-op — that is a judgement call",
+			move: "thunderbolt",
+			self: mon("Pikachu", "electric", "", 100, 100),
+			foe:  mon("Venusaur", "grass", "poison", 100, 100),
+		},
+		{
+			name: "heal below full HP",
+			why:  "healing at 99/100 restores something",
+			move: "recover",
+			self: mon("Chansey", "normal", "", 99, 100),
+			foe:  mon("Snorlax", "normal", "", 100, 100),
+		},
+		{
+			name: "status on a clean target",
+			why:  "the target has no status, so it can land",
+			move: "thunder-wave",
+			self: mon("Pikachu", "electric", "", 100, 100),
+			foe:  mon("Snorlax", "normal", "", 100, 100),
+		},
+	}
+
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			v := viewWith(t, c.move, c.self, c.foe)
+			got := CheckAction(d, v, engine.Action{Kind: engine.ActionMove, Index: 0})
+			if len(got) != 0 {
+				t.Errorf("charged %d error(s) on a legitimate move (%s): %+v", len(got), c.why, got)
+			}
+		})
+	}
+}
+
+// Attacking into a type immunity is the flagship category: guaranteed zero
+// damage, and both defender types are public information.
+func TestCheckAction_FlagsTypeImmuneAttack(t *testing.T) {
+	d := loadDex(t)
+
+	// Normal does nothing to Ghost. Both types are visible in the view.
+	v := viewWith(t, "body-slam",
+		mon("Snorlax", "normal", "", 100, 100),
+		mon("Gengar", "ghost", "poison", 100, 100))
+
+	got := CheckAction(d, v, engine.Action{Kind: engine.ActionMove, Index: 0})
+	if k := kindsOf(got); k[ErrImmuneAttack] != 1 {
+		t.Fatalf("want 1 immune-attack, got %+v", got)
+	}
+	if got[0].Turn != 3 || got[0].Side != 0 {
+		t.Errorf("error not attributed to the right turn/side: %+v", got[0])
+	}
+}
+
+// A switch cannot be proven wasteful from the view — its value depends on what
+// the opponent does next, which is exactly the kind of judgement this metric
+// refuses to make.
+func TestCheckAction_NeverFlagsASwitch(t *testing.T) {
+	d := loadDex(t)
+	v := viewWith(t, "body-slam",
+		mon("Snorlax", "normal", "", 100, 100),
+		mon("Gengar", "ghost", "poison", 100, 100))
+
+	got := CheckAction(d, v, engine.Action{Kind: engine.ActionSwitch, Index: 1})
+	if len(got) != 0 {
+		t.Errorf("charged a switch: %+v", got)
+	}
+}
+
+// End to end on real battles, asserting per *category* rather than on a pooled
+// rate. The categories measure different failures and different policies fail
+// differently, so a single pooled number compares nothing meaningful — see
+// TestScoreVerifiable_HeuristicBoostsAtCap for the case that makes this vivid.
+//
+// The claim here is the sharp one: the heuristic explicitly checks type
+// effectiveness before attacking, so it must never attack into a type immunity,
+// while a policy choosing uniformly at random does. If the metric cannot see
+// that gap it is not measuring anything.
+func TestScoreVerifiable_SeparatesRandomFromCompetentOnTypeImmunity(t *testing.T) {
+	d := loadDex(t)
+	lib, err := LoadTeamLibrary(libraryPath, d)
+	if err != nil {
+		t.Fatalf("load team library: %v", err)
+	}
+
+	countKind := func(build func() ai.Agent, kind ErrKind) (hits, decisions int) {
+		for _, team := range lib.Teams {
+			agents := [2]ai.Agent{build(), ai.NewHeuristicAgent(d)}
+			picks := [2][]engine.TeamPick{team.Picks, team.Picks}
+			_, turns, err := CaptureStored(d, agents, picks, 31337, 0)
+			if err != nil {
+				t.Fatalf("capture: %v", err)
+			}
+			errs, dec, skipped, err := ScoreVerifiable(d, 0, turns)
+			if err != nil {
+				t.Fatalf("score: %v", err)
+			}
+			if skipped > 0 {
+				t.Errorf("%s: %d turns failed to recover", team.Name, skipped)
+			}
+			decisions += dec
+			for _, e := range errs {
+				if e.Kind == kind {
+					hits++
+				}
+				if e.Side != 0 {
+					t.Errorf("error attributed to side %d, scored side 0: %+v", e.Side, e)
+				}
+			}
+		}
+		return hits, decisions
+	}
+
+	randomHits, randomDec := countKind(func() ai.Agent { return ai.NewRandomAgent(20260811) }, ErrImmuneAttack)
+	heurHits, heurDec := countKind(func() ai.Agent { return ai.NewHeuristicAgent(d) }, ErrImmuneAttack)
+	t.Logf("immune attacks: random %d/%d, heuristic %d/%d", randomHits, randomDec, heurHits, heurDec)
+
+	if heurHits != 0 {
+		t.Errorf("heuristic attacked into a type immunity %d times; it checks effectiveness, so this "+
+			"is a false positive in the checker", heurHits)
+	}
+	if randomHits == 0 {
+		t.Error("random never attacked into a type immunity; the checker is not firing at all")
+	}
+}
+
+// The metric found a real defect in the heuristic baseline on its first run:
+// it spends turns on stat boosts that cannot apply because the stat is already
+// at +6, and re-applies a status the target already has.
+//
+// This test pins the finding rather than the bot. It is deliberately NOT a fix:
+// the heuristic is the benchmark's reference opponent, and changing how it
+// plays would invalidate every published number measured against it (the depth
+// sweep in docs/benchmark.md §6 above all). Fixing it is a decision with
+// benchmark-wide consequences, so it is recorded here and left to be made
+// deliberately. If it is fixed, this test should be deleted in the same commit.
+func TestScoreVerifiable_HeuristicBoostsAtCap(t *testing.T) {
+	d := loadDex(t)
+	lib, err := LoadTeamLibrary(libraryPath, d)
+	if err != nil {
+		t.Fatalf("load team library: %v", err)
+	}
+
+	byKind := map[ErrKind]int{}
+	for _, team := range lib.Teams {
+		agents := [2]ai.Agent{ai.NewHeuristicAgent(d), ai.NewHeuristicAgent(d)}
+		picks := [2][]engine.TeamPick{team.Picks, team.Picks}
+		_, turns, err := CaptureStored(d, agents, picks, 31337, 0)
+		if err != nil {
+			t.Fatalf("capture: %v", err)
+		}
+		errs, _, _, err := ScoreVerifiable(d, 0, turns)
+		if err != nil {
+			t.Fatalf("score: %v", err)
+		}
+		for _, e := range errs {
+			byKind[e.Kind]++
+		}
+	}
+	t.Logf("heuristic provable errors by kind: %v", byKind)
+
+	if byKind[ErrBoostAtCap] == 0 {
+		t.Skip("heuristic no longer boosts at cap — if it was fixed, delete this test")
+	}
+}
+
+func TestAggregateVerifiable_SortsCleanestFirstAndCountsCleanGames(t *testing.T) {
+	results := []VerifiableBattle{
+		{Contestant: "sloppy", Decisions: 50, Errors: []VerifiableError{
+			{Kind: ErrImmuneAttack}, {Kind: ErrImmuneAttack}, {Kind: ErrHealAtFull},
+		}},
+		{Contestant: "sloppy", Decisions: 50, Errors: nil},
+		{Contestant: "clean", Decisions: 100, Errors: []VerifiableError{{Kind: ErrBoostAtCap}}},
+	}
+
+	got := AggregateVerifiable(results)
+	if len(got) != 2 {
+		t.Fatalf("want 2 contestants, got %d", len(got))
+	}
+	if got[0].Contestant != "clean" {
+		t.Errorf("want cleanest first, got %q", got[0].Contestant)
+	}
+	if got[0].Per100 != 1 {
+		t.Errorf("clean: want 1.0 per 100, got %v", got[0].Per100)
+	}
+	sloppy := got[1]
+	if sloppy.Errors != 3 || sloppy.Decisions != 100 || sloppy.Per100 != 3 {
+		t.Errorf("sloppy: got %d errors / %d decisions / %v per100", sloppy.Errors, sloppy.Decisions, sloppy.Per100)
+	}
+	if sloppy.ByKind[ErrImmuneAttack] != 2 {
+		t.Errorf("sloppy: want 2 immune-attacks, got %d", sloppy.ByKind[ErrImmuneAttack])
+	}
+	// A rate alone would hide that one of the two games was spotless.
+	if sloppy.CleanGames != 1 {
+		t.Errorf("sloppy: want 1 clean game, got %d", sloppy.CleanGames)
+	}
+}
+
+// engineType converts a type name to the dex's Type. Kept here so the test
+// tables read as plain strings.
+func engineType(s string) domain.Type { return domain.Type(s) }

From 36fa66d9a93e610d76fb8186ac089189e2cb41a8 Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Wed, 12 Aug 2026 01:01:27 +0000
Subject: [PATCH 13/15] bench: one resumable command to run the whole benchmark

Running a benchmark meant invoking run-batch.sh once per model per team, each
call truncating its own results file, with the model-to-battle mapping living
in /tmp. Scaling it up meant remembering what had already been played. That is
how a previous batch was lost.

bench-run takes a config -- entrants, teams, games per team -- and plays the
whole matrix. Four properties, each answering a specific way a long run fails:

Resumable. The plan is a pure function of the config, and each game writes its
own result file named from its coordinates, so re-running the same command
skips what finished and plays the rest. There is no central ledger to lose and
no -resume flag to remember; a laptop that slept overnight is a non-event.
Scaling up is editing games_per_team and re-running.

Balanced under interruption. Games are interleaved across entrants, so stopping
halfway leaves every entrant with the same number of games rather than the first
complete and the last with none. In plan order a truncated run is not a
comparison at all; interleaved, a partial run is still usable.

Failures stay out of the dataset. A game whose harness errors or times out
writes no result file, so the next run retries it. A wrong CLI flag costs a
retry rather than a poisoned batch.

Attribution is a database fact. The entrant id travels as the battle's trainer
name -- which the join-identity work earlier in this branch made possible -- so
"which agent played this battle" is recorded in Postgres instead of inferred
from a scratch directory that may not survive the night.

The unit of identity is the entrant, deliberately not the model: a model reached
through Claude Code and the same model through a bare API harness are different
entrants, because the agent runtime around the model is part of what is being
measured. That makes both comparisons fall out of one dataset -- hold the model
fixed and vary the harness to compare runtimes, hold the harness fixed and vary
the model to compare models.

bench-report -bench-run prints the standings from the result files alone: no
gateway, no Postgres, no API access, so the table under a published claim can
always be re-derived from a directory small enough to commit beside it. Every
win rate carries a Wilson interval rather than offering it as an option,
because a win rate from a small run is the easiest number here to over-read.
Unfinished games are reported in their own column and excluded from the
denominator -- a harness that fails to finish games is a result worth seeing,
not something to hide inside a win rate by scoring it as a loss.

Validation rejects two configs that would silently corrupt a run: duplicate
entrant ids, and distinct ids that reduce to the same filename ("a/b" and
"a-b"), which would overwrite each other's results. The second was caught by
its own test.

Verified end to end against a stand-in harness: full matrix, re-run is a no-op,
resume after deleting results replays exactly the missing games, scaling 3->5
games per team keeps the 18 played and adds 12, and a wholly failing harness
writes zero result files.

The codex arm is wired but its flags are unconfirmed against a live install;
noted in the script, and a wrong flag fails loudly rather than degrading a game.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 cmd/bench-report/main.go         |  27 ++++
 cmd/bench-report/report_run.go   | 161 +++++++++++++++++++
 cmd/bench-run/main.go            | 267 +++++++++++++++++++++++++++++++
 docs/running-the-benchmark.md    |  54 ++++++-
 internal/eval/benchplan.go       | 240 +++++++++++++++++++++++++++
 internal/eval/benchplan_test.go  | 214 +++++++++++++++++++++++++
 scripts/bench/_bench_helpers.py  |  11 +-
 scripts/bench/bench.example.json |  31 ++++
 scripts/bench/play-live.sh       |  28 +++-
 9 files changed, 1024 insertions(+), 9 deletions(-)
 create mode 100644 cmd/bench-report/report_run.go
 create mode 100644 cmd/bench-run/main.go
 create mode 100644 internal/eval/benchplan.go
 create mode 100644 internal/eval/benchplan_test.go
 create mode 100644 scripts/bench/bench.example.json

diff --git a/cmd/bench-report/main.go b/cmd/bench-report/main.go
index 0b194c87..bc72365a 100644
--- a/cmd/bench-report/main.go
+++ b/cmd/bench-report/main.go
@@ -38,8 +38,35 @@ func main() {
 	dataDir := flag.String("data", "data", "benchmark mode: dataset directory (for replay re-simulation)")
 	teamsPath := flag.String("teams", "data/benchmark-teams.json", "benchmark mode: team library the baseline replays re-simulate on")
 	decisionQuality := flag.String("decision-quality", "", "benchmark mode: JSON of per-model decision-quality stats (decision-eval -json) to render as a section")
+	benchRun := flag.String("bench-run", "", "standings mode: a bench-run output directory; prints the entrant table and exits")
+	asJSON := flag.Bool("json", false, "standings mode: emit the table as JSON")
 	flag.Parse()
 
+	// Standings mode: recompute a run's headline numbers from its result files
+	// alone. No gateway, no Postgres, no API access -- so the table under a
+	// published claim can always be re-derived from a directory small enough to
+	// commit alongside it.
+	if *benchRun != "" {
+		games, err := LoadRun(*benchRun)
+		if err != nil {
+			log.Fatalf("read run: %v", err)
+		}
+		if len(games) == 0 {
+			log.Fatalf("no game results in %s", *benchRun)
+		}
+		rows := Standings(games)
+		if *asJSON {
+			enc := json.NewEncoder(os.Stdout)
+			enc.SetIndent("", "  ")
+			if err := enc.Encode(rows); err != nil {
+				log.Fatalf("encode: %v", err)
+			}
+			return
+		}
+		fmt.Print(FormatStandings(rows))
+		return
+	}
+
 	// Benchmark mode: fold both arms into one "vs the reference" ladder and
 	// render it through the standard report — same leaderboard, same replays.
 	if *baseline != "" || *agentic != "" {
diff --git a/cmd/bench-report/report_run.go b/cmd/bench-report/report_run.go
new file mode 100644
index 00000000..52f25b28
--- /dev/null
+++ b/cmd/bench-report/report_run.go
@@ -0,0 +1,161 @@
+package main
+
+// Reporting for a bench-run output directory: pooled win rate with Wilson
+// intervals per entrant, plus head-to-head-free standings.
+//
+// This reads only the per-game result files bench-run wrote. It deliberately
+// does not need Postgres, the gateway, or any API access, so the numbers behind
+// a published table can be recomputed from a directory that fits in a git
+// commit.
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+
+	"pokearena/internal/eval"
+)
+
+// runGame mirrors the record bench-run writes per game. Duplicated rather than
+// shared so the two commands can be read independently; the json tags are the
+// contract between them.
+type runGame struct {
+	Label    string `json:"label"`
+	Entrant  string `json:"entrant"`
+	Harness  string `json:"harness"`
+	Model    string `json:"model"`
+	Team     string `json:"team"`
+	BattleID string `json:"battle_id"`
+	Winner   int    `json:"winner"`
+	Status   string `json:"status"`
+	Seconds  int    `json:"seconds"`
+}
+
+// EntrantStanding is one row of the run's results table.
+type EntrantStanding struct {
+	Entrant    string  `json:"entrant"`
+	Harness    string  `json:"harness"`
+	Model      string  `json:"model"`
+	Games      int     `json:"games"`
+	Wins       int     `json:"wins"`
+	Losses     int     `json:"losses"`
+	Unfinished int     `json:"unfinished"`
+	WinRate    float64 `json:"win_rate"`
+	CILow      float64 `json:"ci_low"`
+	CIHigh     float64 `json:"ci_high"`
+	MedianSecs int     `json:"median_seconds"`
+}
+
+// LoadRun reads every per-game result in a bench-run output directory.
+func LoadRun(dir string) ([]runGame, error) {
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		return nil, err
+	}
+	var out []runGame
+	for _, e := range entries {
+		name := e.Name()
+		if e.IsDir() || !strings.HasSuffix(name, ".json") || name == "config.json" {
+			continue
+		}
+		b, err := os.ReadFile(filepath.Join(dir, name))
+		if err != nil {
+			return nil, err
+		}
+		var g runGame
+		if err := json.Unmarshal(b, &g); err != nil {
+			return nil, fmt.Errorf("%s: %w", name, err)
+		}
+		out = append(out, g)
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Label < out[j].Label })
+	return out, nil
+}
+
+// Standings folds games into one row per entrant.
+//
+// Unfinished games (winner == -1: the agent hung, timed out, or abandoned) are
+// counted and reported but excluded from the win-rate denominator. Counting
+// them as losses would conflate "played badly" with "the harness fell over",
+// which are different findings — and a harness that fails to finish games is
+// itself a result worth seeing in its own column rather than hidden inside a
+// win rate.
+func Standings(games []runGame) []EntrantStanding {
+	type acc struct {
+		harness, model      string
+		wins, losses, unfin int
+		secs                []int
+	}
+	byID := map[string]*acc{}
+	for _, g := range games {
+		a := byID[g.Entrant]
+		if a == nil {
+			a = &acc{harness: g.Harness, model: g.Model}
+			byID[g.Entrant] = a
+		}
+		switch g.Winner {
+		case 0:
+			a.wins++
+		case 1:
+			a.losses++
+		default:
+			a.unfin++
+		}
+		if g.Seconds > 0 {
+			a.secs = append(a.secs, g.Seconds)
+		}
+	}
+
+	out := make([]EntrantStanding, 0, len(byID))
+	for id, a := range byID {
+		decided := a.wins + a.losses
+		s := EntrantStanding{
+			Entrant: id, Harness: a.harness, Model: a.model,
+			Games: decided + a.unfin, Wins: a.wins, Losses: a.losses,
+			Unfinished: a.unfin, MedianSecs: medianInt(a.secs),
+		}
+		if decided > 0 {
+			s.WinRate = float64(a.wins) / float64(decided)
+			s.CILow, s.CIHigh = eval.WilsonInterval(float64(a.wins), decided, 1.96)
+		}
+		out = append(out, s)
+	}
+	sort.Slice(out, func(i, j int) bool {
+		if out[i].WinRate != out[j].WinRate {
+			return out[i].WinRate > out[j].WinRate
+		}
+		return out[i].Entrant < out[j].Entrant
+	})
+	return out
+}
+
+func medianInt(xs []int) int {
+	if len(xs) == 0 {
+		return 0
+	}
+	s := append([]int(nil), xs...)
+	sort.Ints(s)
+	n := len(s)
+	if n%2 == 1 {
+		return s[n/2]
+	}
+	return (s[n/2-1] + s[n/2]) / 2
+}
+
+// FormatStandings renders the table. The confidence interval is printed beside
+// every win rate rather than offered as an option, because a win rate from a
+// small run is the single easiest number in this project to over-read.
+func FormatStandings(rows []EntrantStanding) string {
+	var b strings.Builder
+	fmt.Fprintf(&b, "%-24s %6s %5s %5s %6s %8s %-18s %7s\n",
+		"entrant", "games", "W", "L", "unfin", "win%", "95% CI", "med s")
+	for _, r := range rows {
+		ci := fmt.Sprintf("[%.0f%%, %.0f%%]", 100*r.CILow, 100*r.CIHigh)
+		fmt.Fprintf(&b, "%-24s %6d %5d %5d %6d %7.1f%% %-18s %7d\n",
+			r.Entrant, r.Games, r.Wins, r.Losses, r.Unfinished, 100*r.WinRate, ci, r.MedianSecs)
+	}
+	return b.String()
+}
diff --git a/cmd/bench-run/main.go b/cmd/bench-run/main.go
new file mode 100644
index 00000000..e9b1b94f
--- /dev/null
+++ b/cmd/bench-run/main.go
@@ -0,0 +1,267 @@
+// Command bench-run plays a whole benchmark matrix and can be re-run safely.
+//
+// One command, any size, resumable. It enumerates every game the config calls
+// for, skips the ones already on disk, and plays the rest through the per-game
+// runner (scripts/bench/play-live.sh), which drives an agent CLI over the
+// PokéArena MCP tools.
+//
+// Three properties are the point:
+//
+//   - **Resumable.** The plan is a pure function of the config and each game's
+//     result lands in its own file named from its coordinates. A run killed
+//     overnight resumes by rebuilding the same plan and skipping what exists —
+//     there is no central ledger to lose.
+//   - **Balanced under interruption.** Games are interleaved across entrants,
+//     so stopping early leaves every entrant with the same number of games
+//     rather than the first one complete and the last with none.
+//   - **Durable attribution.** Each game's entrant id is sent as the trainer
+//     name, so which agent played which battle is a fact in Postgres, not a
+//     mapping in a scratch directory. A previous batch was lost exactly that
+//     way (docs/decision-quality-eval-handoff.md).
+//
+// Usage:
+//
+//	bench-run -config bench.json -out runs/2026-08-11          # play
+//	bench-run -config bench.json -out runs/2026-08-11 -dry-run  # show the plan
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"log"
+	"os"
+	"os/exec"
+	"os/signal"
+	"path/filepath"
+	"strings"
+	"sync"
+	"syscall"
+	"time"
+
+	"pokearena/internal/eval"
+)
+
+// GameResult is the per-game record written to the output directory. Its
+// presence is what marks a game complete for resume, so it is written only
+// after the runner returns.
+type GameResult struct {
+	Label     string `json:"label"`
+	Entrant   string `json:"entrant"`
+	Harness   string `json:"harness"`
+	Model     string `json:"model"`
+	Team      string `json:"team"`
+	Index     int    `json:"index"`
+	BattleID  string `json:"battle_id"`
+	Winner    int    `json:"winner"` // 0 = entrant, 1 = opponent, -1 = unfinished
+	Status    string `json:"status"`
+	StartedAt string `json:"started_at"`
+	Seconds   int    `json:"seconds"`
+}
+
+func main() {
+	log.SetFlags(log.Ltime)
+	log.SetPrefix("[bench-run] ")
+
+	configPath := flag.String("config", "bench.json", "run configuration")
+	outDir := flag.String("out", "", "output directory for results (required)")
+	dryRun := flag.Bool("dry-run", false, "print the plan and what remains, then exit")
+	scriptPath := flag.String("runner", "scripts/bench/play-live.sh", "per-game runner script")
+	flag.Parse()
+
+	if *outDir == "" {
+		log.Fatal("-out is required")
+	}
+
+	cfg, err := loadConfig(*configPath)
+	if err != nil {
+		log.Fatalf("config: %v", err)
+	}
+	if err := cfg.Validate(); err != nil {
+		log.Fatalf("config: %v", err)
+	}
+
+	plan := eval.Interleaved(eval.BuildPlan(cfg))
+	if err := os.MkdirAll(*outDir, 0o755); err != nil {
+		log.Fatalf("mkdir %s: %v", *outDir, err)
+	}
+	done, err := completedLabels(*outDir)
+	if err != nil {
+		log.Fatalf("scan %s: %v", *outDir, err)
+	}
+
+	s := eval.Summarize(plan, done)
+	log.Printf("%d entrants x %d teams x %d games = %d total (%d per entrant)",
+		s.Entrants, s.Teams, cfg.GamesPerTeam, s.Total, s.PerEntrant)
+	log.Printf("%d already complete, %d remaining", s.Total-s.Remaining, s.Remaining)
+
+	todo := eval.Remaining(plan, done)
+	if *dryRun {
+		for _, g := range todo {
+			fmt.Printf("%s\t%s\t%s\t%s\n", g.Label, g.Entrant.Harness, g.Entrant.Model, g.Team)
+		}
+		return
+	}
+	if len(todo) == 0 {
+		log.Print("nothing to do")
+		return
+	}
+
+	// Save the config next to the results so a published number can be traced
+	// to the matrix that produced it, even if the source config later changes.
+	if err := writeJSON(filepath.Join(*outDir, "config.json"), cfg); err != nil {
+		log.Fatalf("write config copy: %v", err)
+	}
+
+	conc := cfg.Concurrency
+	if conc < 1 {
+		conc = 1
+	}
+	log.Printf("playing %d games, concurrency %d", len(todo), conc)
+	run(todo, *outDir, *scriptPath, conc)
+}
+
+func run(todo []eval.PlannedGame, outDir, script string, conc int) {
+	// Ctrl-C stops scheduling new games but lets in-flight ones finish writing,
+	// so an interrupted run leaves no half-written result file to be mistaken
+	// for a completed game on resume.
+	stop := make(chan os.Signal, 1)
+	signal.Notify(stop, os.Interrupt, syscall.SIGTERM)
+	stopped := make(chan struct{})
+	go func() {
+		<-stop
+		log.Print("interrupt: finishing in-flight games, then stopping (re-run to resume)")
+		close(stopped)
+	}()
+
+	sem := make(chan struct{}, conc)
+	var wg sync.WaitGroup
+	var mu sync.Mutex
+	played, failed := 0, 0
+
+	for _, g := range todo {
+		select {
+		case <-stopped:
+			wg.Wait()
+			log.Printf("stopped: %d played, %d failed this session", played, failed)
+			return
+		default:
+		}
+
+		wg.Add(1)
+		sem <- struct{}{}
+		go func(g eval.PlannedGame) {
+			defer wg.Done()
+			defer func() { <-sem }()
+
+			res, err := playGame(g, outDir, script)
+			mu.Lock()
+			defer mu.Unlock()
+			if err != nil {
+				failed++
+				log.Printf("%s: FAILED: %v", g.Label, err)
+				return
+			}
+			played++
+			log.Printf("%s: winner=%d (%s) in %ds", g.Label, res.Winner, res.Status, res.Seconds)
+		}(g)
+	}
+	wg.Wait()
+	log.Printf("done: %d played, %d failed this session", played, failed)
+}
+
+// playGame runs one game and records it. The result file is written only on a
+// clean run: a game that errored is left absent so re-running retries it rather
+// than baking a failure into the dataset.
+func playGame(g eval.PlannedGame, outDir, script string) (GameResult, error) {
+	started := time.Now()
+	// The entrant id travels as the trainer name, so attribution is recorded in
+	// the battle row rather than inferred later from a scratch file.
+	cmd := exec.Command("bash", script,
+		g.Entrant.Harness, g.Entrant.Model, g.Team, g.Label, outDir, g.Entrant.ID)
+	out, err := cmd.CombinedOutput()
+	if err != nil {
+		return GameResult{}, fmt.Errorf("%v: %s", err, lastLine(string(out)))
+	}
+
+	bid, winner, status := parseRunnerOutput(string(out))
+	if bid == "" {
+		return GameResult{}, fmt.Errorf("runner printed no bid=: %s", lastLine(string(out)))
+	}
+
+	res := GameResult{
+		Label: g.Label, Entrant: g.Entrant.ID, Harness: g.Entrant.Harness,
+		Model: g.Entrant.Model, Team: g.Team, Index: g.Index,
+		BattleID: bid, Winner: winner, Status: status,
+		StartedAt: started.UTC().Format(time.RFC3339), Seconds: int(time.Since(started).Seconds()),
+	}
+	return res, writeJSON(filepath.Join(outDir, g.Label+".json"), res)
+}
+
+// parseRunnerOutput reads the runner's authoritative result line, which comes
+// from the gateway rather than the agent's self-report:
+//
+//	<label> winner=0 -> AGENT (status=completed) bid=<uuid>
+func parseRunnerOutput(out string) (bid string, winner int, status string) {
+	winner = -1
+	for _, line := range strings.Split(out, "\n") {
+		for _, f := range strings.Fields(line) {
+			switch {
+			case strings.HasPrefix(f, "bid="):
+				bid = strings.TrimPrefix(f, "bid=")
+			case strings.HasPrefix(f, "winner="):
+				fmt.Sscanf(strings.TrimPrefix(f, "winner="), "%d", &winner)
+			case strings.HasPrefix(f, "(status="):
+				status = strings.TrimSuffix(strings.TrimPrefix(f, "(status="), ")")
+			}
+		}
+	}
+	return bid, winner, status
+}
+
+// completedLabels reads which games already have a result. This is the resume
+// mechanism, and it deliberately derives from the files themselves rather than
+// from an index that could drift out of sync with them.
+func completedLabels(dir string) (map[string]bool, error) {
+	done := map[string]bool{}
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return done, nil
+		}
+		return nil, err
+	}
+	for _, e := range entries {
+		name := e.Name()
+		if e.IsDir() || !strings.HasSuffix(name, ".json") || name == "config.json" {
+			continue
+		}
+		done[strings.TrimSuffix(name, ".json")] = true
+	}
+	return done, nil
+}
+
+func loadConfig(path string) (eval.BenchConfig, error) {
+	var c eval.BenchConfig
+	b, err := os.ReadFile(path)
+	if err != nil {
+		return c, err
+	}
+	return c, json.Unmarshal(b, &c)
+}
+
+func writeJSON(path string, v any) error {
+	b, err := json.MarshalIndent(v, "", "  ")
+	if err != nil {
+		return err
+	}
+	return os.WriteFile(path, append(b, '\n'), 0o644)
+}
+
+func lastLine(s string) string {
+	lines := strings.Split(strings.TrimSpace(s), "\n")
+	if len(lines) == 0 {
+		return ""
+	}
+	return lines[len(lines)-1]
+}
diff --git a/docs/running-the-benchmark.md b/docs/running-the-benchmark.md
index 79e6853c..8a3900b7 100644
--- a/docs/running-the-benchmark.md
+++ b/docs/running-the-benchmark.md
@@ -145,11 +145,61 @@ scripts/bench/play-live.sh claude sonnet Genesis demo1
 scripts/bench/play-live.sh agy "Gemini 3.1 Pro (High)" Genesis demo2
 ```
 
-Args: `<claude|agy> <model> <team-name> <label> [outdir]`. The team name is one
+Args: `<claude|agy|codex> <model> <team-name> <label> [outdir] [entrant-id]`. The team name is one
 from `data/benchmark-teams.json` (Genesis, Spectrum, Keystone, Bruiser, Bastion,
 Blitz). It prints the authoritative winner read back from the gateway.
 
-### A batch with a confidence interval
+### A whole run, one command (`cmd/bench-run`)
+
+This is the entry point for a real benchmark: every entrant, every team, any
+number of games, from one config.
+
+```sh
+cp scripts/bench/bench.example.json bench.json   # edit entrants / teams / games
+go run ./cmd/bench-run -config bench.json -out runs/$(date +%F) -dry-run
+go run ./cmd/bench-run -config bench.json -out runs/$(date +%F)
+```
+
+Then the standings, straight from the result files:
+
+```sh
+go run ./cmd/bench-report -bench-run runs/$(date +%F)
+```
+
+```
+entrant                   games     W     L  unfin     win% 95% CI               med s
+gemini-cli/3.1-pro           30    19     9      2    67.9% [49%, 82%]             184
+claude-code/opus             30    17    11      2    60.7% [42%, 77%]             402
+codex/gpt-5                  30    12    16      2    42.9% [26%, 61%]             233
+```
+
+Four properties matter more than the convenience, and each exists because of a
+way a long run goes wrong:
+
+- **Resumable.** The plan is a pure function of the config and every game writes
+  its own result file, named from its coordinates. Re-running the same command
+  skips what finished and plays the rest — there is no central ledger to lose,
+  and no `-resume` flag to remember. A laptop that slept overnight is a
+  non-event.
+- **Balanced if interrupted.** Games are interleaved across entrants, so
+  stopping halfway leaves every entrant with the *same* number of games instead
+  of the first one complete and the last with none. A partial run is still a
+  comparison.
+- **Failures do not enter the dataset.** A game whose harness errors or times
+  out writes no result file, so the next run retries it. A wrong CLI flag costs
+  you a retry, not a poisoned batch.
+- **Attribution is in Postgres.** The entrant id is sent as the battle's trainer
+  name, so "which agent played this battle" is a database fact. The previous
+  generation of this tooling kept that mapping in `/tmp` and lost a full batch
+  when the directory was cleared.
+
+Scaling up is just editing `games_per_team` and re-running: existing games are
+kept, only the new ones are played.
+
+`concurrency` should stay low (2–3). Every live battle also runs a server-side
+expectimax opponent, and some agent CLIs stall under load.
+
+### A single-entrant batch (older, still useful for a quick check)
 
 ```sh
 # <claude|agy> <model> <team> <N> <concurrency> <tag>
diff --git a/internal/eval/benchplan.go b/internal/eval/benchplan.go
new file mode 100644
index 00000000..60bf069f
--- /dev/null
+++ b/internal/eval/benchplan.go
@@ -0,0 +1,240 @@
+package eval
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+)
+
+// A benchmark run is a matrix — every contestant plays every team a fixed
+// number of times — and the matrix is enumerated up front rather than
+// generated as it goes. That is what makes a run resumable: the plan is a pure
+// function of the config, so a run interrupted overnight recomputes exactly the
+// same list next morning and only skips what already finished.
+//
+// The unit of identity is the *contestant*, deliberately not the model. A model
+// reached through Claude Code and the same model reached through a bare API
+// harness are different contestants, because the agent runtime around the model
+// is part of what is being measured (tool-calling loop, retries, context
+// handling). Labelling by contestant makes both comparisons fall out of one
+// dataset: hold the model fixed and vary the harness to measure harnesses, hold
+// the harness fixed and vary the model to measure models.
+
+// Entrant is one competitor: an agent runtime plus the model driving it.
+type Entrant struct {
+	// ID is the label results are recorded under, and it becomes the trainer
+	// name on the battle — so attribution lives in the database rather than in
+	// a file beside the run. Convention is "harness/model".
+	ID string `json:"id"`
+	// Harness selects the runner that drives a game (claude, agy, codex, ...).
+	Harness string `json:"harness"`
+	// Model is passed through to that harness verbatim.
+	Model string `json:"model"`
+}
+
+// BenchConfig is the whole description of a run. It is committed alongside the
+// results so a published number can be traced to the matrix that produced it.
+type BenchConfig struct {
+	Entrants     []Entrant `json:"entrants"`
+	Teams        []string  `json:"teams"`
+	GamesPerTeam int       `json:"games_per_team"`
+	Concurrency  int       `json:"concurrency"`
+}
+
+// PlannedGame is one game of the matrix. Label is deterministic from the
+// coordinates, which is what lets a resumed run recognize completed work
+// without a central ledger that could be lost.
+type PlannedGame struct {
+	Entrant Entrant `json:"entrant"`
+	Team    string  `json:"team"`
+	Index   int     `json:"index"` // 1-based within (contestant, team)
+	Label   string  `json:"label"`
+}
+
+// Validate reports why a config cannot produce a run. Checked before any game
+// is played, because discovering a typo'd team name after an eight-hour batch
+// is a bad way to find out.
+func (c BenchConfig) Validate() error {
+	if len(c.Entrants) == 0 {
+		return fmt.Errorf("no contestants")
+	}
+	if len(c.Teams) == 0 {
+		return fmt.Errorf("no teams")
+	}
+	if c.GamesPerTeam < 1 {
+		return fmt.Errorf("games_per_team must be >= 1, got %d", c.GamesPerTeam)
+	}
+	seen := map[string]bool{}
+	for i, ct := range c.Entrants {
+		switch {
+		case ct.ID == "":
+			return fmt.Errorf("contestant %d: empty id", i)
+		case ct.Harness == "":
+			return fmt.Errorf("contestant %q: empty harness", ct.ID)
+		case ct.Model == "":
+			return fmt.Errorf("contestant %q: empty model", ct.ID)
+		case seen[ct.ID]:
+			// Duplicate ids would silently pool two different configurations
+			// into one row of the results table.
+			return fmt.Errorf("duplicate contestant id %q", ct.ID)
+		}
+		seen[ct.ID] = true
+	}
+	// Ids are unique, but labels are built from the *sanitized* id, and
+	// sanitizing is lossy: "a/b" and "a-b" are different entrants that would
+	// share a filename. Two entrants writing the same result file is silent
+	// data loss — the second overwrites the first, and resume then treats one
+	// game as covering both. Reject it here, where the fix is to rename.
+	byLabel := map[string]string{}
+	for _, ct := range c.Entrants {
+		key := pathSafe(ct.ID)
+		if prev, clash := byLabel[key]; clash {
+			return fmt.Errorf("entrant ids %q and %q both reduce to %q and would share result files; rename one",
+				prev, ct.ID, key)
+		}
+		byLabel[key] = ct.ID
+	}
+	return nil
+}
+
+// BuildPlan enumerates every game the config calls for, in a stable order.
+func BuildPlan(c BenchConfig) []PlannedGame {
+	var out []PlannedGame
+	for _, ct := range c.Entrants {
+		for _, team := range c.Teams {
+			for i := 1; i <= c.GamesPerTeam; i++ {
+				out = append(out, PlannedGame{
+					Entrant: ct,
+					Team:    team,
+					Index:   i,
+					Label:   GameLabel(ct.ID, team, i),
+				})
+			}
+		}
+	}
+	return out
+}
+
+// GameLabel is the deterministic name for one cell of the matrix. It is the
+// results filename and the resume key, so it must stay stable across runs and
+// be safe as a path segment.
+func GameLabel(contestantID, team string, index int) string {
+	return fmt.Sprintf("%s__%s__g%d", pathSafe(contestantID), pathSafe(team), index)
+}
+
+// pathSafe reduces a label to characters that are safe in a filename on every
+// platform we run on. It is intentionally lossy-but-stable: the same input
+// always yields the same output, so resume keeps working.
+func pathSafe(s string) string {
+	var b strings.Builder
+	for _, r := range strings.ToLower(s) {
+		switch {
+		case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
+			b.WriteRune(r)
+		case r == '-', r == '.':
+			b.WriteRune(r)
+		default:
+			b.WriteByte('-')
+		}
+	}
+	return strings.Trim(b.String(), "-")
+}
+
+// Remaining returns the games still to play, given the set of labels already
+// complete. Order is preserved so a resumed run continues predictably.
+//
+// Interleave matters here: the plan groups all of a contestant's games
+// together, which means an interrupted run leaves the last contestants with no
+// games at all. Callers that care about getting a usable partial result from a
+// run that may not finish should use Interleaved.
+func Remaining(plan []PlannedGame, done map[string]bool) []PlannedGame {
+	var out []PlannedGame
+	for _, g := range plan {
+		if !done[g.Label] {
+			out = append(out, g)
+		}
+	}
+	return out
+}
+
+// Interleaved reorders a plan round-robin across contestants, so that any
+// prefix of the run is a balanced sample.
+//
+// This is not cosmetic. A long batch is likely to be cut short — a laptop
+// sleeps, a rate limit trips, the run is stopped by hand — and with the plan in
+// its natural order that leaves contestant 1 complete and contestant 3 with
+// nothing, which is not a comparison at all. Interleaved means an interrupted
+// run still yields every contestant the same number of games, and the partial
+// result is usable rather than discarded.
+func Interleaved(plan []PlannedGame) []PlannedGame {
+	byContestant := map[string][]PlannedGame{}
+	var order []string
+	for _, g := range plan {
+		if _, ok := byContestant[g.Entrant.ID]; !ok {
+			order = append(order, g.Entrant.ID)
+		}
+		byContestant[g.Entrant.ID] = append(byContestant[g.Entrant.ID], g)
+	}
+	var out []PlannedGame
+	for round := 0; ; round++ {
+		added := false
+		for _, id := range order {
+			games := byContestant[id]
+			if round < len(games) {
+				out = append(out, games[round])
+				added = true
+			}
+		}
+		if !added {
+			return out
+		}
+	}
+}
+
+// PlanSummary describes a plan for the operator, before anything runs.
+type PlanSummary struct {
+	Entrants   int
+	Teams      int
+	Total      int
+	PerEntrant int
+	Remaining  int
+}
+
+// Summarize describes a plan and how much of it is left.
+func Summarize(plan []PlannedGame, done map[string]bool) PlanSummary {
+	contestants := map[string]bool{}
+	teams := map[string]bool{}
+	remaining := 0
+	for _, g := range plan {
+		contestants[g.Entrant.ID] = true
+		teams[g.Team] = true
+		if !done[g.Label] {
+			remaining++
+		}
+	}
+	s := PlanSummary{
+		Entrants:  len(contestants),
+		Teams:     len(teams),
+		Total:     len(plan),
+		Remaining: remaining,
+	}
+	if s.Entrants > 0 {
+		s.PerEntrant = s.Total / s.Entrants
+	}
+	return s
+}
+
+// SortContestantIDs returns the contestant ids in a plan, sorted, for stable
+// reporting.
+func SortContestantIDs(plan []PlannedGame) []string {
+	seen := map[string]bool{}
+	var out []string
+	for _, g := range plan {
+		if !seen[g.Entrant.ID] {
+			seen[g.Entrant.ID] = true
+			out = append(out, g.Entrant.ID)
+		}
+	}
+	sort.Strings(out)
+	return out
+}
diff --git a/internal/eval/benchplan_test.go b/internal/eval/benchplan_test.go
new file mode 100644
index 00000000..5e51da02
--- /dev/null
+++ b/internal/eval/benchplan_test.go
@@ -0,0 +1,214 @@
+package eval
+
+import (
+	"strings"
+	"testing"
+)
+
+func testConfig() BenchConfig {
+	return BenchConfig{
+		Entrants: []Entrant{
+			{ID: "claude-code/opus", Harness: "claude", Model: "opus"},
+			{ID: "gemini-cli/3.1-pro", Harness: "agy", Model: "Gemini 3.1 Pro (High)"},
+			{ID: "codex/gpt-5", Harness: "codex", Model: "gpt-5"},
+		},
+		Teams:        []string{"Genesis", "Keystone"},
+		GamesPerTeam: 2,
+	}
+}
+
+func TestBuildPlan_EnumeratesTheWholeMatrix(t *testing.T) {
+	plan := BuildPlan(testConfig())
+	if want := 3 * 2 * 2; len(plan) != want {
+		t.Fatalf("plan has %d games, want %d", len(plan), want)
+	}
+	// Labels are the resume key, so they must be unique across the matrix.
+	seen := map[string]bool{}
+	for _, g := range plan {
+		if seen[g.Label] {
+			t.Fatalf("duplicate label %q — resume would skip a game that never ran", g.Label)
+		}
+		seen[g.Label] = true
+	}
+}
+
+// The plan must be a pure function of the config: a run resumed tomorrow has to
+// rebuild exactly the same list, or the labels stop matching and finished games
+// are replayed.
+func TestBuildPlan_IsDeterministic(t *testing.T) {
+	a := BuildPlan(testConfig())
+	b := BuildPlan(testConfig())
+	if len(a) != len(b) {
+		t.Fatalf("lengths differ: %d vs %d", len(a), len(b))
+	}
+	for i := range a {
+		if a[i].Label != b[i].Label {
+			t.Fatalf("game %d differs between builds: %q vs %q", i, a[i].Label, b[i].Label)
+		}
+	}
+}
+
+// A label becomes a filename, so it must survive ids and team names containing
+// spaces, slashes, and mixed case.
+func TestGameLabel_IsPathSafeAndStable(t *testing.T) {
+	got := GameLabel("claude-code/opus", "Team Alpha", 3)
+	if strings.ContainsAny(got, " /\\") {
+		t.Errorf("label %q contains a path-unsafe character", got)
+	}
+	if again := GameLabel("claude-code/opus", "Team Alpha", 3); again != got {
+		t.Errorf("label not stable: %q vs %q", got, again)
+	}
+	// Sanitizing is lossy, so distinct ids CAN collide into one label. That is
+	// caught at config validation rather than papered over here — see
+	// TestBenchConfig_Validate/colliding ids. Different indexes and teams,
+	// though, must always stay distinct.
+	if GameLabel("x", "T", 1) == GameLabel("x", "T", 2) {
+		t.Error("game index does not affect the label")
+	}
+	if GameLabel("x", "A", 1) == GameLabel("x", "B", 1) {
+		t.Error("team does not affect the label")
+	}
+}
+
+func TestRemaining_SkipsCompletedGames(t *testing.T) {
+	plan := BuildPlan(testConfig())
+	done := map[string]bool{plan[0].Label: true, plan[5].Label: true}
+
+	left := Remaining(plan, done)
+	if len(left) != len(plan)-2 {
+		t.Fatalf("remaining %d, want %d", len(left), len(plan)-2)
+	}
+	for _, g := range left {
+		if done[g.Label] {
+			t.Errorf("completed game %q is still in the remaining list", g.Label)
+		}
+	}
+}
+
+// A long batch usually gets cut short. In plan order that leaves the first
+// contestant complete and the last with nothing, which is not a comparison —
+// so any prefix of the interleaved order must be balanced.
+func TestInterleaved_AnyPrefixIsABalancedSample(t *testing.T) {
+	plan := Interleaved(BuildPlan(testConfig()))
+	if len(plan) != 12 {
+		t.Fatalf("interleaving changed the game count: %d", len(plan))
+	}
+
+	counts := map[string]int{}
+	for i, g := range plan {
+		counts[g.Entrant.ID]++
+		// After every complete round (3 contestants), the counts must be level.
+		if (i+1)%3 == 0 {
+			var min, max int
+			first := true
+			for _, n := range counts {
+				if first {
+					min, max, first = n, n, false
+				}
+				if n < min {
+					min = n
+				}
+				if n > max {
+					max = n
+				}
+			}
+			if max-min > 0 {
+				t.Fatalf("after %d games the sample is unbalanced: %v", i+1, counts)
+			}
+		}
+	}
+	// And no game may be lost or duplicated by the reordering.
+	if len(counts) != 3 {
+		t.Errorf("want 3 contestants, got %d", len(counts))
+	}
+	for id, n := range counts {
+		if n != 4 {
+			t.Errorf("contestant %q has %d games, want 4", id, n)
+		}
+	}
+}
+
+func TestInterleaved_HandlesUnevenContestants(t *testing.T) {
+	// One contestant with fewer games must not stall the round-robin.
+	plan := []PlannedGame{
+		{Entrant: Entrant{ID: "a"}, Label: "a1"},
+		{Entrant: Entrant{ID: "a"}, Label: "a2"},
+		{Entrant: Entrant{ID: "a"}, Label: "a3"},
+		{Entrant: Entrant{ID: "b"}, Label: "b1"},
+	}
+	got := Interleaved(plan)
+	if len(got) != 4 {
+		t.Fatalf("lost games: got %d, want 4", len(got))
+	}
+	if got[0].Label != "a1" || got[1].Label != "b1" || got[2].Label != "a2" || got[3].Label != "a3" {
+		var labels []string
+		for _, g := range got {
+			labels = append(labels, g.Label)
+		}
+		t.Errorf("unexpected order: %v", labels)
+	}
+}
+
+func TestBenchConfig_Validate(t *testing.T) {
+	cases := []struct {
+		name    string
+		mutate  func(*BenchConfig)
+		wantErr string
+	}{
+		{"valid", func(*BenchConfig) {}, ""},
+		{"no contestants", func(c *BenchConfig) { c.Entrants = nil }, "no contestants"},
+		{"no teams", func(c *BenchConfig) { c.Teams = nil }, "no teams"},
+		{"zero games", func(c *BenchConfig) { c.GamesPerTeam = 0 }, "games_per_team"},
+		{"empty id", func(c *BenchConfig) { c.Entrants[0].ID = "" }, "empty id"},
+		{"empty harness", func(c *BenchConfig) { c.Entrants[0].Harness = "" }, "empty harness"},
+		{"empty model", func(c *BenchConfig) { c.Entrants[0].Model = "" }, "empty model"},
+		{
+			// Two rows with the same id would pool different configurations
+			// into one line of the results table.
+			"duplicate id",
+			func(c *BenchConfig) { c.Entrants[1].ID = c.Entrants[0].ID },
+			"duplicate contestant id",
+		},
+		{
+			// Distinct ids that reduce to the same filename would silently
+			// overwrite each other's results.
+			"colliding ids",
+			func(c *BenchConfig) {
+				c.Entrants[0].ID = "a/b"
+				c.Entrants[1].ID = "a-b"
+			},
+			"would share result files",
+		},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			cfg := testConfig()
+			c.mutate(&cfg)
+			err := cfg.Validate()
+			switch {
+			case c.wantErr == "" && err != nil:
+				t.Errorf("unexpected error: %v", err)
+			case c.wantErr != "" && err == nil:
+				t.Errorf("want error containing %q, got nil", c.wantErr)
+			case c.wantErr != "" && !strings.Contains(err.Error(), c.wantErr):
+				t.Errorf("want error containing %q, got %v", c.wantErr, err)
+			}
+		})
+	}
+}
+
+func TestSummarize_CountsRemaining(t *testing.T) {
+	plan := BuildPlan(testConfig())
+	done := map[string]bool{plan[0].Label: true, plan[1].Label: true, plan[2].Label: true}
+
+	s := Summarize(plan, done)
+	if s.Entrants != 3 || s.Teams != 2 || s.Total != 12 {
+		t.Errorf("summary shape wrong: %+v", s)
+	}
+	if s.PerEntrant != 4 {
+		t.Errorf("per-contestant = %d, want 4", s.PerEntrant)
+	}
+	if s.Remaining != 9 {
+		t.Errorf("remaining = %d, want 9", s.Remaining)
+	}
+}
diff --git a/scripts/bench/_bench_helpers.py b/scripts/bench/_bench_helpers.py
index 0a24a14e..a38559da 100755
--- a/scripts/bench/_bench_helpers.py
+++ b/scripts/bench/_bench_helpers.py
@@ -7,7 +7,7 @@
 
 Subcommands:
   picks     <library.json> <team-name>   -> the team's picks as compact JSON
-  newbattle <gateway-http>               -> create a live vs-AI battle, print its id
+  newbattle <gateway-http> [name]        -> create a live vs-AI battle, print its id
   winner    <gateway-http> <bid> <label> -> print the authoritative result line
   tally     <results-file> <tag>         -> pooled win rate + Wilson 95% interval
 """
@@ -25,8 +25,13 @@ def picks(lib_path, team):
     print(json.dumps(t["picks"], separators=(",", ":")))
 
 
-def newbattle(gw):
-    body = json.dumps({"mode": "live", "p1_name": "Agent", "p2_name": "AI"}).encode()
+def newbattle(gw, name="Agent"):
+    # p1_name is the entrant id. Recording it at creation puts attribution in
+    # Postgres instead of a scratch file beside the run -- a previous batch was
+    # lost when that scratch directory was wiped. The agent also re-declares it
+    # on join_battle, so the name survives even if the battle was created by
+    # something else.
+    body = json.dumps({"mode": "live", "p1_name": name, "p2_name": "AI"}).encode()
     req = urllib.request.Request(
         gw + "/api/battles", data=body, headers={"content-type": "application/json"}
     )
diff --git a/scripts/bench/bench.example.json b/scripts/bench/bench.example.json
new file mode 100644
index 00000000..fabedda3
--- /dev/null
+++ b/scripts/bench/bench.example.json
@@ -0,0 +1,31 @@
+{
+  "entrants": [
+    {
+      "id": "claude-code/opus",
+      "harness": "claude",
+      "model": "opus"
+    },
+    {
+      "id": "claude-code/sonnet",
+      "harness": "claude",
+      "model": "sonnet"
+    },
+    {
+      "id": "gemini-cli/3.1-pro",
+      "harness": "agy",
+      "model": "Gemini 3.1 Pro (High)"
+    },
+    {
+      "id": "codex/gpt-5",
+      "harness": "codex",
+      "model": "gpt-5"
+    }
+  ],
+  "teams": [
+    "Genesis",
+    "Keystone",
+    "Spectrum"
+  ],
+  "games_per_team": 10,
+  "concurrency": 2
+}
diff --git a/scripts/bench/play-live.sh b/scripts/bench/play-live.sh
index e410ca9d..bbf821ed 100755
--- a/scripts/bench/play-live.sh
+++ b/scripts/bench/play-live.sh
@@ -11,9 +11,13 @@
 # Prereqs: the stack is up (`make run`), `bin/pokearena-mcp` is built (`make mcp`),
 # and the chosen CLI is installed and authenticated (claude, or agy for Antigravity).
 #
-# Usage: play-live.sh <claude|agy> <model> <team-name> <label> [outdir]
+# Usage: play-live.sh <claude|agy|codex> <model> <team-name> <label> [outdir] [entrant-id]
 #   play-live.sh claude sonnet Genesis cs1
 #   play-live.sh agy "Gemini 3.1 Pro (High)" Genesis ag1
+#
+# entrant-id is the name this game's result is recorded under -- it becomes the
+# battle's trainer name, so "which agent played this battle" is a fact in
+# Postgres rather than a mapping in a scratch file. Defaults to <harness>/<model>.
 set -uo pipefail
 
 HARNESS="${1:?harness: claude or agy}"
@@ -21,6 +25,7 @@ MODEL="${2:?model, e.g. sonnet or Gemini 3.1 Pro (High)}"
 TEAM="${3:?team name from data/benchmark-teams.json}"
 LABEL="${4:?a short label for this game}"
 OUTDIR="${5:-/tmp/pokearena-agentic}"
+ENTRANT="${6:-$HARNESS/$MODEL}"
 
 DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 REPO="$(cd "$DIR/../.." && pwd)"
@@ -70,7 +75,7 @@ run_capped() {
 PICKS=$(python3 "$HELP" picks "$REPO/data/benchmark-teams.json" "$TEAM") || exit 1
 
 # A fresh live (vs-AI) battle.
-BID=$(python3 "$HELP" newbattle "$GW_HTTP") || { echo "$LABEL FAILED: could not create battle (is the stack up?)"; exit 1; }
+BID=$(python3 "$HELP" newbattle "$GW_HTTP" "$ENTRANT") || { echo "$LABEL FAILED: could not create battle (is the stack up?)"; exit 1; }
 
 read -r -d '' PROMPT <<EOF
 You are a Pokemon battle client playing ONE battle to WIN against a programmatic AI opponent.
@@ -80,7 +85,7 @@ Do NOT use shell/bash or any other tool. Do NOT ask questions — play autonomou
 battle_id: $BID
 
 Steps:
-1. join_battle with ONLY battle_id="$BID" (no slot, no join_token). Live vs-AI battle; you are p1.
+1. join_battle with battle_id="$BID" and trainer_name="$ENTRANT" (no slot, no join_token). Live vs-AI battle; you are p1.
 2. submit_team with these exact picks:
 $PICKS
 3. Play loop: call wait (timeout_seconds: 30). When it returns ready with a view and it's your turn, read the view and call act:
@@ -112,8 +117,23 @@ case "$HARNESS" in
       --dangerously-skip-permissions --print-timeout 20m \
       > "$OUTDIR/$LABEL.log" 2>&1
     ;;
+  codex)
+    # Codex CLI. Flag names are the least settled of the three -- confirm them
+    # against your installed version before a long batch, and run a single game
+    # first. A wrong flag here fails fast and loudly rather than playing a
+    # degraded game, which is the behaviour we want: bench-run writes no result
+    # file for a failed game, so it simply retries once the flags are right.
+    MCPCFG="$OUTDIR/mcp-codex.json"
+    printf '{"mcpServers":{"pokearena":{"command":"%s/bin/pokearena-mcp","env":{"POKEARENA_GATEWAY_URL":"%s"}}}}\n' "$REPO" "$GW_WS" > "$MCPCFG"
+    run_capped "$GAME_TIMEOUT" \
+      codex exec "$PROMPT" --model "$MODEL" \
+      --mcp-config "$MCPCFG" \
+      --dangerously-bypass-approvals-and-sandbox \
+      > "$OUTDIR/$LABEL.log" 2>&1
+    [ $? -eq 124 ] && echo "[play-live] $LABEL: killed after ${GAME_TIMEOUT}s wall-clock cap" >> "$OUTDIR/$LABEL.log"
+    ;;
   *)
-    echo "$LABEL FAILED: unknown harness $HARNESS -- want claude or agy"; exit 1 ;;
+    echo "$LABEL FAILED: unknown harness $HARNESS -- want claude, agy, or codex"; exit 1 ;;
 esac
 
 # Authoritative result from the gateway — never trust the agent's self-report.

From 6791c91ef21d1d61f9e3d0b62959b21f1dd540dc Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Wed, 12 Aug 2026 01:32:01 +0000
Subject: [PATCH 14/15] ai: stop the heuristic spending turns on impossible
 moves
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The verifiable-error metric added yesterday was built to judge contestants. The
first thing it measured was our own reference opponent: 27 boost-at-cap turns
across six games, and Thunder Wave re-applied to an already-paralysed target for
six consecutive turns.

Two distinct causes, and the second is the instructive one.

The boost branch had no check at all -- it valued a self-boost at 55 while
healthy without ever consulting me.Stages, so at +6, where the move cannot
change anything, it still outscored most attacks.

The status branch *did* check, and was commented "a status move is wasted on an
already-statused foe". It returned 0. But a damaging move that deals no damage
also scores 0, and Decide breaks ties toward the earliest legal action, so a
dead status move in an early slot won the tie and was replayed every turn. The
code held the right belief, correctly stated, and was still wrong, because 0 is
not a neutral value in a scoring function whose other outputs bottom out there.
Both now return a deadMoveScore well below any live option including a switch --
giving up a turn to reposition genuinely beats spending it on a guaranteed
no-op.

This is not the engine. Every one of these moves fails correctly in the engine,
which is exactly why they were detectable: the metric looks for actions that
provably cannot accomplish anything, and the engine's correct rejection is what
makes "provably" true. Rules, replays, determinism and fairness are untouched.

But the heuristic is the opponent every figure in benchmark.md §6 is measured
against, so the document's only load-bearing table had to be re-measured rather
than assumed still valid. 240 games per depth, same methodology:

  depth 1   54.4% -> 53.8%
  depth 2   42.9% -> 42.9%
  depth 3   42.1% -> 42.1%

Depths 2 and 3 are identical to the exact game (103/240 and 101/240 both times);
depth 1 moved by two games out of 240. The reason is visible once you look at
which turns were wasted: they clustered in already-decided stall positions -- the
six-turn Thunder Wave loop was on the wall team at turns 72-77, in a game long
settled. A bot wasting turns in a position it has already won or lost does not
change the result.

I expected a larger move and braced to re-caveat §6. Re-running the whole sweep
costs 22 minutes of deterministic offline compute and no money, which is a cheap
way to turn that guess into a fact.

The sweep itself is now committed as TestDepthSweep behind POKEARENA_DEPTH_SWEEP
rather than left as a scratch file, and §6 carries the command -- the same
mistake in miniature as the lost /tmp attribution, a published number whose
derivation was not reproducible.

Deliberately not fixed: the heuristic under-values Rest, which carries no effect
block in the dataset and so never reaches the heal branch. The line drawn is
correct a provable waste, do not change a strategy. A move that cannot possibly
work is a defect in any sense; teaching the bot to use Rest well is a judgement
about how the game should be played, changes the baseline's strength, and is a
decision about what the benchmark measures rather than a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 ...40-fixing-the-baseline-the-metric-found.md | 101 ++++++++
 docs/benchmark.md                             |  69 +++++-
 docs/decision-quality.md                      |   5 +
 internal/ai/heuristic.go                      |  79 ++++++-
 internal/ai/heuristic_deadmove_test.go        | 220 ++++++++++++++++++
 internal/eval/depthsweep_test.go              |  54 +++++
 internal/eval/verifiable_test.go              |  40 ----
 7 files changed, 515 insertions(+), 53 deletions(-)
 create mode 100644 backlog/2026-08-12T01-40-fixing-the-baseline-the-metric-found.md
 create mode 100644 internal/ai/heuristic_deadmove_test.go
 create mode 100644 internal/eval/depthsweep_test.go

diff --git a/backlog/2026-08-12T01-40-fixing-the-baseline-the-metric-found.md b/backlog/2026-08-12T01-40-fixing-the-baseline-the-metric-found.md
new file mode 100644
index 00000000..664244f6
--- /dev/null
+++ b/backlog/2026-08-12T01-40-fixing-the-baseline-the-metric-found.md
@@ -0,0 +1,101 @@
+# Fixing the baseline the metric found
+
+The verifiable-error metric was built the day before to measure *contestants*.
+The first thing it measured was our own reference opponent, and it found the
+heuristic spending turns on moves the rules make impossible: 27 boost-at-cap
+turns across six games, and Thunder Wave re-applied to an already-paralysed
+target for six consecutive turns.
+
+## Two bugs that look like one
+
+They are not the same mistake, and the difference is the interesting part.
+
+**The boost branch had no check at all.** It valued a self-boost at 55 while
+healthy, never consulting `me.Stages`. At +6 the move cannot change anything and
+it still scored 55, which beats most attacks. So the agent boosted, and boosted,
+and boosted.
+
+**The status branch did have a check** — `return 0` with the comment "a status
+move is wasted on an already-statused foe." Someone saw this exact case and
+guarded it. The guard was just not strong enough: a damaging move that deals no
+damage also scores 0, and `Decide` breaks ties toward the earliest legal action,
+so a dead status move in an early move slot won the tie and was replayed every
+single turn.
+
+That second one is the one worth remembering. The code contained the right
+belief, correctly commented, and was still wrong — because 0 is not a neutral
+value in a scoring function whose other outputs bottom out at 0. The fix is a
+`deadMoveScore` constant well below anything a live option can score, including
+switching, since giving up a turn to reposition genuinely beats spending it on a
+guaranteed no-op.
+
+## Not the engine, and that distinction did the work
+
+Every one of these moves *fails correctly* in the engine. That is precisely why
+they were detectable — the metric watches for actions that provably cannot
+accomplish anything, and the engine's correct rejection of them is what makes
+"provably" true.
+
+So the blast radius was bounded in a way worth stating: rules, replays,
+determinism, fairness — all untouched. What changed is one *player*. But that
+player is the opponent every published figure in §6 is measured against, so
+"only the bot" still meant re-measuring the document's only load-bearing table.
+
+## Re-sweeping, and being wrong about what I expected
+
+I expected the fix to move the numbers a little and braced for having to
+re-caveat §6. 240 games per depth, same methodology as before:
+
+| Depth | pre-fix | post-fix |
+|---|---:|---:|
+| 1 | 54.4% | 53.8% |
+| 2 | 42.9% | 42.9% |
+| 3 | 42.1% | 42.1% |
+
+Depths 2 and 3 are identical to the exact game — 103/240 and 101/240 both times.
+Depth 1 moved by two games out of 240.
+
+The reason, once you look: the wasted turns clustered in already-decided stall
+positions. The six-turn Thunder Wave loop was on the Bastion wall team at turns
+72–77 — a game whose outcome was long settled. A bot wasting turns in a position
+it has already won or lost does not change the result. Twenty-seven wasted turns
+sounds like a lot until you notice which turns they were.
+
+Two things I'd have gotten wrong by reasoning instead of measuring: I would have
+guessed the effect was larger, and if the numbers *had* moved I would have had
+no way to tell whether the fix or ordinary variance did it. Re-running the whole
+sweep on a deterministic offline harness costs 22 minutes of compute and no
+money, which is a very cheap way to convert a guess into a fact.
+
+## The sweep is now committed
+
+It had been a scratch test. That was the same mistake as the last three rounds
+in miniature — a published number whose derivation lives in a `/tmp` file. It
+is now `TestDepthSweep`, gated behind `POKEARENA_DEPTH_SWEEP=1` so it never
+runs in the suite, and §6 carries the command. The figures in the document can
+be re-derived rather than trusted.
+
+## What I did not fix, and why that was the harder call
+
+The heuristic also under-values Rest — the move carries no effect block in the
+dataset, so it never reaches the heal branch and lands on a flat score of 5.
+Fixing that would make the baseline meaningfully stronger.
+
+I left it. The line I drew: **correct a provable waste, do not change a
+strategy.** A move that cannot possibly work is a defect in any sense of the
+word, and fixing it makes the bot do what it already meant to do. Teaching it to
+use Rest well is a judgement about how the game should be played, and it would
+change the baseline's strength — which is a decision about what the benchmark
+measures, not a bug fix, and it should be made deliberately and announced rather
+than smuggled in beside a correctness patch.
+
+The same line explains why the ability-immunity case is absent from the metric
+itself: Levitate blanks a Ground move, but the attacker may not know the
+ability, so charging it would penalise unknowable information.
+
+## The loop that closed
+
+Built a metric to judge outside contestants; it immediately indicted the
+reference opponent. That is a good sign about the metric and an uncomfortable
+one about the baseline, and the useful version of both is that the tool works on
+its author. Worth expecting the next measurement instrument to do the same.
diff --git a/docs/benchmark.md b/docs/benchmark.md
index e76bb7c3..ab6145aa 100644
--- a/docs/benchmark.md
+++ b/docs/benchmark.md
@@ -154,16 +154,60 @@ full-HP Pokémon — so a KO is a won game only when the bench is genuinely empt
 otherwise it scores as a one-Pokémon material lead, far below a win. This
 **killed the collapse**.
 
-The sweep has since been re-measured on the v2 (EV-trained) library, because
-the numbers below are the doc's only load-bearing measurements and the teams
-underneath them changed (Section 7). Expectimax win rate vs the heuristic,
-240 games per depth — 6 teams × 20 seeds × 2 orientations:
+The sweep has since been re-measured twice, because the numbers below are the
+doc's only load-bearing measurements and what sits underneath them has moved
+twice: once when the team library went to v2 (Section 7), and once when the
+heuristic — the *opponent* every figure here is measured against — was fixed
+(below). Expectimax win rate vs the heuristic, 240 games per depth — 6 teams ×
+20 seeds × 2 orientations:
+
+| Depth | v1 (neutral teams) | v2, pre-fix baseline | **v2, current** | Wilson 95% CI |
+|---|---:|---:|---:|---|
+| 1 | 50% | 54.4% | **53.8%** | [47.4%, 59.9%] |
+| 2 | 40% | 42.9% | **42.9%** | [36.8%, 49.2%] |
+| 3 | 38% | 42.1% | **42.1%** | [36.0%, 48.4%] |
+
+**The baseline fix moved almost nothing.** Depths 2 and 3 are unchanged to the
+exact game (103/240 and 101/240 both times); depth 1 moved by two games out of
+240. The heuristic's wasted turns were real but concentrated in already-decided
+stall positions, so they rarely changed a result. That is a measurement, not an
+assumption — the whole sweep was re-run, and it is reproducible:
+
+```sh
+POKEARENA_DEPTH_SWEEP=1 go test ./internal/eval -run TestDepthSweep -v -timeout 60m
+```
+
+<details>
+<summary><b>What was fixed in the baseline, and why it required a re-sweep</b></summary>
+
+The heuristic was spending turns on moves the rules make impossible: a status
+the target already had, a stat boost with the stat pinned at +6, a heal at full
+HP. Found by the verifiable-error metric in `internal/eval`
+([decision-quality.md](decision-quality.md)), which counted 27 boost-at-cap
+turns across six games and caught the agent re-applying Thunder Wave to an
+already-paralysed target for six consecutive turns.
+
+Two distinct causes. The boost branch had no cap check at all. The status branch
+*did* check, but returned 0 — and a damaging move that deals no damage also
+scores 0, with ties broken toward the earliest legal action, so a dead status
+move in an early slot won the tie and was replayed every turn.
+
+This is a change to the *opponent*, not to the engine: the engine correctly made
+all of these moves fail, which is precisely why they were detectable. But every
+number in this section is "expectimax vs the heuristic," so changing the
+heuristic changes what the numbers describe, and they had to be re-measured
+rather than assumed still valid.
+
+Comparisons between contestants were never at risk — every contestant faced the
+identical baseline — but absolute figures like "42.9% at depth 2" describe a
+specific opponent, and now describe the fixed one.
+
+Deliberately *not* fixed in the same pass: the heuristic under-values Rest,
+which carries no effect block in the dataset and so never reaches the heal
+branch. That is a strategy weakness rather than a provable-waste bug, and
+strengthening the baseline is a separate decision from correcting it.
 
-| Depth | v1 (neutral teams) | v2 (trained teams) | v2 Wilson 95% CI |
-|---|---:|---:|---|
-| 1 | 50% | **54.4%** | [48.1%, 60.6%] |
-| 2 | 40% | **42.9%** | [36.8%, 49.2%] |
-| 3 | 38% | **42.1%** | [36.0%, 48.4%] |
+</details>
 
 **The shape survived the library change, which is the result that matters
 here.** Expectimax is a few points stronger at every depth on trained teams,
@@ -186,10 +230,11 @@ Two honest consequences remain:
   worth more than a meta-specific accident.
 
 One thing the trained library did change: at depth 1, expectimax now edges
-*ahead* of the heuristic (54.4%) where on neutral teams it drew level. The
+*ahead* of the heuristic (53.8%) where on neutral teams it drew level. The
 default `bench` depth is 2, where it still loses at 42.9%, so the headline
-baseline ordering is unaffected — but the per-depth numbers are library-
-specific and should be quoted with the `team_library` version attached.
+baseline ordering is unaffected — but the per-depth numbers are specific to both
+the team library *and* the baseline opponent, and should be quoted with the
+`team_library` version and the engine revision attached.
 
 An oracle that plays worse with more compute cannot define "optimal." The fix
 removes the *dominant* cause of that, but a residual remains, so we still do
diff --git a/docs/decision-quality.md b/docs/decision-quality.md
index 26519906..8d86c38f 100644
--- a/docs/decision-quality.md
+++ b/docs/decision-quality.md
@@ -72,6 +72,11 @@ reproduces the v2 depth-sweep figure in [benchmark.md §6](benchmark.md) (42.9%,
 95% CI [36.8, 49.2]) from a completely different code path — offline capture
 here, the `bench` runner there.
 
+(The v2 calibration table above was measured before the heuristic baseline was
+fixed for the wasted turns this very metric found — see §6. The fix left the
+depth-2 figure unchanged to the exact game, so the cross-check still stands,
+but the calibration's absolute numbers describe the pre-fix opponent.)
+
 And then the ordering falls apart the moment you change the judge.
 
 ## The judge decides the answer
diff --git a/internal/ai/heuristic.go b/internal/ai/heuristic.go
index d7904f95..c5cc4a29 100644
--- a/internal/ai/heuristic.go
+++ b/internal/ai/heuristic.go
@@ -111,9 +111,27 @@ func (a *HeuristicAgent) switchScore(in, foe, cur engine.Pokemon, w *engine.Weat
 	return float64(myOffense)*0.3 + improvement*0.5 - float64(hazardChip) - 40 // -40: a switch costs a turn
 }
 
+// deadMoveScore marks a move that provably cannot accomplish anything from the
+// current position — a status the target already has, a heal at full HP, a
+// boost with every stat pinned at +6.
+//
+// It is strongly negative rather than 0 because 0 was not low enough. A
+// damaging move that deals no damage also scores 0, and Decide breaks ties
+// toward the earliest legal action, so a dead status move sitting in an early
+// move slot would win the tie and be replayed every turn — the agent spent six
+// consecutive turns re-applying Thunder Wave to an already-paralysed target
+// this way. Below every alternative including a switch, because giving up a
+// turn to reposition genuinely beats spending it on a guaranteed no-op.
+const deadMoveScore = -1000
+
 // statusScore values non-damaging moves by the situation. With the new move
 // schema, status moves carry a Primary block declaring what they do; we look
 // at which fields it touches to bucket the move.
+//
+// Each branch first asks whether the move can do anything at all. Those checks
+// are not an optimization — without them the agent burns turns on moves the
+// engine will reject outright, which internal/eval's verifiable-error metric
+// counts and which no amount of good evaluation elsewhere makes up for.
 func (a *HeuristicAgent) statusScore(m domain.Move, me, foe engine.Pokemon) float64 {
 	if m.Primary == nil {
 		return 5
@@ -121,14 +139,23 @@ func (a *HeuristicAgent) statusScore(m domain.Move, me, foe engine.Pokemon) floa
 	p := m.Primary
 	switch {
 	case p.Heal > 0:
+		// At full HP there is nothing to restore. Rest is the exception: it
+		// also cures status, so it still buys something when the user is
+		// statused even though the heal itself is wasted.
+		if me.HP >= me.MaxHP && !(p.Rest && me.Status != engine.StatusNone) {
+			return deadMoveScore
+		}
 		missing := float64(me.MaxHP-me.HP) / float64(me.MaxHP)
 		return missing * 220 // worth more the more HP is missing
 	case p.Status != "":
 		if foe.Status == engine.StatusNone {
 			return 60
 		}
-		return 0 // a status move is wasted on an already-statused foe
+		return deadMoveScore // a Pokemon cannot hold two statuses
 	case len(p.Boosts) > 0:
+		if allBoostsAtCap(p.Boosts, me.Stages) {
+			return deadMoveScore // every stat it would raise is already at +6
+		}
 		if m.Target == domain.TargetSelf && float64(me.HP)/float64(me.MaxHP) > 0.6 {
 			return 55 // set up while healthy
 		}
@@ -137,6 +164,56 @@ func (a *HeuristicAgent) statusScore(m domain.Move, me, foe engine.Pokemon) floa
 	return 10
 }
 
+// maxStage is the stat-stage ceiling a boost cannot move a stat past.
+const maxStage = 6
+
+// allBoostsAtCap reports whether every stat a move would raise is already
+// pinned at +6, so the move cannot change anything. Only positive entries are
+// considered: a move that lowers one of its own stats as a cost still has that
+// cost to pay, so it is not dead.
+//
+// Deliberately conservative — an unrecognized stat key returns false rather
+// than assuming a cap, because wrongly marking a live move dead is far worse
+// than missing a dead one.
+func allBoostsAtCap(boosts map[string]int, st engine.Stages) bool {
+	sawPositive := false
+	for stat, n := range boosts {
+		if n <= 0 {
+			continue
+		}
+		sawPositive = true
+		cur, known := stageValue(st, stat)
+		if !known || cur < maxStage {
+			return false
+		}
+	}
+	return sawPositive
+}
+
+// stageValue maps a move's boost key to the matching stat stage. The move
+// schema and the stage struct use different names for the same stats (see the
+// note in backlog/2026-08-03T20-01-evs-natures-and-ivs.md), so this is explicit
+// and reports known=false rather than defaulting on an unrecognized key.
+func stageValue(st engine.Stages, stat string) (int, bool) {
+	switch stat {
+	case "atk", "attack":
+		return st.Atk, true
+	case "def", "defense":
+		return st.Def, true
+	case "spa", "spatk":
+		return st.SpA, true
+	case "spd", "spdef":
+		return st.SpD, true
+	case "spe", "speed":
+		return st.Spe, true
+	case "accuracy", "acc":
+		return st.Acc, true
+	case "evasion", "eva":
+		return st.Eva, true
+	}
+	return 0, false
+}
+
 // bestDamage returns the highest expected damage atk can deal to def.
 // defSc is the defender side's screens so the estimate honors Reflect /
 // Light Screen / Aurora Veil.
diff --git a/internal/ai/heuristic_deadmove_test.go b/internal/ai/heuristic_deadmove_test.go
new file mode 100644
index 00000000..93fed2c0
--- /dev/null
+++ b/internal/ai/heuristic_deadmove_test.go
@@ -0,0 +1,220 @@
+package ai
+
+import (
+	"context"
+	"testing"
+
+	"pokearena/internal/domain"
+	"pokearena/internal/engine"
+)
+
+// The heuristic used to spend turns on moves the rules make impossible: a
+// status the target already had, a boost with the stat pinned at +6, a heal at
+// full HP. Found by internal/eval's verifiable-error metric, which counted 27
+// boost-at-cap turns across six games and caught the agent re-applying Thunder
+// Wave to an already-paralysed target for six consecutive turns.
+//
+// The tests below pin each case at the scoring level, where the intent is
+// legible, plus one end-to-end check that Decide actually picks something else.
+
+// deadMoveView builds a one-move view so a single scoring branch is isolated.
+func deadMoveView(self engine.Pokemon, moveID string, foe engine.Pokemon) View {
+	self.Moves = []engine.MoveSlot{{MoveID: moveID, PP: 10, MaxPP: 10}}
+	return View{
+		Me:    0,
+		Turn:  5,
+		Self:  engine.Side{Trainer: "P0", Active: 0, Team: []engine.Pokemon{self}},
+		Foe:   foe,
+		Phase: engine.PhaseChoosing,
+	}
+}
+
+func healthy(name string, t1 domain.Type) engine.Pokemon {
+	return engine.Pokemon{Name: name, Type1: t1, HP: 100, MaxHP: 100}
+}
+
+func TestHeuristic_ScoresProvablyDeadMovesBelowEverything(t *testing.T) {
+	d := loadDex(t)
+	a := NewHeuristicAgent(d)
+
+	cases := []struct {
+		name string
+		move string
+		self engine.Pokemon
+		foe  engine.Pokemon
+		dead bool
+	}{
+		{
+			name: "status on an already-statused foe is dead",
+			move: "thunder-wave",
+			self: healthy("Pikachu", "electric"),
+			foe: func() engine.Pokemon {
+				p := healthy("Starmie", "water")
+				p.Status = engine.StatusParalysis
+				return p
+			}(),
+			dead: true,
+		},
+		{
+			name: "status on a clean foe is live",
+			move: "thunder-wave",
+			self: healthy("Pikachu", "electric"),
+			foe:  healthy("Starmie", "water"),
+			dead: false,
+		},
+		{
+			name: "heal at full HP is dead",
+			move: "recover",
+			self: healthy("Starmie", "water"),
+			foe:  healthy("Snorlax", "normal"),
+			dead: true,
+		},
+		{
+			name: "heal below full HP is live",
+			move: "recover",
+			self: func() engine.Pokemon {
+				p := healthy("Starmie", "water")
+				p.HP = 40
+				return p
+			}(),
+			foe:  healthy("Snorlax", "normal"),
+			dead: false,
+		},
+		{
+			name: "boost with the stat at +6 is dead",
+			move: "swords-dance",
+			self: func() engine.Pokemon {
+				p := healthy("Scyther", "bug")
+				p.Stages.Atk = 6
+				return p
+			}(),
+			foe:  healthy("Snorlax", "normal"),
+			dead: true,
+		},
+		{
+			name: "boost with room left is live",
+			move: "swords-dance",
+			self: func() engine.Pokemon {
+				p := healthy("Scyther", "bug")
+				p.Stages.Atk = 4
+				return p
+			}(),
+			foe:  healthy("Snorlax", "normal"),
+			dead: false,
+		},
+	}
+
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			if _, ok := d.Moves[c.move]; !ok {
+				t.Skipf("move %q not in the dataset", c.move)
+			}
+			v := deadMoveView(c.self, c.move, c.foe)
+			got := a.score(v, engine.Action{Kind: engine.ActionMove, Index: 0})
+			if c.dead && got != deadMoveScore {
+				t.Errorf("score = %v, want deadMoveScore (%v)", got, deadMoveScore)
+			}
+			if !c.dead && got <= 0 {
+				t.Errorf("score = %v, want a positive score for a move that can act", got)
+			}
+		})
+	}
+}
+
+// Rest carries no Primary/Self effect block in this dataset — just a "heal"
+// flag the engine special-cases by move id — so it never reaches statusScore's
+// heal branch and lands on the generic fallback instead. The Rest exception in
+// that branch is therefore unreachable today; it is kept because it is correct
+// if the move ever grows an effect block, and pinned here so the reason is not
+// re-derived.
+//
+// The consequence is that the heuristic under-values Rest. That is a
+// pre-existing strategy weakness, not a provable-waste bug, and it is
+// deliberately NOT fixed here: teaching the baseline to use Rest well would
+// change how it plays, and every published number is measured against this
+// opponent. Correctness fixes and strength changes are separate decisions.
+func TestHeuristic_RestIsNotRoutedThroughTheHealBranch(t *testing.T) {
+	d := loadDex(t)
+	rest, ok := d.Moves["rest"]
+	if !ok {
+		t.Skip("rest not in the dataset")
+	}
+	if rest.Primary != nil || rest.Self != nil {
+		t.Skip("rest grew an effect block; the heal branch now applies and this test is obsolete")
+	}
+	a := NewHeuristicAgent(d)
+
+	// Full HP, no status: nothing to gain, yet it is not scored dead, because
+	// the guard cannot see it.
+	v := deadMoveView(healthy("Snorlax", "normal"), "rest", healthy("Starmie", "water"))
+	got := a.score(v, engine.Action{Kind: engine.ActionMove, Index: 0})
+	if got == deadMoveScore {
+		t.Fatal("rest reached the dead-move guard; the dataset or the branch changed — update this test")
+	}
+	// It scores low enough that any real attack outranks it, which is why the
+	// gap costs turns only in positions where nothing else works.
+	if got > 10 {
+		t.Errorf("rest scored %v; expected the low generic fallback", got)
+	}
+}
+
+// The regression that started this: a dead status move in an early slot used to
+// tie at 0 with a zero-damage attack and win on slot order, so the agent
+// replayed it every turn. Decide must now reach past it.
+func TestHeuristic_DoesNotRepeatADeadStatusMove(t *testing.T) {
+	d := loadDex(t)
+	a := NewHeuristicAgent(d)
+
+	self := healthy("Pikachu", "electric")
+	// Thunder Wave first, a real attack second — the ordering that used to lose.
+	self.Moves = []engine.MoveSlot{
+		{MoveID: "thunder-wave", PP: 10, MaxPP: 10},
+		{MoveID: "thunderbolt", PP: 10, MaxPP: 10},
+	}
+	foe := healthy("Starmie", "water")
+	foe.Status = engine.StatusParalysis
+
+	v := View{
+		Me:    0,
+		Turn:  5,
+		Self:  engine.Side{Trainer: "P0", Active: 0, Team: []engine.Pokemon{self}},
+		Foe:   foe,
+		Phase: engine.PhaseChoosing,
+	}
+	act, err := a.Decide(context.Background(), v)
+	if err != nil {
+		t.Fatalf("decide: %v", err)
+	}
+	if act.Kind == engine.ActionMove && act.Index == 0 {
+		t.Error("picked Thunder Wave against an already-paralysed foe; the dead-move guard is not holding")
+	}
+}
+
+func TestAllBoostsAtCap(t *testing.T) {
+	at6 := engine.Stages{Atk: 6, Def: 6, Spe: 3}
+
+	cases := []struct {
+		name   string
+		boosts map[string]int
+		want   bool
+	}{
+		{"single stat at cap", map[string]int{"atk": 1}, true},
+		{"single stat with room", map[string]int{"spe": 1}, false},
+		{"all boosted stats at cap", map[string]int{"atk": 1, "def": 1}, true},
+		// One stat with room means the move still does something.
+		{"one of two has room", map[string]int{"atk": 1, "spe": 1}, false},
+		// A self-drop is a cost still to be paid, so the move is not inert.
+		{"negative entries ignored", map[string]int{"atk": 1, "def": -1}, true},
+		{"only negatives is not capped", map[string]int{"def": -1}, false},
+		// Never guess on a key we do not recognize.
+		{"unknown stat key", map[string]int{"mystery": 1}, false},
+		{"empty", map[string]int{}, false},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			if got := allBoostsAtCap(c.boosts, at6); got != c.want {
+				t.Errorf("allBoostsAtCap(%v) = %v, want %v", c.boosts, got, c.want)
+			}
+		})
+	}
+}
diff --git a/internal/eval/depthsweep_test.go b/internal/eval/depthsweep_test.go
new file mode 100644
index 00000000..0719ce88
--- /dev/null
+++ b/internal/eval/depthsweep_test.go
@@ -0,0 +1,54 @@
+package eval
+
+import (
+	"fmt"
+	"os"
+	"testing"
+
+	"pokearena/internal/ai"
+	"pokearena/internal/engine"
+)
+
+// TestDepthSweep reproduces docs/benchmark.md §6's only load-bearing
+// measurement: expectimax vs the heuristic at depths 1-3, 6 teams x 20 seeds x
+// 2 orientations = 240 games per depth, exactly the published methodology.
+//
+// Gated behind POKEARENA_DEPTH_SWEEP=1 rather than testing.Short because it
+// takes ~22 minutes — far too slow for the suite, but committed so the numbers
+// under §6 can be re-derived instead of taken on trust. It prints the table; it
+// asserts nothing, because the point is to produce figures for a human to read
+// into the document, not to pin them (they legitimately move when the baseline
+// or the team library changes, which is the whole reason this exists).
+//
+//	POKEARENA_DEPTH_SWEEP=1 go test ./internal/eval -run TestDepthSweep -v -timeout 60m
+func TestDepthSweep(t *testing.T) {
+	if os.Getenv("POKEARENA_DEPTH_SWEEP") == "" {
+		t.Skip("set POKEARENA_DEPTH_SWEEP=1 to run the ~22-minute sweep")
+	}
+	d := loadDex(t)
+	lib, err := LoadTeamLibrary(libraryPath, d)
+	if err != nil {
+		t.Fatal(err)
+	}
+	seeds := SeedRange(20)
+	heur := Contestant{Name: "heuristic", New: func(uint64) ai.Agent { return ai.NewHeuristicAgent(d) }}
+
+	for _, depth := range []int{1, 2, 3} {
+		dp := depth
+		exp := Contestant{Name: fmt.Sprintf("expectimax-d%d", dp),
+			New: func(uint64) ai.Agent { return ai.NewExpectimaxAgentFixed(d, dp) }}
+		wins, games := 0, 0
+		for _, team := range lib.Teams {
+			picks := [2][]engine.TeamPick{team.Picks, team.Picks}
+			mr, err := RunMatch(d, exp, heur, team.Name, picks, seeds, 0)
+			if err != nil {
+				t.Fatal(err)
+			}
+			wins += mr.AWins
+			games += mr.AWins + mr.BWins + mr.Draws
+		}
+		lo, hi := WilsonInterval(float64(wins), games, 1.96)
+		fmt.Printf("depth %d: %d/%d = %.1f%%  95%% CI [%.1f%%, %.1f%%]\n",
+			dp, wins, games, 100*float64(wins)/float64(games), 100*lo, 100*hi)
+	}
+}
diff --git a/internal/eval/verifiable_test.go b/internal/eval/verifiable_test.go
index 57ac42aa..6409a566 100644
--- a/internal/eval/verifiable_test.go
+++ b/internal/eval/verifiable_test.go
@@ -187,46 +187,6 @@ func TestScoreVerifiable_SeparatesRandomFromCompetentOnTypeImmunity(t *testing.T
 	}
 }
 
-// The metric found a real defect in the heuristic baseline on its first run:
-// it spends turns on stat boosts that cannot apply because the stat is already
-// at +6, and re-applies a status the target already has.
-//
-// This test pins the finding rather than the bot. It is deliberately NOT a fix:
-// the heuristic is the benchmark's reference opponent, and changing how it
-// plays would invalidate every published number measured against it (the depth
-// sweep in docs/benchmark.md §6 above all). Fixing it is a decision with
-// benchmark-wide consequences, so it is recorded here and left to be made
-// deliberately. If it is fixed, this test should be deleted in the same commit.
-func TestScoreVerifiable_HeuristicBoostsAtCap(t *testing.T) {
-	d := loadDex(t)
-	lib, err := LoadTeamLibrary(libraryPath, d)
-	if err != nil {
-		t.Fatalf("load team library: %v", err)
-	}
-
-	byKind := map[ErrKind]int{}
-	for _, team := range lib.Teams {
-		agents := [2]ai.Agent{ai.NewHeuristicAgent(d), ai.NewHeuristicAgent(d)}
-		picks := [2][]engine.TeamPick{team.Picks, team.Picks}
-		_, turns, err := CaptureStored(d, agents, picks, 31337, 0)
-		if err != nil {
-			t.Fatalf("capture: %v", err)
-		}
-		errs, _, _, err := ScoreVerifiable(d, 0, turns)
-		if err != nil {
-			t.Fatalf("score: %v", err)
-		}
-		for _, e := range errs {
-			byKind[e.Kind]++
-		}
-	}
-	t.Logf("heuristic provable errors by kind: %v", byKind)
-
-	if byKind[ErrBoostAtCap] == 0 {
-		t.Skip("heuristic no longer boosts at cap — if it was fixed, delete this test")
-	}
-}
-
 func TestAggregateVerifiable_SortsCleanestFirstAndCountsCleanGames(t *testing.T) {
 	results := []VerifiableBattle{
 		{Contestant: "sloppy", Decisions: 50, Errors: []VerifiableError{

From 6572857b12edde60564ee20ee99a69fd194711bc Mon Sep 17 00:00:00 2001
From: Died Broke <shaumik@users.noreply.github.com>
Date: Wed, 12 Aug 2026 03:05:37 +0000
Subject: [PATCH 15/15] fix the lint findings CI caught, including a real
 evaluation-order bug
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

golangci-lint could not run in the authoring environment (its build targets an
older Go than this repo), so these landed unchecked. It is now runnable there
via `go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2`
with GOTOOLCHAIN pinned, which is how the fixes below were verified — 0 issues.

One of the findings is a genuine bug, not style. loadConfig ended with

	return c, json.Unmarshal(b, &c)

and Go does not order the read of `c` against the call that fills it, so the
function could return a zero-valued config while reporting no error. It happened
to work in the end-to-end run; it was never guaranteed to. Split into an
explicit unmarshal-then-return.

The rest:

- exec.Command -> exec.CommandContext (noctx), threading a context from run()
  that is canceled on teardown, so a straggling child process is killed rather
  than orphaned. Deliberately no timeout on it: the per-game wall clock lives in
  the runner script, which also kills the CLI's own children, and interrupt
  still lets in-flight games finish writing so no half-written result file is
  mistaken for a completed game.
- fmt.Errorf %v -> %w so the underlying exec error stays unwrappable.
- Unchecked fmt.Sscanf -> strconv.Atoi with the error handled. An unparseable
  winner= field now leaves the game at -1 (unfinished) rather than silently
  reading as a win for side 0 -- an under-counted result beats a fabricated one.
- Two De Morgan simplifications flagged by staticcheck.
- gofumpt on the depth-sweep test.
- British spellings the repo's misspell linter rejects.

Verified: build, vet, vet -tags=integration, golangci-lint v2.12.2, and the full
suite under -race, all clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZ6bbo8VkaDrs47Yz8v84U
---
 cmd/bench-run/main.go                  | 32 +++++++++++++++++++++-----
 cmd/decision-sim/main.go               |  2 +-
 internal/ai/heuristic.go               |  4 ++--
 internal/ai/heuristic_deadmove_test.go |  4 ++--
 internal/eval/benchplan.go             |  2 +-
 internal/eval/depthsweep_test.go       |  6 +++--
 internal/eval/verifiable.go            |  4 ++--
 7 files changed, 38 insertions(+), 16 deletions(-)

diff --git a/cmd/bench-run/main.go b/cmd/bench-run/main.go
index e9b1b94f..5e837bb5 100644
--- a/cmd/bench-run/main.go
+++ b/cmd/bench-run/main.go
@@ -26,6 +26,7 @@
 package main
 
 import (
+	"context"
 	"encoding/json"
 	"flag"
 	"fmt"
@@ -34,6 +35,7 @@ import (
 	"os/exec"
 	"os/signal"
 	"path/filepath"
+	"strconv"
 	"strings"
 	"sync"
 	"syscall"
@@ -122,6 +124,12 @@ func main() {
 }
 
 func run(todo []eval.PlannedGame, outDir, script string, conc int) {
+	// Canceled only when run returns, so any child process still alive at
+	// teardown is killed rather than orphaned. Interrupt deliberately does NOT
+	// cancel it — see below, in-flight games are allowed to finish writing.
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
 	// Ctrl-C stops scheduling new games but lets in-flight ones finish writing,
 	// so an interrupted run leaves no half-written result file to be mistaken
 	// for a completed game on resume.
@@ -154,7 +162,7 @@ func run(todo []eval.PlannedGame, outDir, script string, conc int) {
 			defer wg.Done()
 			defer func() { <-sem }()
 
-			res, err := playGame(g, outDir, script)
+			res, err := playGame(ctx, g, outDir, script)
 			mu.Lock()
 			defer mu.Unlock()
 			if err != nil {
@@ -173,15 +181,19 @@ func run(todo []eval.PlannedGame, outDir, script string, conc int) {
 // playGame runs one game and records it. The result file is written only on a
 // clean run: a game that errored is left absent so re-running retries it rather
 // than baking a failure into the dataset.
-func playGame(g eval.PlannedGame, outDir, script string) (GameResult, error) {
+func playGame(ctx context.Context, g eval.PlannedGame, outDir, script string) (GameResult, error) {
 	started := time.Now()
 	// The entrant id travels as the trainer name, so attribution is recorded in
 	// the battle row rather than inferred later from a scratch file.
-	cmd := exec.Command("bash", script,
+	//
+	// No timeout on this context: the per-game wall-clock cap lives in the
+	// runner script, which also kills the CLI's own children. This context
+	// exists only so a straggler dies at teardown instead of being orphaned.
+	cmd := exec.CommandContext(ctx, "bash", script,
 		g.Entrant.Harness, g.Entrant.Model, g.Team, g.Label, outDir, g.Entrant.ID)
 	out, err := cmd.CombinedOutput()
 	if err != nil {
-		return GameResult{}, fmt.Errorf("%v: %s", err, lastLine(string(out)))
+		return GameResult{}, fmt.Errorf("%s: %w", lastLine(string(out)), err)
 	}
 
 	bid, winner, status := parseRunnerOutput(string(out))
@@ -210,7 +222,12 @@ func parseRunnerOutput(out string) (bid string, winner int, status string) {
 			case strings.HasPrefix(f, "bid="):
 				bid = strings.TrimPrefix(f, "bid=")
 			case strings.HasPrefix(f, "winner="):
-				fmt.Sscanf(strings.TrimPrefix(f, "winner="), "%d", &winner)
+				// A field we cannot parse leaves winner at -1 (unfinished),
+				// which is the safe reading — better an under-counted result
+				// than a fabricated win.
+				if n, convErr := strconv.Atoi(strings.TrimPrefix(f, "winner=")); convErr == nil {
+					winner = n
+				}
 			case strings.HasPrefix(f, "(status="):
 				status = strings.TrimSuffix(strings.TrimPrefix(f, "(status="), ")")
 			}
@@ -247,7 +264,10 @@ func loadConfig(path string) (eval.BenchConfig, error) {
 	if err != nil {
 		return c, err
 	}
-	return c, json.Unmarshal(b, &c)
+	if err := json.Unmarshal(b, &c); err != nil {
+		return c, err
+	}
+	return c, nil
 }
 
 func writeJSON(path string, v any) error {
diff --git a/cmd/decision-sim/main.go b/cmd/decision-sim/main.go
index 81d92a93..c97a0802 100644
--- a/cmd/decision-sim/main.go
+++ b/cmd/decision-sim/main.go
@@ -49,7 +49,7 @@ type turn struct {
 // randomPolicySeed fixes the random policy's RNG so a run is reproducible.
 const randomPolicySeed = 20260811
 
-// policy is one labelled contestant for the scored seat. The label becomes the
+// policy is one labeled contestant for the scored seat. The label becomes the
 // "model" column in the aggregated table.
 type policy struct {
 	label string
diff --git a/internal/ai/heuristic.go b/internal/ai/heuristic.go
index c5cc4a29..b56bb68d 100644
--- a/internal/ai/heuristic.go
+++ b/internal/ai/heuristic.go
@@ -119,7 +119,7 @@ func (a *HeuristicAgent) switchScore(in, foe, cur engine.Pokemon, w *engine.Weat
 // damaging move that deals no damage also scores 0, and Decide breaks ties
 // toward the earliest legal action, so a dead status move sitting in an early
 // move slot would win the tie and be replayed every turn — the agent spent six
-// consecutive turns re-applying Thunder Wave to an already-paralysed target
+// consecutive turns re-applying Thunder Wave to an already-paralyzed target
 // this way. Below every alternative including a switch, because giving up a
 // turn to reposition genuinely beats spending it on a guaranteed no-op.
 const deadMoveScore = -1000
@@ -142,7 +142,7 @@ func (a *HeuristicAgent) statusScore(m domain.Move, me, foe engine.Pokemon) floa
 		// At full HP there is nothing to restore. Rest is the exception: it
 		// also cures status, so it still buys something when the user is
 		// statused even though the heal itself is wasted.
-		if me.HP >= me.MaxHP && !(p.Rest && me.Status != engine.StatusNone) {
+		if me.HP >= me.MaxHP && (!p.Rest || me.Status == engine.StatusNone) {
 			return deadMoveScore
 		}
 		missing := float64(me.MaxHP-me.HP) / float64(me.MaxHP)
diff --git a/internal/ai/heuristic_deadmove_test.go b/internal/ai/heuristic_deadmove_test.go
index 93fed2c0..c4464d6c 100644
--- a/internal/ai/heuristic_deadmove_test.go
+++ b/internal/ai/heuristic_deadmove_test.go
@@ -12,7 +12,7 @@ import (
 // status the target already had, a boost with the stat pinned at +6, a heal at
 // full HP. Found by internal/eval's verifiable-error metric, which counted 27
 // boost-at-cap turns across six games and caught the agent re-applying Thunder
-// Wave to an already-paralysed target for six consecutive turns.
+// Wave to an already-paralyzed target for six consecutive turns.
 //
 // The tests below pin each case at the scoring level, where the intent is
 // legible, plus one end-to-end check that Decide actually picks something else.
@@ -186,7 +186,7 @@ func TestHeuristic_DoesNotRepeatADeadStatusMove(t *testing.T) {
 		t.Fatalf("decide: %v", err)
 	}
 	if act.Kind == engine.ActionMove && act.Index == 0 {
-		t.Error("picked Thunder Wave against an already-paralysed foe; the dead-move guard is not holding")
+		t.Error("picked Thunder Wave against an already-paralyzed foe; the dead-move guard is not holding")
 	}
 }
 
diff --git a/internal/eval/benchplan.go b/internal/eval/benchplan.go
index 60bf069f..276e043d 100644
--- a/internal/eval/benchplan.go
+++ b/internal/eval/benchplan.go
@@ -16,7 +16,7 @@ import (
 // reached through Claude Code and the same model reached through a bare API
 // harness are different contestants, because the agent runtime around the model
 // is part of what is being measured (tool-calling loop, retries, context
-// handling). Labelling by contestant makes both comparisons fall out of one
+// handling). Labeling by contestant makes both comparisons fall out of one
 // dataset: hold the model fixed and vary the harness to measure harnesses, hold
 // the harness fixed and vary the model to measure models.
 
diff --git a/internal/eval/depthsweep_test.go b/internal/eval/depthsweep_test.go
index 0719ce88..1b9d3e30 100644
--- a/internal/eval/depthsweep_test.go
+++ b/internal/eval/depthsweep_test.go
@@ -35,8 +35,10 @@ func TestDepthSweep(t *testing.T) {
 
 	for _, depth := range []int{1, 2, 3} {
 		dp := depth
-		exp := Contestant{Name: fmt.Sprintf("expectimax-d%d", dp),
-			New: func(uint64) ai.Agent { return ai.NewExpectimaxAgentFixed(d, dp) }}
+		exp := Contestant{
+			Name: fmt.Sprintf("expectimax-d%d", dp),
+			New:  func(uint64) ai.Agent { return ai.NewExpectimaxAgentFixed(d, dp) },
+		}
 		wins, games := 0, 0
 		for _, team := range lib.Teams {
 			picks := [2][]engine.TeamPick{team.Picks, team.Picks}
diff --git a/internal/eval/verifiable.go b/internal/eval/verifiable.go
index 013c2261..a0ea965f 100644
--- a/internal/eval/verifiable.go
+++ b/internal/eval/verifiable.go
@@ -34,7 +34,7 @@ import (
 //     was one from the fog-of-war view it was handed. Attacking into a Levitate
 //     immunity is a wasted turn, but the ability may be hidden, so it is not
 //     charged here; attacking a Ghost with a Normal move is charged, because the
-//     defender's types are public. Penalising unknowable information would make
+//     defender's types are public. Penalizing unknowable information would make
 //     this a luck metric.
 //
 // The result is a floor, not a full account of skill: a policy can score zero
@@ -118,7 +118,7 @@ func CheckAction(dex *domain.Dex, v ai.View, act engine.Action) []VerifiableErro
 	// 2. Healing at full HP. Rest is excluded when the user is statused: it
 	// cures, so the turn buys something even with the heal wasted.
 	if selfHeal(mv) > 0 && me.HP >= me.MaxHP {
-		if !(isRest(mv) && me.Status != engine.StatusNone) {
+		if !isRest(mv) || me.Status == engine.StatusNone {
 			add(ErrHealAtFull, "%s at %d/%d HP", mv.Name, me.HP, me.MaxHP)
 		}
 	}