feat(control): inspect ctl sample messages (design + implementation)

CHANGELOG.md

@@ -3,9 +3,11 @@
 - Eval: `EvalSample` and `EvalSampleSummary` now record `turn_count` and the sample's token limit (`token_limit`, `token_limit_type`, and metered `token_limit_usage`).
 - Analysis: `samples_df` gains default `turn_count` and `token_limit_usage` columns, and `evals_df` configuration columns gain `token_limit_type`.
 - Control Channel: `inspect ctl sample list` now shows per-sample turn count and, when a token limit is configured, its computed usage and configured ceiling.
+- Control Channel: New `inspect ctl sample messages TASK SID [EPOCH]` reads a running (or buffered-but-unlogged) sample's current conversation as a snapshot, with `--tail`/`--all`/`--full`.
 - Control Channel: `inspect ctl sample errors` now filters errored/retried samples server-side, so triage polls of large evals no longer build and ship the full sample listing.
 - Control Channel: `inspect ctl` sample commands given an exact full task id now stop discovery at the process hosting it, skipping busy-retry delays from older sibling processes.
 - Control Channel: `inspect ctl config --key NAME LIMIT` retunes any named `concurrency()` limit mid-flight, without the registering code opting in; the config view lists the registered keys.
+- Control Channel: `inspect ctl sample events --full` now pretty-prints the raw events instead of rendering a mostly-empty summary table.
 - Control Channel: `inspect ctl config` can now retune `--timeout`, `--attempt-timeout` and `--max-retries` mid-flight, reaching even generate calls already retrying (pass `clear` to restore launch config).
 - Control Channel: `inspect ctl sample list` now caps its listing at the 100 most relevant rows by default (`--limit`/`--all` to adjust, `--status` to filter) and reports a complete status histogram plus a `truncated` flag in its envelope.
 - Model API: Fixed `max_retries` to permit N retries (N+1 total attempts); previously it counted total attempts, so e.g. `max_retries=1` never actually retried.

design/control-channel.md

@@ -79,6 +79,9 @@ inspect ctl sample errors [TASK]            # triage: samples that errored or we
 inspect ctl sample events TASK SID [EPOCH]  # one sample's transcript events (cursored pull)
                                             #   --cursor CURSOR / --tail N / --type / --full / --since-time / --until
                                             #   (-f / --follow push is phase 4)
+
+inspect ctl sample messages TASK SID [EPOCH]  # one sample's current conversation, snapshot
+                                              #   --tail N / --all / --full (see "Sample messages read")
 inspect ctl process list                    # running Inspect processes (pid / keep-alive / tasks)
 inspect ctl process keep [PID]              # keep a running process alive after its eval finishes
 inspect ctl process release [PID]           # release a lingering keep-alive process
@@ -136,6 +139,7 @@ inspect ctl
 │   ├── show TASK SID [EPOCH]       # was: sample
 │   ├── errors [TASK]               # was: errors (no TASK = across all tasks)
 │   ├── events TASK SID [EPOCH]     # was: events (phase-4 --follow lands here unchanged)
+│   ├── messages TASK SID [EPOCH]   # shipped: current-conversation snapshot (see "Sample messages read")
 │   ├── cancel TASK SID [EPOCH]     # shipped (EPOCH required when the task runs >1 epoch)
 │   └── requeue TASK SID [EPOCH]    # planned: requeue
 ├── config [TASK] [...]         # was: limits + buffer; view/retune launch config, scope per knob
@@ -207,6 +211,7 @@ Known accepted edge in the busy handling: the sole-server rule counts *discovere
 | `ctl cancel-sample <id> <sid>` | **shipped** (as noun form only) | `ctl sample cancel TASK SID [EPOCH]` | compound verb dissolves; `EPOCH` required when the task runs >1 epoch |
 | `ctl requeue <id> <sid>` | planned | `ctl sample requeue TASK SID [EPOCH]` | `EPOCH` required when the task runs >1 epoch |
 | `ctl set-limit <id> --time N` | planned | fold into `ctl config TASK --time-limit/--token-limit/--message-limit` | all retunable launch flags in one surface, same spellings as `inspect eval`; task-scoped knobs, so `TASK` follows the mutation selector rule |
+| — | **shipped** | `ctl sample messages TASK SID [EPOCH]` | new read: the sample's current conversation, summary-projected — see "Sample messages read" |
 | — | new | `ctl process list` | pids, keep-alive status, hosted task_ids; shipped with the reorg — the group's other verbs take a `PID` selector, so pids must be enumerable in-group |
 | — | later | `ctl eval-set list / show / cancel` | group slot ready-made |
 
@@ -273,6 +278,7 @@ The conceptual surface, independent of wire protocol. Each operation becomes eit
 - List running evals (id, task, status, started_at, sample counts).
 - Eval status detail (config, current limits, sample queue depth, in-flight samples, completed counts, model usage so far).
 - List samples within an eval (status, started_at, current model/tool, token usage).
+- Read one sample's current conversation — the live `TaskState.messages`, summary-projected (see "Sample messages read").
 
 **Read (eval-set-level)**
 - List eval-sets.
@@ -351,6 +357,7 @@ The URL scheme has one rule — **three scopes, three roots**: process-scoped op
 | Errored/retried samples only (skips pending-row synthesis) | `GET /evals/<id>/samples?filter=errors` (`filter` is an extensible enum; unrecognized values are rejected with 422) | 2 ✅ |
 | Sample transcript events (pull) | `GET /evals/<id>/sample/events?sample_id=<sid>&epoch=<n>&since=<cursor>` (JSON) | 2 ✅ |
 | Sample transcript events (push) | the pull URL with `Accept: text/event-stream` (SSE) | 4 |
+| Sample conversation messages (snapshot) | `GET /evals/<id>/sample/messages?sample_id=<sid>&epoch=<n>` | 2 ✅ |
 | Eval-wide transcript fan-in (push only) | `GET /evals/<id>/samples/events` (SSE) | 4 |
 | Flush buffered samples to the log | `POST /tasks/<task-id>/log-flush` | 3 ✅ |
 | Read / modify retunable config (concurrency limits, buffer params) | `GET`+`PATCH /config` (process) and `/tasks/<task-id>/config` (task) | 3 ✅ (max-samples / max-sandboxes / max-subprocesses / max-connections / named `concurrency()` keys via `--key` / log-buffer / log-shared) |
@@ -741,6 +748,34 @@ A wall-clock window is a snapshot query (no exactly-once requirement), which is
 
 Unlocks watchdog agents (cursored polling). Live-render TUIs follow once phase-4 push lands.
 
+### Sample messages read (shipped)
+
+> **Status: shipped** (extends the phase-2 read surface). Motivated by issue #70: agents live-watching a sample want the *conversation* (`.messages`), not just the transcript firehose — in particular for samples that exist only in the process buffer, not yet written to the `.eval` log.
+
+**The gap.** A human watching a live sample opens `inspect view` and reads the conversation. An agent has no equivalent: the closest surface is `sample events`, and reconstructing "what does the conversation look like right now" from the transcript is genuinely hard — the messages ride inside `ModelEvent.input` / `output` (the default projection omits the `input` messages entirely — only a truncated completion survives — and `--full` is enormous), solver / agent code can assign `state.messages` without emitting any model event, and compaction rewrites the conversation while the event stream records only a `compaction` marker. Meanwhile the thing the agent wants already exists, materialized, in the running sample's `TaskState.messages` — serve it directly. The buffered case is what makes this control-channel work rather than `inspect log` work: for a running sample — or a completed one still sitting in the log buffer awaiting a flush — the conversation exists only inside the eval process, and the control channel is the only surface that can see it.
+
+**Surface.**
+
+- **HTTP:** `GET /evals/<id>/sample/messages?sample_id=<sid>&epoch=<n>` — attempt-scoped beside `sample` and `sample/events`, with `sample_id` a query param per the reserved-characters rule.
+- **CLI:** `inspect ctl sample messages TASK SID [EPOCH]` — a new verb in the `sample` noun group. Standard selector conventions apply: `TASK` is required (a `SAMPLE_ID` follows it), and `EPOCH` defaults to 1 on reads with the resolved `{sample_id, epoch}` echoed in the envelope.
+
+**Snapshot, not stream — deliberately no cursor.** Transcript events earned a cursor because they're append-only; messages are not. Compaction replaces a prefix of the list with a summary, and solver / agent code can reorder, edit, or wholesale reassign `state.messages` between reads. An index cursor over a rewritable sequence would promise the exactly-once resume it can't deliver (the phase-2 cursor design's whole job is "monotonic, gap-free" — this sequence is neither). So `messages` is a **snapshot read**: each call returns the current conversation (or a tail of it), enveloped with `as_of`, the resolved `{sample_id, epoch}`, the sample's `status`, and the total message `count`. The watch loop composes with the existing reads: poll `sample list --active-since` for "what changed", then drill into `messages` on the sample of interest; the `count` moving (or not) between polls is the cheap staleness signal. Incremental *event*-grain watching stays `sample events`' job. If a delta shape is ever wanted here, the right token is a state version invalidated by any rewrite — never an index — but that's deferred until a consumer actually needs it; for the same reason there is no phase-4 SSE `--follow` on messages (follow the events stream, or poll).
+
+**Summary-shaped by default.** A conversation is typically the largest object in a sample — a full multi-turn agentic `ChatMessage` list with tool outputs can exceed a watching agent's entire context. Mirroring the events projection:
+
+- **Default: compact projection per message** — index, message id, role, truncated text (non-text content items summarized as `[image]` / `[audio]` / ...), tool-call function names + truncated arguments on assistant messages, truncated output on tool messages.
+- **`--tail N`** — the last N messages; the unseeded default is a recent tail (e.g. the last 20), per the "never an empty or overwhelming first page" rule from the events read. `--all` for the whole list. (This is not the selector-widening `--all` rejected in "Selector conventions" — the sample is already pinned by its positional selectors, so the flag can only widen over *messages*; there is no scope ambiguity to smuggle into a flag.)
+- **`--full`** — raw `ChatMessage` JSON instead of the projection (combines with `--tail` to keep it bounded).
+- **`--json`** throughout, per the agent output contract.
+
+**Data source: the same running-vs-terminal split as the sibling reads — with one new plumbing piece on the running side.** Running samples serve the live `TaskState.messages` — in-memory, no log involved, which is exactly why buffered-only samples work. But unlike the sibling reads, whose running-side data sources already hang off `ActiveSample` (`transcript` for `events`, the counter fields for `show` / `list`), nothing the control server can reach holds the live `TaskState` today: it lives in the `_sample_state` ContextVar (`solver/_task_state.py`), set inside the sample's own async context — invisible from the server's request-handler task. The slice therefore adds a live-state handle on `ActiveSample`. Capturing it once at sample start isn't enough: a solver can *replace* the `TaskState` object outright by returning a new one (e.g. a `fork()` result or a deepcopy), so the handle must be refreshed wherever state is threaded step-to-step — the `Chain` / `Plan` step loops call `set_sample_state` after each solver step, and `set_sample_state` refreshes the `ActiveSample` field, the same running-sample update flow `set_active_sample_total_messages` already rides. The step-boundary refresh is a compare-and-swap (the step passes the state it superseded; the handle only moves if it currently points at that state): the handle is a plain attribute on the shared `ActiveSample`, which a `fork()` subtask's copied context still reaches through `sample_active()` — so without the guard a `Chain` / `Plan` running *inside* a fork branch (an advertised pattern; `solver_subtask` special-cases `Chain`) would overwrite the handle with its branch's conversation, non-deterministically (last-finishing branch wins), until the enclosing solver step returned. The guard makes ownership lineage-based: a fork branch threads a deepcopy the handle never pointed at, so its refreshes can't land, while the main thread's always do. (Neither of the mechanisms one might expect to carry this refresh actually fires per step: the `@solver` decorator's solver-boundary `set_sample_state` is dead code — it patches `__call__` as an *instance* attribute, which Python's type-level dunder lookup bypasses — and the `sample_state() or state` re-reads in `_eval/task/run.py` are error/interrupt-recovery paths, not per-step.) Two consistency notes fall out: between those replacement points the conversation is mutated *in place* (the turn loop appends to the same list; even `state.messages = ...` assignment lands on the same `TaskState` object), so the handle reads live at message grain, not solver grain; and since the control server shares the eval's event loop, a snapshot copied in the request handler can never observe a half-applied append or rewrite. Terminal samples need nothing new: the completed sample's messages come from the recorder's buffer via `EvalState.live` (`read_sample`), falling back to the on-disk log — the `samples` / `sample` / `events` pattern, unchanged. In the drill-down family each verb then answers one question: `sample show` — "how is it doing", `sample events` — "what happened, in order", `sample messages` — "what does the model see".
+
+**Scope: the sample's main conversation.** `TaskState.messages` is the sample's top-level thread. Nested agents whose conversations don't share that thread (e.g. an agent invoked as a tool, whose inner exchange never lands in `state.messages`) remain visible through `sample events` (span-tagged); a span / agent selector on `messages` is a plausible follow-on if live subagent-watching becomes a real ask — not part of the first slice.
+
+**Why not fold into `sample show`?** `show` is the summary; the conversation is the detail. Folding it in would break the summary-then-detail shape constraint the whole read surface is built on (and make `show` unusable as the cheap poll target). Separate verb, same relationship as `show` vs `events`.
+
+**Sequencing.** Read-only — no new directive machinery, no security-hardening dependency — so it extends the phase-2 read surface and can land independently of the remaining phase-3 directives. One shared-infrastructure note: the compact message projection should share its truncation helpers with the events projection (which already truncates model / tool event content — completions, tool arguments / results) so the two renderings of the same objects don't drift.
+
 ### Phase 3 — modification (direct) methods
 
 The first endpoints that **mutate the run**, each idempotent and supporting `?dry_run=true` / `--dry-run` from day one. Shipped so far: the log-flush and buffer-params directives, the concurrency-config directive (`max_samples` / `max_sandboxes` / `max_subprocesses` / `max_connections`) — surfaced through `ctl task log-flush` and `ctl config` — and the cancel directives (`ctl task cancel` / `ctl sample cancel`); adding a task to a running eval, then drain / requeue and the per-sample time/token/message limits follow. The Security model's "future hardening" (SO_PEERCRED UID check, self-targeting guard) lands with this phase, since it introduces the first state-mutating writes.