test(director): the boot stretch hands the keyboard over - #208
Merged
Conversation
Roadmap line 0 no longer reproduces under its own recorded conditions. Three runs on 2026-09-04 with the real brain, a seeded persona, --plain and piped stdin, one line typed after the second beat plus a 3s settle: the line echoed every time, and runs 2 and 3 were read at the deterministic seam — settings.json gained musicEnabled:false, and src/steer-tools.ts is the only writer of that key, so the line genuinely crossed the Director's steer path. This is cannot-reproduce, NOT fixed, and the difference is kept everywhere it matters. No cause was ever found: the `settled` guard predates the August 25 loss by a week and LineQueue's take/peek semantics have not changed since, so the guard is pinned by test, never credited with the cure. The new test is green on the base revision too — a characterization pin over the hand-over, not a red-to-green regression, which the test says of itself so a later reader cannot mistake it for proof the defect is dead. guide.test.ts already pins the reader's own guard on the quit and eof arms. What was missing is the hand-over: once a pre-broadcast read has settled, the next line typed is the Director's, on a real LineQueue. Removing `settled` turns it red with the steer brain never seeing the line — the reported symptom exactly. Line 0 leaves the roadmap because nothing about it is schedulable — no repro means no red test to write and no cause to fix — and its record stays: the contract and the history in spec 01 §3.3, the disposition under "Not on this roadmap". A first line going missing again reopens it with the new repro. Line 4's eval half (#98) is now read first among the P2s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018nkjmN4XQdDbffSRnnAaEi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap line 0 — "the listener's first typed line is silently dropped" (issue
#145) — no longer reproduces under its own recorded conditions. This retires it
as cannot-reproduce, not fixed, and spends most of its effort making sure
that distinction survives the deletion.
The evidence
Three runs on 2026-09-04 reproducing the August 25 setup exactly: the real
Claude brain, a seeded
persona.md(so first-run onboarding is skipped),--plain, piped stdin, one line typed after the second beat aired plus a 3ssettle. Run B's shape — a single line, so there is no second line to confuse
the result.
settings.jsonmusicEnabled: falseafter 35.2smusicEnabled: falseafter 5.0smusicEnabledhas exactly one writer in the tree,src/steer-tools.ts— theDirector's steer tool. So runs 2 and 3 are read at a deterministic seam: the
line did not merely echo, it crossed the steer path.
Why this is not "fixed"
No cause was ever identified.
lineReader'ssettledguard predates theAugust 25 loss by a week, and
LineQueue's take/peek semantics have notchanged since — so the guard is pinned by test here, never credited with the
cure. Either an unrelated change closed the window, or the timing drifted, or
the original conditions held a variable nobody wrote down.
That makes the pin more worthwhile, not less: a race that closed without a
known cause returns silently — no error, no log line, just a listener's first
sentence going missing.
What landed
test/director-steer.test.ts— the hand-over the issue asked for: once apre-broadcast reader (the crash-report offer, the setup conversation) has
settled, the next line typed reaches the Director's steer path, on a real
LineQueue.test/guide.test.tsalready pins the reader's own guard on thequit and eof arms; the seam between them was not covered.
Mutation-verified — delete
settledfromsrc/guide.tsand this test goesred with the steer brain never seeing the line, which is the reported
symptom exactly.
specs/spec01/01-core-loop.md§3.3 — the violated contract now carriesits own history, so a first line going missing again is recognisable as this
defect returning rather than a fresh one.
ROADMAP.md— line 0 and its §0 are deleted; the disposition is recordedunder "Not on this roadmap". Line 4's eval half (Steer tool-choice eval (Ollama): switch/end/reply routing has no repeatable LLM-in-the-loop test [spec 11 §5] #98) is now read first among
the P2s, being the only line overdue rather than upcoming.
Peer review
codex exec review(gpt-5.6-sol, xhigh): 1 finding, applied. It observedthat the new test is green on the base revision — it exercises a guard that was
already there, not the unexplained failure — and that deleting line 0 risked
dropping the record from planning.
Both halves are correct as facts. The remedy it proposed (keep line 0 until a
base-revision-red test drives the recorded startup path) is not reachable: you
cannot write a red test for a defect that will not reproduce, so the item would
sit at the top of the roadmap permanently undischargeable. The response instead
states the limitation in the places a future reader will actually look — the
test's own comment, the commit message, and the spec — and keeps the record in
two places rather than keeping a phantom P0.
Verification
vitest: 1413 passed, 1 skipped, 69 filestsc --noEmit: cleanoxlint src test scripts .github/scripts: cleanscratch/and is not committed.AI coding brief
Original request — the user asked what was left on the roadmap, then, on
seeing a P0 they had no memory of, asked what it actually was so they could try
to reproduce it and "otherwise just mark it done". The why: an unremembered P0
blocking every by-ear judgement above it is worse than no P0.
Manual interventions — the user asked for concrete commands plus an
investigation the agent could run itself, rather than a description handed back
to them. Mid-run they asked directly whether the bug was fixed, which is what
forced the cannot-reproduce / fixed distinction to be made explicit instead of
being quietly rounded to "done".
Retro — two things cost a round each and were self-inflicted. The first
probe quit the moment it saw the
⌨echo, before the brain could act, whichproved nothing:
⌨has two printers (director.tson the steer path,guide.tswhen a reader consumes), so only thesettings.jsonseam decides.Asking "which seam proves this, and who else can write it" before running would
have skipped that. The first test draft also fired the esc pulse synchronously,
before the read registers on a microtask —
test/guide.test.tsalready had thesetImmediateline, so reading the neighbouring tests first would have avoidedit. Both are the same lesson the repo already states: read the seam, not the
narration.
🤖 Generated with Claude Code
https://claude.ai/code/session_018nkjmN4XQdDbffSRnnAaEi