Repository navigation
epic: Fable 5 parity — post-#100 verification findings & follow-ups #196
Description
Why
Epic #100 ported the Claude Code Fable 5 harness to aitm (29 children, all merged, shipped in v0.0.23). This epic tracks a full post-merge verification pass of that work — behavioral probes of every compat primitive against the real harness, a four-part component audit against the decomposition, and seven live aitm start runs on sandbox repos — and the defects and parity gaps it surfaced.
Headline: the ported surface is broad and mostly faithful, and all eight second-pass runtime-failure classes (#122–#129) are closed in code. But two P0 bugs share one root cause — ai@6's onStepFinish/prepareStep expose response.messages as a cumulative list, not a per-step delta, and two consumers misread it — which silently guts the context-management arc in production. Several P1 gaps mean a few #100 checkboxes shipped only the library half (Skills, background execution).
Verification method and full findings (evidence, file:line, live-run logs) are recorded outside the tracker; each item below is independently reproduced. Baseline at time of audit: typecheck + lint clean, aitm 902 / compat 250 node tests green.
P0 — context-management arc broken in production
- fix(state): transcript step records duplicate the prior conversation — record per-step deltas #175 transcript step records duplicate the prior conversation (cumulative
response.messages) → O(N²) files + duplicated resume context - fix(compaction): tail sizing reads cumulative step messages — compaction never fires in production #176 compaction tail sizing reads cumulative step messages → compaction never fires in production
P1 — behavior gaps / library-only deliveries
- fix(compat): unify Read/Edit content hashing — non-UTF-8 files lock into a permanent stale-read loop #177 non-UTF-8 files lock into a permanent stale-read loop (Read/Edit hash mismatch)
- fix(compat): slash-free grep glob filters must match by basename at any depth #178 slash-free grep
globfilters miss nested files (full-path anchoring vs rg basename) - fix(cli): take-over flow bypasses bash command rules, tool hooks, and usage metering #179 take-over flow (
aitm merge-pr) bypasses bash rules, hooks, and usage metering - fix(compat): withHooks must decorate toModelOutput — hook feedback and block reasons are invisible to the model #180
withHooksdoesn't decoratetoModelOutput→ hook feedback / block reasons invisible to the model - feat(loop): mount the Skills surface into subagent tool sets — index block + Skill tool + built-in skills #181 Skills are library-only — zero aitm wiring (mount index block + Skill tool + built-ins)
- feat(loop): wire ProcessManager into worker tool sets — real background bash + monitoring tools #182 background execution off in production — ProcessManager never constructed; background tools unmounted
- feat(compat): salvage JSON-string submit payloads before failing schema validation #185 salvage JSON-string submit payloads before failing schema validation (live run-6 failure, one step from success)
P2 — filed follow-ups
- docs(config): document the seven shipped-but-undocumented config keys #183 document the seven shipped-but-undocumented config keys
- feat(testing): graded e2e benchmark scenarios with per-run usage JSONL + compare command #184 graded e2e benchmark scenarios + per-run usage JSONL + compare command (epic-epic: Claude Code Fable 5 harness parity — decomposition & port plan #100 promised, never filed)
- feat(compat): wire the identity + safety-preamble prompt block into subagent prompts #186 wire the identity + safety-preamble prompt block into subagent prompts
- fix(compat): Write must reject a seen-then-externally-modified file (staleness check) #187 Write must reject a seen-then-externally-modified file (staleness check)
- feat(compat): plain-text toModelOutput for web-fetch and fetch-html tools #188 plain-text
toModelOutputfor web-fetch / fetch-html tools - feat(subagents): bound the per-file editor fan-out concurrency #189 bound the per-file editor fan-out concurrency
- feat(loop): run-level cost/token ceiling as an unattended-run guardrail #190 run-level cost/token ceiling as an unattended-run guardrail
- fix(compat): command deny rules bypassed by command/env/absolute-path prefixes #191 command deny rules bypassed by
command/env/ absolute-path prefixes - feat(agent-config): lazy on-touch loading of nested CLAUDE.md via the reminder channel #192 lazy on-touch loading of nested CLAUDE.md via the reminder channel (feat(agent-config): layered CLAUDE.md discovery — user-global + nested directories #117 follow-up, unblocked by feat(compat): file-state tracker — read-before-edit enforcement, stale-read rejection, cat -n windowed Read #104)
- feat(mcp): extend deferred tool loading to the CI-fix session (and evaluate Planner) #193 extend deferred MCP tool loading to the CI-fix session (and evaluate Planner) (feat(mcp): deferred MCP tool loading — name-only stubs + tool_search activation #119 follow-up)
- feat(subagents): mount .claude/agents definitions as read-only spawnable agents #194 mount
.claude/agentsdefinitions as read-only spawnable agents - feat(config): web_search domain filters (allowed/excluded domains) #195 web_search domain filters (allowed/excluded domains)
P2/P3 — tracked, not filed (deliberately)
Real but low-value, covered by another item, or needing a design decision before a ticket is worthwhile:
- Compaction event → transcript recorder seam (A6×feat(state): per-subagent transcript persistence — messages JSONL + in-run resume #108): the
TranscriptRecorder.compaction()seam has no producer. Folded into fix(state): transcript step records duplicate the prior conversation — record per-step deltas #175/fix(compaction): tail sizing reads cumulative step messages — compaction never fires in production #176 scope (moot until fix(compaction): tail sizing reads cumulative step messages — compaction never fires in production #176 fires); split out only if those PRs don't cover it. - Re-summarize economics (A6): above threshold, every step re-summarizes the growing prefix with a fast-tier call. An optimization, only meaningful after fix(compaction): tail sizing reads cumulative step messages — compaction never fires in production #176; revisit with compaction telemetry.
- Scratchpad directive (A1): subagents have no temp-dir concept (stray-file-in-commit risk). Needs a design choice (where the scratch dir lives, cleanup) before filing.
- hook feedback not wrapped in
<system-reminder>(A5): correct home once fix(compat): withHooks must decorate toModelOutput — hook feedback and block reasons are invisible to the model #180 makes hook output model-visible; fold into fix(compat): withHooks must decorate toModelOutput — hook feedback and block reasons are invisible to the model #180. - take-over legs also lack memory index + compactor + context reminder: the reminder half is feat(loop): extend the #106 system-reminder channel to the CI-fix and take-over subagents #141, the rules/hooks/usage half is fix(cli): take-over flow bypasses bash command rules, tool hooks, and usage metering #179; the memory/compactor half rides whichever take-over PR lands first.
- knowledgeCutoff never populated (A2): feat(compat): envBlock model-identity fields + auto git-repo detection #116 scoped it out (OpenRouter catalog doesn't expose it); would need an optional per-tier config map — file only if a user asks.
- grep/glob multiline mode + type filters (B3): feat(compat): grep/glob parity — context lines, brace alternation, mtime sort, .gitignore #110 deferred with rationale; file if demand appears.
- glob truncation ordering (B3): >2000 matches walks lexicographically then sorts by mtime, so "newest-first" isn't the global newest-N; comment-documented, cosmetic.
P3 cosmetic (report-only, models tolerate — no ticket planned): Read line numbers are N\t rather than right-aligned cat -n padding; env-block labels differ slightly ("Working directory" vs "Primary working directory"); zod tool schemas silently strip unknown input keys (a model emitting a Claude-Code-only param gets the default path silently).
Already-open, pre-audit follow-ups (not part of this pass, listed for context)
#131 (run Logger into loop), #132 (typecheck:tests gate), #139 (step-timeout dedup), #141 (reminder contract for ci-fix/take-over), #150 (cache_control vs routing), #154 (bashOutput partial-line buffer), #168 (usage ledger latency+retry).
Out of scope (verified correct, no action)
The audit re-checked every deliberate scope-out in #100 against its issue text and confirmed the rationale holds: sandboxed bash + dangerouslyDisableSandbox (no-containers stance), interactive flow-control tools (B11), scheduling/wakeup daemon surfaces (B12), Artifact/ReportFindings (B13 — PR-prose/JSONL remaps verified present), MCP server surface, image/PDF/notebook Read rendering, model-facing task registry (B5), Workflow JS orchestration (B10 — stage machine + PlanGraph fill the slot).
Priority order
Fix P0 first (#175, #176 — small changes, revive the whole context arc), then the XS/S P1 bugs (#177, #178, #185), then #179/#180, then the library-only wirings (#181, #182). P2 as capacity allows; docs (#183) is cheap and high-visibility.
Activity
fix(compaction): tail sizing reads cumulative step messages — compact…
fix(state): transcript step records duplicate the prior conversation …
fix(compat): unify Read/Edit content hashing — non-UTF-8 files lock i…
fix(compat): slash-free grep/glob filters match by basename at any de…
feat(compat): salvage JSON-string submit payloads before failing sche…
fix(compat): withHooks must decorate toModelOutput — hook feedback an…
fix(compat): command deny rules bypassed by command/env/absolute-path…
3 remaining items
feat(config): web_search allowed/excluded domain filters (#195) (#261)
feat(loop): run-level cost/token ceiling guardrail (#190) (#267)
feat(compat): lead every subagent prompt with the identity + safety p…
feat(loop): extend the #106 reminder channel to the CI-fix worker and…
OGtwelve commentedon Jul 24, 2026
Status update — 2026-07-24
Closed since the audit was filed: #179 (take-over rules/hooks/usage parity), #186 (identity + safety-preamble block), #194 (
.claude/agentsas read-only spawnable — landed differently via the specialist registry), and pre-audit follow-up #141 (reminder channel now reaches the CI-fix worker + take-over reviewer). Checkboxes above updated.Remaining, grouped
P1 parity — advertised-but-unwired (the substantive gaps):
P2 follow-ups:
Pre-audit follow-ups still open:
typecheck:testsgate (aitm test files are currently unchecked in CI, so test drift isn't caught).OnUsagechange — the largest remaining item).Everything in the "tracked, not filed" and "out of scope" sections below still holds.
feat(testing): graded e2e benchmark harness with per-run usage JSONL …
OGtwelve commentedon Aug 3, 2026
All 21 items are landed — closing candidates and what outlived the epic
The five boxes still open (#181, #182, #184, #192, #193) were all closed on the code side; the checklist just was never ticked behind them. Done now, so the list reads 21/21.
Landed most recently:
Two things this epic produced that outlive it, both filed rather than left implicit:
init.toolsis shared — so the fix is a tools factory onScoutAgentInit, which serves both features at once.ghcredentials, and safe isolation needs containers, which CLAUDE.md bans. Parked with the safer direction recorded (read the pass/fail from aitm's own guarded run instead of re-executing).Nothing else from the #100 verification pass is outstanding. This can close whenever you're happy with the two follow-ups standing on their own.
OGtwelve commentedon Aug 3, 2026
Closing: 21/21 items landed and the checklist is ticked. The two things this epic produced that outlive it — #333 (deferred loading for the remaining roles, plus the per-agent tool-record seam #192's review turned up) and #308 (benchmark exec, deferred on sandboxing grounds) — stand on their own issues and do not need the umbrella.