refactor(codex): facet-scoped ingestion layer - #667
Draft
Octane0411 wants to merge 25 commits into
Draft
Conversation
Introduce the arbitration layer for Codex session state: observations, six orthogonal facets, and a declarative authority matrix that ranks sources per facet instead of ranking sources globally. No source dominates — the app-server sees turn boundaries but not the terminal, hooks see approvals but not thread closure — so a scalar priority lets a source's blind spot overwrite another's authority. Ranking per facet keeps each source winning only where it can observe the truth. Also decodes session_meta fields the previous parser ignored, including the polymorphic `source` field that threw on its object form and took roughly a fifth of real transcripts down with it. - CodexObservation: the single shape crossing into projection - CodexAuthorityMatrix: who may write what, and who wins - CodexFacetStore: per-facet provenance and conflict counting - CodexIdentityResolver: session keys, surface classification - CodexRecordDecoder/Table: versioned, drift-reporting rollout decoding - CodexSessionProjector: facets to AgentEvent - CodexDiagnostics: unknown types and rejected writes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the collection layer — hook, app-server, and rollout adapters that translate their native protocols into observations and nothing more — plus regression coverage built from anonymized real transcripts. The fixture generator redacts by allow-list, not deny-list. A deny-list was tried first and leaked immediately: transcripts nest user content under dozens of key names, including ones that did not exist when the list was written. Only keys the decoder branches on survive, and their values are shape-checked before being kept. Running the corpus through the decoder surfaced two real defects: - A `response_item` message is typed "message" and identifies its speaker by `role`. The decoder stopped at the type and treated all 800 of them as unrecognized. - Seven further record types Codex writes today were missing from the table (thread_goal_updated, thread_settings_applied, item_completed, thread_rolled_back, context_compacted, compacted, image_generation_call). With both fixed the corpus decodes with zero unknown records, so the test asserts exactly that — a future Codex release now breaks a test instead of quietly changing behaviour. Also splits workspace out of placement. Both were one facet until implementation showed they have different witnesses: `session_meta.cwd` is authoritative for the working directory, while only a hook knows the terminal. Fused, Codex.app sessions — which have no terminal at all — could never acquire a workspace and would never appear. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wires the source adapters, facet store, and projector into one seam the rest of the app can hold, and mounts it in BridgeServer alongside the existing path. The mode switch is the point. In .shadow the pipeline observes every hook the legacy path observes and assembles full session state, but emits nothing — so the rewrite can be exercised against real sessions before anything user-visible depends on it. Divergence comparison looks only at what a user can perceive (event kind, session, phase); summary wording and timestamps differ harmlessly between implementations and would otherwise drown the signal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Running the full 928-transcript corpus through the pipeline exposed three defects that fixtures alone did not. Only Codex.app sessions were restored. Terminal identity comes from hooks, and at cold start none have fired, so every CLI session failed to produce a jump target and was never announced — 250 of 484 sessions appeared. Terminal identity is now derived from the surface: a desktop thread jumps to Codex.app by thread id, anything else takes the house "Unknown" sentinel and jumps by folder, matching how transcript-restored Claude sessions already behave. The previous implementation instead labelled everything "Codex.app", which is why CLI sessions showed up as desktop ones. Restore read every transcript whole — 449 seconds over 6.2 GB. Cold start only needs the two ends of a file: the header and opening prompt at the front, recent activity at the back. Bounded reads bring the same restore to 24 seconds. The head read grows on demand because session_meta can embed full system instructions and run past 64 KB; without that, 65 transcripts silently yielded no session at all. Two further record types (entered_review_mode, exited_review_mode) were missing from the table. The corpus now decodes with zero unknown records and restores 484 of 484 visible sessions, with all 416 spawned threads correctly withheld. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
CI failed on CodexAppServerTimeoutTests, which asserts a 0.1s request timeout completes within 2s. That test is untouched by this branch and passes on main — the corpus suites were re-reading dozens of fixture files per test, and Swift Testing runs suites in parallel, so the disk load pushed the timeout assertion past its jitter allowance. Fixture bytes are now read once and replayed from memory, and the cold-start suite is serialized. Two tests keep direct coverage of the file path itself, including one asserting that a session_meta larger than the head read is still recovered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The timeout test kept failing intermittently on CI — 8.8s, then 4.9s, then passing — which tracked the amount of parallel work this branch added rather than anything in the code under test. Each corpus suite was walking and decoding all 39 fixtures separately per test, five and four times over respectively. Both now do one pass, cache what they observed, and assert against that. Suite wall time drops from 6.5s to 3.8s locally, against a 2.0s baseline before this branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Root cause of the CI failures, which two rounds of guessing did not reach: CodexAppServerClient implements its request timeout as a Task awaiting Task.sleep, so it needs the cooperative pool to schedule it after 0.1s. The corpus suites decode fixtures synchronously, occupying cooperative threads. On a runner with a few cores that is enough to delay the sleep past the test's 2s allowance. That is why cutting total work only moved the number (8.8s, 4.9s, 4.2s) without fixing anything — peak blocking is what matters, not the sum. Responsibilities are now split by what each suite is actually for. Decode coverage across every Codex version stays with the corpus suite, which walks all 39 fixtures once. Cold-start tests assert pipeline behaviour — idempotent restore, spawned threads withheld, workspaces present — none of which varies by version, so they run against a 10-fixture slice taken from both ends of the corpus. Suite wall time: 6.5s → 1.9s, level with the 2.0s baseline before this branch. Three consecutive local runs pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Trimming work was treating the symptom. Swift Testing runs tests on the cooperative thread pool, and decoding fixtures is synchronous CPU work that holds those threads for as long as it runs. CodexAppServerClient implements its request timeout as a Task awaiting Task.sleep, so a held pool delays that sleep — which is why the unrelated timeout test kept failing on runners with few cores, and why each round of trimming only moved the number (8.8s, 4.9s, 4.2s, 2.7s) instead of fixing it. Both corpus decodes are now async and yield between fixtures, cached behind an actor so the work still happens once per run. Wall time goes up (1.9s to 4.1s) because yielding costs something, but wall time was never the problem — holding the pool was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…bsent The fixtures are generated from the maintainer's own Codex session history. Even redacted, they stay off GitHub: the directory is now git-ignored and the two suites that consume it are gated on its presence with .enabled(if:), so CI — which never has the corpus — skips them cleanly rather than failing. Tests that need no corpus (a hand-built transcript, a temporary file) move to their own always-on suite so that coverage survives on CI. The gate condition is a free function rather than a static member: a @suite trait cannot reference the type it decorates without the macro expansion becoming circular. Verified in both states: corpus present runs 431 tests across 47 suites; corpus absent skips the two gated suites and passes the rest. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Octane0411
force-pushed
the
refactor/codex-ingestion
branch
from
August 26, 2026 06:58
1614a5f to
eb697cb
Compare
…on hook Codex fires PermissionRequest regardless of the user's approval mode — it cannot know whether an observer intends to weigh in. BridgeServer treated every request as one to hold and display, so a user running with bypassPermissions still saw each action queue for up to an hour waiting for a click (#559). The permission_mode field was already parsed; it was never consulted. Route by mode before anything is shown: bypassPermissions, dontAsk → allow at once, no card acceptEdits → allow apply_patch, ask about the rest plan → deny at once with a reason default → hold and ask, as before The PermissionRequest contract offers only allow and deny — there is no way to hand the question back to Codex's native prompt — so every mode must resolve to an answer rather than wait for the timeout. This is the rule #638 asks for: Codex's own setting is authoritative, and Open Island steps in only where the user asked to be asked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codex 0.148 exposes eleven hook events; the managed install registered four. The omission was deliberate — per-command PreToolUse/PostToolUse echo into the terminal — but it predates SessionEnd, SubagentStart/Stop, TurnStart and PreCompact, none of which fire per command. Those five are now installed by default. What they buy the user: - SessionEnd is the first signal that actually means the session ended. Stop only closes a turn; reading it as an ending is one way sessions were left showing as running after the user had quit. - SubagentStart/Stop give a live subagent count (#584, #579). Hooks report boundaries, so the source keeps the tally and emits a total. - PreCompact explains an otherwise silent pause. Handled on both paths — the legacy BridgeServer switch and the new pipeline's hook source — so shadow comparison stays meaningful. Payload gains agent_id, agent_type, trigger and reason; CodexSessionMetadata gains an optional activeSubagentCount that older persisted records decode without. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ources Until now the pipeline observed hooks only, and its shadow mode returned nothing — so there was no legacy output to compare against and no way to justify ever switching it live. This makes the comparison real. - ingest() always returns the candidate events; the caller applies them only when the pipeline drives the UI. BridgeServer captures the legacy handler's emits per hook (via a defer, so every exit path flushes) and hands both lists to recordDivergence. - The app-server coordinator reports each notification together with the events the legacy path produced for it; AppModel runs the pipeline on the same notification and compares. - Cold-start discovery feeds every transcript it finds into the pipeline and records a session-level comparison — spawned threads leaking in was the headline defect, and that is where it shows. Two matrix corrections surfaced by tests, not by reasoning: - The app-server must be able to write a session's workspace. Its thread listing carries the Codex-authored cwd, and without it every desktop session projected with "Codex.app" as its workspace name. - The app-server may raise a placeholder approval card when a thread reports it is waiting. It ranks below hooks, so a hook's card — which carries the actual tool and path — replaces it and can never be displaced by a later status. The hook source now applies the same approval routing as the bridge, so a user in bypass mode gets no card from either path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The comparison data was being collected but was visible nowhere, which made the shadow run pointless. The Codex ingestion section now names the pipeline mode and lists the most recent divergences alongside the drift counters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
session-terminals.json holds what hooks told the previous run — above all which terminal a session lives in, which no other source can supply. Cold start now feeds those records into the pipeline before reading transcripts, attributed to the hook source since that is where they came from. A later transcript read cannot downgrade the terminal (the matrix bars rollout from placement), and a live hook for the same session arrives with a later sequence and supersedes it. Remembered data does not end replay mode; only a live source does. The plan had called for adding a Codex registry. That was based on a wrong observation — the store already existed under a different filename — and the doc has been corrected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A multi-day shadow run is only useful if it survives app restarts. Every divergence — event-level and cold-start — is now also appended to Application Support/open-island/codex-shadow.log, so the report can be read from the shell without opening Settings and is not lost when the app relaunches. In-memory stays bounded; the file keeps everything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ests The first minutes of the shadow log surfaced two problems. The pipeline ignored ~/.codex/archived_sessions. Archiving a thread is an identity signal, not a liveness one: the thread leaves the active list, it does not end. Cold start now leaves archived transcripts out rather than restoring them and marking them ended. The comparison line names the sessions that differ, so the log can be acted on directly. Tests build BridgeServer freely, and BridgeServer was constructing its pipeline with the real Application Support log path — fixture session IDs showed up in the user's log. The path is now injected by AppModel and defaults to nil. The rewritten hook source now mirrors the bridge on PreToolUse: a card regardless of Codex's approval mode. PreToolUse is not installed by default, so registering it is the user opting into per-command gating (Full Control, #364); only PermissionRequest follows the mode. Without this the two paths disagreed in shadow, and in live mode the held hook would have waited on a card the pipeline never showed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three things the log itself showed within minutes. App tests build a real AppModel, which pointed the pipeline at the user's Application Support directory — fixture session IDs landed in the shadow log. There is no test isolation of that directory anywhere in the project, so the log URL is now withheld whenever the process is a test runner. The cold-start comparison set the legacy side to transcript discovery alone, but the legacy path restores persisted records as well. A placeholder record with no transcript therefore showed as a phantom "only-candidate" every time. Both sides now compare persisted ∪ discovered. The periodic rescan re-recorded the same comparison every ten seconds. It now feeds the pipeline and records nothing; only the startup scan writes the line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OPEN_ISLAND_CODEX_PIPELINE=live|shadow|off overrides the compiled default, so the rewritten path can be tried on a real machine — and switched back — without a rebuild. Unset or unrecognized leaves the caller's choice alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shadow mode caught this against real usage before it could ship: on the Stop hook the rewritten path emitted activityUpdated(.completed) where the legacy path emits sessionCompleted. The two look equivalent at the state level — both set phase and summary, neither marks the session ended — but two consumers switch on the event kind and nothing else: IslandSurface pops the island only for sessionCompleted WatchNotificationRelay pushes to the watch only for sessionCompleted Reporting a finished turn as activity silently dropped both, so Codex would have finished a turn with no island and no watch notification. A finished turn is now sessionCompleted with isSessionEnd false; a real ending keeps isSessionEnd true. That preserves the design distinction — Stop ends a turn, SessionEnd ends the session — without changing what the user perceives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… blocks Two defects the running app surfaced immediately. Sending a message in Codex.app produced a session and a completion notice while the agent was still working. Codex.app opens an ephemeral thread to generate the conversation title; the legacy coordinator skips those with an early return, but the shadow hand-off was a defer, which runs on that return too — so the internal thread reached the pipeline anyway. The source now drops ephemeral threads and remembers their ids, because turn/completed carries only an id and would otherwise look like a real session finishing. Session titles read "<recommended_plugins> Here is a list…". Codex prepends blocks to the transcript as if the user had typed them, and the first one was taken as the title. The legacy reducer filters five such prefixes; the rewritten rollout source filtered none, and the legacy list predates this block anyway. The filter now covers the known prefixes plus a catch-all for an opening tag alone on the first line, so blocks Codex has yet to invent do not become titles either. Text that merely contains a tag is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… list The list showed rows reading "/ · <recommended_plugins> Here…". Two independent legacy defects produced that single line, and the rewritten pipeline could not mask either: cold-start discovery and rediscovery assign SessionState directly rather than emitting events, so live mode never touches them. The reducer filtered injected blocks on the response_item path but not on event_msg/user_message, which is the path current Codex uses — the text went straight into the summary. And makeRecord never set a jump target, so the working directory was dropped and callers deriving a workspace name from it resolved "/". Both paths now share one injected-block list, so a block Codex adds later cannot be filtered on one path and shown on the other. Verified against the real corpus: 11 discovered records, no injected text, no bad workspace names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The list showed a session summarised as "You: You are a helpful assistant. You will be pres…". That text is a `developer` message — the harness's own setup, alongside multi-agent wiring and desktop app context — and in a live transcript those outnumber real user messages 13 to 9. The rewritten decoder mapped every non-assistant role to user_message, so all of it became things the user supposedly said. The legacy reducer switches on "user" and "assistant" and ignores the rest; this was a regression introduced by the rewrite, not an inherited defect. Only user and assistant are conversation now. Verified against the live corpus: no harness text reaches a prompt, and real questions come through intact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Octane0411
force-pushed
the
refactor/codex-ingestion
branch
from
August 27, 2026 06:47
3efb42a to
d6d7bfd
Compare
Sending a message to a new Codex session raised a completion notice as soon as the model's first reply landed, while the agent was still working. My previous fix over-corrected. To restore the island popup on a finished turn I made every lifecycle change to .completed emit sessionCompleted — but Codex.app reports a thread idle, and reports turns completed, at moments during a turn. The legacy path had this right and split it by source: only the Stop hook emits sessionCompleted; app-server turn and status events emit activityUpdated, which never surfaces. The projector sees a facet, not its origin, so that distinction was lost. Rather than have the projector sniff which source wrote the facet, the facet now carries the meaning: announcesCompletion is set by the Stop and SessionEnd hooks and by nothing else. A thread that merely is not running a turn right now stays an activity update. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lback Two findings from auditing the legacy path's guards against the rewritten pipeline. CodexLiveness carried an `archived` end reason that nothing produces, and the projector used it to emit isSessionEnd: false. The reconciler that actually handles archived threads marks them isSessionEnd: true. Had anything ever set the reason, archived sessions would have lingered as merely completed. The exception is gone; every end reason ends the session. Failing to locate Codex.app returned silently, which is indistinguishable from Codex.app not running — desktop sessions would lose their only real-time source with nothing said. Both failure paths now report. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A Codex.app session sat at "running" for 27 hours. Its transcript held only session_meta — a conversation Codex opened and the user never used — so the file existed and the vanished-transcript rescue, the only thing that recovers a stuck turn, never fired. Nothing else resolves phase, so the row stayed running and inflated the running count. A turn that is genuinely running keeps appending to its transcript as it reasons and calls tools. A transcript untouched for half an hour while the session still claims to run means the ending was missed. That now recovers the session the same way a vanished transcript does. The threshold is deliberately generous — a long turn still writes as it works — and an actively written transcript is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrites the Codex ingestion layer around facet-scoped authority instead of a
global source priority, and validates it against a real 928-transcript corpus.
This is step 1 of the plan and is deliberately inert: the new pipeline runs in
.shadowmode insideBridgeServer, observing every hook the legacy pathobserves and assembling full session state while emitting nothing. Only the hook
source is wired so far. Nothing user-visible changes.
Opening as a draft for design review before the remaining sources are connected.
Why the current design fails
No source dominates, and each is blind to what the others see:
Today all four write the whole session object with no arbitration, so whatever
lands last wins — including values written from a source's blind spot. That single
mechanism produces the title corruption, the sessions stuck in
running, and theapprovals that never round-trip.
A scalar priority cannot fix it: ranking app-server above hooks would let its
missing terminal knowledge overwrite the only source that has any.
The design
Session state is split into seven orthogonal facets. Each source may write only
the facets it can actually observe, ranked per facet in one declarative table
(
CodexAuthorityMatrix). Cold start runs in replay mode, where the rollout mayfill everything provisionally and the first authoritative write replaces it.
Every open Codex issue should map to a row of that matrix. If one doesn't, the
facet split is wrong and the design needs another pass — that's the intended
self-check.
What the real corpus found
Fixtures alone were not enough. Running all 928 transcripts (6.2 GB) end-to-end
exposed three defects, all now fixed:
cold start none have fired — every CLI session failed to build a jump target and
vanished. Now derived from the surface; the previous implementation instead
labelled everything
Codex.app, which is why CLI sessions appeared as desktopones.
reads, with the head growing on demand because
session_metacan embed fullsystem instructions and exceed 64 KB.
messagerecords (800 of them) were unrecognized. Aresponse_itemmessageis typed
"message"and identifies its speaker byrole; the decoder stopped atthe type. Plus nine further record types Codex writes today.
Corrections to the design doc
400-file sample; the full 928 give 442. Nearly half of all transcripts are threads
Codex spawned for itself.
workspacehad to split fromplacement:session_meta.cwdis authoritative for the working directory while only a hook knows the terminal.
Fused, Codex.app sessions — which have no terminal — could never acquire a
workspace and would never appear.
Codex.app compatibility
Codex Desktopandcodex_work_desktopboth map to desktop. Real corpora holdboth spellings (152 : 60) and will indefinitely; matching only the current one
orphans half a user's history. Explicit allow-list, unknown values reported — no
prefix guessing.
are still open; treating that as an ending is what stranded desktop rows in
running.them later (Subagent permission prompts should surface on the island #584, Claude background subagent progress disappears after parent Stop #579).
Fixtures and privacy
The transcript corpus is generated from the maintainer's own session history and
is not committed —
Tests/Fixtures/codex-rollouts/is git-ignored, and the twosuites that consume it are gated with
.enabled(if:)so CI (which never has thecorpus) skips them cleanly. The three self-contained cold-start tests run everywhere.
Locally, regenerate with
python3 scripts/codex-fixtures.py --source ~/.codex/sessions --out Tests/Fixtures/codex-rollouts. The script redacts by allow-list — a deny-listwas tried first and leaked immediately — and exits non-zero if anything long, path-like,
or base64-shaped survives. Redaction still matters for files that sit in a working tree.
Branch history was rewritten to remove the fixtures from the one commit that had
carried them; no commit on this branch has ever contained them.
Testing
swift build— passeszsh scripts/test-clt.sh— 431 tests / 47 suites pass. On CI, which has no corpus, the two gated suites (11 tests) report as skipped and are counted in the total; CI and a local corpus-free run agree exactlyzsh scripts/harness.sh docs— passesNew coverage includes property tests for the matrix invariants: an authoritative write
is never overwritten by a non-authoritative one, out-of-order delivery cannot rewind
state, subagents never produce
sessionStarted, and provisional cold-start valuesalways yield.
Not in this PR
BridgeServer(hook only so far).liveisCodexAppSession/refreshCodexAppClassificationremovedfrom
SessionStateReview focus
Sources/OpenIslandCore/CodexAuthorityMatrix.swiftis ~130 lines and is the file toread first.
workspace/placementsplit correct, or should cwd stay part of placement?Are the fixtures acceptable to commit?Resolved: they stay local, never committed.🤖 Generated with Claude Code