tui: a failing runtime store is a visible notice, not a log line - #5972
tui: a failing runtime store is a visible notice, not a log line#5972Hmbown wants to merge 2 commits into
Conversation
A runtime turn whose own store record could not be read, parsed, or written surfaced only as a 'Failed to read turn ...' log line while the operator's task sat waiting on a turn.completed that could never arrive (#5931, store half; the session-id half landed as #5960). The store's load/save paths now attach a typed RuntimeStoreRecordFailure (operation, record kind, record id, path) as anyhow context, and the monitor failure paths publish a runtime.store_failure event through the existing emit_event -> event_tx -> RuntimeEventRecord channel, following the SnapshotsDisabledNotice / SessionIdDivergedNotice precedent: the payload names the file, the root cause, and the next action (move the file aside, or check free space and permissions). The TUI drains the manager's event tap each tick and shows a 12 s warning toast plus a transcript line (RuntimeStoreUnreadableNotice / RuntimeStoreUnwritableNotice, translated in all 15 packs), the task timeline records the fault, and the compat SSE stream forwards the event. When the turn's own record is unreadable or unwritable the notice is terminal, so the task driver fails at once instead of idling out. Gates (run in this worktree, on this commit after the rebase): - cargo fmt --all: clean - cargo clippy --workspace --all-targets --all-features --locked -D warnings (with the standing three -A allowances): clean; its first post-rebase run caught start_with_executor as dead code in the lib target (its remaining callers are tests), fixed with #[cfg(test)] - targeted store-failure tests: 6 passed, 0 failed - cargo test -p codewhale-tui --lib --locked: 11876 passed, 0 failed, 13 ignored. Two earlier full runs under heavy machine contention (concurrent same-binary suites from other worktrees) each had a few deadline-sensitive failures (one runtime_chat_relay, six runtime_threads/runtime_api sequencing tests); every one of them passes in isolation and none touches code changed here. Signed-off-by: CodeWhale Bot <bot@codewhale.net>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_04cdee42-2057-4017-b5d8-1b28a59fd83a) |
|
Claude finished @Hmbown's task in 2m 45s —— View job Claude finished @Hmbown's taskTodo list
SummaryThis is a clean, well-scoped change that does exactly what the title says: it turns five previously log-only store-failure sites into a typed, structured What I checked closely, and why it holds up:
Minor observations (not blocking):
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| let mut pending_translations = 0usize; | ||
| // #5931: the background runtime's own store faults arrive on its event | ||
| // channel, which nothing else in this loop reads. | ||
| let mut runtime_event_rx = task_manager.subscribe_runtime_events(); |
There was a problem hiding this comment.
🟡 Startup store warnings can disappear
When restored tasks fail immediately, subscribe_runtime_events attaches after the worker loop starts. Broadcast events have no replay, so the TUI can miss the warning entirely.
Prompt for agents
Remove the startup gap between TaskManager worker execution and the TUI runtime-event subscription. The manager starts workers in TaskManager::start_with_executor_and_runtime before run_event_loop calls TaskManager::subscribe_runtime_events. Because tokio broadcast receivers only observe later sends, an immediately resumed queued task can publish runtime.store_failure before the TUI receiver exists. Establish the receiver before workers can emit, or add durable replay with sequence-based deduplication when the TUI tap starts.
Was this helpful? React with 👍 or 👎 to provide feedback.
| self.report_store_failure(&thread_id, Some(&turn_id), &failure, &error, false) | ||
| .await; |
There was a problem hiding this comment.
🟡 Store failures produce duplicate notices
When monitoring hits a store fault, report_store_failure publishes it before settlement retries the same record. Settlement reports it again, duplicating timeline errors, toasts, and transcript lines.
Prompt for agents
Deduplicate runtime.store_failure publication across monitor_claimed_turn and settle_claimed_turn_failure. A typed error is currently reported once in monitor_claimed_turn, then settlement commonly encounters the same unreadable item or unwritable turn and reports it again. Preserve the terminal=true signal when settlement proves no turn.completed can follow, but emit one user-visible fault per underlying record failure, for example by deferring publication to settlement or carrying the original failure into settlement and upgrading its terminal classification there.
Was this helpful? React with 👍 or 👎 to provide feedback.
| - A background runtime turn whose own store record could not be read, parsed | ||
| or written (`Failed to read turn …`, `Failed to read item …`) was only a log | ||
| line. The runtime now publishes a `runtime.store_failure` event naming the | ||
| file, the root cause and the next action (move the file aside, or check free | ||
| space and permissions); the TUI shows it as a warning toast and a transcript | ||
| line, the task timeline records it, and the runtime API streams it. A turn | ||
| whose own record is unreadable or unwritable is reported as terminal, so its | ||
| task fails at once instead of idling out (#5931). |
There was a problem hiding this comment.
🔍 Remove branch changelog edits
Repository rules reserve both changelogs for merge-time updates on main. These additions create the conflicts that policy explicitly avoids.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
🟡 Changes recommended
Store-failure event detection and emission has two concrete edge cases that can suppress typed detection under added context and can duplicate user-visible notices for a single underlying store fault.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Implements the runtime-store portion of #5931 by turning on-disk runtime thread store read/parse/write failures into a typed, user-visible notice instead of relying on log lines, and propagating that notice through existing runtime event channels.
Changes:
- Adds typed
RuntimeStoreRecordFailurecontext to runtime store I/O errors and introduces aruntime.store_failureevent payload. - Surfaces store-failure events in the TUI (toast + transcript), task timeline (error + terminal fail), and compat SSE stream.
- Documents the new event and adds localized strings across all locale packs.
File summaries
| File | Description |
|---|---|
| docs/RUNTIME_API.md | Documents runtime.store_failure event semantics/payload. |
| crates/tui/src/tui/ui/tests.rs | Adds UI test coverage for store-failure toast/transcript. |
| crates/tui/src/tui/ui/event_loop.rs | Subscribes/drains runtime events and renders store-failure notices. |
| crates/tui/src/task_manager.rs | Ingests store-failure events; exposes runtime event subscription. |
| crates/tui/src/runtime_threads/tests.rs | Adds tests for typed context + event publication behavior. |
| crates/tui/src/runtime_threads.rs | Implements typed store failure context + event emission plumbing. |
| crates/tui/src/runtime_api/tests.rs | Adds compat SSE mapping test for store-failure events. |
| crates/tui/src/runtime_api.rs | Forwards runtime.store_failure over compat SSE mapping. |
| crates/tui/src/localization.rs | Adds new MessageIds for store-failure notices. |
| crates/tui/locales/zh-Hant.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/zh-Hans.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/vi.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/uk.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/ru.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/pt-BR.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/ko.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/ja.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/id.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/hi.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/fr.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/es-419.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/en.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/de.json | Adds translations for new store-failure notice strings. |
| crates/tui/locales/ca.json | Adds translations for new store-failure notice strings. |
| crates/tui/CHANGELOG.md | Notes new visible behavior for runtime store failures. |
| CHANGELOG.md | Notes new visible behavior for runtime store failures. |
Review details
- Files reviewed: 26/26 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| /// The typed store fault behind an error, if any layer of it is one. | ||
| #[must_use] | ||
| pub fn from_error(error: &anyhow::Error) -> Option<&Self> { | ||
| error.downcast_ref::<Self>() | ||
| } |
| Ok(Err(error)) => { | ||
| let failure = format!("Failed to monitor {}: {error}", kind.label()); | ||
| // An unreadable item or turn under the monitor is the | ||
| // operator's own state: name the file before settling (#5931). | ||
| self.report_store_failure(&thread_id, Some(&turn_id), &failure, &error, false) | ||
| .await; | ||
| failure | ||
| } |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ffde4b7a6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "{record}", | ||
| &format!( | ||
| "{} {}", | ||
| notice.failure.record_kind, notice.failure.record_id | ||
| ), |
There was a problem hiding this comment.
When a non-English locale is active, {record} is still constructed from RuntimeStoreRecordKind's English Display implementation, so otherwise translated notices contain phrases such as thread <id> or item <id>. Add typed message IDs for these nouns or select a fully localized template rather than composing user-visible prose from the enum display.
AGENTS.md reference: crates/tui/AGENTS.md:L25-L26
Useful? React with 👍 / 👎.
| // #5931: the background runtime's own store faults arrive on its event | ||
| // channel, which nothing else in this loop reads. | ||
| let mut runtime_event_rx = task_manager.subscribe_runtime_events(); |
There was a problem hiding this comment.
When startup restores queued tasks, TaskManager::start_with_executor_and_runtime spawns its workers before returning, but this broadcast subscription is created only later after additional TUI setup. Because Tokio broadcast receivers do not replay earlier messages, a recovered task can publish runtime.store_failure before this line and the promised toast/transcript notice is permanently missed; establish the receiver before workers start or replay durable failure events on subscription.
Useful? React with 👍 / 👎.
| Err(TryRecvError::Lagged(skipped)) => { | ||
| tracing::warn!( | ||
| skipped, | ||
| "runtime event tap lagged; a store-failure notice may have been missed" | ||
| ); |
There was a problem hiding this comment.
When more than the runtime channel's 1,024 events accumulate before this tap drains them, TryRecvError::Lagged advances past the skipped records and only writes a log warning. If runtime.store_failure was among those records, it never reaches the toast or transcript—the exact failure this change is intended to surface. Use a dedicated failure channel or replay the durable event log after lag instead of accepting the loss.
Useful? React with 👍 / 👎.
| let record: TurnRecord = serde_json::from_str(&raw).with_context(|| { | ||
| RuntimeStoreRecordFailure::new( | ||
| RuntimeStoreOperation::Parse, | ||
| RuntimeStoreRecordKind::Turn, | ||
| turn_id, | ||
| &path, | ||
| ) | ||
| })?; |
There was a problem hiding this comment.
When a turn record was written by a newer Codewhale version, deserialization succeeds and the subsequent schema-version check fails outside this new typed context. RuntimeStoreRecordFailure::from_error therefore returns None; the monitor logs the failure, settlement fails to load the same turn again, and no terminal runtime.store_failure is emitted, leaving task drivers waiting until their guard timeout without the actionable path notice. Include post-deserialization validation failures in the typed load context as well.
Useful? React with 👍 / 👎.
| // The operator's own store failed; the payload names the file and | ||
| // the next action, so compat clients see it too (#5931). | ||
| crate::runtime_threads::RUNTIME_STORE_FAILURE_EVENT => Some(sse_json( | ||
| crate::runtime_threads::RUNTIME_STORE_FAILURE_EVENT, | ||
| payload.clone(), | ||
| )), |
There was a problem hiding this comment.
When this mapped event carries terminal: true, take_compat_turn_event still considers only turn.completed terminal. Since this notice explicitly means no turn.completed can follow, the one-turn compatibility SSE handler remains open sending keepalives indefinitely instead of emitting done; existing compatibility clients that wait for the stream's normal terminal marker hang even though the runtime has already declared the turn failed. Treat a terminal runtime.store_failure as a terminal stream event.
Useful? React with 👍 / 👎.
| match self.operation { | ||
| RuntimeStoreOperation::Read | RuntimeStoreOperation::Parse => format!( | ||
| "Move {} aside (or delete it) and retry; the thread's other records stay in place.", | ||
| self.path.display() | ||
| ), |
There was a problem hiding this comment.
When a read fails because of permissions, a transient I/O error, file-descriptor exhaustion, or another non-corruption cause, this branch still tells the operator to move or delete the record even though its contents may be healthy. Following that advice can destroy the only copy of a turn or item while doing nothing to fix the actual access problem; distinguish parser corruption from generic read failures and give read errors a non-destructive permissions/access remedy.
AGENTS.md reference: AGENTS.md:L28-L29
Useful? React with 👍 / 👎.
| .replace("{path}", ¬ice.failure.path.display().to_string()) | ||
| .replace("{reason}", ¬ice.reason) |
There was a problem hiding this comment.
When the runtime-store path contains the literal substring {reason}—which is legal in a configured home or data-directory name—the {path} replacement inserts it and the following replacement interprets that inserted text as another template placeholder. The notice therefore displays a different, nonexistent path and gives unusable recovery guidance. Substitute placeholders in one pass, as the existing brace-preserving localization formatter does, rather than chaining replace over already-inserted values.
Useful? React with 👍 / 👎.
![codewhale-agent[bot]](https://avatars.githubusercontent.com/u/101357273?s=60&v=4){kind=small}
There was a problem hiding this comment.
Codewhale review
PR makes runtime store read/parse/write faults visible through typed RuntimeStoreRecordFailure context, a runtime.store_failure event, TUI toasts, task timeline errors, and SSE/docs. The implementation is mostly solid and well-tested, but schema-version load failures are not typed, leaving a task-idling blind spot.
Findings
- [WARNING] Schema-version load failures bypass typed context and can still idle tasks (
crates/tui/src/runtime_threads.rs:1678)
In load_turn, the schema_version bail after a successful parse returns a plain anyhow error without a RuntimeStoreRecordFailure context. settle_claimed_turn_failure relies on RuntimeStoreRecordFailure::from_error to publish a terminal runtime.store_failure when the turn record cannot be loaded. A turn record that parses but has schema_version > CURRENT_RUNTIME_SCHEMA_VERSION therefore fails with an untyped error, no runtime.store_failure is published, and the driver can wait/idle instead of failing fast. The same visibility gap applies to load_item/load_thread, but load_turn is the terminal-idle case.
Assessment
The change is well-structured, localized, documented, and covered by focused tests. The terminal-behavior blind spot for future-schema turn records should be fixed before merge so every failed load_turn in the terminal path publishes a terminal runtime.store_failure.
Advisory review by Codewhale (codewhale review --pr 5972 --post, head 6ffde4b7a6ec0ab5138e5a3d8da782b3d6f498bd). Line-specific findings are also posted as inline review comments; mechanical fixes arrive as committable suggestions you can apply from the Files tab. CODEOWNERS approval still governs merge.
| &path, | ||
| ) | ||
| })?; | ||
| if record.schema_version > CURRENT_RUNTIME_SCHEMA_VERSION { |
There was a problem hiding this comment.
[WARNING] Schema-version load failures bypass typed context and can still idle tasks
In load_turn, the schema_version bail after a successful parse returns a plain anyhow error without a RuntimeStoreRecordFailure context. settle_claimed_turn_failure relies on RuntimeStoreRecordFailure::from_error to publish a terminal runtime.store_failure when the turn record cannot be loaded. A turn record that parses but has schema_version > CURRENT_RUNTIME_SCHEMA_VERSION therefore fails with an untyped error, no runtime.store_failure is published, and the driver can wait/idle instead of failing fast. The same visibility gap applies to load_item/load_thread, but load_turn is the terminal-idle case.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_3f1a047d-3c39-4d27-846d-a77fdb1433c0) |
|
Claude finished @Hmbown's task in 7m 7s —— View job Claude's review of PR #5972I independently traced the diff ( Correctness bugs (highest priority)
Judgment call worth a second look
Lower-priority, confirmed but narrow
What's solidLock discipline around |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| crate::runtime_threads::RUNTIME_STORE_FAILURE_EVENT => Some(sse_json( | ||
| crate::runtime_threads::RUNTIME_STORE_FAILURE_EVENT, | ||
| payload.clone(), | ||
| )), |
There was a problem hiding this comment.
Confirmed gap (traced independently): the mapped SSE event is forwarded fine here, but the stream-closing decision lives one level up in take_compat_turn_event (runtime_api.rs:5453), which only treats event.event == "turn.completed" as terminal. A runtime.store_failure notice with payload["terminal"] == true — which by this PR's stated contract means "no turn.completed can follow" — still leaves take_compat_turn_event's terminal bool false. All three call sites (lines ~5328, ~5349, ~5405) gate yield Ok(sse_json("done", ...)); return; on that bool, so the one-turn compat stream never sends done and stays open on 15s keepalives after a terminal store failure — exactly the "driver waits/idles instead of failing fast" scenario this PR exists to prevent, just for this one consumer.
The one test that touches this event (stream_compat_mapping_forwards_runtime_store_failures, runtime_api/tests.rs:11337) calls map_compat_stream_event directly and never exercises take_compat_turn_event, so the gap has no coverage. Suggest folding this into take_compat_turn_event's terminal check, e.g. event.event == "turn.completed" || (event.event == crate::runtime_threads::RUNTIME_STORE_FAILURE_EVENT && event.payload.get("terminal").and_then(Value::as_bool).unwrap_or(false)).
| Ok(Err(error)) => { | ||
| let failure = format!("Failed to monitor {}: {error}", kind.label()); | ||
| // An unreadable item or turn under the monitor is the | ||
| // operator's own state: name the file before settling (#5931). | ||
| self.report_store_failure(&thread_id, Some(&turn_id), &failure, &error, false) | ||
| .await; |
There was a problem hiding this comment.
Confirmed via trace (agrees with Devin's and Copilot's flags on this line): this report_store_failure call (terminal=false) fires on the raw monitor_turn error, then settle_claimed_turn_failure runs immediately after (line 7323) and independently re-runs self.store.list_items_for_turn(turn_id) (line 7073) and self.store.load_turn(turn_id) (line 7116) against the same on-disk fault. If those re-reads hit the same unreadable file — the common case, since nothing repaired it in between — report_store_failure fires again at line 7102 or 7149 (this time terminal=true for the load-turn case), producing two runtime.store_failure events, two toasts, two transcript lines, and two task-timeline Error entries for one underlying fault.
Since the two calls differ only in message text and the terminal flag, one option: have monitor_claimed_turn skip its own report_store_failure when it's about to call settle_claimed_turn_failure anyway (settlement already re-derives the typed failure with the correct terminal classification from its own load/list attempts), and reserve this call site's report for panics/errors that don't route through settlement.
| let record: TurnRecord = serde_json::from_str(&raw).with_context(|| { | ||
| RuntimeStoreRecordFailure::new( | ||
| RuntimeStoreOperation::Parse, | ||
| RuntimeStoreRecordKind::Turn, | ||
| turn_id, | ||
| &path, | ||
| ) | ||
| })?; |
There was a problem hiding this comment.
Confirming the Codewhale bot finding with the concrete downstream trace: the schema_version bail! three lines below this (unchanged by this PR) produces a plain anyhow::Error with no RuntimeStoreRecordFailure context, so RuntimeStoreRecordFailure::from_error (line 5405) returns None for it. In settle_claimed_turn_failure (line 7116), this is exactly the "turn's own record is unreadable" case that's supposed to be terminal: true — but here report_store_failure (line 7149) silently returns None and only logs. The turn's on-disk record is never marked Failed either (the load itself failed, so there's nothing to mutate and save), so this one fault mode — a turn record written by a newer binary than the one now reading it — produces zero visible signal: no toast, no transcript line, no task-timeline error, no terminal SSE event. A waiting driver idles out instead of failing fast, which is the exact failure mode this PR exists to close. The same gap applies to load_thread's and load_item's schema checks just above and below this function.
| pub fn next_action(&self) -> String { | ||
| match self.operation { | ||
| RuntimeStoreOperation::Read | RuntimeStoreOperation::Parse => format!( | ||
| "Move {} aside (or delete it) and retry; the thread's other records stay in place.", | ||
| self.path.display() | ||
| ), | ||
| RuntimeStoreOperation::Write => format!( | ||
| "Check free space and permissions for {}, then retry; nothing was overwritten.", | ||
| self.path.display() | ||
| ), | ||
| } | ||
| } |
There was a problem hiding this comment.
Agreeing with Codex's flag on this line: Read and Parse share the same "move it aside (or delete it)" remedy, but they're different failure classes. Parse (JSON deserialization failed) is genuine corruption — deletion is reasonable. Read (read_store_file failed — permission denied, ENOSPC on a temp/lock file, a transient EIO, an NFS hiccup) says nothing about the content being bad; the file may be perfectly healthy and the fix is a permissions/mount problem, not deleting the operator's only copy of a turn or item record. This is the one place in the PR where the notice text can actively cause data loss for a fixable, non-corruption failure — worth splitting the two operations' remedies (e.g. "check permissions and retry; if the file is confirmed corrupt, move it aside" for Read, keep the current wording for Parse).
Closes #5931
The runtime-store half of #5931. The session-id-divergence half landed earlier as #5960, and the approval-receipt half (approval log path + error kind + next action in the tool error) already landed on main in 8edc6cb.
What changed
load_thread/load_turn/load_item/save_*/list_items_for_turnpaths attach a typedRuntimeStoreRecordFailure(operation, record kind, record id, path) asanyhowcontext, so a monitor recognizes a store fault by type instead of by message text. The context'sDisplaykeeps the previous message text (Failed to read turn <path>), so anything matching on the old strings still matches.RuntimeThreadManagergainsreport_store_failure, which logs and — when the error carries the typed context — publishes aruntime.store_failureevent through the existingemit_event->event_tx->RuntimeEventRecordchannel (theSnapshotsDisabledNotice/SessionIdDivergedNoticeprecedent from tui: a diverged engine session id is a visible notice, not a log line #5960). The paths that used to be log lines only are wired in: "Failed to terminalize item after monitor failure", "Failed to list turn items", "Failed to load turn after monitor failure", "Failed to persist terminal monitor failure", and theFailed to monitor turnerror path. A fault that also blocks event publication stays a log line, since the event log lives in the same store.TaskManager::subscribe_runtime_events) once per event-loop tick and shows each notice as a 12 s warning toast plus a transcript system line:RuntimeStoreUnreadableNotice/RuntimeStoreUnwritableNotice, added asMessageIds and translated in all 15 locale packs exactly as tui: a diverged engine session id is a visible notice, not a log line #5960 did.task_manager::ingest_runtime_event) records the notice as a taskErrorevent, and aterminal: truenotice resolves the turn asFailed, so the driver stops waiting instead of idling out.map_compat_stream_event) forwards the event, anddocs/RUNTIME_API.mddocuments the payload.How a store failure surfaces now
A corrupted or unwritable record under the session's runtime store produces one event with:
operation(read|parse|write),record_kind(thread|turn|item),record_id,path, the fullerrorchain, the root-causereason, a concretenext_action(move that file aside, or check free space and permissions there), a one-linemessage, andterminalwhen the turn's own record is unreadable/unwritable — meaning noturn.completedcan follow and waiting drivers should treat it as failed.Gates (worktree
tmp/wt-5931, commit 6ffde4b, rebased on main at 2ef252c)cargo fmt --all: cleancargo clippy --workspace --all-targets --all-features --locked -- -D warnings(with the standing-A uninlined_format_args/too_many_arguments/unnecessary_map_or): clean. Its first post-rebase run caughtstart_with_executoras dead code in the lib target (only test callers remain after this refactor), fixed with#[cfg(test)]cargo test -p codewhale-tui --lib --locked: 11876 passed, 0 failed, 13 ignored (208.9 s). Two earlier full runs under heavy machine contention (other worktrees running the same suite concurrently) each had a few deadline-sensitive failures (oneruntime_chat_relay, sixruntime_threads/runtime_apisequencing tests); each of those passes in isolation and none touches code changed hereNote
Medium Risk
Touches durable runtime persistence and turn lifecycle termination on store I/O failures; behavior change is intentional but affects failure modes when disk state is corrupt or full.
Overview
When the background runtime cannot read, parse, or write its own thread/turn/item JSON on disk, the change stops treating that as log-only noise and makes it actionable across the stack.
Runtime: Load/save paths attach typed
RuntimeStoreRecordFailurecontext (operation, record kind, id, path). Monitor/settle paths callreport_store_failure, which emitsruntime.store_failurewith reason,next_action, andterminal: truewhen the turn’s record is unreadable/unwritable so tasks do not idle waiting forturn.completed.Consumers: The task timeline records the notice (terminal faults fail the turn). The TUI subscribes to runtime events and shows a warning toast plus transcript line via new locale strings. Compat SSE forwards the event; RUNTIME_API docs describe the payload.
Reviewed by Cursor Bugbot for commit ea45b97. Bugbot is set up for automated code reviews on this repo. Configure here.