Skip to content

feat(trusty-mpm): retire aged-out session records and color tm ls NUM/NAME - #4993

Merged
bobmatnyc merged 6 commits into
mainfrom
feat/tm-ls-retention-and-color
Aug 6, 2026
Merged

feat(trusty-mpm): retire aged-out session records and color tm ls NUM/NAME#4993
bobmatnyc merged 6 commits into
mainfrom
feat/tm-ls-retention-and-color

Conversation

@bobmatnyc

Copy link
Copy Markdown
Owner

The defect

tm ls printed NUM 107 at the bottom of a 31-row listing. The slot registry (#3034) hands a number to every record it observes; the default view then hides decommissioned/deleted rows. Measured on the reporting machine: 111 records, 76 terminal, 35 shown — 76 of the assigned slots were held by rows nobody could see.

Nothing had ever left the store. The auto-prune path calls /decommission with record_only=true, which leaves the record in place. So lowest-free slot allocation on its own is a provable no-op: the free set is permanently empty. Freeing numbers requires evicting the records behind them.

Change 1 — record retention (7 days)

Timestamp: a new SessionRecord::terminal_at, stamped when the record enters Decommissioned/Deleted. Not created_at — a session born in January and decommissioned today is a one-day-old tombstone, and a clock keyed off birth would delete it immediately. It is written at exactly one place, SessionRecord::enter_terminal_state, so a future terminal transition cannot silently skip it. Re-entering the same terminal state does not refresh the stamp (a repeated decommission must not extend retention); moving back to a live state clears it.

Trigger: the daemon's existing orphan-GC tick (daemon/mod.rs), placed ahead of the TmuxDriver::discover() guard that continues the tick — a store-only sweep must not be disabled by a machine having no tmux. AUTO_PRUNE_CAP=5 in session_picker_prune.rs is untouched: that is client-side rate limiting on the CLI's decommission path, a different mechanism at a different layer.

Window: TERMINAL_RECORD_RETENTION_DAYS = 7, a named const, pinned by retention_window_is_seven_days.

How #3034's tombstone intent survives

#3034 asks that a deleted session's slot stay reserved and render as -- deleted -- rather than being handed to the next session. Three things keep that true:

  • Inside the window, nothing changes. A record that becomes terminal today is untouched for 7 days. Its slot stays held, and it still tombstones exactly as before if an explicit prune removes it — pinned by sweep_preserves_the_in_window_tombstone_slot.
  • The registry is in-memory and resets on daemon restart (unchanged). A number captured before a restart was already meaningless, so the guarantee only ever had to hold for one daemon lifetime — a span the 7-day window comfortably covers.
  • Beyond the window there is no record left to point at. numbered_snapshot now walks the registry's held slots instead of 1..=max, so a released slot leaves the listing entirely rather than becoming a phantom tombstone at a number no operator ever saw.

The guarantee narrows from "forever within a daemon lifetime" to "for the whole retention window", which in practice is the same thing.

Eviction touches the record store and nothing else

sweep_terminal_records calls SessionStore::remove_many (an entry in sessions.json) and SlotRegistry::release (an in-memory map). It opens no other path for writing. The one filesystem call it makes is an existence check, and that check can only ever make it evict fewer records.

That check is load-bearing, not tidiness. prune_orphaned_worktrees protects a worktree from deletion by finding its path among the workspace_paths read from the store — a set deliberately unfiltered by state, because a record carrying a terminal state is routinely a live session with unsaved work (see the measured case documented in prune::reap_orphaned_worktrees). Both of that sweep's two independent reads come from the store, so deleting a record would remove the path from both at once, collapsing a defense-in-depth pair into nothing and handing the worktree to an unattended dry_run: false timer. A record whose workspace directory still exists is therefore never evicted, however old. sweep_never_touches_the_filesystem seeds a 400-day-old terminal record with a real directory and a canary file, and asserts the sweep evicts nothing, leaves the directory standing, and leaves the file's bytes unchanged.

Legacy records are dated, not deleted

Every pre-existing record has terminal_at == None. That means UNKNOWN, not old: it is stamped with the current time and the window starts there. Inferring death from last_activity_at/created_at would be wrong in the common direction — the auto-prune (#4384/#4702) decommissions long-idle records, so a record whose last activity is a month old may have become terminal seconds ago, and 27 of the 76 terminal records here look "7+ days old" by that measure while their actual death dates are unknown. Guessing would evict them on the first sweep with zero retention.

Measured before / after

records terminal max NUM
before 111 76 107
after first sweep 111 (76 stamped) 76 107
after the window elapses 43 8 ≤ 43

The 8 survivors are the terminal records whose workspace directory still exists (67 have no workspace_path, 1 points at a path that is gone). One write per sweep, not per record — upsert_many/remove_many batch the reload and the save.

Change 2 — color NUM and NAME

Two distinct hues: NUM magenta (35), NAME cyan (36). Not one hue — a name's trailing serial comes from allocate_serial (per-project, 01–99, reuses gaps) while NUM is a global slot, so tm-foo-01 at NUM 1 is coincidence; 57 currently listed sessions end in -01, and retention makes an accidental match more likely, not less. Both are plain (non-bold, non-bright) mid-tones, legible on light and dark backgrounds — 33 was avoided because light themes wash it out.

  • Gate: a new table_use_color(stdout_tty), not picker_use_color. The table prints with println!; the picker with eprintln!. Reusing the picker's stderr gate would leak escapes into tm ls | grep whenever stderr stays a TTY. Both delegate to one color_enabled so the NO_COLOR rule (any set value, including empty, disables) cannot drift.
  • Alignment: {:<5}/{:<24} measure the formatted value, so an ANSI-wrapped column came out ~9 chars narrow. pad_visible colorizes the text and appends the padding outside the escape. format_tombstone_row gets the same treatment and its NUM column still lines up with a live row's.
  • Plain output is byte-identical. ls_row_plain_when_color_disabled asserts the plain row has no escape at all and that stripping the escapes from the colored row reproduces it exactly.

Rendered (\x1b shown literally):

before:  7      11111111-…  active          tm-trusty-tools-01        do the thing
after:   \x1b[35m7\x1b[0m      11111111-…  active          \x1b[36mtm-trusty-tools-01\x1b[0m        do the thing
piped:   7      11111111-…  active          tm-trusty-tools-01        do the thing   ← unchanged

managed.rs was 477/500 SLOC, so the table renderer moved to commands::managed_render.

Gate — rung 5 (record persistence, irreversible deletion)

cargo fmt --check && cargo check -p trusty-mpm && cargo clippy -p trusty-mpm --all-targets -- -D warnings
EXIT=0

cargo test -p trusty-mpm --no-fail-fast
test result: ok. 4667 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 47.60s
  (+ 44 further binaries, all ok, 0 failed)
EXIT=0

cargo test -p trusty-mpm -- --include-ignored
test result: FAILED. 4672 passed; 2 failed; 0 ignored
  ensure_managed_config_dir_emits_the_frozen_skill_warning          (#4931, known flake)
  daemon::managed_routes::tests::stale_assets_for_many_*            (#4619, known flake)
Both pass in isolation:
  cargo test -p trusty-mpm --lib -- --include-ignored --test-threads=1 <both>
  test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 4669 filtered out

Nine lint gates, all EXIT=0:

script exit
check_line_cap.sh 0 — 3725 files, 0 violations
check_sld.sh 0 — 0 errors, 0 warnings
check_doc_numbers.sh 0
check_test_pointers.sh 0 — 21921 citations, 0 dangling
check_capabilities.sh 0 — up to date (no CLI command added or renamed)
check_changelog_fragment.sh 0 — fragment present and valid
check_generation_artifacts.sh 0
check_agent_assets.sh 0
check_buildrs_sync.sh 0

New coverage: retention_tests.rs (10 cases — the pure verdict for every guard, eviction, slot release and reuse, the no-duplicate-NUM invariant across that cycle, the filesystem no-touch assertion, and the legacy stamp-then-evict sequence), slot_registry_release_*, store_{upsert,remove}_many_*, enter_terminal_state_*, and the three ls_row_* color/alignment cases.

Refs #3034

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

…/NAME

`tm ls` showed `NUM 107` at the bottom of a 31-row listing. The slot registry
(#3034) numbers every record it observes and the default view then hides the
terminal ones — 76 of those 107 slots were held by rows nobody could see.
Nothing had ever left the store: the auto-prune path decommissions
`record_only`, which keeps the record. Freeing numbers therefore needs the
records evicted, not a smarter allocator; with the free set permanently empty,
lowest-free allocation alone is a provable no-op.

Retention: `SessionRecord::terminal_at` records when a record entered
`Decommissioned`/`Deleted`, stamped at the single `enter_terminal_state` entry
point. `SessionManager::sweep_terminal_records` evicts records past
`TERMINAL_RECORD_RETENTION_DAYS` (7) and releases their slots; it runs on the
daemon's existing GC tick, ahead of the tmux probe so a machine without tmux
still sweeps. Two guards keep it safe: a record whose workspace directory still
exists is never evicted (it is what keeps that path in
`prune_orphaned_worktrees`'s protected set — dropping the record would collapse
both of that sweep's independent reads at once), and a record with no
`terminal_at` is dated rather than deleted, because the auto-prune decommissions
long-idle records so inferring death from `created_at`/`last_activity_at` would
evict the whole legacy backlog with no retention at all.

Color: `NUM` renders magenta and `NAME` cyan — distinct hues, because a name's
trailing serial comes from `allocate_serial` (per-project, reuses gaps) while
`NUM` is a global slot, so any match is coincidence. The gate keys off stdout's
TTY-ness, not the picker's stderr gate, so `tm ls | grep` stays clean; padding
now measures visible width so ANSI escapes cannot stagger the table.

The table renderer moved to `commands::managed_render` — `managed.rs` was 23
SLOC under the cap.

Refs #3034

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools
@bobmatnyc bobmatnyc added the trusty-mpm trusty-mpm platform and related work label Aug 6, 2026
@bobmatnyc bobmatnyc self-assigned this Aug 6, 2026
@bobmatnyc bobmatnyc added the trusty-mpm trusty-mpm platform and related work label Aug 6, 2026
@bobmatnyc

Copy link
Copy Markdown
Owner Author

Verdict: WARN

Adversarial review at head 5b2fd03e1029aeab9c6c20cded950196fbcf2d07, rung 5, reviewed against the spec only. One HIGH, no CRITICAL. The central design — evict the record, keep the worktree — holds up; the guard that makes it hold up has one hole.

Findings

Severity File Line Issue Fix Disposition
HIGH session_manager/retention.rs 179 Path::exists() returns false on any stat error, not only absence, so the load-bearing guard evicts when it cannot determine try_exists().unwrap_or(true) + a debounce Fix here
MEDIUM session_manager/reactivate.rs 112 The only production terminal→live transition bypasses enter_terminal_state, so the documented "revival clears the stamp" invariant is not enforced call enter_terminal_state(Active, now) or clear terminal_at Fix here
MEDIUM session_manager/retention.rs 202 remove_many hard-deletes from a stale snapshot without re-validating against the store it reloads; and the on-disk guard ignores cwd, which mark_reactivated treats as revivable re-derive the verdict after the reload; align the two guards on cwd Fix here
LOW changelog.d/4988-tm-ls-retention-and-color.md Named for #4988, which is a different open PR rename to 4993- (or 3034-) Fix here
LOW daemon/mod.rs 594 The placement comment claims the sweep cannot be silently disabled; TRUSTY_MPM_ORPHAN_GC=0 disables it one clause in the comment Fix here

HIGH — retention.rs:179 — the existence guard fails open

let on_disk = record
    .workspace_path
    .as_deref()
    .is_some_and(std::path::Path::exists);

Path::exists() is documented to return false when the metadata cannot be read at all — a permission error on an ancestor, a stale NFS/SMB handle, EIO on a volume that went away. "Cannot determine" and "not there" produce the same answer, and that answer is the one that evicts. This function's own doc (retention.rs:150-151) states the opposite:

the one filesystem call it makes is an existence check that can only ever make it evict FEWER records

In the error case it makes it evict more. That matters because the two sweeps run in the same orphan-GC tick, retention first: daemon/mod.rs:605, then daemon/mod.rs:709. One sweep taken while the volume is unreachable evicts the record permanently — nothing re-adds it when the volume returns — and from then on the path is absent from both of prune_orphaned_worktrees's deliberately-unfiltered store reads (prune.rs:1092, prune.rs:881). That is the pair prune.rs:1070 says data loss requires collapsing.

The two sibling destructive sweeps in this same loop both require two consecutive observations before acting (OrphanGc at daemon/mod.rs:582, PidOrphanGc at daemon/mod.rs:586). Retention acts on one.

Not CRITICAL: reaching a real remove_dir_all still needs the #3649 sentinel gate to name a provably-ownerless owner, git_worktree_list_agrees, and the #4091 dirty gate, which refuses on modified, untracked, unpushed, or unreadable trees. I could not construct a path at >80% confidence to destroying content git cannot reconstruct.

.is_some_and(|p| p.try_exists().unwrap_or(true));

unwrap_or(true) means "cannot determine → assume present → keep", which is what the doc already promises. Worth a test that an unreadable path yields Keep, and worth considering the two-pass debounce the rest of this loop uses.

MEDIUM — reactivate.rs:112enter_terminal_state is not the only write path

record.rs:420 says terminal_at is "Written at exactly one place — Self::enter_terminal_state", and record.rs:454 clears it on a non-terminal transition so "a record resurrected out of a tombstone does not carry a stale death time". enter_terminal_state_clears_stamp_on_revival tests that branch.

No production caller reaches it. mark_reactivated is the only path that moves a Decommissioned record back to Active (reactivate.rs:86, mark_reactivated_flips_decommissioned_to_active), and it assigns record.state = ManagedSessionState::Active directly. The revived record keeps its old stamp.

Today that is inert — retention_verdict short-circuits on !is_terminal() at retention.rs:111, so an Active record is never evicted whatever its stamp says. The defect is that the invariant is documented and tested but not enforced where it is actually exercised, and the next person to add a terminal transition, or to read terminal_at outside the is_terminal gate, inherits a stale value.

MEDIUM — retention.rs:202 — hard delete from a stale snapshot, and two guards that disagree

The verdict comes from a cached_all() snapshot at retention.rs:171. remove_many (store.rs:294) then reloads from disk and deletes every id in evicted unconditionally — it never re-runs retention_verdict against the record it is about to drop. A full upsert_many file write sits between the two (retention.rs:193), so the window is real rather than theoretical.

mark_reactivated can flip a record to Active inside that window; it gets hard-deleted anyway. Reaching it is easier than it looks, because the two guards disagree about what counts as on-disk: mark_reactivated resolves resume_dir as workspace_path falling back to cwd (reactivate.rs:98-101), so a record with workspace_path == None — exactly what retention evicts, and exactly what decommission_record_only produces at decommission.rs:531-535 — is revivable whenever its cwd still exists. That half needs no race at all.

This repo already solved this shape next door: prune_orphaned_worktrees takes "ONE fresh snapshot immediately before the deletion loop (#1845 item 9)" and re-runs the dirty check "immediately before each individual remove_session_worktree" (prune.rs:750-754, prune.rs:852-857). The retention sweep should do the same.


What I attacked and could not break

Stated explicitly, because "no finding" is information:

  • No NUM collision is constructible. observe returns lowest_free(), which by construction is not a key of slots (slots.rs:783-792), and release (slots.rs:844) is called only for ids the sweep evicted. Even the cross-process case — the supervisor rewriting sessions.json and resurrecting an evicted record after its slot was freed — re-observes it into a free slot, never a held one.
  • No phantom tombstone. numbered_snapshot walks assigned_slots() (numbering.rs:52), and a released slot is gone from slots, so a gap renders as nothing rather than as a number nobody held.
  • enter_terminal_state is the only production path that writes a TERMINAL state. Every other .state = site is either a test or a non-terminal transition, and set_workspace's generic new_state parameter (manager.rs:1133) is only ever passed Active.
  • No unstamped record can be evicted. retention_verdict returns Stamp for terminal_at == None before any age comparison (retention.rs:114-118), and a record stamped by a sweep is not re-examined by that same sweep.
  • remove_many has no partial-failure mode beyond the one upsert/remove already have. It reloads, mutates the map, saves once, and propagates a save error. The in-memory/on-disk divergence on a failed save() is the pre-existing shape of upsert, not something this PR introduces.
  • No ANSI can reach a non-TTY stdout. render_session_table resolves table_use_color(std::io::stdout().is_terminal()) once at the I/O boundary (managed_render.rs:79); color_enabled is tty && NO_COLOR unset (session_picker_render.rs:164-166), shared with the picker gate. ls_row_plain_when_color_disabled asserts byte-identity after stripping escapes, and ls_row_alignment_matches_with_and_without_color covers the tombstone row's NUM column against a live row's.
  • No coverage removed, #[ignore]d, cfg-gated, --excluded, or narrowed to --lib. The only fn deletions in the diff are the five moved into managed_render.rs.

#4978 collision — trivial

One line: the use crate::commands::session_picker_render::{…} import at session_picker_tests.rs:17-19. This PR adds table_use_color; #4978 adds command_legend. A union merge, resolved in seconds.

Nothing else touches. #4978's session_picker.rs hunks (67, 601-1013) never reach this PR's two lines (205, 1117), and #4978 does not reference render_session_table — it imports filter_live_sessions, which stays in managed.rs. #4978 appends to session_picker_render.rs at line 164+, below this PR's edits at 44-80 and 124-166. Whichever lands second rebases cleanly.

Gates I ran at head 5b2fd03e

cargo fmt --check && cargo clippy -p trusty-mpm --all-targets -- -D warnings && cargo test -p trusty-mpm --no-fail-fast
EXIT=101

test result: FAILED. 4666 passed; 1 failed; 7 ignored; 0 measured; 0 filtered out; finished in 58.39s
failures:
    core::managed_config::tests::ensure_managed_config_dir_emits_the_frozen_skill_warning

That is the documented #4931 baseline flake. git diff origin/main...HEAD --name-only -- crates/trusty-mpm/src/core/ returns agent_reset_workspace.rs and session_assets.rs only — this PR touches zero lines of managed_config.

fmt and clippy passed (the chain is &&-gated). Every new test passed:

test session_manager::retention::tests::retention_window_is_seven_days ... ok
test session_manager::retention::tests::retention_verdict_keeps_live_states ... ok
test session_manager::retention::tests::retention_verdict_keeps_record_whose_workspace_still_exists ... ok
test session_manager::retention::tests::retention_verdict_stamps_undated_terminal_record ... ok
test session_manager::retention::tests::retention_verdict_keeps_record_inside_window ... ok
test session_manager::retention::tests::retention_verdict_evicts_record_outside_window ... ok
test session_manager::retention::tests::sweep_evicts_only_records_past_the_window ... ok
test session_manager::retention::tests::sweep_preserves_the_in_window_tombstone_slot ... ok
test session_manager::retention::tests::sweep_releases_the_evicted_slot ... ok
test session_manager::retention::tests::sweep_never_touches_the_filesystem ... ok
test session_manager::retention::tests::sweep_stamps_legacy_records_instead_of_evicting_them ... ok
test session_manager::record::tests::enter_terminal_state_stamps_once ... ok
test session_manager::record::tests::enter_terminal_state_clears_stamp_on_revival ... ok
test session_manager::slots::tests::slot_registry_release_frees_the_slot_for_reuse ... ok
test session_manager::slots::tests::slot_registry_release_of_unknown_id_is_a_noop ... ok
test session_manager::store::tests::store_upsert_many_writes_all_in_one_pass ... ok
test session_manager::store::tests::store_remove_many_removes_present_ids_only ... ok
test commands::managed::tests::ls_row_colors_num_and_name_in_distinct_hues ... ok
test commands::managed::tests::ls_row_plain_when_color_disabled ... ok
test commands::managed::tests::ls_row_alignment_matches_with_and_without_color ... ok
test commands::session_picker::tests::table_use_color_requires_stdout_tty ... ok
test commands::session_picker::tests::table_use_color_false_when_no_color_set_even_on_tty ... ok
bash scripts/check_line_cap.sh
LINECAP_EXIT=0
line-cap: measured 3728 tracked .rs file(s) (floor 500); 7 allowlisted, 0 violations — OK.

Required before re-review

  1. retention.rs:179try_exists().unwrap_or(true), plus a test that an unreadable workspace path yields Keep. This is the one that must land.
  2. reactivate.rs:112 — clear terminal_at on the Decommissioned → Active flip, with a test.
  3. retention.rs:202 — re-derive the verdict against the reloaded record before deleting; decide whether cwd belongs in the on-disk guard or out of mark_reactivated's fallback.
  4. Rename the changelog fragment to 4993-.
  5. Trim the daemon/mod.rs:594 comment to what the placement actually guarantees.

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

… sweep

Code-critic WARN on #4993 — 1 HIGH, 2 MEDIUM, 2 LOW.

HIGH: `Path::exists()` maps every stat error to `false`, so a permission error
on an ancestor, a stale NFS handle, or `EIO` from a departed volume read as "not
there" — and "not there" evicts. The `workspace_path` that leaves with the
record is what keeps `prune_orphaned_worktrees` from deleting the worktree, so
failing open there could hand live work to an unattended sweep. The probe is now
`try_exists().unwrap_or(true)` behind `workspace_present`, whose two non-Ok(false)
answers both mean KEEP; the doc that claimed the check "can only evict fewer
records" now describes what the code does. The probe is injected, so the error
branch is tested deterministically rather than provoked with real permissions.

Retention also now debounces like the two sibling destructive sweeps in the same
orphan-GC tick: a record must be evictable on TWO consecutive sweeps. One
observation was not enough because "the workspace is gone" is a live filesystem
reading, and an external volume that unmounts between ticks answers Ok(false)
truthfully — the error handling above cannot catch that.

MEDIUM: `mark_reactivated` is the sole production Decommissioned -> Active
transition and assigned `record.state` directly, so a revived record kept its
`terminal_at` stamp and the invariant on the field was false. It routes through
the setter — renamed `set_lifecycle_state`, since it now governs both directions
— and a test drives the real reactivation path instead of the setter alone.

MEDIUM: the delete ran against the phase-1 snapshot, which this sweep's own
`upsert_many` and any other process's write have already invalidated. Each
candidate is now re-read from the store and re-judged with a fresh probe
immediately before deletion, following `prune_orphaned_worktrees`'s #1845 item-9
shape.

LOW: the changelog fragment was named for #4988, a different open PR. LOW: the
comment claimed the sweep's placement prevents silent disablement, but
`TRUSTY_MPM_ORPHAN_GC=0` kills the whole loop.

Refs #3034

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools
@bobmatnyc

Copy link
Copy Markdown
Owner Author

Critic round 1 — all five addressed (head 067a362c)

HIGH 1 — the existence check failed open

Confirmed and fixed. Path::exists() maps every stat error to false, so a permission error on an ancestor, a stale NFS/SMB handle, or EIO from a departed volume read as "not there" — and "not there" evicts, taking the workspace_path that keeps prune_orphaned_worktrees from deleting the worktree. The doc claimed the opposite.

The probe is now try_exists() behind a named helper:

fn workspace_present(
    path: Option<&Path>,
    probe: impl Fn(&Path) -> std::io::Result<bool>,
) -> bool {
    path.is_some_and(|p| probe(p).unwrap_or(true))
}

Both non-Ok(false) answers — present, and undetermined — mean KEEP. The doc at what was line 150 now says what the code does. The probe is injected rather than provoked with real filesystem permissions, so workspace_present_treats_an_undetermined_path_as_present exercises the Err branch deterministically on every platform and under any uid, root included, and the same test carries the reading through retention_verdict to assert Keep.

Two consecutive observations — implemented, not argued against

You were right that the asymmetry with the two siblings in the same tick mattered, and the reason is sharper than "be consistent": try_exists cannot save the case that actually worries me. An external volume that unmounts between ticks answers Ok(false) truthfully — the mountpoint's child genuinely is not there — so no error handling catches it. Only a second observation does.

RetentionDebounce is owned by orphan_gc_loop alongside gc and pid_gc; confirm() returns the candidates seen on the previous call and re-arms with the current set, so a record that stops being a candidate (reactivated, volume remounted) disarms itself rather than staying primed. debounce_requires_two_consecutive_observations proves sweep one evicts nothing and sweep two does; debounce_disarms_a_candidate_that_lapses proves a gap re-arms from zero rather than counting as the second observation.

MEDIUM 2 — enter_terminal_state was not the only write path

Correct, and the test was covering a branch no caller reached — the second one tonight, as you say. mark_reactivated (reactivate.rs:112) now routes through the setter, renamed set_lifecycle_state since it governs both directions rather than only entry. mark_reactivated_clears_the_terminal_stamp drives the real reactivation path end to end and asserts the cleared stamp is what gets persisted.

The invariant is now stated as what it is: every transition into or out of a terminal state goes through the setter; a live→live transition may still assign state directly, because a live record's terminal_at is already None — which is exactly what the setter's clearing branch guarantees. I audited the other 20 .state = sites: reconcile.rs:104 continues on terminal records before reaching its Active assignment, and no production site assigns a terminal state outside decommission/delete (agent_reset_workspace.rs:530 is a test).

MEDIUM 3 — deleting from a stale snapshot

Fixed in the shape you pointed at. Phase 3 re-reads each confirmed candidate via self.get() (which reloads from disk) and re-runs the verdict with a fresh probe immediately before deleting; anything that no longer says Evict is left alone and logged. sweep_revalidates_before_deleting_a_reactivated_record arms the gate, calls the real mark_reactivated between the two sweeps, and asserts the record survives with terminal_at == None.

On the cwd half: mark_reactivated falling back to cwd and retention keying only on workspace_path is a deliberate asymmetry, not an oversight. prune_orphaned_worktrees's protected set is built from workspace_path exclusivelycwd is not in it — so evicting a record with workspace_path: None cannot unprotect anything on disk. Widening retention's guard to cwd would make nearly every record permanently un-evictable, since cwd is almost always a real directory. What is genuinely lost is the ability to reactivate a >7-day-old decommissioned session, which is the retention ruling working as intended.

LOW 4 — fragment renamed

changelog.d/4988-…changelog.d/4993-….

LOW 5 — comment corrected

It now says the placement keeps a store-only sweep from being tied to tmux discoverability, and states explicitly that TRUSTY_MPM_ORPHAN_GC=0 disables the whole loop, retention included.

Gate — rung 5, re-run at 067a362c

cargo fmt --check && cargo check -p trusty-mpm && cargo clippy -p trusty-mpm --all-targets -- -D warnings
EXIT=0

cargo test -p trusty-mpm --no-fail-fast
test result: ok. 4672 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 49.27s
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
  (+ 42 further binaries, all ok)
EXIT=0

cargo test -p trusty-mpm --lib -- --include-ignored
test result: ok. 4679 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 49.72s
EXIT=0

The full-workspace --include-ignored run additionally hits meta_run_demo_writes_and_verifies_artifact (baseline row 40). No coverage was deleted, ignored, cfg-gated, or narrowed.

Nine lint gates, all EXIT=0: check_line_cap (3728 files, 0 violations), check_sld, check_doc_numbers, check_test_pointers (21955 citations, 0 dangling), check_capabilities, check_changelog_fragment, check_generation_artifacts, check_agent_assets, check_buildrs_sync.

check_line_cap did catch one thing on the way: the inlined retention block pushed daemon/mod.rs to 501 SLOC. The defaults moved to SessionManager::sweep_default_retention, which is where they belonged anyway — the daemon has no use for the injectable clock and window that exist for the tests.

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

@bobmatnyc

Copy link
Copy Markdown
Owner Author

Verdict: APPROVE

Re-review at head 067a362c, scoped to the fix round (5b2fd03e..067a362c) and what it could have disturbed. All five prior findings resolved. Two new findings, both below the HIGH bar.

Findings

Severity File Line Issue Fix Disposition
MEDIUM session_manager/retention.rs 237 The Test: pointer names sweep_revalidates_before_deleting_a_reactivated_record as covering phase 3; that test never reaches phase 3 Correct the pointer, or add a seam that exercises the intra-sweep window Fix here
LOW session_manager/record.rs 420 "every transition INTO or OUT OF a terminal state goes through the setter" — the out-of-terminal half is unenforced at mark_errored and set_workspace Say the enforced invariant: every transition INTO a terminal state goes through the setter Fix here

1. HIGH — fail-open existence check: RESOLVED

workspace_present (retention.rs:155-160) resolves both non-Ok(false) answers to KEEP:

path.is_some_and(|p| probe(p).unwrap_or(true))

Errtrue → present → retention_verdict short-circuits Keep. Ok(true) → same. Only Ok(false) and None reach the age comparison.

The production call site does pass try_existsverdict_for at retention.rs:344:

let on_disk = workspace_present(record.workspace_path.as_deref(), |p| p.try_exists());

and it is the only path into retention_verdict from the sweep, used by both phase 1 (retention.rs:254) and phase 3 (retention.rs:287). Nothing calls Path::exists in this module any more.

The doc at what was line 150 now states what the code does (retention.rs:143-154, and retention.rs:210-213 on the sweep), including the reason it matters — the guard feeds prune_orphaned_worktrees's protected set.

On the injected probe: the generic parameter exists only for the test, but it is not a test-only path — production reaches the same three-line body through verdict_for's closure. What the injection buys is a deterministic Err, which a real-filesystem provocation cannot give you under root. workspace_present_treats_an_undetermined_path_as_present covers all four inputs and then carries the Err reading through retention_verdict to assert Keep. That is the right trade.

2. Two-observation gate: sound, and the reasoning is right

The premise checks out. try_exists distinguishes "cannot determine" from "not there"; it cannot help when a volume unmounts cleanly between ticks, because the mountpoint's child genuinely is absent and the answer is a truthful Ok(false). Only a second observation separates that from a real deletion.

I attacked the implementation on the four axes you would expect:

  • Can a candidate be evicted after ONE observation, through any path? No. sweep_terminal_records is the sole eviction site and always routes through debounce.confirm (retention.rs:277) before phase 3 can populate evicted. store.remove_many has exactly one production caller — retention.rs:301 — and one test. There is one production caller of the sweep (daemon/mod.rs:607), holding one RetentionDebounce, so no second instance can arm independently.
  • Does a lapsed candidate genuinely disarm? Yes. confirm replaces armed with the current candidate set unconditionally (retention.rs:198), so a record absent for one tick re-arms from zero. debounce_disarms_a_candidate_that_lapses drives arm → lapse → re-appear → confirm and asserts only the consecutive pair passes.
  • Unbounded growth? No. armed is overwritten each call, never extended; its size is bounded by the number of evict candidates in the store.
  • Key collision? No. ManagedSessionId is pub struct ManagedSessionId(pub Uuid) with derived Hash/Eq (record.rs:29-30) — one key per record.
  • Surviving a restart when it should not? It does not survive: constructed per orphan_gc_loop (daemon/mod.rs:588). That is the safe direction — nothing is evictable on the first tick after a restart. Cost is one tick, and ORPHAN_GC_INTERVAL_SECS = 60 (daemon/mod.rs:452), so ~60s against a 7-day window.

One behavior worth naming rather than flagging: if phase 1 errors out (reload_if_changed at retention.rs:248 propagates with ?), confirm is never called and armed survives the failed tick untouched. A record armed on tick A, skipped on tick B, confirmed on tick C is evicted after two observations separated by a tick that observed nothing. That is the correct reading of "two consecutive observations".

3. MEDIUM 2 — set_lifecycle_state: audit verified independently

I did not take the count on trust. Grepping every .state = and state: ManagedSessionState:: site in crates/trusty-mpm/src, then reading each production one:

Site State assigned Guard
manager.rs:575 stop Stopped returns InvalidState on is_terminal() (manager.rs:560)
manager.rs:745 mark_runtime_exited_stopped Stopped returns InvalidState unless state == Active (manager.rs:707)
manager.rs:912 resume Active Stopped | Errored only (manager.rs:829-839)
manager.rs:1112 mark_errored Errored none
manager.rs:1133 set_workspace new_state none; all 5 production callers pass Activelaunch_on_main.rs:176, lifecycle.rs:726, :1030, :1229, :1310
reconcile.rs:104 / :109 Active / Stopped continues on is_terminal() at reconcile.rs:81-99 — confirmed
adopt.rs:344 Stopped selector matches Active | Provisioning only (adopt.rs:336-342)

Every other hit is a test file (agent_reset_workspace.rs:530, managed_routes/reactivate.rs:461, mcp_proxy.rs:167, runtime_reap.rs:451, task_inject.rs:537/:686, resume_workdir.rs:342-422) or a different type entirely (control/actor.rs's SessionState, worktree_reconcile.rs's ReconcileState, core/circuit.rs).

The load-bearing half of the invariant is true: no production site assigns a terminal state outside set_lifecycle_state. That is the half that matters, because a direct terminal assignment is what would leave a stale terminal_at on a tombstone and make it immediately evictable.

The other half is not. mark_errored (manager.rs:1106-1114) and set_workspace (manager.rs:1125-1135) both take a record from get() with no terminality guard and assign a live state directly. If either were ever handed a tombstone, the record would leave the terminal state still carrying its stamp — the exact shape you just fixed in mark_reactivated. In practice neither is reachable that way: mark_errored runs on provision/spawn failure and set_workspace on spawn, both on records that have never been terminal. And it is self-healing even if it happened — a live record is Keeped unconditionally (retention.rs:112), and a later re-terminalization goes through set_lifecycle_state, whose !was_terminal branch restamps fresh (record.rs:462-464).

So: LOW, and the fix is the doc, not the code. record.rs:420-424 should claim what is enforced — every transition INTO a terminal state — rather than a symmetric invariant two unguarded call sites could violate.

mark_reactivated_clears_the_terminal_stamp is the right test: it drives the real path and asserts on the persisted record, not just the in-memory return.

4. MEDIUM 3 — re-validation before delete

The re-read is genuine. SessionManager::get (manager.rs:325-336) takes the store write lock, calls reload_if_changed, then cached_get. reload_if_changed (store.rs:196-210) re-stats and re-reads whenever the (mtime, len) fingerprint differs, treating a None on either side as changed. Same-process writes are already reflected in self.data; cross-process writes trip the fingerprint. Not a cached snapshot.

The window is as small as claimed — the phase-3 loop, then remove_many at retention.rs:301. A record confirmed at the top of the loop and deleted at the bottom is separated by at most the remaining get() calls plus one lock acquisition.

The cwd claim holds. prune_orphaned_worktrees builds its protected set from workspace_path exclusively — prune.rs:883 (let Some(p) = r.workspace_path else) and prune.rs:1096 (.filter_map(|r| r.workspace_path)). cwd appears nowhere in prune.rs. So evicting a record with workspace_path: None removes nothing from that set, and the asymmetry with mark_reactivated's cwd fallback is not a hole. Widening retention to cwd would, as you say, make almost every record permanently un-evictable. Withdrawn.

New MEDIUM — the phase-3 test does not reach phase 3

sweep_revalidates_before_deleting_a_reactivated_record is named in the Test: pointer at retention.rs:237 as covering phase 3. It does not reach it.

mark_reactivated sets the record to Active. On the second sweep, phase 1's retention_verdict short-circuits at retention.rs:112 (!record.state.is_terminal()), so the record is never pushed to candidates, confirm returns empty, and the phase-3 loop body never executes. The record is saved by phase 1 plus the debounce — the same two mechanisms debounce_requires_two_consecutive_observations already covers.

Proven rather than argued. I put eprintln!("CRITIC_PHASE3_REACHED id={id}") at the top of the phase-3 loop in a throwaway worktree and ran the retention module:

test session_manager::retention::tests::debounce_requires_two_consecutive_observations ... CRITIC_PHASE3_REACHED id=b549a47a-…
test session_manager::retention::tests::sweep_evicts_only_records_past_the_window ... CRITIC_PHASE3_REACHED id=6a10120b-…
CRITIC_PHASE3_REACHED id=2a91ffd8-…
test session_manager::retention::tests::sweep_releases_the_evicted_slot ... CRITIC_PHASE3_REACHED id=0df7c920-…
test session_manager::retention::tests::sweep_stamps_legacy_records_instead_of_evicting_them ... CRITIC_PHASE3_REACHED id=ee342254-…
test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 4664 filtered out

Four tests reach it; sweep_revalidates_… is not among them. (Instrumentation reverted; git status --porcelain clean.)

So phase 3's Evict arm is covered incidentally by every eviction test, but its two protective arms — Ok(_) => warn!(…leaving it in place) at retention.rs:291 and Err(_) at :296 — are unexercised.

This is structural, not an oversight in the test: because phase 1 re-derives candidates from a fresh snapshot on every sweep, anything that changes between sweeps is caught at phase 1 and never reaches phase 3. Phase 3 only fires for a write landing within one sweep's execution, which has no seam to drive from a test.

Not a defect in the code — phase 3 reads correctly and is the right belt-and-braces behind the debounce. The defect is the pointer claiming coverage the test does not provide, which is the same shape as the enter_terminal_state test you just fixed: a documented guarantee with no caller behind it. Either correct the pointer to say phase 3's protective branches are uncovered and why, or add the seam. The doc fix is the cheaper and, given this repo's ban on test-only production methods, probably the right one.

5. LOW 4/5 — both confirmed

  • crates/trusty-mpm/changelog.d/4993-tm-ls-retention-and-color.md — renamed, one Changed category, sits directly in changelog.d/.
  • daemon/mod.rs:598-604 — now states the placement keeps a store-only sweep off tmux discoverability, and that TRUSTY_MPM_ORPHAN_GC=0 disables the loop, retention included. Matches daemon/mod.rs:133-137.

6. Did the fixes break anything?

The relocation is behavior-preserving. sweep_default_retention (retention.rs:328-335) computes Duration::days(TERMINAL_RECORD_RETENTION_DAYS) and Utc::now() — the same two values the daemon inlined before, in the same order, passed to the same function. The daemon's match arms are unchanged apart from renaming outcome to o.

No coverage removed, ignored, or gated. git diff origin/main...067a362c -- 'crates/trusty-mpm/**' removes five fns, all of them the managed_render.rs moves already cleared in round 1 (render_session_table, truncate, short_timestamp, format_tombstone_row, format_state_column). Zero added #[ignore], zero added #[cfg(not…)].

Gates at 067a362c

cargo fmt --check && cargo clippy -p trusty-mpm --all-targets -- -D warnings && cargo test -p trusty-mpm --no-fail-fast
EXIT=0

test result: ok. 4672 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 49.56s
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 4.22s
  (+ 43 further binaries, all ok)

The #4931 baseline flake (ensure_managed_config_dir_emits_the_frozen_skill_warning) did not fire on this run.

Every test the fix round added or renamed passed:

test session_manager::retention::tests::workspace_present_treats_an_undetermined_path_as_present ... ok
test session_manager::retention::tests::debounce_requires_two_consecutive_observations ... ok
test session_manager::retention::tests::debounce_disarms_a_candidate_that_lapses ... ok
test session_manager::retention::tests::sweep_revalidates_before_deleting_a_reactivated_record ... ok
test session_manager::record::tests::set_lifecycle_state_stamps_once ... ok
test session_manager::record::tests::set_lifecycle_state_clears_stamp_on_revival ... ok
test session_manager::reactivate_tests::mark_reactivated_clears_the_terminal_stamp ... ok
bash scripts/check_line_cap.sh
LINECAP_EXIT=0
line-cap: measured 3728 tracked .rs file(s) (floor 500); 7 allowlisted, 0 violations — OK.

Notes

Neither finding blocks. The HIGH is closed, the two-observation gate is correct and correctly reasoned, the re-read is real, and the cwd asymmetry is justified by prune.rs's protected-set construction. Both remaining findings are documentation claiming more than the code enforces — worth a two-line correction, not another round.

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

…verclaiming it

Code-critic round 2, MEDIUM: `sweep_revalidates_before_deleting_a_reactivated_record`
did not exercise phase 3. Confirmed with an `eprintln!` marker in the phase-3
loop — four sibling tests printed, that one did not.

Structural, not sloppiness: phase 1 re-derives candidates from scratch every
sweep, so a record reactivated BETWEEN sweeps is classified `Keep` there and
never becomes a candidate. The debounce plus phase 1 is what saved that record;
the re-validation arms sat unexercised on an irreversible-deletion path.

Phase 3 is now `revalidate_for_eviction`, a named method the tests can reach.
Taking the candidate list as a plain `Vec<Id>` is not a test hook — a list
computed from an older view of the store IS the stale snapshot the guard exists
for, and passing one directly is the only way to observe the two non-evicting
arms fire. Three tests: a still-evictable candidate survives re-validation (so
the guard cannot degrade into "never delete"), a record reactivated after the
snapshot is dropped, and one a concurrent prune already removed is dropped
rather than counted as evicted. Mutation-checked: stubbing the guard to return
its input turns both "drops" tests red and leaves the first green.

The misnamed test is renamed `sweep_spares_a_record_reactivated_between_sweeps`
and its doc now names the guard that actually protects it.

LOW: the `terminal_at` invariant was stated symmetrically but enforced one way.
`mark_errored` and `set_workspace` assign a live state with no terminality
guard. The doc now claims only the entering half, names the two exceptions, and
says why a stale stamp on a live record is inert.

Refs #3034

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools
@bobmatnyc

Copy link
Copy Markdown
Owner Author

Round 2 — both closed (head ad0d1063)

MEDIUM — the phase-3 test never reached phase 3

Reproduced before fixing. eprintln!("PHASE3-MARKER {id}") in the loop body, running the retention module:

PHASE3-MARKER df7c65f1-…
PHASE3-MARKER 08b8fd46-…
test sweep_never_touches_the_filesystem ... ok
PHASE3-MARKER ece711ad-…
test debounce_requires_two_consecutive_observations ... ok
test sweep_stamps_legacy_records_instead_of_evicting_them ... ok
test sweep_revalidates_before_deleting_a_reactivated_record ... ok   ← no marker

Exactly as you called it. And the diagnosis is the important part: phase 1 re-derives candidates from scratch every sweep, so a record reactivated between sweeps is classified Keep there and never becomes a candidate. The debounce plus phase 1 saved that record; the re-validation arms had no coverage at all.

Phase 3 is now revalidate_for_eviction, a named method tests can reach. Taking the candidate list as a plain Vec<ManagedSessionId> is not a test hook — that is precisely what phase 2 produces, and a list computed from an older view of the store is the stale snapshot the guard exists for. Passing one directly is the only way to observe the two non-evicting arms fire; the doc on the method says so, including why the end-to-end route cannot reach them.

Three tests:

test arm
revalidate_keeps_a_still_evictable_record the passing arm — stops the guard degrading into "never delete"
revalidate_drops_a_record_reactivated_after_the_snapshot Ok(_), via the real mark_reactivated
revalidate_drops_a_record_a_concurrent_prune_already_removed Err(_)

Mutation-checked rather than assumed. Stubbing the guard to return its input:

test revalidate_keeps_a_still_evictable_record ... ok
test revalidate_drops_a_record_a_concurrent_prune_already_removed ... FAILED
test revalidate_drops_a_record_reactivated_after_the_snapshot ... FAILED

The misnamed test is now sweep_spares_a_record_reactivated_between_sweeps, and its doc names the guard that actually protects it — including the marker experiment, so the next reader does not have to redo it.

On the pattern — three tests tonight that looked like coverage

They share one shape: assert an outcome that more than one mechanism produces, without pinning which one produced it. sweep_spares_… asserted "record survives" — true under the debounce, phase 1, and phase 3, so it passed while testing none of them specifically. enter_terminal_state_clears_stamp_on_revival asserted "stamp cleared" against a setter no caller invoked. reinstall_preserves_a_customized_agent_without_force asserted "file preserved" from a hand-seeded state.

The check that catches all three, and the one I skipped: break the thing the test names and confirm it goes red. That is cheap — one stub, one run — and it is what turned the three new tests above from plausible into proven. I am treating it as required for any test whose subject is a guard on a destructive path, not optional diligence.

LOW — invariant stated symmetrically, enforced one-way

Correct: mark_errored (manager.rs:1106) and set_workspace (manager.rs:1125) assign a live state with no terminality guard. No enforcement added. The doc now claims only the entering half, names both exceptions explicitly, and states why a stale stamp on a live record is inert — retention_verdict short-circuits on !is_terminal(), and the next terminal transition restamps rather than trusting the old value.

Gate — rung 5, at ad0d1063

cargo fmt --check && cargo check -p trusty-mpm && cargo clippy -p trusty-mpm --all-targets -- -D warnings
EXIT=0

cargo test -p trusty-mpm --no-fail-fast
test result: FAILED. 4674 passed; 1 failed; 7 ignored; 0 measured; 0 filtered out; finished in 50.02s
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
  (+ 42 further binaries, all ok)
  failure: daemon::managed_routes::tests::stale_assets_for_many_reads_shared_agent_dir_once_for_the_whole_fleet  (baseline row 38)

cargo test -p trusty-mpm --lib -- --test-threads=1 daemon::managed_routes::tests::stale_assets_for_many
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 4678 filtered out
EXIT=0

An earlier --include-ignored --lib run in this round also hit ensure_managed_config_dir_emits_the_frozen_skill_warning (#4931) and bench_stale_assets_for_many; both pass under --test-threads=1 alongside the above. No coverage deleted, ignored, cfg-gated, or narrowed — the change is net +3 tests.

Nine lint gates, all EXIT=0:

script result
check_line_cap.sh 3728 files, 0 violations
check_sld.sh 0 errors, 0 warnings
check_doc_numbers.sh 0 violations
check_test_pointers.sh 21958 citations, 0 dangling
check_capabilities.sh up to date
check_changelog_fragment.sh fragment present and valid
check_generation_artifacts.sh 0 leaked artifacts
check_agent_assets.sh all match
check_buildrs_sync.sh in sync

check_test_pointers did fail once on the way: renaming the test left a dangling Test: pointer in sweep_terminal_records's doc. Fixed, and it now points at both the renamed test and the revalidate_* set with a note on which guard each covers.

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

Stamping `terminal_at = now` on records that predate the field grandfathered
the entire pre-existing backlog for a further seven days — 76 of 116 records on
the reporting machine, holding 76 of the slot numbers that put NUM in three
digits. The owner would install the fix and still see NUM 116.

`inferred_terminal_at` dates such a record from its latest evidence of life:
`last_activity_at` when set, else `created_at`, never earlier than `created_at`.
A timestamp in the future is not evidence — clock skew or corruption — and falls
back to `now`, so such a record gets a full window rather than being deleted on
the strength of a value that cannot be true. `RetentionVerdict::Stamp` now
carries the inferred time so the one call site that acts on it cannot silently
substitute its own.

The earlier reasoning against inference was right about the mechanism and wrong
about the goal: the auto-prune does decommission long-idle records, so this can
date a record earlier than it truly died — but only for records that predate the
field, and only toward clearing a backlog already weeks old. Ordering is
unchanged: the inferred date is written to disk first and only a LATER sweep can
evict, so the value is auditable before anything acts on it and the record still
passes the debounce and phase-3 re-validation.

Every safety property is untouched. `retention_verdict` still returns Keep on
`!is_terminal() || workspace_on_disk` BEFORE reading `terminal_at`, so the
workspace guard runs ahead of the backfill; `workspace_present` still resolves an
undetermined `try_exists` to present; the two-observation debounce and the phase-3
re-read are unchanged.

Measured against the live store (116 records, 40 shown): 27 evicted, 8 spared by
the workspace guard, 41 spared as genuinely recent — 36 of those created AND
decommissioned inside the last week. Max NUM 116 -> 89 on the first sweep cycle,
falling toward the shown-row count as the recent tombstones age out.

Refs #3034

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools
@bobmatnyc

Copy link
Copy Markdown
Owner Author

One-time backfill added (head ab90cd4c)

You were right and my earlier reasoning was the wrong shape. I argued that inferring a death date from last_activity_at "would evict the whole backlog with zero retention" — which is true, and is the goal. The window protects records that recently became terminal; it was never meant to grandfather 76 records that have been terminal for weeks.

The signal, and why

inferred_terminal_at takes the latest evidence the record carries that it was alive: last_activity_at when set, else created_at, never earlier than created_at. Latest rather than earliest, so the inference is as conservative as the data allows.

I checked for a better signal and there isn't one. SessionRecord carries no decommission timestamp — decommission writes state and clears pending_decision/proposed_default, nothing dated. Store-file mtime is one value for the whole file, so per-record it would read as "now" for everything. last_activity_at / created_at is all there is.

No usable signal → now. created_at is non-optional so a signal always exists, but it can be unusable: a timestamp in the future is clock skew or corruption, not evidence, and dating from it would compute a negative age. That falls back to now, so the record gets a full window rather than being deleted on the strength of a value that cannot be true.

RetentionVerdict::Stamp now carries the inferred time rather than letting the sweep pick, so the one call site that acts on it cannot silently substitute its own.

Ordering is unchanged and deliberate: the inferred date is written to disk first, and only a LATER sweep can evict. The operator can see what date a record was assigned rather than discovering it by the record's absence, and an old legacy record still goes through the debounce and phase-3 re-validation like any other. Backfill, arm, evict — three ticks, about two minutes at the 60s interval, not seven days.

Every safety property still holds

property status
Workspace-exists guard Unchanged. retention_verdict returns Keep on !is_terminal() || workspace_on_disk before it reads terminal_at, so the guard runs ahead of the backfill and no inferred date can reach a record with a live directory. Pinned by sweep_backfill_still_spares_a_legacy_record_whose_workspace_exists, which seeds a 400-day legacy record with a real directory and a canary file and asserts three sweeps stamp nothing, evict nothing, and leave the file intact.
Fail-closed try_exists Unchanged. workspace_present still resolves Err to present. Not touched by this commit.
Two-observation debounce Unchanged. A backfilled record is a candidate only on the sweep after it is stamped, then needs one more to confirm.
Phase-3 re-validation Unchanged. Every eviction still re-reads the record and re-probes the filesystem immediately before deleting.

The change is confined to which timestamp a record with terminal_at == None receives. It adds no path that bypasses a guard, and no new caller of remove_many.

Mutation-checked, both branches

Reverting the inference to the old now behaviour:

inferred_terminal_at_uses_the_latest_evidence_of_life ... FAILED
retention_verdict_stamps_undated_terminal_record ... FAILED
sweep_backfills_an_old_legacy_record_and_evicts_it_without_waiting ... FAILED
sweep_gives_a_recent_legacy_record_its_full_window ... FAILED
test result: FAILED. 15 passed; 4 failed

The guard tests stayed green under it, correctly — they do not depend on the inference.

Removing the future-timestamp fallback:

inferred_terminal_at_falls_back_to_now_for_a_future_timestamp ... FAILED
sweep_stamps_now_when_a_legacy_record_has_no_usable_signal ... FAILED
test result: FAILED. 17 passed; 2 failed

What the owner will actually see — measured, and it is not the whole story

Against the live store as it stands (116 records, 40 rows shown):

count
evicted by inferred age 27
spared — workspace still on disk 8
spared — inferred age under 7 days 41
live (never in scope) 40

Max NUM: 116 → 89 on the first sweep cycle. Materially better, and it arrives in minutes rather than a week. But it is still 89 against 40 visible rows, so the owner will not see the two-digit-in-a-fifty-item-list result he asked for on day one, and I would rather say that now than have him find it.

The reason is not a defect. Of the 41 records spared as recent, 36 were created AND decommissioned inside the last seven days — they have no last_activity_at at all and a created_at from this week. Those are genuinely fresh tombstones and the window is doing exactly its job. They age out on their own, and NUM lands near the shown-row count about a week from now, permanently.

If the owner wants it tighter on day one, the lever is TERMINAL_RECORD_RETENTION_DAYS — his call, not mine, and I have not touched it.

Gate — rung 5, at ab90cd4c

cargo fmt --check && cargo check -p trusty-mpm && cargo clippy -p trusty-mpm --all-targets -- -D warnings
EXIT=0

cargo test -p trusty-mpm --no-fail-fast
test result: FAILED. 4679 passed; 1 failed; 7 ignored; 0 measured; 0 filtered out; finished in 48.86s
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
  (+ 42 further binaries, all ok)
  failure: daemon::managed_routes::tests::stale_assets_for_many_reads_shared_agent_dir_once_for_the_whole_fleet  (baseline row 38)

cargo test -p trusty-mpm --lib -- --include-ignored
test result: FAILED. 4686 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 47.75s
  failure: core::managed_config::tests::ensure_managed_config_dir_emits_the_frozen_skill_warning  (#4931)

cargo test -p trusty-mpm --lib -- --include-ignored --test-threads=1 core::managed_config::tests::ensure_managed_config_dir_emits_the_frozen_skill_warning
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 4686 filtered out
EXIT=0

Nine lint gates, all EXIT=0: check_line_cap (3728 files, 0 violations), check_sld, check_doc_numbers, check_test_pointers (21963 citations, 0 dangling), check_capabilities, check_changelog_fragment, check_generation_artifacts, check_agent_assets, check_buildrs_sync.

One thing I caught in myself, worth recording

My first pass at this replaced a region of the test file by text range and silently took four unrelated tests with it — workspace_present_treats_an_undetermined_path_as_present, both debounce_*, and sweep_spares_a_record_reactivated_between_sweeps. cargo test went green on 4675 tests and I nearly pushed it. What caught it was a clippy unused-import error on workspace_present, not the suite, because a deleted test cannot fail.

That is the "never make a red gate green by deleting coverage" rule violated by accident rather than intent, and the accidental version is harder to see. I diffed the test-function list against HEAD before and after, restored all four, and confirmed the only remaining deletion is sweep_stamps_legacy_records_instead_of_evicting_them — whose premise ("an old legacy record is stamped now and NOT evicted") is precisely what this commit reverses, replaced by two tests. Net for this commit: +6 tests. I am adding that before/after function-list diff to how I do range edits on test files.

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

…OME mutation

CI `Test` failed at ab90cd4 on `isolated_managed_state_guard_panics_on_production_root`
— "test did not panic as expected", 27 passed / 1 failed. Not one of the
documented baseline flakes.

The test reads `dirs::home_dir()` twice: once in its body to build the
production path, once inside `with_root_isolated_managed`'s guard. Eight tests
in the same binary hold a `HomeGuard` that overrides `$HOME` process-wide, and
`#[serial_test::serial]` only serialises those against each OTHER — this test
carried no such attribute. When a HomeGuard boundary lands between the two
reads they disagree about the production path, `assert_ne!` passes, nothing
panics, and the `should_panic` test fails.

Proven rather than assumed: widening the gap between the two reads to 1500ms
reproduces the CI signature exactly (27 passed, 1 failed, "did not panic as
expected"); adding `#[serial]` makes that same widened case pass 28/28. The
diagnostic sleep is not part of this commit.

Pre-existing, not caused by this branch: `git diff origin/main...HEAD --
crates/trusty-mpm/tests/` is empty, so the test file was byte-identical to main
when it failed. Same `$HOME`-mutation-race class as baseline rows 38 and 39,
which cover the sibling cases in the lib crate.

Fixed rather than re-rolled — a re-run that happened to pass would have proven
nothing and left the defect for the next PR.

Refs #3034

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools
@bobmatnyc

Copy link
Copy Markdown
Owner Author

CI Test failure diagnosed and fixed (head 0b2ebbd5)

Not a baseline flake, and not caused by this branch. It is a real pre-existing test defect that this branch happened to be unlucky enough to surface.

The actual failure

test isolated_managed_state_guard_panics_on_production_root - should panic ... FAILED

---- isolated_managed_state_guard_panics_on_production_root stdout ----
note: test did not panic as expected at crates/trusty-mpm/tests/session_manager_mvp.rs:2250:10

test result: FAILED. 27 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out; finished in 2.01s
error: test failed, to rerun pass `-p trusty-mpm --test session_manager_mvp`

None of the documented baseline flakes — not stale_assets_for_many_* (row 38), not #4931, not meta_run_demo_* (row 40).

Root cause

The test reads dirs::home_dir() twice: once in its body to build ~/.trusty-mpm, and once inside with_root_isolated_managed's guard, which recomputes the same path to compare against. Both reads resolve $HOME afresh.

Eight tests in the same binary hold a HomeGuard (#3965/#4206) that overrides $HOME process-wide. They carry #[serial_test::serial] — which serialises them against each other, not against everything else. This test carried no such attribute, so it ran in parallel with them.

When a HomeGuard boundary lands between the two reads, they disagree about the production path, assert_ne! passes, nothing panics, and a should_panic test that does not panic fails.

Proven, not assumed

Widening the gap between the two reads to 1500ms — simulating a slower runner — reproduces the CI signature exactly:

test isolated_managed_state_guard_panics_on_production_root - should panic ... FAILED
test result: FAILED. 27 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out

Same counts, same message. Adding #[serial_test::serial] makes that same widened case pass:

test isolated_managed_state_guard_panics_on_production_root - should panic ... ok
test result: ok. 28 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out

The diagnostic sleep is not part of the commit — the diff is one attribute plus the comment explaining it.

Ten unwidened runs of the full binary passed before the fix and eight after, which is why it took a deliberately widened window to see it: the window is the boundary crossing itself, so it is rare rather than load-dependent.

Pre-existing

$ git diff --stat origin/main...HEAD -- crates/trusty-mpm/tests/
(empty)

The test file was byte-identical to origin/main when it failed. This is the same $HOME-mutation-race class the baseline already documents at rows 38 and 39 for the lib crate — this is the integration-binary sibling, previously unrecorded.

Why I fixed it instead of reporting and stopping

You asked me to push nothing if it were purely a baseline flake. It is not — it is undocumented, and the fix is the file's own established convention: the comment above HomeGuard already says every caller pairs it with #[serial_test::serial], and this test reads the same shared mutable state without doing so. Re-running until green would have proven nothing and left it to fire on the next PR.

No coverage was removed, ignored, or narrowed. One attribute added.

Gate — rung 5, at 0b2ebbd5

cargo fmt --check && cargo check -p trusty-mpm && cargo clippy -p trusty-mpm --all-targets -- -D warnings
EXIT=0

cargo test -p trusty-mpm --no-fail-fast
test result: ok. 4680 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 47.97s
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
test result: ok. 1485 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
test result: ok. 28 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out   <- session_manager_mvp
  (+ 41 further binaries, all ok)
EXIT=0

cargo test -p trusty-mpm --lib -- --include-ignored
test result: FAILED. 4686 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
  failure: daemon::managed_routes::staleness_bench_tests::bench_stale_assets_for_many  (baseline row 38 family)

cargo test -p trusty-mpm --lib -- --include-ignored --test-threads=1 <that test>
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 4686 filtered out
EXIT=0

8x cargo test -p trusty-mpm --test session_manager_mvp  ->  0/8 failed

Nine lint gates, all EXIT=0: check_line_cap (3728 files, 0 violations), check_sld, check_doc_numbers, check_test_pointers (21963 citations, 0 dangling), check_capabilities, check_changelog_fragment, check_generation_artifacts, check_agent_assets, check_buildrs_sync.

🤖🤖🤖 Generated with trusty-mpm — https://github.com/bobmatnyc/trusty-tools

@bobmatnyc
bobmatnyc merged commit ab03fe6 into main Aug 6, 2026
28 checks passed
@bobmatnyc
bobmatnyc deleted the feat/tm-ls-retention-and-color branch August 6, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

trusty-mpm trusty-mpm platform and related work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant