[theQuert]

theQuert/stima-api#5886 guards G2, G3 and G4 (EL-DISK-5886-CONSUMERS-1739). The plan is in https://github.com/theQuert/stima-api/issues/5886. The root cause:

• reclaim-byproducts is the only live trimmer of go-build and the Claude scratchpads.
• It hung in a per-path grep over 21.7 GB of transcripts, then was frozen.
• Its backups only log.

Every new deleting step defaults to spotcheck (log-only). Nothing deletes more until a separate change flips a mode after 5 clean rounds (SCRIPT-SAFE).

G2: go-build cap (shell/disk-janitor.sh, config/disk-janitor.conf)

• go-build-age now also runs from the hourly --check when either of these holds:
  • go-build is over CC_DJ_GO_BUILD_CAP_GB (default 8);
  • free space is under CC_DJ_GO_BUILD_FREE_GB (default 35).
• Before any delete, a system-wide listing runs: lsof -nP -b -Fn, time-bounded by CC_DJ_HOLDER_TIMEOUT_SECONDS (default 120).
  • A timeout, or a non-zero rc even with partial output, deletes nothing (reason=lsof-unusable).
  • An unmeasurable du does not count as over the cap.
• Under auto, act runs below 30 GB free, so go-build-age's effective act trigger is 30 GB. This is documented in CLAUDE.md.

G3: a scratchpad liveness lookup that cannot hang or fail open (shell/reclaim-byproducts.sh)

• One index of transcript mentions replaces the per-path grep. It is cached under a fingerprint of the corpus and of the extraction rule (v2).
• The lookup has a hard budget (CC_RB_MENTION_BUDGET_SECONDS, default 600) and runs in its own process group. TERM, INT and HUP also take that group down.
• Any lookup that does not finish cleanly keeps every directory and writes no cache and no stamp. That covers a timeout, a find rc other than 0, a grep rc of 2 or more, a missing binary, or a signal.
• Each path-in-use.sh call is bounded by the phase budget left.
• On a 108 MB fixture, the old per-path grep took 102 s and the new lookup takes 5–6 s. The base HEAD took 13 s.

G4: stalled-reclaimer alarm (shell/resource-watch.sh)

• It raises ALERT:reclaim-stalled once when the last-finished stamp is older than 2× the LaunchAgent StartInterval. With the installed 7200 s interval, that is 14400 s.
• With no stamp, the age comes from the plist's mtime (else the script's), so a first sweep that never finishes still alerts.
• It re-arms when the stamp is fresh again. It kills nothing and deletes nothing.

Review

• Opus review of the full diff found:
  • one P1, a G3 fail-open that the reviewer reproduced: a grep exiting 2 or missing gave an empty index that was cached, and 5 referenced dirs were reaped in the default config;
  • an unbounded lsof;
  • a kill test that could pass with zero proof;
  • four P3s.
• All are fixed in 448e0e5, 2fe4027 and 296353f. The Opus delta review re-ran the repro on 296353f: 14 of 14 dirs kept, no cache, no stamp. P1 is closed, and it found no new fail-open path.
• That delta review's P2 (a reclaimer with no stamp never alerts) is fixed in 396c9e4.
• Follow-ups, not blocking:
  • A per-transcript incremental cache, because any live transcript append changes the corpus fingerprint.
  • A transcript deleted mid-lookup makes that run keep everything. That is safe, and G4 catches it if it persists.

Tests (foreground, temp dirs and fakes only)

• tests/reclaim-byproducts.sh: rc=0, passed: 314, failed: 0 (on 296353f; 396c9e4 does not touch reclaim)
• tests/disk-janitor.sh: rc=0, 434 ok, 0 not ok, disk-janitor validation passed (on 296353f; 396c9e4 does not touch the janitor)
• tests/resource-watch.sh: rc=0, 166 ok, 0 not ok, resource-watch validation passed (on 396c9e4)
• New fail-closed cases:
  • grep exits 2, grep missing, grep killed by a signal, an unreadable dir for find: each keeps 10 of 10, with no cache or stamp;
  • a hung lsof and a partial lsof: each deletes nothing;
  • a direct bounded_group test that the grandchild is killed by timeout and by TERM;
  • no stamp, old and fresh plist.

Deploy (not done in this PR)

• After merge: install.sh on the Mac mini. The live disk-janitor.conf needs no change, since the new knobs have defaults and the modes stay spotcheck.
• Rollback: reinstall from the previous commit.
• Not touched: Docker, volumes, pids 70987/4565, ~/.cc-reaper.

[theQuert]

#113 live spot-check round 8 on the Mac mini (read 02:50–02:56Z, read-only). Log timestamps are local (+0800). Load was 159 (1-minute average) at 02:51Z.

• G2 (go-build cap): one --check since 00:48Z. It started at 01:13:42Z, with its own check: disk free=11% line. At 01:53:00Z it logged not run, free 40.8G is not under 25 GB (soft floor 35) and the cache is 2.3G, not over the 8 GB cap.
  • Both numbers agree with the next record. The reclaim sweep that started at 01:53:26Z logged free 40.8GB -> 39.6GB and a go cache of 2GB -> 2GB (rounded).
  • At 02:50Z, df -h /System/Volumes/Data shows 40Gi available and du -sg ~/Library/Caches/go-build shows 3. That is consistent with the cache growing back after the sweep.
  • No new step deleted anything.
• G3 (scratchpad lookup): one sweep since 00:48Z, pid 81696 (scheduled, 1h gate). It ran 01:53:26–02:16:22Z, 1377 s.
  • It logged the transcript lookup did not finish within 600s; no directory judged, every one kept; the next run tries again.
  • This is the second timeout in a row, again under sustained load (over 100 all morning). It is still the designed fail-closed path: no verdicts, nothing removed, no cache and no stamp. pass-when allows it ("or kept everything").
  • The sweep's closing at least one reap failed line has no other failure line above it, so it comes from this abstain.
  • There are no new verdicts to sample this round.
• G4 (stall alarm): it fired correctly, and this is the first true alert.
  • The last-finished stamp is still 21:32:38Z, so 14400 s is reached at 01:32:38Z.
  • The 01:26:22Z check is at 13424 s with no alert. The 01:37:00Z check is at 14662 s and logs ALERT:reclaim-stalled. It did not repeat on any later check, because the alert is raised once and re-arms on a fresh stamp.

SCRIPT-SAFE count: G2 8 clean rounds. G3 8 clean, 0 misses; rounds 7 and 8 are safe abstains (kept everything), not judged samples. G4 8 clean, including its first true alert. The rate stays every two hours, and the modes stay spotcheck.

Follow-up, not blocking (this is the second timeout, as named in round 7): under the current load the lookup cannot finish in 600 s, so G3 judges nothing. Nothing is lost while the mode is spotcheck, but it would block a later flip to act. The next step is a cheaper lookup, the per-transcript incremental cache already listed in the PR follow-ups. The budget will not be raised. G3 stays in spotcheck until a lookup finishes and a judged sample passes.

check-result: PASS

[theQuert]

check-by 2026-10-29T04:50:00Z: #113 live spot-check round 9 on the Mac mini. pass-when: every disk-janitor --check after 02:50Z logs a go-build guard verdict that matches du and df at its time (or its own check: disk free line when df cannot be read at that minute), with no deletion by a new step in spotcheck; every reclaim sweep since 02:50Z finished its lookup inside budget (or kept everything), and a re-read sample of any verdicts matches the 6 h transcript rule; resource-watch raised reclaim-stalled if and only if the last-finished stamp was older than 14400 s at its check, at most once until a fresh stamp. on-fail: roll back the faulty script from .bak-113, record the miss, fix it in cc-reaper, and return to hourly. Not shorter: 8 clean rounds hold the two-hour rate under SCRIPT-SAFE.