perf(supervisor): memoize GetMeta within one reconcile tick (sys-yre7dj) - #6
Merged
Merged
Conversation
beadReconcileTick reaches GetMeta from seven independent call sites per session per tick (worker-handle resolution, drain-ack check, restart-request check, pending-create-info matching, drain-ack-queue dedup, stale-drain-ack detection), each forking an uncached `tmux show-environment -t <session> <key>`. `sample` on the live 9.5-day supervisor showed GetEnvironment as the dominant leaf (365 weighted samples), scaling with session count. Add a tick-scoped memo to tmux.Provider: ResetTickCache() (called once per beadReconcileTick via the new local tickCacheResetter capability interface, so providers that don't support it are unaffected) starts a window in which GetMeta fetches a session's whole environment once and serves every key from it; SetMeta/RemoveMeta invalidate the affected session's entry so a same-tick write is never masked by a stale read. Deliberately NOT a Provider-wide decorator: cmd/gc asserts several optional Provider capabilities (IdleWaitProvider, RelaunchProvider, ...) on the exact `sp` value threaded through the same reconcile-tick call graph (session_reconciler.go), and a wrapping decorator would fail those type assertions silently, degrading idle-probe wake detection and drift-relaunch. Caching inside tmux.Provider's own GetMeta/SetMeta/ RemoveMeta leaves sp's concrete type and identity untouched, so every existing type assertion elsewhere in the codebase is unaffected. Tests (internal/runtime/tmux/tick_meta_cache_test.go): memoization within a tick, per-session isolation, a fresh window per ResetTickCache() call (the correctness requirement — a memo that outlived its tick would serve stale drain-ack/restart-request state), SetMeta/RemoveMeta invalidation, unchanged uncached behavior when ResetTickCache is never called, and error-classification parity with the original per-key GetMeta. Verified on the Nomad cluster (crun, golang:1.26): full internal/runtime/tmux suite passes, cmd/gc builds and vets clean, and the reconciler/drain-ack/restart/wake/worker-handle test subset in cmd/gc passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R8V78VNFSCPbidXmWwJuHQ
atbrace
force-pushed
the
fix/sys-yre7dj-getmeta-tick-cache
branch
from
September 4, 2026 20:57
b87a904 to
2739aaa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
beadReconcileTickreachesruntime.Provider.GetMetafrom seven independentcall sites per session per tick (worker-handle resolution, drain-ack check,
restart-request check, pending-create-info matching, drain-ack-queue dedup,
stale-drain-ack detection), each forking an uncached
tmux show-environment -t <session> <key>. Asamplecapture on the live 9.5-day supervisor (PID 2160)showed
GetEnvironmentas the dominant leaf (365 weighted samples), scalingwith session count — the fork cost is dominated by
os/exec's envconstruction for the child (60+ vars in a gc session env).
Root cause
cmd/gc/city_runtime.go'sbeadReconcileTick→reconcileSessionBeadsTracedWithNamedDemandfans out to seven call sites that each independently call
sp.GetMeta(name, key)for the same session within one tick, with no memoization:
workerHandleForSessionTargetWithRuntimeHintsWithConfig,providerDrainOps.isDrainAcked,providerDrainOps.isRestartRequested,runningSessionMatchesPendingCreateInfo,queueDrainAckAsyncStop,staleReconcilerDrainAckInfo,reconcilerDrainAckMatchesSessionInfo.Every one of them terminates at
tmux.Provider.GetMeta(internal/runtime/tmux/adapter.go),regardless of the seam-wrapping layers above it.
Fix
Add a tick-scoped memo to
tmux.Provider:ResetTickCache()starts a window inwhich
GetMetafetches a session's whole environment once (GetAllEnvironment)and serves every key from it;
SetMeta/RemoveMetainvalidate the affectedsession's entry so a same-tick write is never masked by a stale read.
CityRuntime.beadReconcileTickcallsResetTickCache()once at the top of eachtick via a local
tickCacheResettercapability interface, so providers thatdon't implement it are unaffected.
Deliberately NOT a Provider-wide decorator.
cmd/gctype-asserts severaloptional
Providercapabilities (IdleWaitProvider,RelaunchProvider, andothers) on the exact
spvalue threaded through this same reconcile-tick callgraph (
session_reconciler.go). A decorator wrapping the wholeProviderinterface would silently fail those assertions, degrading idle-probe wake
detection and drift-relaunch — a correctness regression worse than the CPU
cost it fixes. Caching inside
tmux.Provider's ownGetMeta/SetMeta/RemoveMetaleavessp's concrete type and identity untouched, so everyexisting type assertion elsewhere in the codebase is unaffected.
Tests
internal/runtime/tmux/tick_meta_cache_test.go:ResetTickCache()call — the correctness requirement: amemo that outlived its tick would serve stale drain-ack/restart-request state
SetMeta/RemoveMetainvalidate the memo so a same-tick write is visibleResetTickCacheis never called (tests,one-off CLI invocations)
GetMeta(
ErrSessionNotFound/ErrNoServerpropagate; everything else →("", nil))go test ./internal/runtime/tmux/...andgo vet ./cmd/gc/... ./internal/runtime/tmux/...pass.Validation
Verified on the Nomad cluster (
crun,golang:1.26, since this box cannotcompile
cmd/gc/dolt's cgo deps locally without the icu4c toolchain and perthis repo's convention that
cmd/gccompiles are offloaded):internal/runtime/tmuxsuite passes (no regressions).cmd/gcbuilds andgo vets clean.cmd/gcpasses (
-run "Reconcile|DrainAck|Restart|Wake|WorkerHandle|PendingCreate").Not yet dogfooded against the live jonesy supervisor (would require installing
a new binary onto the live control plane — out of scope for this branch;
flagged for the human-attended install/measurement step per sys-yre7dj).
sys-yre7dj
🤖 Generated with Claude Code
https://claude.ai/code/session_01R8V78VNFSCPbidXmWwJuHQ