Design principles and rejected alternatives for LUMI's execution cache.
| Document | Answers |
|---|---|
| Brief | What — executive summary |
| Philosophy (this doc) | Why — principles and tradeoffs |
| Caching model | When — hit/miss semantics |
| Whitepaper | How — full specification |
The foundational vocabulary rule.
| Memory implies | Cache implies |
|---|---|
| Identity, continuity, recall | Locality, speed, bounded lifetime |
| Narrative history | Invalidation and eviction |
| Cross-session persistence | Session-scoped TTL |
| "What the agent knows" | "What the agent recently computed" |
JoyRide uses operational terms only. See Glossary for banned vocabulary.
Lineage: Bazel action cache and Turborepo task cache are not "build memory." JoyRide applies the same discipline to agent execution.
Fast stale state is worse than slow correct state.
JoyRide optimizes repeated provably safe work. Incomplete proof, unknown policy, or fingerprint mismatch → execute normally.
Lineage: Bazel hermetic inputs · Nx input hashing · HTTP ETag revalidation
Allowlist for active reuse. Unknown commands never skip. Unsafe syntax never skips. Failed verification never reused as truth.
| Tier | Skip? | Store diagnostic? |
|---|---|---|
safe-readonly |
Yes (fingerprint match) | Yes |
verification |
Only with complete proof | Yes |
diagnostic-store-only |
Never | Sometimes |
no-store |
Never | Never |
Lineage: OpenSSH AllowUsers over implicit trust · default-deny security models
Every lookup returns JoyRideCacheDecision:
type— discriminantcanReuse— skip eligibilityreasonCode— stable vocabulary (JOYRIDE_REASON)fallbackBehavior— caller obligation
Lineage: Rust Result<T,E> · HTTP status + reason · OpenTelemetry span status
Legacy boolean | undefined APIs were removed. Ambiguity at the call site caused policy invention.
| Limit | Default |
|---|---|
| Total cache | 32 MiB |
| Per entry | 512 KiB |
| Per task | 8 MiB |
| Command summary | 12 KiB (head/tail truncate) |
| Decision log | 128 entries |
Under pressure: TTL → LRU → pressure trim → emergency trim (35% target).
Lineage: Redis maxmemory-policy · Memcached slab limits
No UI. UX = behavior + explainability:
- Decision log on every lookup
- Audit trail on every active skip
- Bug-report snapshots (bounded, no secrets)
- Reason codes stable enough for contract tests
Lineage: Linux perf/dmesg · OpenTelemetry without dashboards
Internal JoyRide failure must not block the agent:
- Suspend active reuse
- Continue normal execution
- Record degraded reason
- Include in bug snapshots
Lineage: Circuit breaker — fail optimization path, not request path
Frozen export surface (JOYRIDE_FROZEN_EXPORTS). Import boundary tests. Forbidden raw cache calls. Contract drift tests in CI.
Prevents the most common failure mode: internal cache layers eroding under maintenance pressure.
Lineage: Protobuf field stability · Semver + API compat tests
Reject secrets at the door. Count rejections. Never log raw rejected content. Diagnostics show counts, not credentials.
Scratch requires cleanupHandler. No handler → no admission. Flush on task end, cancel, shutdown, pressure. Idempotent cleanup.
Hidden temp buildup is an agent-runtime failure mode JoyRide explicitly prevents.
Documented for reviewers — these approaches were considered and rejected.
| Alternative | Why rejected |
|---|---|
| Boolean cache hit API | Call sites invented unsafe policy; no audit trail |
| Blocklist command classifier | Fails open as shell surface grows |
| "Trust last test output" verification | Highest correctness risk in agent loops |
| Compatibility wrappers for legacy API | Two APIs = two bugs; modern-only enforced |
Export JoyRideCache to integrations |
Raw .get()/.set() bypasses typed gates |
| Agent memory framing | Wrong semantics; unbounded retention expectation |
| Dashboard / status bar UX | Scope creep; logs + snapshots sufficient |
| Cross-session persistence (active reuse) | Stale state risk across workspace changes |
| Semantic/fuzzy search cache | Non-deterministic keys; false hits |
| Distributed remote JoyRide tier | Complexity; session-local sufficient for v1 |
| GC-only scratch cleanup | Orphaned files; no accountability |
| Unbounded decision log | Memory leak in long sessions |
| Anti-pattern | JoyRide response |
|---|---|
| Cache everything, filter later | Classifier gates admission |
| Boolean cache hit | Typed decision + reason |
| Trust last test output | Verification proof required |
| Memory for context | Bounded TTL + flush |
| Dashboard for debug | Structured logs + snapshots |
| Internal import for speed | Import boundary tests |
| Vague reason codes | JOYRIDE_FORBIDDEN_VAGUE_REASONS contract |
JoyRide adapts proven cache-system patterns — not copies:
| System | Concept borrowed |
|---|---|
| Bazel action cache | Content-addressable keys; input hashing |
| Turborepo caching | Task hash inputs; hit/miss semantics |
| Nx computation cache | Input-based invalidation |
| HTTP caching RFC 9111 | Validator mismatch → revalidate |
| Redis eviction | Bounded memory + policies |
| OpenTelemetry | Structured observability |
JoyRide is a session-scoped, fail-closed, typed decision cache for VS Code agent runtime — documented with the same rigor those systems established.
Before adding JoyRide capability, answer:
- Is it cache or memory? — if memory semantics, reject
- What are the inputs to the hash? — document in CACHING.md
- What proof is required for reuse? — default: complete proof
- What happens when proof fails? — default: execute normally
- What is the budget and TTL? — default: bounded
- What reason codes apply? — add to
JOYRIDE_REASON - What tests prove fail-closed behavior? — contract + dogfood
- Does it need UI? — default: no
JoyRide should feel like better momentum during coding sessions — not like the agent has a mind.
Fast when safe. Silent when irrelevant. Explicit when questioned. Disabled when needed. Degraded when suspicious. Fail-closed always.
Implementation changes must follow CONTRIBUTING.md: typed decisions, contract tests, documentation updates, no legacy APIs, no memory vocabulary.
MIT License — Copyright (c) CardSorting.