Record objects' HTTP traffic as typeId-keyed cassettes - #12
Open
andreBurnt wants to merge 3 commits into
Open
Conversation
HttpClient's four message entry points now emit a changed('httpExchange')
event after each completed request, carrying the caller's id
(msg.routing.from), the request, and the response as text. Redaction is
stem-based on NAMES at the emission boundary: headers, query params, and
JSON/form body fields whose name contains a secret stem (key, token,
secret, passw, credential, session, signature, cookie, auth) are replaced
with REDACTED before anything crosses the bus. A false positive redacts
something harmless; a false negative persists a live credential - so the
matcher errs toward matching. Bodies are capped at 64K characters with a
truncated flag.
Nothing on this path parses JSON. With no dependents subscribed the
emission is skipped entirely (new hasDependents accessor on Abject), so
an unobserved HttpClient does no serialization work at all. Emission
failures are logged and never break the reply to the caller. Requests
that throw before a response exists (timeout, DNS, the SSRF guard)
produce no event - evidence of failed transport is a different shape
and a deliberate non-goal here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q23ProuK7YuZ4wtBbBmQGQ
A CassetteRecorder abject subscribes to HttpClient's httpExchange events (addDependent, the universal dependents protocol) and persists each exchange under cassettes:<typeId> in Storage. The caller's identity is resolved against the registry via resolveCallerIdentity - never trusted from the payload - and exchanges whose caller has no durable typeId are not recorded. A live AbjectId dies with its object; the typeId survives restarts, so the evidence does too. Retention is per endpoint bucket (method + path, query stripped, FIFO cap 5) with an overall cap of 50 per typeId. Global overflow always comes out of the LARGEST bucket, so a rare endpoint's only recording survives no matter how old it is. Failure independence: HttpClient discovery retries with capped backoff for as long as the recorder lives - a permanent recorder that silently gives up has failed its one job - and the subscription never waits on Storage. While Storage is missing, recording continues in memory; persistence catches up once it appears, merging what the store already held so a restart never clobbers accumulated evidence. Everything crosses the bus as messages. HttpClient and this recorder may be hosted on different worker threads (both are workerEligible), which is why no in-process seam could work. See mempko#11. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q23ProuK7YuZ4wtBbBmQGQ
Same shape as HealthMonitor: registered constructor, workerEligible, supervisedSpawn permanent with a system typeId. Recording is on from boot - evidence accumulates without anyone asking for it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q23ProuK7YuZ4wtBbBmQGQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Record objects' HTTP traffic as typeId-keyed cassettes
Piece 1 of the #11 series, per the order we agreed there: the recorder first, so evidence exists before anything judges it. No judging in this PR, no manifest declarations, no new dependencies. 3 commits, 4 files, +385/-5.
What this adds
Every HTTP request an object makes through
HttpClientnow leaves a trace: a cassette undercassettes:<typeId>inStorage, written by aCassetteRecorderabject spawned permanent at boot. Restart the object and the evidence is still there, because it is keyed by the durable typeId, not the ephemeral AbjectId.The seam
I asked in #11 whether you wanted an open exchange event or a scoped subscribe message. Reading the dependents protocol answered it:
changed()already IS both.HttpClientemitschanged('httpExchange', ...), and only objects that sentaddDependentreceive it. Anything on the bus may subscribe (your everything-on-the-bus spirit), nothing sees traffic without subscribing (my scoping concern). If you want a different shape, the emission is one method and moves cheaply.Everything crosses the bus as messages: the subscription (
addDependent), the exchanges (events), the persistence (Storagerequests), the attribution lookups (Registryrequests).HttpClientand the recorder are both workerEligible and may sit on different threads. There is no module state anywhere in the path.One line lands in
core/abject.ts: a protectedhasDependentsaccessor, so an unobservedHttpClientskips building event payloads entirely. Zero work when nobody subscribed.What the redaction does, and does not, promise
Redaction is stem-based on NAMES, applied before emission inside
HttpClient: headers, query params, and JSON/form body fields whose name contains a secret stem (key,token,secret,passw,credential,session,signature,cookie,auth) are replaced withREDACTEDbefore anything crosses the bus. That catchesauthorization,client_secret,refresh_token,x-amz-security-token,X-Amz-Signature, anaccess_tokenin an OAuth response body, apasswordin a form post - and whatever similarly-named header a generated object invents. A false positive redacts something harmless; a false negative persists a live credential, so the matcher errs toward matching.What it does NOT do: find a secret stored under an innocent name, or scan free text. It is a name-pattern scrub, not a secret scanner. If you want value-based redaction against what
SecretsVaultactually holds, that is a conversation I would have before building it - it means the vault's values reachHttpClientfor comparison, which has its own threat model. The request handed to the network is untouched either way; only the emitted copy is scrubbed.Two more honest boundaries: bodies are capped at 64K characters (not bytes) with a
truncatedflag, and requests that throw before a response exists (timeout, DNS failure, your SSRF guard) produce no cassette - only resolved responses are evidence, including 4xx/5xx. If you want transport-failure evidence too, that is an event-shape question and I would rather design it with you than guess.Design choices you should check
Attribution is registry-verified. The event carries
msg.routing.from, and the recorder resolves it throughresolveCallerIdentity- your existing anti-spoofing helper - rather than trusting the payload. A caller with no durable typeId is not recorded: a cassette that cannot be tied to a type is evidence about nothing.Eviction is bucketed by endpoint (method + path, query stripped): FIFO cap 5 per bucket, 50 per typeId, and global overflow always comes out of the LARGEST bucket. Your Fitness gate: judge generated objects by evidence they cannot edit #11 bucketing suggestion, landed here where
_httpvolume makes it real - a hot parameterized endpoint cannot evict the one recording of a rarely-hit one, no matter how old that recording is.The recorder never silently gives up. HttpClient discovery retries with capped backoff for as long as the recorder lives, and the subscription does not wait on Storage. While Storage is missing, recording continues in memory; persistence catches up when it appears, merging what the store already held so a restart never clobbers evidence. Both behaviors exist because scripted verification caught the failure modes, not because I thought of them.
Discovery is by registered name (
discoverDep('HttpClient')/discoverDep('Storage')), the same patternHealthMonitorand theAbjectbase class already use. I know your rule about naming specific objects - the sandbox hardcoding you flagged in Fitness gate: judge generated objects by evidence they cannot edit #11 special-cased behavior per name, which this does not. If you want capability-based discovery here instead, say so and I will follow whateverHealthMonitormigrates to.The cassette shape is contributor-authored infrastructure, not generator-authored evidence. Your Fitness gate: judge generated objects by evidence they cannot edit #11 concern was the generator declaring the standards it is judged by. Nothing here is written by a generator: the recorder records what actually crossed the wire, and a judged object cannot influence what gets recorded about it. The shape itself is yours to review like any other code.
fetchBase64does not emit. Base64 image fetches are not replay evidence and would blow the body cap for nothing. Say the word if you want it covered.Verification
tsc --noEmitclean. Three one-shot scripts (below, not committed - your convention), 16 checks total, run withnpx tsx --test <file>:recorder-unit.ts- recorder behaviors against the real bus and real dependents protocol: typeId keying, no-typeId skip, merge-not-clobber on restart, per-bucket eviction, largest-bucket overflow eviction, late-registering HttpClient, recording through a Storage outage.http-emit-unit.ts- emission behaviors with onlymakeRequeststubbed (your SSRF guard blocks loopback, so no live socket): caller attribution, header/query/body redaction incl. the stem cases above, truncation, all four entry points, zero work when nobody subscribes.recorder-live.ts- end to end with production classes: realRegistry, realStorage, realHttpClient, real recorder. Boot-style spawn and register, discovery, subscription, a caller's request, cassette read back fromStoragewith the secret gone.lab/recorder-unit.ts
lab/http-emit-unit.ts
lab/recorder-live.ts
What comes next in the series
The blocklist PR (
Atomics/SharedArrayBuffer, its own reasoning), then the judge as an abject, evaluated inside the deploy ops. Once cassettes accumulate here, the learned-schema conversation from #11 has real data to stand on.