feat(telemetry-edge): self-referential tempo edge — ADR + schema + working JS edge (Small + Medium) - #7
Draft
yanmo42 wants to merge 5 commits into
Draft
feat(telemetry-edge): self-referential tempo edge — ADR + schema + working JS edge (Small + Medium)#7yanmo42 wants to merge 5 commits into
yanmo42 wants to merge 5 commits into
Conversation
Open the self-referential tempo edge lane (no executable code yet): - docs/23_self_referential_tempo_edge.md ADR for a JS edge that ingests gameplay + input streams and emits structured tempo points to the Python research engine. Bridges SC-CONCEPT-0003 (observer coupling) and SC-CONCEPT-0010 (closed-loop causal leverage) to a real embedded operator (operator's own sessions), not just toy models. Coupling/decoupling framing tagged plausible. - docs/24_tempo_edge_redaction_policy.md Normative v0 policy. Hard rules: no raw keystrokes on disk; in-memory ring buffer only; aggregate-only emission; no identifiers; structural game state only; reproducible builds before pointing at real input. - schemas/tempo_point_v0.schema.json Wire format. additionalProperties:false at root and in aggregates; the top-level `not` block explicitly rejects raw_*, key_sequence, text, chat, username, hostname, etc. window_ms capped at 10 min so unbounded windows can't be smuggled in. schema_version and redaction_version are const so corpus consumers can filter by policy generation. Verification: - jsonschema metaschema check passes - 6 acceptance/rejection assertions pass (valid sample, raw_keys rejected, username rejected, unknown root field rejected, unknown aggregate field rejected, window cap enforced, schema_version const enforced) - existing test suite 236/236 pass on this branch Scope discipline: this commit is the Small step from the proposal. Medium (one Node module emitting valid JSONL from SC2 replays + sampled input) is gated on operator review of this ADR. No npm, no public publication. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…, validated emit Medium step on top of the ADR + schema (PR #7 small step): - src/types.ts: TS mirror of tempo_point_v0 + RawEvent union (never serialized) - src/schema.ts: ajv 2020-12 loader, assertValid, TempoPointValidationError - src/ring-buffer.ts: bounded by capacity + retentionMs, injectable now() - src/aggregator.ts: APM, intervals (mean/p50/p95/max), modal categorical classes (never literal keys), summed game_state deltas — output type physically cannot carry raw_* fields - src/emit.ts: openJsonlEmitter validates with assertValid before any write, no fallback serialization path - src/sources/synthetic.ts: deterministic mulberry32 PRNG, categorical tags only, distributes events across a duration at a target mean APM - bin/emit-sample.ts: CLI that runs the synthetic source through the full seam and writes validated JSONL; uses a simulated clock so the ring buffer's time eviction matches live-capture semantics - 28 tests across schema / ring buffer / aggregator / redaction / integration Cross-language verification: emitted JSONL passes Python jsonschema against the same canonical schemas/tempo_point_v0.schema.json (6/6 records). Redaction guarantee is structural: no raw_keys, raw_input, chat, transcript, username, hostname, text, or password tokens can appear in any output, even if attached as extra properties on inputs. The schema's not/anyOf block rejects them at the wire, and the aggregator never copies them through. This turns the ADR claim "a JS edge can produce valid tempo_point_v0 JSONL" from a promise into a tested contract. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
yanmo42
changed the title
The runtime-adoption comparison (3-query workflow fixture) was returning 0/3 hit rate for both topology and flat modes because the three expected nodes — N_SCRIPT_AUTOMATION_ORCH, N_SCRIPT_ORCH_AUTOSPAWN, N_PLAN_TODO — did not exist in graph_v0.json. Root cause: the graph only modeled research/validation artifacts for the topological-memory lane itself, but not the actual automation scripts and master TODO surface that real workflow queries point to. Fix: added the 3 missing nodes with vocabulary matched to the workflow query language, 5 semantic edges connecting them into the existing graph topology, and 4 calibrated traces. Corrected trace weight regressions (Q-015, Q-020, Q-025) by reducing script trace weights and adding a dedicated eval-module trace. Results after fix: - Frozen 30-query benchmark: keyword hit@3=1.000, topology hit@3=1.000 (maintained); topology MRR improved 0.833→0.906 - Workflow adoption queries: topology hit@3=1.000, flat hit@3=1.000 (was 0.000 / 0.000) - Promotion gate updated to v0.1 with new workflow condition Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Two atomic steps on top of the same branch:
Small (3ab3f29): ADR + redaction policy + canonical schema.
Medium (dbf0144): working Node.js edge that produces validated
tempo_point/v0JSONL from a synthetic source.Why
The ADR claimed "a JS edge can produce valid tempo_point_v0 JSONL." Until that ran end-to-end, every downstream conversation (containerization, npm scope, IoT extension, live SC2 capture) was built on a promise. This PR turns the claim into a tested contract.
What landed
Small step (docs + schema)
docs/23_self_referential_tempo_edge.md— ADR bridging SC-CONCEPT-0003 and SC-CONCEPT-0010, coupling/decoupling framing, claim tiers, failure conditions, explicit non-decisionsdocs/24_tempo_edge_redaction_policy.md— normative v0 with 6 hard MUST rules, permitted aggregate categories, forbidden field list, audit checklistschemas/tempo_point_v0.schema.json— canonical wire format, Draft 2020-12,additionalProperties: false, top-levelnot/anyOfrejecting raw_keys / raw_input / chat / transcript / username / hostname / ip_address / handle / etc.Medium step (working seam)
telemetry-edge/src/types.ts— TS mirror of the schema +RawEventunion (the latter never serializes)telemetry-edge/src/schema.ts— ajv 2020-12 loader,assertValid,TempoPointValidationErrortelemetry-edge/src/ring-buffer.ts— bounded by capacity + retentionMs, injectablenow()for deterministic testingtelemetry-edge/src/aggregator.ts— APM, intervals (mean/p50/p95/max), modal categorical classes (never literal keys), summedgame_statedeltastelemetry-edge/src/emit.ts—openJsonlEmittervalidates withassertValidbefore any write; no fallback serialization pathtelemetry-edge/src/sources/synthetic.ts— deterministic mulberry32 PRNG, categorical tags onlytelemetry-edge/bin/emit-sample.ts— CLI running the full seam with a simulated clock so ring-buffer eviction semantics match live captureVerification receipts
npm test→ 28/28 passnpm run emit-sample -- --out /tmp/tempo-sample.jsonl→ 6 records with realistic APM (146–150), non-trivial inter-event intervals, summedgame_statedeltas, modal classes likemovement/hotkeyjsonschema.validate()against the same canonical schema → 6/6 validraw_keys,raw_input,chat,transcript,username,hostname,password) and asserts none appearRedaction guarantee (structural, not advisory)
assertValidbefore write — invalid records throw, nothing reaches disknot/anyOfblock rejects forbidden fields at the wire even if they slip past the type systemclear()on shutdownWhat's still open
Test plan
npm testfromtelemetry-edge/passes (28/28)npm run emit-sampleproduces validated JSONLschemas/tempo_point_v0.schema.jsondocs/23+docs/24for the framingtelemetry-edge/src/aggregator.tsfor the redaction-by-construction shape🤖 Generated with Claude Code