Skip to content

test(golden): cross-language parity-oracle characterization tests (#225 item 2)#227

Merged
rado0x54 merged 2 commits into
developfrom
test/golden-parity-oracle
Jul 1, 2026
Merged

test(golden): cross-language parity-oracle characterization tests (#225 item 2)#227
rado0x54 merged 2 commits into
developfrom
test/golden-parity-oracle

Conversation

@rado0x54

@rado0x54 rado0x54 commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Summary

Implements item 2 of #225 — golden / characterization tests that capture what the Node backend actually returns across REST, MCP, WebSocket, and audit, as committed normalized JSON fixtures. These are the cross-language parity oracle for the Go rewrite (#210): the Go server replays the same requests, applies the same normalization, and diffs against the identical files. They complement the hand-authored spec from #226 (spec = what the shape should be; goldens = what it is, byte-for-byte).

What's included

Harnesssrc/test/helpers/golden.ts: a normalization layer that folds per-run volatile values to stable placeholders (<TS>, sess_<ID>, <UUID>, <CURSOR>, <FINGERPRINT>, <PORT>, <BASE_URL>, <REDACTED>) and an expectGolden() matcher (assert, or rewrite under UPDATE_GOLDENS=1).

4 suites / 21 cases / 28 fixtures under src/test/integration/:

  • golden-http — OAuth + RFC 9728 discovery docs, REST success envelopes, and the 401/404/400 matrix (incl. the /mcp 401 resource_metadata hint). externalUrl is pinned so discovery bodies are port-independent.
  • golden-mcp — every tool's JSON payload + the isError/message error shape; read_output is asserted structurally only (echo-shell output is non-deterministic).
  • golden-ws — connect-time sessions:changed snapshot + the terminal:attach reply sequence (terminal:statusterminal:mode, ui→control).
  • golden-audit — the highest-value target: DB-backed seed rows exercising paged { rows, nextCursor }, keyset pagination across two pages, single-row lookup, all nullable columns, and the invalid-filter 400.

Plumbing

  • startTestApp gains optional sessionLifecycleRepo/signingRequestsRepo passthrough so the audit routes (otherwise unmounted) can be wired with a seeded in-memory DB.
  • Scripts: pnpm test:golden (assert) and pnpm test:golden:update (regenerate). Goldens also run inside pnpm test:integration (CI).
  • __goldens__/README.md documents the normalization contract and how the Go harness consumes the fixtures.

Characterization findings (surfaced while capturing)

  • POST /api/endpoints does not validate label/host presence — a host-only body still returns {status:"created"}, though the OpenAPI marks them required. A spec-vs-handler gap worth reconciling (not fixed here — goldens capture reality).

Validation

  • pnpm test:golden → 21/21 ✓; stable across repeated runs (fresh SSH keys + ports each time).
  • Full pnpm test:integration → 144/144 ✓ (the startTestApp change is additive).
  • pnpm typecheck ✓, pnpm spdx:check ✓.

Not in this cut (follow-ups)

Deliberately deferred, each needs extra wiring: passkey/step-up ceremonies (needs the fake authenticator from #162), Hydra login/consent provider bodies, /agent-proxy framing, and pending-action/push routes. Easy to add once those harnesses land.

Relates to #210, #225 (item 2); builds on #226.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant