diff --git a/AGENTS.md b/AGENTS.md index c246d39..f0676ba 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -41,7 +41,7 @@ Guidance for any agent (or human) working in this repo. `CLAUDE.md` is a symlink 2. **Fixture-semantics** — *"does this fixture actually test what it claims?"* See the **Fixture quality bar** in `fixtures/asyncapi/README.md` (no vacuous values; claim↔content; full-path/both-direction coverage; internal consistency; negative cases). The validity-only angle misses all of these. ## Status & next -All build tiers and v1 gates are `tested` (38 of 40 requirements): `model` → `broker`/`registry`/`ingestion` → `engine`/`validation` → `scenarios`/`control-plane` → `cli` (the full verb set incl. `up`/`down` process management over the G14 runfile, watch modes, `init`), plus the four cross-cutting v1 gates (R-028–R-031). The open items: the two empirical spikes, `R-006` (WS-fidelity) and `R-007` (capture the browser application's `connect()`), which need the real browser application against `broker/`'s Aedes defaults and are hard gates on calling v1 done; plus `R-033` (`tested`): the `demo-app/` spike-harness webapp + connect fingerprint (`docs/specs/demo-app.md`), which rehearses both spikes — the at-work capture is now a no-app-change procedure (point the real client at offbook, read `offbook logs`). The adoption surface is `tested` (R-034–R-036: README + guides with executable quickstart/cookbook gates, `offbook doctor`, the first-run error audit — docs/specs/adoption.md). The AsyncAPI support range is declared and hardened (R-037–R-039, D-018): **2.0.0–2.6.0, 3.0.0, 3.1.0**, payloads validated under **draft-07**, with the R-028 gate extended over `multi-format.yaml` (3.1.0) and `v2-oldest.yaml` (2.0.0). The per-channel initial-state opt-out is `tested` (R-040, D-025): `topicOverrides.
.initialState: false` declares a reactive-only channel (no L1 floor; four spec-load warnings; handler-wins warn-log; `GET /v1/topics` marks suppressed channels). +**v1 is complete: all 40 requirements are `tested`**, including the two empirical spikes `R-006` (WS-fidelity) and `R-007` (capture the browser application's `connect()`), closed 2026-08-03 by the authoritative runs of the real browser application against `broker/`'s Aedes defaults (D-026): go on the defaults unchanged — MQTT 3.1.1 (level 4) over ws, no auth, QoS 0/1 only; the sanitized capture is `fixtures/connect/real-client.json` (deployment-specific values are not published), pinned by `src/broker/connect-profile.test.ts`. The build tiers: `model` → `broker`/`registry`/`ingestion` → `engine`/`validation` → `scenarios`/`control-plane` → `cli` (the full verb set incl. `up`/`down` process management over the G14 runfile, watch modes, `init`), plus the four cross-cutting v1 gates (R-028–R-031). `R-033` (`tested`): the `demo-app/` spike-harness webapp + connect fingerprint (`docs/specs/demo-app.md`), which rehearsed both spikes and stays the rehearsal surface for re-runs. Next up: the `aedes` 1.x bump, now unblocked (D-021 obligation (1), de-risked by D-024's Bun probe) — its own branch/PR/decision entry with an R-033 rehearsal re-run. The adoption surface is `tested` (R-034–R-036: README + guides with executable quickstart/cookbook gates, `offbook doctor`, the first-run error audit — docs/specs/adoption.md). The AsyncAPI support range is declared and hardened (R-037–R-039, D-018): **2.0.0–2.6.0, 3.0.0, 3.1.0**, payloads validated under **draft-07**, with the R-028 gate extended over `multi-format.yaml` (3.1.0) and `v2-oldest.yaml` (2.0.0). The per-channel initial-state opt-out is `tested` (R-040, D-025): `topicOverrides..initialState: false` declares a reactive-only channel (no L1 floor; four spec-load warnings; handler-wins warn-log; `GET /v1/topics` marks suppressed channels). ## Working notes - **Git identity is the user's to set** — don't run `git config user.*` on their behalf. Commit/push **only when asked**. diff --git a/DECISIONS.md b/DECISIONS.md index eb98a13..fb2780c 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -245,3 +245,14 @@ Append-only. Each decision has a stable never-reused `D-###` id, what was decide **Obligations**: if services.yaml ever becomes re-readable mid-process, grow the F21 cache key with a ServiceConfig fingerprint (see the `src/cli/boot.ts` comment). **From**: docs/archive/intake/2026-08-01-initial-state-optout.md (design dialog + 4-lens adversarial review, 2026-08-01). **Folds into**: docs/specs/contracts.md §1/§2/§5/§6, docs/specs/design.md §7a, docs/guides/wiring-your-service.md, REQUIREMENTS.md (R-040), src/model/index.ts, src/config/fixtures/services.yaml, src/registry/index.ts, src/engine/index.ts, src/compose/index.ts, src/control-plane/index.ts, src/cli/index.ts, src/cli/boot.ts + +### D-026: R-006/R-007 closed by the authoritative real-client runs — go on the defaults, capture sanitized +**Date**: 2026-08-03 +**What**: The two empirical spikes are closed. The real (unmodifiable) browser application was pointed at offbook per the demo-app.md §9 procedure, and the run was a **go on the broker's current defaults, unchanged**: MQTT 3.1.1 (protocol level 4) over ws, **no auth**, **QoS 0/1 only — no QoS 2 use**, keepalive 30 with a persistent session (`clean: false`), and the listener's first-offered subprotocol echo sufficed. The ws listener config that build-plan §5 held **provisional** (subprotocol/path/auth) is now **final as-is** — zero listener changes came back. The R-007 capture is committed as `fixtures/connect/real-client.json` (the demo-app.md §6 shape, `"source": "real-client"`), **sanitized**: deployment-specific values (`wsUrl`, `path`, `clientIdPattern`) are private and not published — null and listed under `redacted` — and fields not recorded during the runs (the `subprotocol` string, `retainUsed`) are null and listed under `notCaptured` — a null never silently poses as a measurement. `src/broker/connect-profile.test.ts` pins the closure from both sides: the fixture's internal consistency (redacted/notCaptured stay null; no auth; no QoS 2), and the broker's acceptance of the measured profile live (level 4, no auth, keepalive 30 with a persistent session, a non-root stand-in path, SUBACK + retained receipt, QoS 0/1 both ways, the fingerprint's server-side view matching). +**Why**: R-006/R-007 were the last two open v1 requirements and hard gates on calling v1 done; only the real application run could close them (the R-033 harness was expressly a rehearsal, never the gate). The private ws path costs nothing because the listener never routes on path — it upgrades every request and records `pathname` purely as a fingerprint fact — so the real path needs no repo accommodation and never has to be published. R-007's "broker ws port default" artifact is **deliberately narrowed**: the real target cannot be published, so the public default stays `9001` and the deployment's port lives in a local `brokerWsPort` override, which is what config overrides are for. +**Measured / evidence**: connect and subscribe facts were read back off the fingerprint surface during the runs (protocol level 4, no auth, keepalive 30, `clean: false`, QoS 0/1 observations). **Retained receipt is owner-affirmed via application behavior rather than log-transcribed** — the fingerprint surface records connects/subscribes/publishes, not deliveries to the client, so app-side receipt has no server-side log line to quote; the owner accepted it as working. **Not settled**: DUP-on-redelivery (D-006) — the broker in the loop was offbook itself on Aedes' in-memory persistence, which cannot redeliver an in-flight unacked QoS-1 message, so the run physically could not exercise the DUP flag. +**Mitigations / notes**: `fixtures/connect/README.md` states the sanitization rules (a null means redacted-or-not-captured, never "the client sent nothing" — except `auth`, where absence *was* the finding) and forbids hand-editing the fixture to appease tests. The pinning test derives its client options from the fixture at runtime, so fixture drift and listener drift both flip it red. +**Consequences for earlier entries**: **D-021 obligation (1) is unblocked** — the aedes 1.x bump's precondition (spikes measured against 0.51.x defaults) is met; the bump proceeds as its own branch/PR/decision with the R-033 rehearsal re-run D-024 records. **D-006 is narrowed, not discharged**: its delegation of the DUP contract to "the WS-fidelity spike scope" is exhausted — the spike ran without a redelivering persistence — so the DUP clause now rests solely on D-006's persistence obligation (re-point the tripwire if the broker ever adopts a redelivering persistence). D-015's fingerprint surface fulfilled its R-007 purpose as designed. +**Obligations**: if the real client's profile ever changes (auth appears, QoS 2 shows up, the protocol level moves, the transport changes), re-run the capture per demo-app.md §9 and update `fixtures/connect/real-client.json` + this entry's facts; none blocking. +**From**: the authoritative spike runs against the real browser application (reported 2026-08-03), executed per docs/specs/demo-app.md §9. +**Folds into**: REQUIREMENTS.md (R-006, R-007, R-033 text, seeding note), fixtures/connect/real-client.json, fixtures/connect/README.md, src/broker/connect-profile.test.ts, docs/specs/build-plan.md §5, AGENTS.md (Status & next) diff --git a/REQUIREMENTS.md b/REQUIREMENTS.md index c96c148..3c64528 100644 --- a/REQUIREMENTS.md +++ b/REQUIREMENTS.md @@ -52,15 +52,19 @@ A browser-style `mqtt.js` client connects to the Aedes ws listener over MQTT 3.1 #### WS-fidelity spike is the authoritative connect gate **UID**: R-006 -**STATUS**: specified +**STATUS**: tested **COVERS**: docs/specs/build-plan.md#spikes -The real browser application's `mqtt.js` connects+subscribes+receives-retained against a bare Aedes ws listener, finalizing the broker's listener config (subprotocol/path/auth). +**IMPL**: src/broker/, fixtures/connect/real-client.json +**TEST**: src/broker/connect-profile.test.ts +The real browser application's `mqtt.js` connects+subscribes+receives-retained against a bare Aedes ws listener, finalizing the broker's listener config (subprotocol/path/auth). Closed by the authoritative real-application runs (D-026): **go on the defaults, unchanged** — MQTT 3.1.1 (level 4), no auth, the first-offered subprotocol echo sufficed, and the deployment's private ws path needed no accommodation (the listener never routes on path). The D-006 DUP clause stays a known limitation: no redelivering persistence was in the loop. #### Capture the browser application's connect() **UID**: R-007 -**STATUS**: specified +**STATUS**: tested **COVERS**: docs/specs/build-plan.md#spikes -The client's `connect()` auth fields, ws URL/path, subprotocol, protocol level, and any QoS-2 use are captured into a config fixture + broker ws port default. +**IMPL**: fixtures/connect/real-client.json +**TEST**: src/broker/connect-profile.test.ts +The client's `connect()` auth fields, ws URL/path, subprotocol, protocol level, and any QoS-2 use are captured into a config fixture + broker ws port default. Closed per D-026 with the artifact sanitized: the capture (level 4, no auth, QoS 0/1 only — no QoS 2) is `fixtures/connect/real-client.json`, deployment-specific fields null-and-listed under `redacted`; the public ws port default stays 9001, the real target being a local `brokerWsPort` override. #### M0 walking-skeleton prototype: retained receipt + topic discovery **UID**: R-008 @@ -268,7 +272,7 @@ The engine owns the instance-materialization ledger (contracts §2, F1): `Instan **COVERS**: docs/specs/demo-app.md#demo-app **IMPL**: demo-app/, src/broker/index.ts, src/compose/index.ts, src/cli/boot.ts, src/cli/serve.ts, src/cli/index.ts, src/demo/scenarios/50-thermostat-chain.yaml, src/demo/thermostat.yaml **TEST**: src/broker/fingerprint.test.ts, test/demo-serve.test.ts, test/demo-app.test.ts -A React demo webapp (`demo-app/`, real `mqtt.js` over ws from a real browser) doubles as the showcase and the R-006/R-007 rehearsal harness: thermostat dashboard + `/v1/validation` feed (via a same-origin proxy, no CORS change) + a first-class spike panel (live R-006 checklist, sent-vs-seen comparison, R-007 capture download). `src/broker/` captures a normalized connect fingerprint (ws upgrade + CONNECT facts, password as presence only; deduped subscribe/publish QoS observations) surfaced as structured `offbook.log` lines — the R-007 capture surface (D-015) that also works against the real, unmodifiable browser application at work. `offbook demo --serve` boots the bundled spec + bundled chain scenarios long-running over the G14 machinery. R-006/R-007 remain open: the real browser application run stays the authoritative gate. +A React demo webapp (`demo-app/`, real `mqtt.js` over ws from a real browser) doubles as the showcase and the R-006/R-007 rehearsal harness: thermostat dashboard + `/v1/validation` feed (via a same-origin proxy, no CORS change) + a first-class spike panel (live R-006 checklist, sent-vs-seen comparison, R-007 capture download). `src/broker/` captures a normalized connect fingerprint (ws upgrade + CONNECT facts, password as presence only; deduped subscribe/publish QoS observations) surfaced as structured `offbook.log` lines — the R-007 capture surface (D-015) that also works against the real, unmodifiable browser application. `offbook demo --serve` boots the bundled spec + bundled chain scenarios long-running over the G14 machinery. The authoritative real-application runs have since closed R-006/R-007 (D-026); the harness stays the rehearsal surface for future re-runs (the aedes 1.x bump's R-033 re-run, for one). #### Adopter document set — README front door + guides **UID**: R-034 @@ -327,7 +331,7 @@ Every error reachable on the clone→demo→init→wire→up→first-publish pat `topicOverrides..initialState: false` (services.yaml) declares a reactive-only channel: the registry resolves the flag onto `Channel.initialState` (toClient records only; no spec-binding tier; only `false` is meaningful), the engine's L1 proactive floor skips the channel on every materialization leg (concrete subscribe, eager startup, `seedInstances`, `reset` republish) while the instance ledger, L2/L3 emissions, wildcard retained replay, and the explicit example surfaces stay untouched; an L3 `initialState` handler still wins, with a compose-root warn-log naming channel and handler re-run after a specs refresh; four `spec-load` warnings (`override-dangling-key`, `initial-state-on-from-client`, `initial-state-non-boolean`, `initial-state-cross-service`) make misconfiguration loud; `GET /v1/topics` exposes `initialState: false` on suppressed channels only. These gate/scope the build and are **not** on the module critical path — run them first/alongside (§12.1–2, §12.6): -1. **WS-fidelity spike** — point the real browser application's `mqtt.js` at a bare Aedes ws listener; confirm connect+subscribe+retained receipt at the browser application's actual protocol level/path/subprotocol/auth. *Artifact:* go/no-go + any Aedes listener config; feeds `broker/`. **Ordering (reconciles handoff Step-1 Gate):** `broker/` **may start in parallel against Aedes defaults** — it is buildable now — but this spike is a **gate on the broker's listener config (ws subprotocol/path/auth) being final**: that config is **provisional** until the spike returns. The spike is the **authoritative connect-fidelity gate** (the `broker/` `mqtt.js` acceptance is only a smoke test). -2. **Capture the browser application's `connect()`** — auth fields, ws URL/path, subprotocol, protocol level; note any **QoS 2** use. *Artifact:* a config fixture + the broker ws port default. +1. **WS-fidelity spike** — point the real browser application's `mqtt.js` at a bare Aedes ws listener; confirm connect+subscribe+retained receipt at the browser application's actual protocol level/path/subprotocol/auth. *Artifact:* go/no-go + any Aedes listener config; feeds `broker/`. **Ordering (reconciles handoff Step-1 Gate):** `broker/` **may start in parallel against Aedes defaults** — it is buildable now — but this spike is a **gate on the broker's listener config (ws subprotocol/path/auth) being final**: that config is **provisional** until the spike returns. The spike is the **authoritative connect-fidelity gate** (the `broker/` `mqtt.js` acceptance is only a smoke test). **Resolved (D-026, 2026-08-03):** go on the defaults, unchanged — MQTT 3.1.1 (level 4), no auth, first-offered subprotocol echo sufficed, and the deployment's private ws path needed no accommodation (the listener never routes on path). The provisional config is final as-is; the D-006 DUP clause remains a known limitation (no redelivering persistence was in the loop). +2. **Capture the browser application's `connect()`** — auth fields, ws URL/path, subprotocol, protocol level; note any **QoS 2** use. *Artifact:* a config fixture + the broker ws port default. **Resolved (D-026, 2026-08-03):** the sanitized capture is `fixtures/connect/real-client.json` (level 4, no auth, QoS 0/1 only — no QoS 2; deployment-specific fields redacted); the public ws port default stays 9001, the real target being a local override. 3. **Adopt-vs-build** — already resolved (build justified, §12.6); the residual is an ergonomic fit check against the real specs, not a blocker. 4. **`mqtt-pattern` parity spike (F6/R2)** — confirm the **`{p}`→`+p` rewrite** reproduces AsyncAPI single-segment capture exactly (mqtt-pattern matches `{param}` *literally* — it does **not** read braces natively) and that `matches` implements MQTT `+`/`#` exactly (incl. `#` matching zero trailing levels) on the fixture channel addresses; it is pure-string with no transport deps. *Artifact:* go/no-go for R2 = rewrite round-trip fidelity; on failure `registry/` falls back to a hand-rolled matcher. Gates `registry/`'s matcher only. 5. **json-schema-faker fidelity spike (F8)** — run JSF (pinned version; 0.6.2 at first measurement, re-measured on the 0.6.3 bump, D-020) against every `fixtures/asyncapi/*` bundled `channel.schema` and count Ajv-recheck failures per fixture (esp. `external-ref`'s `oneOf`/external-`$ref`, and `qos-overrides`). *Artifact:* a per-fixture recheck-failure rate; if a §5 bar fixture fails, it decides whether F5's keyed-fallback re-draw is needed (else drop-and-surface stands). Gates L1's CI reliance. diff --git a/fixtures/connect/README.md b/fixtures/connect/README.md new file mode 100644 index 0000000..a5b2e3b --- /dev/null +++ b/fixtures/connect/README.md @@ -0,0 +1,20 @@ +# fixtures/connect/ + +The R-007 capture artifact (demo-app.md §6 shape, §9 step 4 destination): the real +browser application's `connect()` profile as offbook's fingerprint surface saw it +during the authoritative spike runs (D-026). + +**`real-client.json`** is deliberately sanitized. Deployment-specific values +(`wsUrl`, `path`, `clientIdPattern`) are private and not published: they are +`null` and listed under `redacted`; values that were not recorded during the +runs are `null` and listed under `notCaptured`. A `null` therefore never means +"the client sent nothing" — the one field where absence itself was the finding +is `auth` (no username, no password). The published facts — protocol level 4 +(MQTT 3.1.1), ws transport, no auth, QoS 0/1 only with no QoS 2 use, keepalive +30 with a persistent session — are exactly the set the broker is pinned against +in `src/broker/connect-profile.test.ts`, which drives a live ws client from +this file and fails if the fixture and the listener's behavior drift apart. + +If the real client's profile ever changes (auth appears, QoS 2 shows up, the +protocol level moves), re-run the capture per demo-app.md §9 and update this +fixture plus the D-026 obligations — do not hand-edit it to make a test pass. diff --git a/fixtures/connect/real-client.json b/fixtures/connect/real-client.json new file mode 100644 index 0000000..98ff9ef --- /dev/null +++ b/fixtures/connect/real-client.json @@ -0,0 +1,17 @@ +{ + "capturedAt": "2026-08-03", + "source": "real-client", + "wsUrl": null, + "path": null, + "subprotocol": null, + "protocolLevel": 4, + "clientIdPattern": null, + "auth": { "username": null, "passwordPresent": false }, + "keepalive": 30, + "clean": false, + "qosUsed": [0, 1], + "retainUsed": null, + "redacted": ["wsUrl", "path", "clientIdPattern"], + "notCaptured": ["subprotocol", "retainUsed"], + "provenance": "The authoritative R-006/R-007 spike runs: the real browser application against offbook's Aedes defaults (D-026). capturedAt is the date the runs were reported. Fields listed in `redacted` have real values that are private to the deployment and not published; fields listed in `notCaptured` were not recorded. Both kinds are null here, and src/broker/connect-profile.test.ts enforces that they stay null. The listener's first-offered subprotocol echo sufficed for the real browser, no auth was used, no QoS 2 use was observed, and the client runs a persistent session (clean: false) at keepalive 30." +} diff --git a/src/broker/connect-profile.test.ts b/src/broker/connect-profile.test.ts new file mode 100644 index 0000000..c41c5b7 --- /dev/null +++ b/src/broker/connect-profile.test.ts @@ -0,0 +1,179 @@ +// R-006/R-007 — the closed spikes' in-repo pin (D-026). The authoritative runs +// were the real browser application against this broker's defaults; the +// published capture is fixtures/connect/real-client.json, and this file keeps +// the closure honest in both directions. The fixture must stay internally +// consistent (redacted / not-captured fields stay null; no auth; no QoS 2), +// and the broker must keep accepting the measured profile: MQTT 3.1.1 +// (level 4) over ws, no auth, a non-root path (the real one is private — the +// listener must not care), retained receipt on subscribe, a persistent +// session at keepalive 30, QoS 0/1 both ways. +// [itest->R-006] +// [itest->R-007] +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { connectAsync } from "mqtt"; +import { loadConfig } from "#src/config/index.ts"; +import { DEFAULT_CONFIG, type InboundEvent } from "#src/model/index.ts"; +import type { BrokerModule, FingerprintEvent } from "./index.ts"; +import { createBroker } from "./index.ts"; + +// ports unique to this file: ws 19200 / tcp 12950 +const WS = 19200; +const TCP = 12950; + +// A stand-in for the real, undisclosed ws path: any non-root path proves the +// property that makes the redaction affordable — the listener upgrades every +// path and records it only as a fingerprint fact, never routes on it. +const PATH_STAND_IN = "/an/undisclosed/path"; + +interface ConnectProfile { + capturedAt: string; + source: string; + wsUrl: string | null; + path: string | null; + subprotocol: string | null; + protocolLevel: number; + clientIdPattern: string | null; + auth: { username: string | null; passwordPresent: boolean }; + keepalive: number | null; + clean: boolean | null; + qosUsed: number[]; + retainUsed: boolean | null; + redacted: string[]; + notCaptured: string[]; + provenance: string; +} + +let profile: ConnectProfile; +let broker: BrokerModule; +const events: FingerprintEvent[] = []; +const inbound: InboundEvent[] = []; + +beforeAll(async () => { + profile = (await Bun.file( + `${import.meta.dir}/../../fixtures/connect/real-client.json`, + ).json()) as ConnectProfile; + broker = createBroker( + loadConfig({ + brokerWsPort: WS, + brokerTcpPort: TCP, + controlPlanePort: 19201, + }), + ); + broker.onFingerprint((e) => events.push(e)); + broker.onInbound((e) => inbound.push(e)); + await broker.start(); + // retained state the profile connection must receive on subscribe + await broker.emit({ + topic: "state/profile-pin", + payload: { ok: true }, + qos: 1, + retain: true, + }); +}); +afterAll(async () => { + await broker.stop(); +}); + +test("the fixture is internally consistent and carries exactly the disclosed facts", () => { + // the redactions are enforced, not just described: every field the capture + // marks redacted or not-captured must actually be null + for (const key of [...profile.redacted, ...profile.notCaptured]) { + expect((profile as unknown as Record