From f79c075966e867b1ba1f9b3cc8eaecc16f0082d1 Mon Sep 17 00:00:00 2001 From: Ryan Dombrowski Date: Tue, 21 Jul 2026 17:26:44 -0400 Subject: [PATCH] scenario: signup/onboarding ships ready (data-collection) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fifth ready scenario, on the support-triage template: replay-first, zero per-scenario code, shipped atomically. The honesty split is the headline: labels were ALWAYS enforced (the universal rule.input-carries-label), so the old "Labeled inputs, enforced" tagline over-claimed and is rewritten. What the new intent adds is structural — rule.data-collection-requires-input-and-action (component-choice, must): an ask without a field is not a form, and a form that cannot be submitted does not ship. Deliberately no layout, field-count, or form-shape mandates: data collection is not equivalent to signup forms. Rationales quote the Astryx TextInput and Button docs verbatim. Durability check (owner requirement): a non-signup probe — workspace access request: name, work email, team — produced an honest form under the same intent and rules, and, unforced, the scenario's refusal story: the model picked dropdown-menu for the team select, which lints clean but the astryx profile cannot project. The intent generalizes; the select gap is a component-vocabulary limit (the hotel-reservations blocker class), not an intent flaw. Fixtures (Spark, ollama:gpt-oss:latest, published dspack-gen 0.1.2): - fixture-013 argues-back: first --require-repair attempt — a text-only welcome card, S3 fails on the new rule, repair ships a labeled email field and a "Join now" action. - fixture-012 clean: first pass to a labeled name+email form with one primary action (close to the worked example's shape; noted honestly). - fixture-014 refusal: the access-request recording, reproduced on the second roll (the first was a clean pass, discarded). Also fixes a main inconsistency this session introduced: the #17 lockfile carried specifier ^0.1.2 while apps/agent/package.json said ^0.1.1 (pnpm -r up rewrote the specifier; only the lockfile was committed). The agent dep floor is now an explicit ^0.1.2 — which it behaviorally requires — with the lockfile matching. Fail-first: pre-contract recording failed with "intent 'data-collection' is not registered"; all five onboarding.spec.ts tests failed on the old registry. studio-shell's planned-reveal test retargets hotel-reservations (the last planned entry) — a retarget that passes on both builds, not a new assertion. Gates: pnpm -r typecheck/test green; contracts build+drift clean; local Playwright 102 passed / 4 skipped; production config vs the local deploy artifact 82 passed; one live agent run on published 0.1.2 produced a clean labeled form. Co-Authored-By: Claude Fable 5 --- apps/agent/package.json | 2 +- docs/AUDIT.md | 2 +- docs/IMPLEMENTATION_LOG.md | 42 ++ docs/NAV-REFACTOR.md | 8 +- e2e/onboarding.spec.ts | 74 +++ e2e/studio-shell.spec.ts | 3 +- packages/contracts/astryx.dspack.json | 89 +++ packages/replay/fixtures/fixture-012.json | 498 ++++++++++++++++ packages/replay/fixtures/fixture-013.json | 632 +++++++++++++++++++++ packages/replay/fixtures/fixture-014.json | 347 +++++++++++ packages/scenarios/src/break-conditions.ts | 18 +- packages/scenarios/src/registry.ts | 47 +- playwright.production.config.ts | 1 + pnpm-lock.yaml | 2 +- 14 files changed, 1749 insertions(+), 16 deletions(-) create mode 100644 e2e/onboarding.spec.ts create mode 100644 packages/replay/fixtures/fixture-012.json create mode 100644 packages/replay/fixtures/fixture-013.json create mode 100644 packages/replay/fixtures/fixture-014.json diff --git a/apps/agent/package.json b/apps/agent/package.json index 327038d..9603331 100644 --- a/apps/agent/package.json +++ b/apps/agent/package.json @@ -13,7 +13,7 @@ "record:catch": "tsx src/record-catch.ts" }, "dependencies": { - "@aestheticfunction/dspack-gen": "^0.1.1", + "@aestheticfunction/dspack-gen": "^0.1.2", "@dspack-studio/agui-bridge": "workspace:*", "@dspack-studio/contracts": "workspace:*", "@dspack-studio/replay": "workspace:*" diff --git a/docs/AUDIT.md b/docs/AUDIT.md index b0fe135..367076c 100644 --- a/docs/AUDIT.md +++ b/docs/AUDIT.md @@ -18,7 +18,7 @@ followed it. Statuses: Complete, Partial, Missing, Intentionally changed. | X-ray and provenance (FM-4) | Complete | Canvas click-to-trace, provenance card (node, catalog entry, creating/updating events, rules), reverse violet wash for the playhead delivery. This pass added a keyboard path: "trace" buttons in the inspector components tab. | | Governed repair / "argues back" (FM-1) | Complete | fixture-001 carries two real repairs (`rule.destructive-requires-alertdialog`, `rule.alertdialog-action-label-specific`); gate ticker + gates tab show findings and verbatim repair messages. | | Break-it mode (FM-8) | Complete | 7 conditions across 6 failure kinds through the ordinary RunView. This pass added offline behavior: three conditions replay their recorded catch when the local agent is absent (the deployed static site); live-only conditions say so plainly. | -| Scenarios | Partial by plan | 4 ready (project-deletion, appointment-booking, recipe-creator, support-triage), 2 planned (onboarding, hotel-reservations) presented honestly with their blockers. support-triage shipped 2026-07-21 with owner-authored record-collection governance and two live gpt-oss recordings (fixtures 010/011). Remaining scenarios need owner-authored contract intents/rules (plan §17, open question 2) — deferred, not a defect. | +| Scenarios | Partial by plan | 5 ready (project-deletion, appointment-booking, recipe-creator, support-triage, onboarding), 1 planned (hotel-reservations) presented honestly with its blockers. support-triage and onboarding shipped 2026-07-21 with owner-authored governance (record-collection, data-collection) and live gpt-oss recordings (fixtures 010-014). hotel-reservations needs new design-system components (plan §17, open question 2) — deferred, not a defect. | | Recipe shared-state experience (FM-6) | Complete | Real dishes, per-serving amounts, constraint swaps; this pass added numbered cooking instructions that the constraint swaps rewrite, on both the deterministic and generated paths, and re-recorded fixture-006 (live `gpt-oss`, 2026-07-11) with the model laying out an instructions table and the labeled enhancement seeding responder content into it. | | Live BYO-key mode | Intentionally changed | The plan's "browser BYO key" became "no credentials in the browser": modelRef selects `scripted` or local `ollama:*`; hosted keys live only in the agent's env. Stricter than the plan; documented in the UI. | | HITL (FM-7) | Partial | Appointment booking pauses for slot selection and validates round-trips; the agent's question is not itself a governed AlertDialog. Post-MVP. | diff --git a/docs/IMPLEMENTATION_LOG.md b/docs/IMPLEMENTATION_LOG.md index e2c2bee..76d6a3d 100644 --- a/docs/IMPLEMENTATION_LOG.md +++ b/docs/IMPLEMENTATION_LOG.md @@ -959,3 +959,45 @@ old registry (shelf "(planned)", missing fixtures, permalink "unknown or not-yet-ready"); upstream, three new grammar-alignment assertions failed on dspack-gen 0.1.1. shadcn intent parity stays deferred to the standing owner-authored contract-extension item. + +## 2026-07-21 — signup/onboarding: fifth ready scenario (data-collection) + +Same template as support-triage: replay-first, zero per-scenario code, an +atomic PR of governance + recordings + registry + tests + docs. + +Honesty split, stated everywhere it matters: labels were ALWAYS enforced +(`rule.input-carries-label` is universal, no appliesTo) — the scenario's +old "Labeled inputs, enforced" tagline over-claimed and was rewritten. +What the new intent adds is structural: +`rule.data-collection-requires-input-and-action` (component-choice, +must) — an ask without a field is not a form, and a form that cannot be +submitted does not ship. No layout, field count, or form shape is +mandated, deliberately: data collection is not equivalent to signup +forms. Rationales quote the Astryx TextInput and Button docs verbatim. + +Durability check (owner requirement: the intent must remain durable +beyond the signup example): a non-signup probe — "request access to a +private workspace: name, work email, team" — produced an honest form +under the same intent and rules. It also produced, unforced, the +scenario's refusal story: the model picked `dropdown-menu` for the team +select, which lints clean but the astryx profile cannot project. That +recording ships as the third fixture; it documents the vocabulary's +real limit (no select), which is a component gap (the hotel-reservations +blocker class), not an intent-design flaw. + +Recording (Spark, ollama:gpt-oss:latest, dspack-gen 0.1.2 published): +fixture-012 clean — first pass to a labeled name+email form with one +primary action (close to the worked example's shape; the few-shot +steers hard, noted honestly). fixture-013 argues-back — first +`--require-repair` attempt: the model obeys the text-only prompt with a +card and welcome text, S3 fails on the new rule (both missing +components), repair ships a labeled email field and a "Join now" +action. fixture-014 refusal — reproduced on the second roll (first +was a clean pass, discarded). + +Fail-first evidence: pre-contract recording fails with getIntent's +"intent 'data-collection' is not registered" (pasted in the PR); the +five onboarding.spec.ts tests fail on the old registry. +`studio-shell.spec.ts`'s planned-reveal test retargeted +`scenario-onboarding` → `scenario-hotel-reservations` (the last planned +entry) — a retarget, not a new assertion; it passes on both builds. diff --git a/docs/NAV-REFACTOR.md b/docs/NAV-REFACTOR.md index 6377d9e..1da0976 100644 --- a/docs/NAV-REFACTOR.md +++ b/docs/NAV-REFACTOR.md @@ -70,8 +70,8 @@ session file, or run it live and download one."* - **Scenario** = `scenarioId` (React state in `Studio`), canonical ids from `packages/scenarios/src/registry.ts`: ready `recipe-creator`, - `appointment-booking`, `project-deletion`, `support-triage`; planned - (not selectable) `onboarding`, `hotel-reservations`. + `appointment-booking`, `project-deletion`, `support-triage`, + `onboarding`; planned (not selectable) `hotel-reservations`. - **Operation** = `view` state: `"replay" | "live" | "break" | "canvas"`. X-ray and Fork/diff are not top-level views: they are controls inside `RunView` and operate on whatever run is loaded — already @@ -157,8 +157,8 @@ local agent · **gap** = no support yet. Cells marked † change with this work. | `project-deletion` | **recorded** ×3 (fixtures 001/002/003) | **live-only** (generation: `scripted` deterministic or `ollama:*`) | `no-alertdialog` **recorded** (catch=argues-back) · `ok-label` **recorded** (argues-back) · `unsupported-component` **recorded** (refusal) · `malformed-generation` **live-only** (scripted, needs agent) | **recorded** today (the hardcode) → **universal**† via fixture-001 | **universal** | **universal** | | *(any scenario)* | — | — | `malformed-import`: **universal** (client-side validator; mislabeled project-deletion → `scenarioIndependent`†) | — | — | — | | `support-triage` | **recorded** ×2 (fixtures 010/011) | **live-only** (generation: `scripted` deterministic or `ollama:*`; zero agent code, same as project-deletion) | `records-as-prose` **recorded** (catch=argues-back) | **universal** via fixture-011 final surface | **universal** | **universal** | -| `onboarding` (planned) | **gap** (not selectable; shelf states needs) | **gap** | **gap** | **gap** | n/a until a run loads | n/a | -| `hotel-reservations` (planned) | **gap** (same) | **gap** | **gap** | **gap** | n/a | n/a | +| `onboarding` | **recorded** ×3 (fixtures 012/013/014) | **live-only** (generation: `scripted` deterministic or `ollama:*`; zero agent code) | `ask-without-a-form` **recorded** (catch=argues-back) | **universal** via fixture-012 final surface | **universal** | **universal** | +| `hotel-reservations` (planned) | **gap** (not selectable; shelf states needs) | **gap** | **gap** | **gap** | n/a until a run loads | n/a | Unused asset worth noting: `packages/replay/fixtures/fixture-004.json` ("Scheduling, generated live", intent `scheduling`, 12 events) is bundled but diff --git a/e2e/onboarding.spec.ts b/e2e/onboarding.spec.ts new file mode 100644 index 0000000..41c5949 --- /dev/null +++ b/e2e/onboarding.spec.ts @@ -0,0 +1,74 @@ +/** + * Signup / onboarding: the data-collection scenario, replay-first, against + * the static export. All three fixtures are mode:"live" recordings (gpt-oss + * via Ollama) — nothing here is hand-scripted content. + * + * The governance story, honestly split: labels were always enforced (the + * universal rule.input-carries-label); what the new intent adds is + * structural — an ask needs something to fill in and a way to send it + * (rule.data-collection-requires-input-and-action). + */ +import { expect, test, type Page } from "@playwright/test"; + +const scrubber = (page: Page) => page.getByTestId("scrubber"); + +async function scrubToEnd(page: Page) { + await scrubber(page).focus(); + await page.keyboard.press("End"); +} + +test.beforeEach(async ({ page }) => { + await page.goto("/"); +}); + +test("onboarding is a ready, selectable scenario on the shelf", async ({ page }) => { + const shelf = page.getByTestId("scenario-onboarding"); + await expect(shelf).not.toContainText("(planned)"); + await shelf.click(); + await expect(page.getByTestId("fixture-argues-back")).toBeVisible(); + await expect(page.getByTestId("fixture-clean")).toBeVisible(); + await expect(page.getByTestId("fixture-refusal")).toBeVisible(); +}); + +test("clean replay: a live run straight through the gates to a labeled form", async ({ page }) => { + await page.getByTestId("scenario-onboarding").click(); + await page.getByTestId("fixture-clean").click(); + await expect(page.getByTestId("fixture-meta")).toContainText("live run"); + await scrubToEnd(page); + await expect(page.getByTestId("audit-outcome")).toContainText("passed"); + const canvas = page.locator("[data-canvas]"); + await expect(canvas).toContainText("Your name"); + await expect(canvas).toContainText("Email address"); + await expect(canvas).toContainText("Create account"); +}); + +test("repair replay: a text-only ask caught by the form rule, repaired surface ships", async ({ page }) => { + await page.getByTestId("scenario-onboarding").click(); + await page.getByTestId("fixture-argues-back").click(); + await scrubToEnd(page); + const ticker = page.getByTestId("gate-ticker"); + await expect(ticker).toContainText(/attempt 0:\s*S1✓\s*S2✓\s*S3✗\s*→ repair/); + await expect(ticker).toContainText(/attempt 1:\s*S1✓\s*S2✓\s*S3✓/); + await expect(ticker).toContainText("outcome passed"); + // The design system won: the ask got its field and its action. + const canvas = page.locator("[data-canvas]"); + await expect(canvas).toContainText("Email address"); + await expect(canvas).toContainText("Join now"); +}); + +test("refusal replay: a non-signup ask meets the vocabulary's honest limit", async ({ page }) => { + await page.getByTestId("scenario-onboarding").click(); + await page.getByTestId("fixture-refusal").click(); + await scrubToEnd(page); + await expect(page.getByTestId("audit-outcome")).toContainText("failed-gate (exit 3)"); + const panel = page.getByTestId("failure-panel"); + await expect(panel).toContainText("unknown component 'dropdown-menu'"); + await expect(page.getByTestId("canvas-empty")).toContainText("No surface shipped"); +}); + +test("permalink: an onboarding deep link resolves", async ({ page }) => { + await page.goto("/#s=onboarding&f=clean"); + await page.reload(); + await expect(page.getByTestId("fixture-meta")).toContainText("live run"); + await expect(page.getByTestId("link-error")).toHaveCount(0); +}); diff --git a/e2e/studio-shell.spec.ts b/e2e/studio-shell.spec.ts index 669e11e..344899b 100644 --- a/e2e/studio-shell.spec.ts +++ b/e2e/studio-shell.spec.ts @@ -42,7 +42,8 @@ test("the pipeline diagram names the honest pipeline in reading order", async ({ test("planned scenarios reveal what they need, by keyboard", async ({ page }) => { await page.goto("/"); - const planned = page.getByTestId("scenario-onboarding"); + // hotel-reservations is the last remaining planned scenario. + const planned = page.getByTestId("scenario-hotel-reservations"); await expect(planned).toContainText("(planned)"); await planned.focus(); await page.keyboard.press("Enter"); diff --git a/packages/contracts/astryx.dspack.json b/packages/contracts/astryx.dspack.json index 868e24a..6679201 100644 --- a/packages/contracts/astryx.dspack.json +++ b/packages/contracts/astryx.dspack.json @@ -430,6 +430,11 @@ "id": "record-collection", "name": "Record collection", "description": "Surfaces that present many records of the same kind — support tickets, orders, members — for scanning and action. Records share uniform fields, so the collection renders as a table; each record carries a status the user reads at a glance, summarized with badges where attention is needed and plain text otherwise." + }, + { + "id": "data-collection", + "name": "Data collection", + "description": "Surfaces that ask the user for information and submit it — signup, contact, profile edits, an RSVP. The ask is stated, the fields are labeled, and the submit action is explicit: at minimum, something to fill in and a way to send it." } ], "rules": [ @@ -605,6 +610,28 @@ "structure" ], "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/Table/Table.doc.mjs" + }, + { + "id": "rule.data-collection-requires-input-and-action", + "type": "component-choice", + "severity": "must", + "appliesTo": { + "intents": [ + "data-collection" + ] + }, + "require": [ + "text-input", + "button" + ], + "rationale": "\"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4", + "examples": [ + "ex.signup-basic" + ], + "tags": [ + "structure" + ], + "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/TextInput/TextInput.doc.mjs" } ], "examples": [ @@ -970,6 +997,68 @@ ] } } + }, + { + "id": "ex.signup-basic", + "intent": "data-collection", + "name": "Signup, name and email", + "prompt": "a signup form asking for name and email, with a clear call to action", + "description": "A card stating the ask, two labeled fields, and one explicit primary action — something to fill in and a way to send it, every field labeled.", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "signup-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Create your account" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Name and email is all it takes. You can add the rest later." + }, + { + "component": "text-input", + "id": "name-input", + "props": { + "label": "Your name", + "placeholder": "Ada Lovelace", + "isRequired": true + } + }, + { + "component": "text-input", + "id": "email-input", + "props": { + "type": "email", + "label": "Email address", + "placeholder": "ada@example.com", + "description": "We send a confirmation link, nothing else.", + "isRequired": true + } + }, + { + "component": "button", + "id": "submit", + "props": { + "label": "Create account", + "variant": "primary" + } + } + ] + } + } } ] } diff --git a/packages/replay/fixtures/fixture-012.json b/packages/replay/fixtures/fixture-012.json new file mode 100644 index 0000000..43d366e --- /dev/null +++ b/packages/replay/fixtures/fixture-012.json @@ -0,0 +1,498 @@ +{ + "replayFixture": "0.1", + "id": "fixture-012", + "name": "Clean signup form", + "recordedAt": "2026-07-21T21:10:37.666Z", + "mode": "live", + "adapterId": "ollama:gpt-oss:latest", + "intent": "data-collection", + "prompt": "A signup form asking for name and email, with a clear call to action.", + "events": [ + { + "atMs": 2, + "event": { + "type": "RUN_STARTED", + "threadId": "studio", + "runId": "fixture-012" + } + }, + { + "atMs": 2, + "event": { + "type": "CUSTOM", + "name": "dspack.run.start", + "value": { + "intent": "data-collection", + "prompt": "A signup form asking for name and email, with a clear call to action.", + "adapterId": "ollama:gpt-oss:latest", + "ruleIds": [ + "rule.alertdialog-carries-content", + "rule.alertdialog-action-label-specific", + "rule.button-carries-label", + "rule.input-carries-label", + "rule.dialog-no-nested-overlays", + "rule.data-collection-requires-input-and-action" + ] + } + } + }, + { + "atMs": 72050, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-0" + } + }, + { + "atMs": 72050, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 0, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "signup-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Create your account" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Name and email is all it takes. You can add the rest later." + }, + { + "component": "text-input", + "id": "name-input", + "props": { + "label": "Your name", + "placeholder": "Ada Lovelace", + "isRequired": true + } + }, + { + "component": "text-input", + "id": "email-input", + "props": { + "type": "email", + "label": "Email address", + "placeholder": "ada@example.com", + "description": "We send a confirmation link, nothing else.", + "isRequired": true + } + }, + { + "component": "button", + "id": "submit", + "props": { + "label": "Create account", + "variant": "primary" + } + } + ] + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + } + }, + { + "atMs": 72050, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-0" + } + }, + { + "atMs": 72138, + "event": { + "type": "CUSTOM", + "name": "dspack.emit", + "value": { + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ], + "warnings": [ + { + "code": "surface-synthesized-action", + "message": "$.root.children[4]: A2UI requires a declarative action on Button; synthesized event 'submit'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 5 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ] + } + } + }, + { + "atMs": 72139, + "event": { + "type": "TOOL_CALL_START", + "toolCallId": "fixture-012-a2ui-0", + "toolCallName": "generate_a2ui" + } + }, + { + "atMs": 72139, + "event": { + "type": "TOOL_CALL_ARGS", + "toolCallId": "fixture-012-a2ui-0", + "delta": "{\"surfaceId\":\"data_collection\"}" + } + }, + { + "atMs": 72139, + "event": { + "type": "TOOL_CALL_END", + "toolCallId": "fixture-012-a2ui-0" + } + }, + { + "atMs": 72139, + "event": { + "type": "TOOL_CALL_RESULT", + "messageId": "fixture-012-a2ui-0-result", + "toolCallId": "fixture-012-a2ui-0", + "content": "{\"a2ui_operations\":[{\"version\":\"v0.9\",\"createSurface\":{\"surfaceId\":\"data_collection\",\"catalogId\":\"https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json\",\"theme\":{\"agentDisplayName\":\"Astryx via dspack\"}}},{\"version\":\"v0.9\",\"updateComponents\":{\"surfaceId\":\"data_collection\",\"components\":[{\"id\":\"root\",\"component\":\"Card\",\"child\":\"root_col\"},{\"id\":\"title\",\"component\":\"Text\",\"variant\":\"h2\",\"text\":\"Create your account\"},{\"id\":\"intro\",\"component\":\"Text\",\"variant\":\"body\",\"text\":\"Name and email is all it takes. You can add the rest later.\"},{\"id\":\"name_input\",\"component\":\"TextField\",\"label\":\"Your name\",\"placeholder\":\"Ada Lovelace\",\"isRequired\":true},{\"id\":\"email_input\",\"component\":\"TextField\",\"variant\":\"shortText\",\"label\":\"Email address\",\"placeholder\":\"ada@example.com\",\"description\":\"We send a confirmation link, nothing else.\",\"isRequired\":true},{\"id\":\"submit\",\"component\":\"Button\",\"label\":\"Create account\",\"variant\":\"primary\",\"action\":{\"event\":{\"name\":\"submit\",\"context\":{}}}},{\"id\":\"root_col\",\"component\":\"Column\",\"children\":[\"title\",\"intro\",\"name_input\",\"email_input\",\"submit\"]}]}}]}" + } + }, + { + "atMs": 72139, + "event": { + "type": "CUSTOM", + "name": "dspack.audit", + "value": { + "outcome": "passed", + "exitCode": 0, + "report": { + "reportVersion": "1", + "createdAt": "2026-07-21T21:09:25.527Z", + "request": { + "prompt": "A signup form asking for name and email, with a clear call to action.", + "intent": "data-collection", + "contract": { + "name": "Astryx", + "dspack": "0.4", + "sha256": "ec7289f199304b127b4b615483b1980820a73fde30d3c03d82c722a1f2f2cb86" + } + }, + "generation": { + "adapterId": "ollama:gpt-oss:latest", + "schemaSha256": "2d54e53a05663ea6caf22c237027bb82fe0174f68d5b56edc1997b02d345b573", + "maxRepairs": 2, + "ruleSteering": true, + "repairTemplate": "standard" + }, + "attempts": [ + { + "index": 0, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "signup-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Create your account" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Name and email is all it takes. You can add the rest later." + }, + { + "component": "text-input", + "id": "name-input", + "props": { + "label": "Your name", + "placeholder": "Ada Lovelace", + "isRequired": true + } + }, + { + "component": "text-input", + "id": "email-input", + "props": { + "type": "email", + "label": "Email address", + "placeholder": "ada@example.com", + "description": "We send a confirmation link, nothing else.", + "isRequired": true + } + }, + { + "component": "button", + "id": "submit", + "props": { + "label": "Create account", + "variant": "primary" + } + } + ] + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 1732, + "outputTokens": 187 + }, + "meta": { + "provider": "ollama", + "total_duration": 71921067713, + "load_duration": 4110364781, + "prompt_eval_duration": 62909000, + "eval_duration": 11942396000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + ], + "repairMessages": [], + "outcome": "passed", + "emitted": { + "target": "a2ui", + "surfaceMessages": { + "messages": [ + { + "version": "v0.9", + "createSurface": { + "surfaceId": "data_collection", + "catalogId": "https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json", + "theme": { + "agentDisplayName": "Astryx via dspack" + } + } + }, + { + "version": "v0.9", + "updateComponents": { + "surfaceId": "data_collection", + "components": [ + { + "id": "root", + "component": "Card", + "child": "root_col" + }, + { + "id": "title", + "component": "Text", + "variant": "h2", + "text": "Create your account" + }, + { + "id": "intro", + "component": "Text", + "variant": "body", + "text": "Name and email is all it takes. You can add the rest later." + }, + { + "id": "name_input", + "component": "TextField", + "label": "Your name", + "placeholder": "Ada Lovelace", + "isRequired": true + }, + { + "id": "email_input", + "component": "TextField", + "variant": "shortText", + "label": "Email address", + "placeholder": "ada@example.com", + "description": "We send a confirmation link, nothing else.", + "isRequired": true + }, + { + "id": "submit", + "component": "Button", + "label": "Create account", + "variant": "primary", + "action": { + "event": { + "name": "submit", + "context": {} + } + } + }, + { + "id": "root_col", + "component": "Column", + "children": [ + "title", + "intro", + "name_input", + "email_input", + "submit" + ] + } + ] + } + } + ] + }, + "warnings": [ + { + "code": "surface-synthesized-action", + "message": "$.root.children[4]: A2UI requires a declarative action on Button; synthesized event 'submit'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 5 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ], + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ] + }, + "timings": { + "totalMs": 72139 + } + } + } + } + }, + { + "atMs": 72139, + "event": { + "type": "RUN_FINISHED", + "threadId": "studio", + "runId": "fixture-012" + } + } + ] +} diff --git a/packages/replay/fixtures/fixture-013.json b/packages/replay/fixtures/fixture-013.json new file mode 100644 index 0000000..dc2ae86 --- /dev/null +++ b/packages/replay/fixtures/fixture-013.json @@ -0,0 +1,632 @@ +{ + "replayFixture": "0.1", + "id": "fixture-013", + "name": "An ask needs a form", + "recordedAt": "2026-07-21T21:15:23.184Z", + "mode": "live", + "adapterId": "ollama:gpt-oss:latest", + "intent": "data-collection", + "prompt": "Invite people to join the product: just a friendly welcome message telling them to sign up. Keep it to text only, no form fields, no buttons.", + "events": [ + { + "atMs": 1, + "event": { + "type": "RUN_STARTED", + "threadId": "studio", + "runId": "fixture-013" + } + }, + { + "atMs": 1, + "event": { + "type": "CUSTOM", + "name": "dspack.run.start", + "value": { + "intent": "data-collection", + "prompt": "Invite people to join the product: just a friendly welcome message telling them to sign up. Keep it to text only, no form fields, no buttons.", + "adapterId": "ollama:gpt-oss:latest", + "ruleIds": [ + "rule.alertdialog-carries-content", + "rule.alertdialog-action-label-specific", + "rule.button-carries-label", + "rule.input-carries-label", + "rule.dialog-no-nested-overlays", + "rule.data-collection-requires-input-and-action" + ] + } + } + }, + { + "atMs": 51166, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-0" + } + }, + { + "atMs": 51166, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 0, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "welcome-card", + "children": [ + { + "component": "text", + "id": "welcome-text", + "props": { + "type": "body" + }, + "text": "Welcome! Join our product today and start exploring all the features." + } + ] + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "FAIL" + } + ], + "findings": [ + { + "ruleId": "rule.data-collection-requires-input-and-action", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'text-input' does not appear in the surface.", + "rationale": "\"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.signup-basic" + ] + }, + { + "ruleId": "rule.data-collection-requires-input-and-action", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'button' does not appear in the surface.", + "rationale": "\"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.signup-basic" + ] + } + ] + } + } + }, + { + "atMs": 51166, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-0" + } + }, + { + "atMs": 51166, + "event": { + "type": "CUSTOM", + "name": "dspack.repair", + "value": { + "index": 0, + "message": "Your surface violates 2 governance rule finding(s) of the \"Astryx\" design system:\n\nViolation 1: [rule.data-collection-requires-input-and-action / must]\n At: $.root (component: surface)\n Required component 'text-input' does not appear in the surface.\n Why: \"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4\nViolation 2: [rule.data-collection-requires-input-and-action / must]\n At: $.root (component: surface)\n Required component 'button' does not appear in the surface.\n Why: \"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4\n\nA correct example (ex.signup-basic): {\"dspackSurface\":\"0.1\",\"system\":\"Astryx\",\"intent\":\"data-collection\",\"root\":{\"component\":\"card\",\"id\":\"signup-card\",\"children\":[{\"component\":\"text\",\"id\":\"title\",\"props\":{\"type\":\"display-3\"},\"text\":\"Create your account\"},{\"component\":\"text\",\"id\":\"intro\",\"props\":{\"type\":\"body\"},\"text\":\"Name and email is all it takes. You can add the rest later.\"},{\"component\":\"text-input\",\"id\":\"name-input\",\"props\":{\"label\":\"Your name\",\"placeholder\":\"Ada Lovelace\",\"isRequired\":true}},{\"component\":\"text-input\",\"id\":\"email-input\",\"props\":{\"type\":\"email\",\"label\":\"Email address\",\"placeholder\":\"ada@example.com\",\"description\":\"We send a confirmation link, nothing else.\",\"isRequired\":true}},{\"component\":\"button\",\"id\":\"submit\",\"props\":{\"label\":\"Create account\",\"variant\":\"primary\"}}]}}\n\nProduce a corrected dspack surface document that fixes every violation above.\nDo not change parts of the surface that were not flagged.\nRespond with a single JSON object conforming to the provided schema. No commentary." + } + } + }, + { + "atMs": 105367, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-1" + } + }, + { + "atMs": 105367, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 1, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "welcome-card", + "children": [ + { + "component": "text", + "id": "welcome-text", + "props": { + "type": "body" + }, + "text": "Welcome! Join our product today and start exploring all the features." + }, + { + "component": "text-input", + "id": "email-input", + "props": { + "label": "Email address", + "placeholder": "you@example.com", + "isRequired": true + } + }, + { + "component": "button", + "id": "join-button", + "props": { + "label": "Join now", + "variant": "primary" + } + } + ] + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + } + }, + { + "atMs": 105367, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-1" + } + }, + { + "atMs": 105453, + "event": { + "type": "CUSTOM", + "name": "dspack.emit", + "value": { + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ], + "warnings": [ + { + "code": "surface-synthesized-action", + "message": "$.root.children[2]: A2UI requires a declarative action on Button; synthesized event 'join_button'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 3 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ] + } + } + }, + { + "atMs": 105453, + "event": { + "type": "TOOL_CALL_START", + "toolCallId": "fixture-013-a2ui-0", + "toolCallName": "generate_a2ui" + } + }, + { + "atMs": 105453, + "event": { + "type": "TOOL_CALL_ARGS", + "toolCallId": "fixture-013-a2ui-0", + "delta": "{\"surfaceId\":\"data_collection\"}" + } + }, + { + "atMs": 105453, + "event": { + "type": "TOOL_CALL_END", + "toolCallId": "fixture-013-a2ui-0" + } + }, + { + "atMs": 105453, + "event": { + "type": "TOOL_CALL_RESULT", + "messageId": "fixture-013-a2ui-0-result", + "toolCallId": "fixture-013-a2ui-0", + "content": "{\"a2ui_operations\":[{\"version\":\"v0.9\",\"createSurface\":{\"surfaceId\":\"data_collection\",\"catalogId\":\"https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json\",\"theme\":{\"agentDisplayName\":\"Astryx via dspack\"}}},{\"version\":\"v0.9\",\"updateComponents\":{\"surfaceId\":\"data_collection\",\"components\":[{\"id\":\"root\",\"component\":\"Card\",\"child\":\"root_col\"},{\"id\":\"welcome_text\",\"component\":\"Text\",\"variant\":\"body\",\"text\":\"Welcome! Join our product today and start exploring all the features.\"},{\"id\":\"email_input\",\"component\":\"TextField\",\"label\":\"Email address\",\"placeholder\":\"you@example.com\",\"isRequired\":true},{\"id\":\"join_button\",\"component\":\"Button\",\"label\":\"Join now\",\"variant\":\"primary\",\"action\":{\"event\":{\"name\":\"join_button\",\"context\":{}}}},{\"id\":\"root_col\",\"component\":\"Column\",\"children\":[\"welcome_text\",\"email_input\",\"join_button\"]}]}}]}" + } + }, + { + "atMs": 105453, + "event": { + "type": "CUSTOM", + "name": "dspack.audit", + "value": { + "outcome": "passed", + "exitCode": 0, + "report": { + "reportVersion": "1", + "createdAt": "2026-07-21T21:13:37.731Z", + "request": { + "prompt": "Invite people to join the product: just a friendly welcome message telling them to sign up. Keep it to text only, no form fields, no buttons.", + "intent": "data-collection", + "contract": { + "name": "Astryx", + "dspack": "0.4", + "sha256": "ec7289f199304b127b4b615483b1980820a73fde30d3c03d82c722a1f2f2cb86" + } + }, + "generation": { + "adapterId": "ollama:gpt-oss:latest", + "schemaSha256": "2d54e53a05663ea6caf22c237027bb82fe0174f68d5b56edc1997b02d345b573", + "maxRepairs": 2, + "ruleSteering": true, + "repairTemplate": "standard" + }, + "attempts": [ + { + "index": 0, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "welcome-card", + "children": [ + { + "component": "text", + "id": "welcome-text", + "props": { + "type": "body" + }, + "text": "Welcome! Join our product today and start exploring all the features." + } + ] + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 2240, + "outputTokens": 69 + }, + "meta": { + "provider": "ollama", + "total_duration": 51105111125, + "load_duration": 270283227, + "prompt_eval_duration": 63610000, + "eval_duration": 4321977000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "FAIL" + } + ], + "findings": [ + { + "ruleId": "rule.data-collection-requires-input-and-action", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'text-input' does not appear in the surface.", + "rationale": "\"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.signup-basic" + ] + }, + { + "ruleId": "rule.data-collection-requires-input-and-action", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'button' does not appear in the surface.", + "rationale": "\"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.signup-basic" + ] + } + ] + }, + { + "index": 1, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "welcome-card", + "children": [ + { + "component": "text", + "id": "welcome-text", + "props": { + "type": "body" + }, + "text": "Welcome! Join our product today and start exploring all the features." + }, + { + "component": "text-input", + "id": "email-input", + "props": { + "label": "Email address", + "placeholder": "you@example.com", + "isRequired": true + } + }, + { + "component": "button", + "id": "join-button", + "props": { + "label": "Join now", + "variant": "primary" + } + } + ] + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 2592, + "outputTokens": 119 + }, + "meta": { + "provider": "ollama", + "total_duration": 54110475146, + "load_duration": 287157005, + "prompt_eval_duration": 63897000, + "eval_duration": 7473159000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + ], + "repairMessages": [ + "Your surface violates 2 governance rule finding(s) of the \"Astryx\" design system:\n\nViolation 1: [rule.data-collection-requires-input-and-action / must]\n At: $.root (component: surface)\n Required component 'text-input' does not appear in the surface.\n Why: \"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4\nViolation 2: [rule.data-collection-requires-input-and-action / must]\n At: $.root (component: surface)\n Required component 'button' does not appear in the surface.\n Why: \"TextInput collects short-form text like names, emails, or search queries\"; Button \"triggers an action when clicked. Use it for form submissions, confirmations, navigation, or any interaction that needs a clear call to action\". An ask without a field is not a form, and a form that cannot be submitted does not ship: a data-collection surface carries at least one input and an explicit action. No layout, field count, or form shape is mandated. — Astryx TextInput and Button docs, v0.1.4\n\nA correct example (ex.signup-basic): {\"dspackSurface\":\"0.1\",\"system\":\"Astryx\",\"intent\":\"data-collection\",\"root\":{\"component\":\"card\",\"id\":\"signup-card\",\"children\":[{\"component\":\"text\",\"id\":\"title\",\"props\":{\"type\":\"display-3\"},\"text\":\"Create your account\"},{\"component\":\"text\",\"id\":\"intro\",\"props\":{\"type\":\"body\"},\"text\":\"Name and email is all it takes. You can add the rest later.\"},{\"component\":\"text-input\",\"id\":\"name-input\",\"props\":{\"label\":\"Your name\",\"placeholder\":\"Ada Lovelace\",\"isRequired\":true}},{\"component\":\"text-input\",\"id\":\"email-input\",\"props\":{\"type\":\"email\",\"label\":\"Email address\",\"placeholder\":\"ada@example.com\",\"description\":\"We send a confirmation link, nothing else.\",\"isRequired\":true}},{\"component\":\"button\",\"id\":\"submit\",\"props\":{\"label\":\"Create account\",\"variant\":\"primary\"}}]}}\n\nProduce a corrected dspack surface document that fixes every violation above.\nDo not change parts of the surface that were not flagged.\nRespond with a single JSON object conforming to the provided schema. No commentary." + ], + "outcome": "passed", + "emitted": { + "target": "a2ui", + "surfaceMessages": { + "messages": [ + { + "version": "v0.9", + "createSurface": { + "surfaceId": "data_collection", + "catalogId": "https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json", + "theme": { + "agentDisplayName": "Astryx via dspack" + } + } + }, + { + "version": "v0.9", + "updateComponents": { + "surfaceId": "data_collection", + "components": [ + { + "id": "root", + "component": "Card", + "child": "root_col" + }, + { + "id": "welcome_text", + "component": "Text", + "variant": "body", + "text": "Welcome! Join our product today and start exploring all the features." + }, + { + "id": "email_input", + "component": "TextField", + "label": "Email address", + "placeholder": "you@example.com", + "isRequired": true + }, + { + "id": "join_button", + "component": "Button", + "label": "Join now", + "variant": "primary", + "action": { + "event": { + "name": "join_button", + "context": {} + } + } + }, + { + "id": "root_col", + "component": "Column", + "children": [ + "welcome_text", + "email_input", + "join_button" + ] + } + ] + } + } + ] + }, + "warnings": [ + { + "code": "surface-synthesized-action", + "message": "$.root.children[2]: A2UI requires a declarative action on Button; synthesized event 'join_button'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 3 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ], + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ] + }, + "timings": { + "totalMs": 105453 + } + } + } + } + }, + { + "atMs": 105453, + "event": { + "type": "RUN_FINISHED", + "threadId": "studio", + "runId": "fixture-013" + } + } + ] +} diff --git a/packages/replay/fixtures/fixture-014.json b/packages/replay/fixtures/fixture-014.json new file mode 100644 index 0000000..d83c327 --- /dev/null +++ b/packages/replay/fixtures/fixture-014.json @@ -0,0 +1,347 @@ +{ + "replayFixture": "0.1", + "id": "fixture-014", + "name": "The emitter refuses the select", + "recordedAt": "2026-07-21T21:18:07.522Z", + "mode": "live", + "adapterId": "ollama:gpt-oss:latest", + "intent": "data-collection", + "prompt": "A form to request access to a private workspace: your name, work email, and which team you are on, with a clear submit action.", + "events": [ + { + "atMs": 1, + "event": { + "type": "RUN_STARTED", + "threadId": "studio", + "runId": "fixture-014" + } + }, + { + "atMs": 1, + "event": { + "type": "CUSTOM", + "name": "dspack.run.start", + "value": { + "intent": "data-collection", + "prompt": "A form to request access to a private workspace: your name, work email, and which team you are on, with a clear submit action.", + "adapterId": "ollama:gpt-oss:latest", + "ruleIds": [ + "rule.alertdialog-carries-content", + "rule.alertdialog-action-label-specific", + "rule.button-carries-label", + "rule.input-carries-label", + "rule.dialog-no-nested-overlays", + "rule.data-collection-requires-input-and-action" + ] + } + } + }, + { + "atMs": 56096, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-0" + } + }, + { + "atMs": 56096, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 0, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "access-request-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Request Access" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Please provide your details to join our private workspace." + }, + { + "component": "text-input", + "id": "name", + "props": { + "label": "Full name", + "placeholder": "John Doe", + "isRequired": true + } + }, + { + "component": "text-input", + "id": "email", + "props": { + "type": "email", + "label": "Work email address", + "placeholder": "john.doe@company.com", + "description": "We will verify your domain.", + "isRequired": true + } + }, + { + "component": "text", + "id": "team-label", + "props": { + "type": "label" + }, + "text": "Team" + }, + { + "component": "dropdown-menu", + "id": "team-select", + "props": { + "items": [ + { + "label": "Engineering", + "value": "engineering" + }, + { + "label": "Marketing", + "value": "marketing" + }, + { + "label": "Design", + "value": "design" + } + ], + "hasChevron": true + } + }, + { + "component": "button", + "id": "submit", + "props": { + "label": "Submit request", + "variant": "primary" + } + } + ] + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + } + }, + { + "atMs": 56096, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-0" + } + }, + { + "atMs": 56097, + "event": { + "type": "CUSTOM", + "name": "dspack.emit", + "value": { + "validations": [], + "warnings": [] + } + } + }, + { + "atMs": 56098, + "event": { + "type": "CUSTOM", + "name": "dspack.audit", + "value": { + "outcome": "failed-gate", + "exitCode": 3, + "report": { + "reportVersion": "1", + "createdAt": "2026-07-21T21:17:11.423Z", + "request": { + "prompt": "A form to request access to a private workspace: your name, work email, and which team you are on, with a clear submit action.", + "intent": "data-collection", + "contract": { + "name": "Astryx", + "dspack": "0.4", + "sha256": "ec7289f199304b127b4b615483b1980820a73fde30d3c03d82c722a1f2f2cb86" + } + }, + "generation": { + "adapterId": "ollama:gpt-oss:latest", + "schemaSha256": "2d54e53a05663ea6caf22c237027bb82fe0174f68d5b56edc1997b02d345b573", + "maxRepairs": 2, + "ruleSteering": true, + "repairTemplate": "standard" + }, + "attempts": [ + { + "index": 0, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "data-collection", + "root": { + "component": "card", + "id": "access-request-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Request Access" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Please provide your details to join our private workspace." + }, + { + "component": "text-input", + "id": "name", + "props": { + "label": "Full name", + "placeholder": "John Doe", + "isRequired": true + } + }, + { + "component": "text-input", + "id": "email", + "props": { + "type": "email", + "label": "Work email address", + "placeholder": "john.doe@company.com", + "description": "We will verify your domain.", + "isRequired": true + } + }, + { + "component": "text", + "id": "team-label", + "props": { + "type": "label" + }, + "text": "Team" + }, + { + "component": "dropdown-menu", + "id": "team-select", + "props": { + "items": [ + { + "label": "Engineering", + "value": "engineering" + }, + { + "label": "Marketing", + "value": "marketing" + }, + { + "label": "Design", + "value": "design" + } + ], + "hasChevron": true + } + }, + { + "component": "button", + "id": "submit", + "props": { + "label": "Submit request", + "variant": "primary" + } + } + ] + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 2159, + "outputTokens": 245 + }, + "meta": { + "provider": "ollama", + "total_duration": 56045556055, + "load_duration": 316771719, + "prompt_eval_duration": 63883000, + "eval_duration": 15697525000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + ], + "repairMessages": [], + "outcome": "failed-gate", + "emitted": { + "target": "a2ui", + "refusal": "unknown component 'dropdown-menu': not a mapped component of the 'Astryx — A2UI catalog (compiled from dspack)' profile (sub-components are consumed by their compound parent and cannot be emitted standalone) (at $.root.children[5])", + "warnings": [], + "validations": [] + }, + "timings": { + "totalMs": 56098 + } + } + } + } + }, + { + "atMs": 56098, + "event": { + "type": "RUN_FINISHED", + "threadId": "studio", + "runId": "fixture-014" + } + } + ] +} diff --git a/packages/scenarios/src/break-conditions.ts b/packages/scenarios/src/break-conditions.ts index 2a80f68..842df81 100644 --- a/packages/scenarios/src/break-conditions.ts +++ b/packages/scenarios/src/break-conditions.ts @@ -12,7 +12,7 @@ export interface BreakCondition { id: string; label: string; /** The scenario this condition belongs to; absent when scenarioIndependent. */ - scenarioId?: "project-deletion" | "appointment-booking" | "recipe-creator" | "support-triage"; + scenarioId?: "project-deletion" | "appointment-booking" | "recipe-creator" | "support-triage" | "onboarding"; /** * True for conditions that belong to every scenario: pure client-side * demonstrations that never start a run or read scenario state. @@ -139,6 +139,22 @@ export const breakConditions: BreakCondition[] = [ note: "In this recorded real run the model stacked the tickets as plain prose; the table rule caught it and the repaired surface shipped a filled triage table with status badges.", }, }, + { + id: "ask-without-a-form", + label: "data collection without a form", + scenarioId: "onboarding", + intent: "data-collection", + kind: "governed-repair", + expected: + "S3 fails with rule.data-collection-requires-input-and-action (the TextInput and Button docs' own rationale, verbatim), a repair message is sent, and the repaired surface ships a labeled field and an explicit action.", + prompt: + "Invite people to join the product: just a friendly welcome message telling them to sign up. Keep it to text only, no form fields, no buttons.", + recordedCatch: { + scenarioId: "onboarding", + fixtureKey: "argues-back", + note: "In this recorded real run the model shipped a text-only welcome with nothing to fill in; the form rule caught it and the repaired surface added a labeled email field and an explicit action.", + }, + }, { id: "malformed-import", label: "malformed session import", diff --git a/packages/scenarios/src/registry.ts b/packages/scenarios/src/registry.ts index b84627a..bc4add5 100644 --- a/packages/scenarios/src/registry.ts +++ b/packages/scenarios/src/registry.ts @@ -5,8 +5,9 @@ * governance content, per the project plan). * * Order is intentional: ready scenarios first (recipe co-editing leads as the - * default, then booking, then project deletion, then support triage), planned - * ones grouped after. readyScenarios[0] is the studio's default scenario. + * default, then booking, then project deletion, then support triage, then + * signup/onboarding), planned ones grouped after. readyScenarios[0] is the + * studio's default scenario. */ import type { Scenario } from "./types"; import fixture001 from "@dspack-studio/replay/fixtures/fixture-001.json"; @@ -19,6 +20,9 @@ import fixture008 from "@dspack-studio/replay/fixtures/fixture-008.json"; import fixture009 from "@dspack-studio/replay/fixtures/fixture-009.json"; import fixture010 from "@dspack-studio/replay/fixtures/fixture-010.json"; import fixture011 from "@dspack-studio/replay/fixtures/fixture-011.json"; +import fixture012 from "@dspack-studio/replay/fixtures/fixture-012.json"; +import fixture013 from "@dspack-studio/replay/fixtures/fixture-013.json"; +import fixture014 from "@dspack-studio/replay/fixtures/fixture-014.json"; export const scenarios: Scenario[] = [ { @@ -156,12 +160,41 @@ export const scenarios: Scenario[] = [ { id: "onboarding", name: "Signup / onboarding", - tagline: "Labeled inputs, enforced: every field carries a visible label or it does not ship.", + tagline: "An ask needs a form: something to fill in, a way to send it — and every field keeps its label.", intent: "data-collection", - status: "planned", - needs: ["design-system rules for signup and form screens", "a recorded real run to replay"], - seedPrompts: ["A signup form asking for name and email, with a clear call to action."], - fixtures: [], + status: "ready", + seedPrompts: [ + "A signup form asking for name and email, with a clear call to action.", + "Invite people to join the product: just a friendly welcome message telling them to sign up. Keep it to text only, no form fields, no buttons.", + "A form to request access to a private workspace: your name, work email, and which team you are on, with a clear submit action.", + ], + breakItPrompts: [ + { + ruleId: "rule.data-collection-requires-input-and-action", + prompt: + "Invite people to join the product: just a friendly welcome message telling them to sign up. Keep it to text only, no form fields, no buttons.", + }, + ], + fixtures: [ + { + key: "argues-back", + label: "the interface argues back", + blurb: "One governed repair: the model ships a text-only welcome with nothing to fill in, the form rule catches it, and the repaired surface adds a labeled email field and an explicit action.", + fixture: fixture013, + }, + { + key: "clean", + label: "clean first pass", + blurb: "No violations: one attempt, straight through the gates to a labeled name-and-email form with one explicit primary action.", + fixture: fixture012, + }, + { + key: "refusal", + label: "the emitter refuses", + blurb: "A non-signup ask — a workspace access request — lints clean but picks a team select (dropdown-menu) the protocol profile cannot project: the pipeline refuses, with receipts.", + fixture: fixture014, + }, + ], }, { id: "hotel-reservations", diff --git a/playwright.production.config.ts b/playwright.production.config.ts index 3ae2d95..f266d6b 100644 --- a/playwright.production.config.ts +++ b/playwright.production.config.ts @@ -31,6 +31,7 @@ export default defineConfig({ testMatch: [ "replay.spec.ts", "support-triage.spec.ts", + "onboarding.spec.ts", "alive.spec.ts", "permalinks.spec.ts", "tour-xray-wire.spec.ts", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 1f0c567..960b8e4 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -49,7 +49,7 @@ importers: apps/web: dependencies: '@aestheticfunction/dspack-gen': - specifier: ^0.1.2 + specifier: ^0.1.1 version: 0.1.2(zod@4.4.3) '@astryxdesign/core': specifier: 0.1.4