Skip to content

scenario: support-ticket triage ships ready (record-collection)#17

Merged
ryandmonk merged 4 commits into
mainfrom
feat/support-triage
Jul 21, 2026
Merged

scenario: support-ticket triage ships ready (record-collection)#17
ryandmonk merged 4 commits into
mainfrom
feat/support-triage

Conversation

@ryandmonk

Copy link
Copy Markdown
Contributor

What

The fourth ready scenario — replay-first, zero per-scenario code: no agent SCENARIOS entry, no capabilities, no authored surface, no web UI changes. A registry record, owner-authored governance, and two live gpt-oss recordings, shipped atomically (contract + fixtures + registry + break condition + spec + docs), so no intermediate governed-intent-without-scenario state ever exists on main.

Contract (owner-authored governance, astryx.dspack.json)

  • Intent record-collection — records of the same kind (tickets, orders, members), scanned and acted on; badge guidance lives in the intent description and worked example, deliberately not a rule (the linter renders a should-severity miss as "Required component does not appear" under a passing gate — a contradiction — and Badge's own docs endorse plain text when no action is needed).
  • rule.record-collection-requires-table (component-choice, must) — rationale quoted verbatim from the Astryx Table docs via npx astryx component Table --json; explicitly noted as valid while table is the catalog's only collection structure.
  • rule.table-carries-data (required-props, scoped to the intent) — a data-driven table must carry columns and data. Born from observed live failures, see below.
  • Worked example ex.support-ticket-triage: badge strip above a compact tickets table (the contract's table is data-driven strings, so badges live beside it, never in cells). Appended last; examples[0] (the A3 build sample) untouched.
  • shadcn intent parity: deferred to the standing owner-authored contract-extension item. Generation always lints under the astryx catalog (loadContract() is hardcoded), and shadcn already carries table+badge for restyle rendering — verified, nothing blocks.

The generation bug this work surfaced (and its fix)

Across ~20 live runs on two models (gpt-oss:latest, qwen3-coder:30b), the table's nested data rows were never emitted — every surface shipped an empty table shell, and with the data rule active, runs exhausted repairs instead. Root cause upstream: dspack-gen's generation schema marks every prop optional, and grammar-constrained decoders skip optional heavy branches; a repair round cannot fix what the grammar never demands. Proof: calling Ollama directly with data marked required, the same model fills correct rows first try.

Fix: dspack-gen feat/required-props-grammar (targets 0.1.2) — prop descriptors may declare required: true, which reaches the props schema's required and makes props itself required on the node branch. This contract flags table columns+data accordingly.

Draft until dspack-gen 0.1.2 publishes, then the lockfile bumps here. The deployed static site replays recordings and is unaffected; only local Run It Live for this scenario needs ≥0.1.2.

Fixtures (Spark Ollama, ollama:gpt-oss:latest, recorded under the full final contract)

  • fixture-010 "the interface argues back": the model obeys the honest break-it prompt (a stacked prose list) with a lone text node → S3 fails on the table rule → the repaired surface ships badges + a filled 3-row table. --require-repair rejected 3 compliant runs before a genuine violation landed — curation, not distortion; all copy is written from the recorded events.
  • fixture-011 "clean first pass": the neutral seed prompt, one attempt, straight through the gates to a five-row triage table with a status badge strip.
  • A third emitter-refusal fixture was attempted once honestly; the model built a clean table instead of a dropdown, so nothing ships there.

Fail-first evidence

Pre-contract recording failed with intent 'record-collection' is not registered in the contract (known: destructive-action, scheduling, structured-editing). All four support-triage.spec.ts tests failed on the old registry:

3 failed
  support-triage is a ready, selectable scenario on the shelf
  clean replay: a live run straight through the gates to a filled table
  repair replay: prose list caught by the table rule, repaired surface ships
(permalink test, after adding the reload the permalink effect requires:)
1 failed
  permalink: a support-triage deep link resolves

Upstream, three new grammar-alignment assertions fail on dspack-gen 0.1.1.

Gates

  • pnpm -r typecheck / pnpm -r test green; contracts build:catalogs + drift-check clean.
  • Playwright: 94 passed, 4 skipped (90 existing + 4 new; zero existing-spec edits needed — studio-shell.spec.ts pins onboarding and permalinks.spec.ts pins hotel-reservations, verified).
  • Manual: scenario selected in the browser; clean replay renders the five-row table + badge strip on the canvas; repair replay shows attempt 0: S3✗ → repair → attempt 1: ✓.

Merge note: playwright.production.config.ts will conflict trivially with #16 (project split) — resolution is adding support-triage.spec.ts to the studio project's testMatch.

🤖 Generated with Claude Code

ryandmonk and others added 2 commits July 21, 2026 14:44
New owner-authored governance for the record-collection intent (many
records of the same kind, scanned and acted on). One must rule:
collections render as a table — rationale quoted verbatim from the
Astryx Table docs ("consistent dimensionality", "uniform structure"),
valid while table is this catalog's only collection structure. Status
badges are deliberately example-steered, not ruled: the linter's
"Required component does not appear" message under a passing gate would
misread for a should-severity rule, and Badge's own docs say plain text
is fine when no action is needed.

Worked example: badge strip summarizing queue status above a compact
tickets table (the contract's data-driven table is strings-only, so
badges live beside it, not in cells). Appended last; examples[0] (the
A3 build sample) is untouched.

Fail-first: before this change, recording with --intent
record-collection failed with "intent 'record-collection' is not
registered in the contract (known: destructive-action, scheduling,
structured-editing)". After: the scripted smoke plays the worked
example clean through the gates (12 events, passed, no repair).
contracts typecheck/test/build:catalogs/drift-check all clean.

shadcn parity is deferred to the standing owner-authored contract
extension item; generation always lints under this catalog, and the
shadcn side already carries table+badge for restyle rendering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fourth ready scenario, replay-first, zero per-scenario code: a registry
record, owner-authored governance, and two live gpt-oss recordings.

Recording surfaced a real generation bug — across ~20 live runs on two
models the table's nested `data` rows were never emitted, so every
surface shipped an empty table shell. Root cause upstream: dspack-gen's
generation schema marks all props optional, and grammar-constrained
decoders skip optional heavy branches. Fixed in dspack-gen
feat/required-props-grammar (targets 0.1.2): prop descriptors may
declare required:true. This contract now flags table columns+data
required and adds rule.table-carries-data (required-props, scoped to
record-collection) so the gates state the same demand governance-side.

Fixtures (Spark, ollama:gpt-oss:latest, recorded under the full
contract with the fixed generator):
- fixture-010 "argues back": the model obeys a prose-list prompt with a
  lone text node; S3 fails on rule.record-collection-requires-table;
  the repaired surface ships badges + a filled 3-row table.
  --require-repair rejected 3 compliant runs first (curation, not
  distortion). One attempt at an emitter-refusal third fixture produced
  a clean table instead — honestly skipped.
- fixture-011 "clean": first pass straight through the gates to a
  five-row triage table with a status badge strip.

Break-it: records-as-prose (recordedCatch -> argues-back; no
scriptedRef, break-scripts untouched). Docs: AUDIT 4 ready / 2 planned;
NAV-REFACTOR matrix row; IMPLEMENTATION_LOG entry.

Fail-first: all four support-triage.spec.ts tests failed on the old
registry (shelf "(planned)", missing fixtures, permalink "unknown or
not-yet-ready" — outputs in the PR). Full suite: 94 passed, 4 skipped;
pnpm -r typecheck/test green.

Note: Run It Live for this scenario on a local agent needs dspack-gen
>=0.1.2 (the deployed static site replays only and is unaffected);
lockfile bump follows the 0.1.2 publish.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ryandmonk and others added 2 commits July 21, 2026 16:55
Conflict resolution keeps both halves: the first-run/studio project
isolation from #16, with support-triage.spec.ts added to the studio
project's production testMatch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified against the published npm tarball (not the linked checkout):
one Run It Live through the agent server (ollama:gpt-oss:latest on
Spark) produced a clean first pass with a five-row populated triage
table, audit passed. Full gates re-run after the main merge: pnpm -r
typecheck/test green; local Playwright 97 passed / 4 skipped; the
production config against the local deploy artifact 77 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ryandmonk
ryandmonk marked this pull request as ready for review July 21, 2026 21:00
Copilot AI review requested due to automatic review settings July 21, 2026 21:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new ready “support-ticket triage” scenario that is replay-first and governed via contract updates (new intent + rules), backed by two new live recordings and a new Playwright spec, plus a dependency bump to pick up required-props grammar support.

Changes:

  • Add record-collection intent and table-focused governance rules (including required table.columns + table.data) to the Astryx contract.
  • Register the new support-triage scenario as ready, add a new Break-it condition, and ship fixtures fixture-010 / fixture-011.
  • Add e2e/support-triage.spec.ts and include it in the production smoke suite; bump @aestheticfunction/dspack-gen to 0.1.2.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pnpm-lock.yaml Bumps @aestheticfunction/dspack-gen to 0.1.2 for required-props grammar support.
playwright.production.config.ts Includes support-triage.spec.ts in the production “studio” project smoke list.
packages/scenarios/src/registry.ts Promotes support-triage to ready and wires in fixtures 010/011.
packages/scenarios/src/break-conditions.ts Adds a support-triage governed-repair break condition (records-as-prose).
packages/replay/fixtures/fixture-010.json Adds the “argues-back” live recording fixture.
packages/replay/fixtures/fixture-011.json Adds the “clean first pass” live recording fixture.
packages/contracts/astryx.dspack.json Adds record-collection intent, table rules, and marks table columns/data as required.
e2e/support-triage.spec.ts New e2e coverage for shelf readiness, clean replay, repair replay, and permalink resolution.
docs/NAV-REFACTOR.md Updates scenario lists/matrix to include support-triage as ready and documented.
docs/IMPLEMENTATION_LOG.md Adds an implementation log entry for the scenario + required-props grammar motivation.
docs/AUDIT.md Updates audit doc to reflect 4 ready / 2 planned scenarios and support-triage shipping.
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +142 to +146
{
key: "argues-back",
label: "the interface argues back",
blurb: "One governed repair: the model stacks the tickets as plain prose, the table rule catches it, and the repaired surface ships a filled triage table with a status badge strip.",
fixture: fixture010,
Comment on lines +132 to +140
expected:
"S3 fails with rule.record-collection-requires-table (the Table docs' own rationale, verbatim), a repair message is sent, and the repaired surface ships a filled triage table.",
prompt:
"Show the open support tickets as a simple stacked list of short text lines. Keep it minimal: no table, no grid, just plain text per ticket.",
recordedCatch: {
scenarioId: "support-triage",
fixtureKey: "argues-back",
note: "In this recorded real run the model stacked the tickets as plain prose; the table rule caught it and the repaired surface shipped a filled triage table with status badges.",
},
Comment on lines +941 to +948
required. With the fix: fixture-011 (clean) is a first-pass five-row
triage table with a status badge strip; fixture-010 (argues-back) is
the flagship catch — the model obeys a prose-list prompt with a lone
text node, S3 fails on the table rule, and the repaired surface ships
the filled table. `--require-repair` rejected 3 compliant runs before a
genuine violation landed (recorded as rejected, not distorted). The
optional emitter-refusal class was attempted once honestly; the model
built a clean table instead, so no third fixture ships.
@ryandmonk
ryandmonk merged commit dc8bad9 into main Jul 21, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants