Skip to content

[Frontier] [EVID16-01] Run the complete deterministic cross-client acceptance matrix #430

Open

Description

@chayapats

Parent

#392 — #392; complete S16 target #392.

Model tier

Frontier: this slice requires judgment about protected authority, lifecycle/recovery or a new cross-client/service contract; exact new APIs and migrations are not yet settled.

Context to read

AGENTS.md; CONTEXT.md; docs/agents/workflow.md; docs/specs/S16-rakazo-experience.md; docs/specs/S15-conversational-autonomy.md; ADR 0064 and ADRs 0074–0077; docs/design/S16-rakazo-experience.md; docs/design/S16-mobbin-research.md. Reuse the completed design research; do not repeat broad Mobbin discovery.

Starting implementation/contract surfaces: tests/; docs/design/S16-rakazo-experience.md; docs/tickets/S16-rakazo-experience-coverage.json. Read their current definitions and relevant tests before selecting changes. Paths with braces denote existing module families, not settled new signatures.

Deliverable

Maintain and execute the full actual-client composed acceptance matrix at exact builds, with independent live/device rows explicitly pending. Matrix authoring can start now; final execution depends on listed owning deliveries.

Baseline and readiness

Audited stack head 17b814220371cd1e2126be2ec9bd9753537f69e6; main remains the pre-S16 baseline. #398/#399/#403/#404 are open drafts, not merged/live acceptance. Existing S15 services are reusable baseline, and #400/#401 supply draft text/binary custody. No new behavior is claimed here. Branches/PRs and issue ownership were inspected on 2026-11-05; this new ticket has no implementation claim. Recheck at dispatch. Preserve the primary-owned jit/402-s16-file-memory branch.

Needs triage: contract/engineering choices remain; no ready-for-agent label. Resolve named contract decisions and verify prerequisite revisions before unattended implementation. Independent audit/design work may proceed now; final acceptance needs these deliveries.

Implementation steps

  1. Recheck ownership, branches/PRs, prerequisites and exact source baseline. Compare existing implementation, pinned Rakazo reuse and a simpler alternative; record the chosen tradeoff and versioned contract before coding.
  2. Create per-AC scenario/evidence records including every FR/NFR and baseline reuse, fixture versus live classification.
  3. Drive real clients/composed services for file-to-artifact, modern browser, delegated scheduled restart, connected held action and voice/mobile continuity.
  4. Inject cancellation, withdrawal, permission loss, stale generation, crash boundaries and unknown effect recovery.
  5. Record exact source/package/engine/parser/client digests, executed commands and gaps; never certify parity by issue count.
  6. Integrate the authored experience using real service facts, English/Thai dictionary keys in their designated files, accessible loading/error/empty/reconnect states and current capabilities. For non-UI evidence work, verify those states instead of inventing new UI. Record migrations, compatibility and exact build evidence.

Acceptance criteria

  • All AC01–AC22 have actual outcome assertions and evidence owners.
  • Every FR01–42/NFR01–14 is traced without marking inherited evidence as new proof.
  • Fixture transports are labelled; no live credential required for deterministic suite.
  • Any required unsupported case blocks full acceptance rather than disappearing.
  • Deliverable works end to end through its actual owning service/client; source presence, transport success and model prose alone are insufficient.
  • No S14 default, S15 opt-in, installed policy, historical private scope or owner-host configuration changes without its separate explicit transition.
  • New contracts, unresolved limits and exact revision/evidence are recorded; applicable upstream licenses, immutable provenance and modified-file notices ship.

Tests and evidence

  1. All AC01–AC22 have actual outcome assertions and evidence owners. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
  2. Every FR01–42/NFR01–14 is traced without marking inherited evidence as new proof. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
  3. Fixture transports are labelled; no live credential required for deterministic suite. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
  4. Any required unsupported case blocks full acceptance rather than disappearing. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
  5. Run relevant existing module/composed regressions plus the new scenario suite; for rendered behavior, exercise the real bundle in English/Thai, keyboard and mobile widths. Record exact test paths/commands in the PR after implementation selects them.
  6. Keep real host/account/device checks opt-in and evidence-labelled; no absent credential blocks independent deterministic work. Public evidence contains hashes/outcomes only; installation-specific details belong in ignored .local records.

Requirements

S16-AC01, S16-AC02, S16-AC03, S16-AC04, S16-AC05, S16-AC06, S16-AC07, S16-AC08, S16-AC09, S16-AC10, S16-AC11, S16-AC12, S16-AC13, S16-AC14, S16-AC15, S16-AC16, S16-AC17, S16-AC18, S16-AC19, S16-AC20, S16-AC21, S16-AC22; FR-01, FR-02, FR-03, FR-04, FR-05, FR-06, FR-07, FR-08, FR-09, FR-10, FR-11, FR-12, FR-13, FR-14, FR-15, FR-16, FR-17, FR-18, FR-19, FR-20, FR-21, FR-22, FR-23, FR-24, FR-25, FR-26, FR-27, FR-28, FR-29, FR-30, FR-31, FR-32, FR-33, FR-34, FR-35, FR-36, FR-37, FR-38, FR-39, FR-40, FR-41, FR-42, NFR-01, NFR-02, NFR-03, NFR-04, NFR-05, NFR-06, NFR-07, NFR-08, NFR-09, NFR-10, NFR-11, NFR-12, NFR-13, NFR-14. Retain S15 authority decisions and FR/NFR identifiers.

Blocked by

Contracts used/delivered

Use current owning POLICY, DATA15, TEAM, MEM, WORK, CHAT and S16 client/file contracts where relevant, plus Schway's supported public host surface. Specific inputs and proposed delivery: tests/; docs/design/S16-rakazo-experience.md; docs/tickets/S16-rakazo-experience-coverage.json. The Deliverable and ordered steps define the missing contract outcome; freeze version, schemas, migration/recovery and conformance fixtures in the implementation ADR/PR. Proposed operations are not already supported APIs. A demonstrated missing engine seam requires a bounded public-only reproduction and a linked Schway contract ticket, never an internal import or duplicate runtime.

Out of scope

Other tickets' outcomes, hosted multi-user SaaS, marketplace operation, upstream branding, silent billing, replacing Schway, general historical data conversion, and unapproved live deployment. Live credentials, real-device acceptance and owner-host cutover are separately owned by LIVE16-01/CUT16-01; deterministic implementation remains in scope.

Do not

Do not use source/model/tool output as authority; expose credentials, sensitive-site sessions or protected state to working execution (trusted-site sessions are the S16-D09 exception); remove guards before proof; auto-retry unknown effects; weaken or bypass the S16-D07 human-required list or let anything but the in-chat owner decision approve it; treat current stack limitations as scope cuts; add model attribution; use implement-parallel; alter #402 ownership; self-merge or deploy. No placeholder controls or fabricated success.

Definition of done

Run bun install --frozen-lockfile, bun run typecheck, bun test --timeout 30000 for affected deterministic suites, and git diff --check; record exact selected suite commands and results. Add real-browser tests for rendered paths and package checks for shipped artifacts. Pure feasibility documentation runs link/coverage checks and git diff --check; a prototype must run its conformance cases. PR body includes Closes #430 and Refs #392, contract versions, tested head and honest pending evidence. No self-merge; parent #392 remains open until full accepted outcomes.

Owner revision 2026-11-06

Grok-style autonomy (S16-D06–D11, ADR 0084) and Telegram removal (ADR 0085) apply. Only the S16-D07 human-required list waits for the owner, approved by a one-press in-chat card; GitHub push/PR/review replies/merge after required checks are the product default; trusted sites may keep sessions in the working VM browser while sensitive sites stay in the protected browser. Jit has no messaging channel; notifications use Web Push (#441). Policy/approval changes belong to #442.

#424 is closed as not planned; its prerequisite edge now points to #441. S16-AC18 now means notifications.

Activity

  1. marked this as blocked by 20 issues on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageMaintainer needs to evaluate this issue.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions