Skip to content

test(contracts): one typed conformance helper for the runtime-family suites - #2301

Merged
thymikee merged 5 commits into
mainfrom
claude/test-contracts-runtime-family-conformance
Sep 5, 2026
Merged

test(contracts): one typed conformance helper for the runtime-family suites#2301
thymikee merged 5 commits into
mainfrom
claude/test-contracts-runtime-family-conformance

Conversation

@thymikee

@thymikee thymikee commented Sep 5, 2026

Copy link
Copy Markdown
Member

Summary

Ten packages/contracts/src/*-runtime.test.ts files (alert, app-event,
app-switcher, back, clipboard, home, keyboard, orientation, settings,
tv-remote) repeated the same three binding-rule tests by copy for every
interactor-catalog operation. Those three rules now live once, in
interactor-operation-conformance.test.ts, driven by the catalog itself: each
rule walks the exported INTERACTOR_OPERATIONS and binds through the catalog
row's own bind and label, exactly as production does. The per-operation
expectations (interactor method, minimal input) sit in one table typed over
every catalog operation, so a missing, duplicated or unregistered row fails
typecheck and a table-vs-catalog test, and every row is executed by
construction — completeness is structural, not observed from source text.
Each facet's *-runtime.test.ts keeps only its dedicated tests.
app-inventory-runtime.ts/app-state-runtime.ts/
platform-runtime-operations.ts (#2295) and
platform-runtime-unavailable.test.ts (#2291) are untouched.

Validation

Head 453d28e0e5.

  • Focused: npx vitest run packages/contracts/src — 60 files / 304 tests
    passed. oxlint, oxfmt --check, tsc -b packages/contracts, check:layering
    and check:fallow --base origin/main clean.
  • Planted red (row commented out, spelling kept in the comment): commented
    the keyboardEnter row of the expectation table → tsc -b packages/contracts
    fails (Property 'keyboardEnter' is missing) and all four conformance tests
    fail. Reverted.
  • Planted red (registry member removed): dropped back from
    INTERACTOR_OPERATIONS → typecheck fails on the now-excess back row and the
    table-vs-catalog test fails. Reverted.
  • Earlier heads (a87c6984d6 and before) carried the source-scanning gate
    that the review found; CI on those was green except a pre-existing iOS smoke
    flake on d0b11ba2 that cleared on rerun.

@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.44 MB 4.44 MB 0 B
Package (unpacked) 4.44 MB 4.44 MB 0 B
Package (download) 1.32 MB 1.32 MB +9 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.4 ms 26.3 ms -0.2 ms
CLI --help 77.6 ms 78.2 ms +0.6 ms

@thymikee
thymikee force-pushed the claude/test-contracts-runtime-family-conformance branch from 9265102 to d0b11ba Compare September 5, 2026 17:17
thymikee added a commit that referenced this pull request Sep 5, 2026
…eletion blind spot

Review on PR #2301 (P2): the two-direction completeness check in
interactor-operation-conformance.test.ts can't catch a catalog row and its
conformance rows being deleted together in one change — both sets shrink in
lockstep and neither assertion trips. No behavior change; documents the
limitation for a future maintainer who wants to close it with an independent
count.
…suites

Extracts the shared binding-rule mechanics that ten *-runtime.test.ts
files repeated by copy into interactor-operation-conformance.fixtures.ts.
Each module keeps its own expected table and calls the helper with it;
the helper owns no expectations.
…tion conformance

Asserts both directions against the production registry: every catalog
operation has exactly one conformance table naming it, and every named
operation is registered in the catalog.
…eletion blind spot

Review on PR #2301 (P2): the two-direction completeness check in
interactor-operation-conformance.test.ts can't catch a catalog row and its
conformance rows being deleted together in one change — both sets shrink in
lockstep and neither assertion trips. No behavior change; documents the
limitation for a future maintainer who wants to close it with an independent
count.
@thymikee
thymikee force-pushed the claude/test-contracts-runtime-family-conformance branch from c781e4a to a87c698 Compare September 5, 2026 18:04
@thymikee

thymikee commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

Sentinel review at a87c6984d60ef4e75ae1f46e15437ba8cee28398.

P2: completeness reads raw test source and regex-matches operation literals (packages/contracts/src/interactor-operation-conformance.test.ts, lines 14–23), so commenting out the entire keyboardEnter row still counts it as conformed while no tests execute that row. Use typed row data shared by actual test registration and completeness rather than reconstructing coverage from source text. Add a planted-red case removing an executable row while its spelling remains in a comment; completeness must fail. Consolidation otherwise looks sound, but hold readiness until the gate proves executable coverage.

…log itself

The completeness gate read raw test sources and regex-matched operation
literals, so a commented-out row still counted as conformed while nothing
executed it. The three binding rules now live in one test file that walks
INTERACTOR_OPERATIONS and binds through each catalog row's own bind and
label, with the per-operation expectations (interactor method, minimal
input) in a table typed over every catalog operation. A missing, duplicated
or unregistered row fails typecheck and the table-vs-catalog test; every row
is executed by construction. The conformance fixture and the per-facet
conformInteractorOperations blocks are gone; each facet test keeps only its
dedicated tests.
@thymikee

thymikee commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

Addressed in 453d28e. The source-scanning completeness gate is gone. The three binding rules now live in one test that walks INTERACTOR_OPERATIONS and binds through each catalog row's own bind/label (the production path), with per-operation expectations in a table typed over every catalog operation. So the row data executed by the tests is the completeness data: a row that is commented out (spelling kept in the comment) fails tsc -b packages/contracts (Property 'keyboardEnter' is missing) and all four conformance tests; a catalog member dropped fails typecheck on the excess row and the table-vs-catalog test. Both planted and reverted. The per-facet fixture and conformInteractorOperations blocks are removed; each facet test keeps only its dedicated tests.

* origin/main:
  perf: bundle runtime dependencies and report full install size (#2310)
  ci: avoid unrelated Apple runner cache invalidation (#2303)
  fix(web): preserve the backend ref so snapshot refs match actionable refs (#2283)
  test(daemon): session-open-url-prewarm through the request seam (#2304)
  test(daemon): session-devices-batch-runtime through the request seam (#2305)
  chore(gates): layering baselines ratchet against merge-base (#2299)
  test(daemon): one typed conformance helper for the daemon runtime suites (#2298)
  chore(layering): derive the contracts export inventory from package.json (#2297)
  perf: bundle tar-stream to reduce install footprint (#2286)
  docs: simplify agent context and resolve conflicting guidance (#2287)
  refactor(cli): let help resolve command aliases itself and retire R12 (#2293)
@thymikee

thymikee commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

Sentinel review at 453d28e: Independent delta review is source-clean. Source-text completeness scanning is gone: a catalog-keyed typed expectation table is executed through each production bind/label, so a commented-out row cannot count as conformance. Missing/excess rows have planted-red proof. Remaining blocker is exact-head CI: Size encountered the cross-version base-report js.rawBytes failure and rerun checks are pending. No additional device acceptance required for this test-only refactor.

@thymikee
thymikee merged commit 51bed41 into main Sep 5, 2026
18 checks passed
@thymikee
thymikee deleted the claude/test-contracts-runtime-family-conformance branch September 5, 2026 20:11
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-05 20:12 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant