The rules — the ≥90% per-file gate, test placement, the v8 ignore policy — are in AGENTS.md. This page is the reference for what each script covers.
Each client self-validates from its own folder; the root scripts chain them. There is no aggregate root test script — use validate (fast) or coverage (the gate).
This is the canonical description of the CI-vs-local split — everything else in the repo that mentions it points here.
| Tier | How it runs | What it covers |
|---|---|---|
GitHub CI (.github/workflows/main.yml) |
Automatically, on every push | npm install, then validate, verify:skills:cli, verify:build-gate, verify:bundle-externals, smoke (which includes smoke:web:chromium), test:storybook — plus coverage in a parallel job (#2159) |
The local gate (npm run local:gate) |
By hand, before you push | Every check above (the install is yours to run), plus the Firefox engine pass (smoke:web:firefox), and smoke:tui for real rather than self-skipped |
The local gate is a strict superset, not a mirror. Two of its steps have no GitHub CI counterpart, each for its own reason:
| Local-only step | Why it is local-only |
|---|---|
smoke:web:firefox |
Trialled as a CI job and removed (#2086): across a dozen runs it never once disagreed with Chromium, and playwright install --with-deps carries a real flake surface. Kept in front of a human about to push instead. smoke:web:webkit is worse still — it fails two of the three smokes for reasons nobody has identified — so it is in neither tier. See Supported browsers. |
smoke:tui |
The Ink TUI needs a real TTY. It is invoked in CI via npm run smoke and self-skips there on process.env.CI, so it needs no guarding — it handles itself. |
So the direction that matters holds: passing npm run local:gate means CI's gates will pass. The reverse does not.
The gate is deliberately not named ci (#2146). That name collided with npm ci — a built-in that clean-installs from the lockfile, does not run this script, and is one keystroke away — and it invited the cross-engine passes into GitHub CI by looking like the thing a workflow ought to call. There is no back-compat alias: npm run ci now fails with npm's missing-script error, which prints the available scripts.
That is the readable half, and prose rots. The enforced half is scripts/lib/workflow-gate.mjs, run by npm run test:scripts over every file under .github/workflows/. It fails the suite if a workflow invokes a local:* script, a non-Chromium engine pass (smoke:web:firefox, smoke:web:webkit), or smoke:web:engine (whose engine comes from the environment and so cannot be read off the workflow at all), or sets SMOKE_BROWSER to anything but a literal chromium (including setting it to nothing, which is present-and-empty rather than unset, and is rejected rather than defaulted). That last rule respects where it is written — an env: entry whose key is SMOKE_BROWSER (case-insensitively, since Windows environment variables are), or an assignment in a run: script (POSIX, cmd, a >> $GITHUB_ENV append, or PowerShell — the default shell on a Windows runner) — so a script that merely prints the variable's name is not read as setting it. It treats any name it cannot read the same way — a workflow expression (smoke:web:${{ matrix.browser }}) or a shell variable the script expands itself (smoke:web:$ENGINE, %ENGINE%) — since a value the workflow can vary may well be firefox. It deliberately does not forbid npm run smoke, smoke:web:chromium, or smoke:tui — those belong in CI and are there today. It parses the workflow and descends the schema's executable paths — workflow env/defaults.run.shell, job env/container.env/with/defaults.run.shell, step run/shell/env/with — a custom shell: counts, since Actions runs it around the script — with aliases resolved — rather than matching text or key names, so it reads only what actually executes — so a step name explaining the rule is not mistaken for an invocation of it, and the workflow can still document itself.
| Script | What it does |
|---|---|
npm run validate |
Runs the four durable guards first — verify:format-coverage (every tracked source file is format-gated), verify:skills (every .claude/skills manifest parses and declares its invocation mode; the model-invoked ones also carry eval cases — the name-only skills intentionally have none), verify:typecheck-coverage (every tracked TS file lands in a tsconfig project), verify:dep-lockstep (no dependency reaching one tsc program from two installs skews across them) — then test:scripts (the guards' own parser unit tests), then validate:core (the shared core/ format:check + lint gate — including the core/react/** React-hooks block added by #2192, whose plugin is a root devDependency because a client-local copy is unreachable from the root: Node resolution walks up, not down), then per client: format:check + lint + typecheck (cli/tui/launcher; web typechecks via tsc -b inside its build) + build + fast unit tests. The quick inner-loop check. |
npm run coverage |
The per-file ≥90% gate (lines/statements/functions/branches) under v8 instrumentation, per client. CI-enforced. For web this also runs the integration project and covers the shared core/ runtime (including core/json and core/client). |
npm run smoke |
End-to-end smokes through the built launcher (--help dispatch + prod cli/tui/web — the TUI one runs under a pseudoterminal and asserts the Ink app is still running two seconds after its first frame, not merely that it painted one, #2147), plus three headless-browser smokes: a boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests), and an MCP Apps smoke (smoke:web:app) that drives connect → open app → data-app-status="ready" against a composable App server, covering the sandbox proxy and UI-protocol bridge, and an app-rendered elicitation smoke (smoke:web:elicit) that drives one end to end — call the tool, answer inside the sandboxed app, see the app's ElicitResult reach the server — and then the same tool against a server that never advertised the capability, which must fall back to the native elicitation form. Those three take their engine from SMOKE_BROWSER — see Supported browsers. A fourth, smoke:web:tabs, drives the core tabs against one server on one browser launch — Tools runs a tool and asserts its structuredContent section, Resources reads a resource with templates listed, Prompts fetches one (#2148) — asserting the data-* contract in the web README rather than visible copy. It is Chromium-only: those tabs are ordinary React and Mantine, so unlike the sandbox they are not engine-sensitive. |
npm run verify:build-gate |
Runs a real vite build with a Node built-in forced into the browser graph and asserts the build fails via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of npm run local:gate and of GitHub CI. |
npm run verify:bundle-externals |
Guards the must-not-bundle invariant (#2067): for each tsup-bundled client it reads the built build/ output and fails if any package that must stay external was inlined anyway. Candidates are the union of the client's own external array and the root manifest's dependencies — the latter because #2067 was a missing external entry, which a self-referential check would have passed. Detection is via esbuild's // <path> module banners, so it covers both shapes — a separate <pkg>-HASH.js chunk (what a dynamically import()ed CommonJS package produces) and a statically-imported package folded straight into index.js, which emits no chunk at all. A build with no banners fails as such rather than passing clean, so enabling minify cannot silently retire the check. undici was declared only in the root and clients/cli manifests, and tsup auto-externalizes only what the nearest manifest declares, so the web and TUI bundles inlined 1.05MB of it — and CommonJS inlined into an ESM bundle throws Dynamic require of "assert" is not supported on first use, from a specifier no user-side install can satisfy. Reads the output rather than the config because those two disagreed for four releases. Part of npm run local:gate and of GitHub CI. |
npm run verify:format-coverage |
Parses the format:check globs out of every package.json (only those reachable from validate), enumerates all tracked source files, and fails listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in validate. |
npm run verify:skills |
Guards .claude/skills (#2163). A skill's body loads only when it is invoked, so one that stops being reachable loses behavior silently — and the way that happens is frontmatter Claude Code cannot parse, which loads the body with an empty description: /skill-name still works, so a manual spot check passes, while nothing is left for the model to match against. This parses each SKILL.md the same way, requires an explicit invocation mode on every skill and committed eval cases (positives and negatives) on every model-invoked one, and fails when the skill listing exceeds its recorded character budget. Also runs claude plugin validate — the authoritative schema — when the installed CLI is exactly the pinned version, and otherwise says so and moves on. It runs inside validate, which local:gate executes before the pinned step, so accepting any other local CLI would let it reject skills that CI accepts and exit the gate before the reproducible check ever ran. Runs in validate; because that hand-off is best-effort, the authoritative validator also gets a guaranteed step of its own — see verify:skills:cli below. |
npm run verify:skills:cli |
Runs claude plugin validate — the authoritative skill schema — as a guaranteed step, in local:gate and in CI. verify:skills reimplements the parse and skips when the CLI is absent, which is right for validate (fast, offline, no Claude Code required) but adds up to "never runs". This one resolves the CLI instead: an installed one only when it matches the pin exactly, otherwise the pinned package via npx -y @anthropic-ai/claude-code@…. Exact, not a floor: accepting a newer local CLI would validate against a different schema than CI's, so the same local:gate could disagree across machines — which is the failure a pin exists to prevent. Needs no authentication. Not in validate — it can reach the network. |
npm run test:scripts |
Table-driven unit tests (node --test) for the guard's own pure parsers (scripts/lib/npm-scripts.mjs, scripts/lib/tsc-program.mjs + the exported helpers of verify-typecheck-coverage.mjs and verify-dep-lockstep.mjs), one case per rule they encode, plus two suites over shared scripts/lib helpers that no smoke can check itself: resolve-node-bin.test.mjs — the cross-platform bin resolver (#1939), pinned against the real bin/exports shapes of the packages the scripts actually spawn — and announced-child.test.mjs — the spawn/readiness ownership helper (#2000), which drives real node -e children to prove a child that never announces is still published to the caller before the timeout throws, and so is reachable by teardown rather than orphaned. Four more do the same: mcp-app-flow.test.mjs covers the shared MCP Apps flow (#2003) — the deep link's two CSRF gates and appArgs encoding, plus driveAppFlow's failure branches against a stand-in page, all of which are dead code from the happy-path smokes' point of view and would otherwise surface only as opaque timeouts; render-smoke.test.mjs and pty.test.mjs cover the TUI boot harness (#2147) — the former driving real node -e stubs to prove that a child which paints the marker and then dies is a failure, which the old harness reported as OK and which no fixed TUI can reproduce; the latter pinning the three mutually-incompatible script(1) invocations, where a wrong guess stops the pseudoterminal from being allocated at all; and ensure-test-servers.test.mjs pins the #2111 invariant — that test-servers/build is rebuilt even when it already exists — which no smoke can assert about itself, since one driving a stale fixture reports a product failure rather than a staleness one. workflow-gate.test.mjs is a different shape again (#2146): besides the table of parser cases it runs the parser over the repo's real .github/workflows/**, so a workflow that invokes the local-only gate or a non-Chromium engine pass fails here rather than in a CI run nobody expected to be red. Five more arrive with the skills (#2163). skill-manifest.test.mjs pins the frontmatter contract one case per rule — most importantly that a fence which is not a whole line is not a terminator, and that malformed YAML is an error rather than an empty description. verify-skills.main.test.mjs drives that guard's main() over fixture directories, the sibling guards' *.main.test.mjs pattern, so it cannot stop enforcing while the parser tests stay green. skill-eval.test.mjs covers the trigger eval's stream parsing and its run classification, which an eval run's happy path never reaches and which would otherwise report a plausible hit rate for a CLI that failed to run. verify-skills-cli.test.mjs covers which validator gets run — a local CLI only at the exact pin — and the orchestration around it, neither of which an ordinary run on a working machine exercises. claude-cli.test.mjs covers the one place that decides how the CLI is spawned — the Windows .cmd shim needs a shell, and its arguments then need quoting — with the platform injected, since a win32-only bug that no runner here executes is exactly the kind that ships. Runs in validate — and verify:typecheck-coverage guards this gate in turn (reachable from validate, non-empty test set, every test file matched by the test:scripts glob), since node --test silently skips a file its glob misses and still exits 0. |
npm run verify:typecheck-coverage |
The typecheck-coverage analog of the above (#1791): for each Node client (auto-discovered from disk — enrolled via its typecheck script's projects, or for a tsc -b client like clients/web via its tsconfig.json references) it runs those projects with tsc --listFilesOnly, unions them, and fails listing any tracked .ts/.tsx/.mts/.cts under the client that lands in no project (so a new top-level config/helper can't silently go untypechecked). It also requires, deny-by-default, the first-party TS no client owns (test-servers/src, the root vitest.shared.mts, all of core/, and any new top-level location) to land in some client project's tsc pass — so a core *.tsx web's projects don't reach is caught too. Also asserts the gate is wired (each client's typecheck pass — its typecheck script, or web's tsc -b — is reachable from its validate, and the root chain runs each client's validate). Runs in validate. |
npm run verify:dep-lockstep |
Guards the "one version per install-crossing dependency" invariant (#1896). v2 is not a workspace, so a client's test project compiles the shared first-party TypeScript — core/, test-servers/src, and the root-owned vitest.shared.mts, all of which resolve their dependencies from the root install — alongside the client's own sources, putting the same package in one tsc program twice. At the same version that's harmless; skewed, TypeScript must relate two structurally-distinct copies of every type, which for a recursive-generic surface is exponential (zod 4.3.6 vs 4.4.3 exhausted the 4GB tsc heap in clients/web). Derives its candidate set from what actually enters each program (#1965) — every client tsconfig project listed with tsc --listFilesOnly via the shared scripts/lib/tsc-program.mjs, each resolved node_modules file mapped to its owning install, keeping the packages that reach one program from two installs (a package whose declarations arrive only through another package's .d.ts, as @modelcontextprotocol/sdk's do, is invisible to a scan of first-party imports). Prices each copy from the lockfile entry for the exact install path the program resolved, compares only the installs that met in one program, and fails deny-by-default on any disagreement not in the annotated TOLERATED_SKEW allowlist — empty today — with an allowlisted package tolerated only within a major version. Runs in validate. |
npm run local:gate |
Mandatory pre-push command. validate → verify:skills:cli → coverage → verify:build-gate → verify:bundle-externals → smoke → smoke:web:firefox → local:storybook. A strict superset of GitHub CI — see Two tiers. Named local: rather than ci on purpose (#2146); there is no npm run ci alias. |
npm run pack:verify |
Publish smoke — see Publishing. |
Per-client scripts exist too (validate:web, coverage:cli, smoke:tui, …), plus root validate:core / format:core for the shared core/ package, format:scripts for the root scripts/ tooling, and format:shared / lint:shared for the root "shared" surface (test-servers/src/**, vitest.shared.mts, the root eslint.config.js). Run npm run format before committing — the root format fixes core/, the root scripts/, the shared surface, and every client; validate runs the non-fixing format:check and fails CI on any unformatted file.
Linting is type-aware. All five ESLint scopes (clients/{web,cli,tui,launcher} plus the root core/ + shared gate) enable @typescript-eslint/no-floating-promises at error, so a promise that is neither awaited, returned, .catch(…)-terminated, nor explicitly discarded with void fails lint — and therefore validate (#1959). The rule needs type information, so each scope's config names a parser project; the root scope's is tsconfig.lint.json, a lint-only project covering core/**, test-servers/src/**, and vitest.shared.mts, which have no tsconfig of their own. It emits nothing and changes no typecheck — but a new first-party TS location added to the root lint scope must be added to its include. See TypeScript instructions in AGENTS.md for when void is acceptable.
The React hook rules cover core/react/ too. Every copy of eslint-plugin-react-hooks used to be client-local — a devDependency of clients/web and of clients/tui, each reachable only from that client's own eslint . — so the whole hook rule set, set-state-in-effect included, had never looked at core/react/, a directory of nothing but React hooks (#2192). "Neither surface has JSX" was the stated reason and it was never a good one: the rules judge hooks, not JSX. The plugin is now a root devDependency as well, applied by a core/react/** block in the root eslint.config.js — scoped there rather than to all of core/**, since the rules key off the use-prefix convention and a non-React useFoo helper elsewhere would be judged as a hook it is not.
And lint has no warning tier. Every lint script runs with --max-warnings 0, so a warning fails validate exactly as an error does (#2085) — a warn-level react-hooks/exhaustive-deps finding had otherwise let a stale-closure bug pass the mandatory pre-push gate and reach review. Fix the finding rather than silencing it; if a rule genuinely must be waived, use its inline disable comment with a one-line justification.
For the full testing rules — the ≥90% per-file gate, where test files live, the unit vs. integration vs. storybook projects, and the v8 ignore policy — see AGENTS.md.
The three headless web smokes — smoke:web:browser, smoke:web:app, smoke:web:elicit — take their browser engine from SMOKE_BROWSER, which accepts chromium, firefox and webkit (#2086):
SMOKE_BROWSER=webkit npm run smoke:web:app # one smoke, one engine
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes, one engineUnset, the engine is chromium, so npm run smoke is unchanged. npm run local:gate — the mandatory pre-push gate — additionally runs all three smokes under Firefox via smoke:web:firefox; GitHub CI does not. An unrecognized SMOKE_BROWSER is an error, not a fallback: a silent fallback would report a green Chromium run for a command that asked for webkit. A missing browser binary fails naming the engine and a remedy that works from where you are — npm run smoke:web:<engine>, or npx playwright install --with-deps <engine> run from clients/web, where Playwright is pinned.
Firefox passes all three smokes. WebKit fails the two App smokes, for reasons nobody has identified. Two things are known: it does not reproduce in real Safari (an MCP App opens there normally), and an isolated repro of the mechanism it was first blamed on did not reproduce it under Playwright's WebKit either. So it reads as a property of that particular build rather than a bug users hit, and chasing it further was judged not worth the effort — treat a WebKit failure as unexplained rather than as a defect until someone has looked.
Why Firefox is in the pre-push gate rather than in CI. A GitHub Actions job was trialled and was cheap — about two minutes, in parallel with the 15-minute build job, so no added wall-clock. It was dropped anyway, on the honest count: across a dozen runs it never once disagreed with Chromium, so it spent runner minutes on every push from every branch to re-confirm a result already in hand, and carried a real flake surface (playwright install --with-deps runs apt-get update, which fails whenever a third-party repo in the runner image breaks).
Putting it in npm run local:gate instead keeps the check where it is worth most — in front of a human about to push a change they can still reason about — and pays for it once per push rather than once per CI run. If a cross-engine regression ever reaches v2/main because someone skipped the gate, that is the evidence for restoring the CI job; it is a one-job diff.
Why these smokes specifically. Most of the web client's behavior is React and Mantine, where a second engine buys little. The MCP Apps sandbox is the exception — it is built out of the primitives that genuinely diverge between engines: a CSP <meta> injected as the first <head> child of a srcdoc document, a nested sandboxed iframe, a Permissions-Policy allow attribute, and postMessage origin discipline across those two frames. Nothing else covers that: sandbox-csp.test.ts asserts which policy string is built, which passes identically on an engine that ignores <meta> CSP entirely, and no Storybook story reaches the sandbox at all (all three App stories point the iframe at a data: placeholder and hand the renderer a mock bridge). Storybook itself remains Chromium-only — broadening it covers a much larger and differently-shaped surface, and is a separate decision.
⚠️ Playwright's WebKit is a WebKit build, not Safari. It is close enough to catch engine-level CSP and iframe divergence, and not close enough to certify Safari specifically. A green run here is not a Safari guarantee.
pack:verify stays Chromium-only on purpose — it is a packaging check, and the engine question belongs where the sandbox is under test.