feat: add the missing tier-2 CI coverage gates (knowledge, docs, tests)

agent-docs/framework-dev.md

@@ -48,6 +48,12 @@ The scaffold is webjs's primary teaching surface for AI agents, so a new framewo
 
 **When you add or rename a `@webjsdev/core` or `@webjsdev/server` export, or add a routing convention file the router parses, update the manifest** the same way you write a test: add a demo pointer / `{ demoed: true }`, or an honest exemption. All three surfaces are gated (the convention stems are derived from `packages/server/src/router.js`, so a new `stem === '...'` branch auto-appears and must be classified).
 
+The scaffold gate is one of a FAMILY of tier-2 coverage gates that keep the framework's agent-facing surfaces from rotting behind a new feature. The others:
+
+- **Knowledge coverage** (`test/knowledge/knowledge-coverage.test.js`): reconciles the live `webjs check` `RULES` against the troubleshooting page + gotcha docs (a new rule must be explained in a symptom-keyed surface or exempted in `knowledge-coverage.json`), and asserts the AGENTS.md headings the MCP `init` primer sources (DERIVED from the `sectionByHeading(agents, /.../)` calls in `packages/mcp/src/mcp-docs.js`) still exist, so a heading rename cannot silently empty the primer.
+- **API docs + test coverage** (`test/api-coverage/api-coverage.test.js`): every agent-facing `@webjsdev/core` + `@webjsdev/server` export (NON-internal per the scaffold manifest, the single source of truth) must be referenced in the docs corpus (AGENTS.md + agent-docs + docs site) AND in a test. A new public export that ships undocumented or untested turns CI red.
+- **Types** (`test/types/dts-export-coverage.test.mjs`, `server-types.test.mjs`), **elision** (`packages/server/test/elision/lifecycle-coverage.test.js`), and **llms.txt** (`test/docs/llms.test.mjs`) round out the family. Each reads its live surface dynamically so it cannot go stale.
+
 ---
 
 ### Changelog: per-package, per-version, auto-generated

agent-docs/testing.md

@@ -299,6 +299,13 @@ not a test framework: each is a few lines over native `Request` / `Response`,
 and they reuse the REAL cookie / header names and the REAL wire serializer (so a
 test exercises the production contract, never a parallel fake).
 
+For a real-browser test that needs the app served to a live page (rather than a
+node `Response`), `createBrowserTestHandler(appDir)` from `@webjsdev/server`
+is the browser-side counterpart (it takes the app dir as a positional string):
+it stands up the same request pipeline behind a listener the web-test-runner
+harness drives, so a component test loads the app's actual routes and modules in
+Chromium.
+
 ```js
 import { createRequestHandler } from '@webjsdev/server';
 import { testRequest, invokeActionForTest, rawActionRequest, loginAndGetCookies, withSessionCookie }

docs/app/docs/troubleshooting/page.ts

@@ -81,6 +81,21 @@ export default function Troubleshooting() {
     <p><strong>Cause:</strong> a <code>webjs.dev.before</code> / <code>webjs.start.before</code> step (the scaffold ships <code>webjs db migrate</code>) exited non-zero. As of #550, <code>webjs dev</code> / <code>webjs start</code> run these one-shot steps to completion BEFORE serving (replacing the old <code>predev</code> / <code>prestart</code> npm hooks), and they ABORT the boot on a failure so the app never serves a stale client or schema. A bare <code>webjs dev</code> and <code>npm run dev</code> run the same steps, so this is not a "wrong command" problem.</p>
     <p><strong>Fix:</strong> run the failing command directly to read its real error (for the scaffold's steps, <code>webjs db generate</code> or <code>webjs db migrate</code>), fix the cause (a malformed <code>db/schema.server.ts</code>, an unreachable <code>DATABASE_URL</code>), then re-run. A local binary in a step (<code>drizzle-kit</code>, <code>tailwindcss</code>) resolves under a bare <code>webjs dev</code> the same way <code>npm run</code> resolves it (the ancestor <code>node_modules/.bin</code> dirs are on the step's PATH).</p>
 
+    <h2>A <code>'use server'</code> file's exports are not callable from a component</h2>
+    <p><strong>Symptom:</strong> you added <code>'use server'</code> to a plain <code>.ts</code> file and imported one of its functions into a component, but the call fails or <code>webjs check</code> flags <code>use-server-needs-extension</code>.</p>
+    <p><strong>Cause:</strong> the RPC boundary keys on the FILE EXTENSION, not the directive alone. A <code>'use server'</code> directive in a plain <code>.ts</code> file is a lint violation, because the file router only treats <code>.server.{js,ts}</code> files as the server boundary. Without the extension the source would also be served to the browser.</p>
+    <p><strong>Fix:</strong> rename the file to <code>*.server.ts</code>. The extension makes it server-only (source-protected) and the <code>'use server'</code> directive makes its exports RPC-callable. See <a href="/docs/server-actions">Server Actions</a>. The <code>use-server-needs-extension</code> check rule catches this, and <code>use-server-exports-callable</code> flags a <code>'use server'</code> file that exports no callable function (it registers nothing, so the RPC 404s silently).</p>
+
+    <h2>A configured server action file with more than one function is rejected</h2>
+    <p><strong>Symptom:</strong> <code>webjs check</code> flags <code>one-action-per-configured-file</code> on a <code>*.server.ts</code> that exports a config (<code>method</code>, <code>cache</code>, <code>validate</code>, <code>middleware</code>) alongside two callable functions.</p>
+    <p><strong>Cause:</strong> the HTTP-verb and caching config exports (<code>export const method</code>, <code>cache</code>, <code>tags</code>, <code>invalidates</code>, <code>validate</code>, <code>middleware</code>) attach to the ONE action in the file, so a second callable function makes the config ambiguous.</p>
+    <p><strong>Fix:</strong> keep one callable function per configured action file (the convention is one function per action file regardless). Split the second function into its own <code>*.server.ts</code>. See <a href="/docs/server-actions">Server Actions</a>. The <code>one-action-per-configured-file</code> check rule enforces this.</p>
+
+    <h2>A nested layout or page's <code>&lt;html&gt;</code> shell is dropped</h2>
+    <p><strong>Symptom:</strong> you wrote <code>&lt;!doctype&gt;</code> / <code>&lt;html&gt;</code> / <code>&lt;head&gt;</code> / <code>&lt;body&gt;</code> in a non-root layout or a page and the tags vanish from the output, or <code>webjs check</code> flags <code>shell-in-non-root-layout</code>.</p>
+    <p><strong>Cause:</strong> the framework auto-emits the document shell around the whole composition, so only the ROOT layout (<code>app/layout.ts</code> exactly) may write its own shell. A nested shell ends up inside the framework's <code>&lt;body&gt;</code> and the HTML parser drops the stray tags.</p>
+    <p><strong>Fix:</strong> remove the shell tags from the nested layout or page and return just the content. Move any <code>&lt;html lang&gt;</code> / <code>&lt;body class&gt;</code> customization to the root layout, which the framework respects and splices its required tags into. See <a href="/docs/routing">Routing</a>. The <code>shell-in-non-root-layout</code> check rule flags this, framework invariant 8.</p>
+
     <h2>Still stuck</h2>
     <p>The framework source is in <code>node_modules/@webjsdev/</code> with no build step, so what you read is what runs. Grep the relevant file (the SSR pipeline in <code>@webjsdev/server/src/ssr.js</code>, client hydration in <code>@webjsdev/core/src/render-client.js</code>, the convention rules in <code>@webjsdev/server/src/check.js</code>). Run <code>webjs check</code> to surface most of the issues above before they reach the browser, and run <code>webjs check --rules</code> to read what each rule enforces.</p>
   `;

test/api-coverage/api-coverage.json

@@ -0,0 +1,11 @@
+{
+  "$comment": "Exemptions for the API docs/test coverage gate (api-coverage.test.js). The gate reuses test/scaffolds/gallery-coverage.json for the INTERNAL classification (an internal export is exempt from docs + test coverage too), so this file only lists the few extra exemptions the scaffold manifest does not express. Keep each reason honest.",
+  "docsExempt": {
+    "cookieSession": "alias of cookieSessionStorage; documented under the canonical name (docs/app/docs/sessions)",
+    "storeSession": "alias of storeSessionStorage; documented under the canonical name (docs/app/docs/sessions)"
+  },
+  "testExempt": {
+    "cookieSession": "alias of cookieSessionStorage; exercised through the canonical name",
+    "storeSession": "alias of storeSessionStorage; exercised through the canonical name"
+  }
+}

test/api-coverage/api-coverage.test.js

@@ -0,0 +1,159 @@
+// API docs + test coverage gate (tier-2, un-skippable CI enforcement).
+//
+// Types are already gated (test/types/dts-export-coverage.test.mjs: every export
+// is typed) and demos are gated (the scaffold gate). This adds the two remaining
+// coverage dimensions for the public API surface:
+//
+//   1. DOCS coverage: every agent-facing @webjsdev/core + @webjsdev/server export
+//      is mentioned in a doc (AGENTS.md, agent-docs/*.md, or the docs site).
+//   2. TEST coverage: every agent-facing export is referenced by a test file.
+//
+// "Agent-facing" = NOT classified `internal:` in the scaffold manifest
+// (test/scaffolds/gallery-coverage.json), which is the single source of truth for
+// what is framework plumbing vs an API an app author writes. So an export marked
+// internal there is automatically exempt here too; this file only carries the few
+// extra exemptions (aliases documented/tested under a canonical name).
+//
+// A new public export that ships undocumented or untested turns CI red. The
+// corpus checks are word-boundary greps (a rare new name with zero doc/test
+// mention is the signal; common names trivially pass because they are everywhere).
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync, readdirSync } from 'node:fs';
+import { join, dirname, sep } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import * as core from '@webjsdev/core';
+import * as server from '@webjsdev/server';
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+const REPO = join(__dirname, '..', '..');
+const SCAFFOLD_MANIFEST = JSON.parse(readFileSync(join(REPO, 'test', 'scaffolds', 'gallery-coverage.json'), 'utf8'));
+const EXEMPT = JSON.parse(readFileSync(join(__dirname, 'api-coverage.json'), 'utf8'));
+
+function isInternal(name, section) {
+  const e = (SCAFFOLD_MANIFEST[section] || {})[name];
+  return !!(e && typeof e.exempt === 'string' && e.exempt.startsWith('internal:'));
+}
+
+function readAll(dir, exts, out) {
+  let entries;
+  try { entries = readdirSync(dir, { withFileTypes: true }); } catch { return out; }
+  for (const e of entries) {
+    if (e.name === 'node_modules' || e.name === '.git') continue;
+    const f = join(dir, e.name);
+    // Exclude the coverage gate's OWN files so an export name hardcoded in a
+    // counterfactual (e.g. 'html', 'signal') does not self-satisfy the corpus.
+    if (f.includes(`${sep}knowledge${sep}`) || f.includes(`${sep}api-coverage${sep}`)) continue;
+    if (e.isDirectory()) readAll(f, exts, out);
+    else if (exts.some((x) => e.name.endsWith(x))) { try { out.push(readFileSync(f, 'utf8')); } catch { /* skip */ } }
+  }
+  return out;
+}
+
+// Corpora: built once.
+const DOCS = [readFileSync(join(REPO, 'AGENTS.md'), 'utf8')]
+  .concat(readAll(join(REPO, 'agent-docs'), ['.md'], []))
+  .concat(readAll(join(REPO, 'docs', 'app', 'docs'), ['.ts'], []))
+  .join('\n');
+const TESTS = readAll(join(REPO, 'test'), ['.js', '.mjs', '.ts'], [])
+  .concat(readAll(join(REPO, 'packages', 'core', 'test'), ['.js', '.mjs'], []))
+  .concat(readAll(join(REPO, 'packages', 'server', 'test'), ['.js', '.mjs'], []))
+  .join('\n');
+
+// A whole-word match on the export name. This is a deliberately LIGHTWEIGHT
+// signal: it reliably catches a NEW export that appears nowhere in docs/tests
+// (the case the gate exists for), but it does NOT prove depth of coverage, and a
+// common-English-word export name (`cache`, `html`, `json`, `route`, `session`,
+// `headers`, `stream`, `render`) trivially passes because the word saturates the
+// corpora. Strengthening to an import-from-@webjsdev match was tried and rejected:
+// ~55 agent-facing exports are exercised transitively, via subpath imports, or in
+// the browser-test suite this corpus omits, so it produced a large false-positive
+// exempt list with no real gain. Depth of coverage is a code-review concern, the
+// same as with the scaffold gate's demo pointers.
+const hasWord = (corpus, name) => new RegExp(`\\b${name}\\b`).test(corpus);
+
+/**
+ * Pure reconciliation. For each live export not internal and not exempt, require
+ * `present(name)` to be true. Returns error strings.
+ */
+export function reconcileCoverage(liveNames, internalOf, exemptSet, present, kind) {
+  const errors = [];
+  for (const name of liveNames) {
+    if (internalOf(name)) continue;
+    if (exemptSet.has(name)) continue;
+    if (!present(name)) {
+      errors.push(`${kind} gap: agent-facing export "${name}" is not referenced in the ${kind} corpus; document/test it or exempt it (with a reason) in test/api-coverage/api-coverage.json.`);
+    }
+  }
+  return errors;
+}
+
+/**
+ * Exemptions that no longer name a live export. Validated against the UNION of
+ * all live export names (the exempt manifest is global, not per-surface), so a
+ * server-only alias is not flagged stale while auditing the core surface.
+ */
+export function staleExemptions(exemptNames, allLive) {
+  const live = new Set(allLive);
+  return [...exemptNames].filter((n) => !live.has(n)).map((n) => `stale exemption "${n}": no longer a live export, remove it from test/api-coverage/api-coverage.json.`);
+}
+
+const SURFACES = [
+  { label: 'core', names: Object.keys(core), section: 'exports' },
+  { label: 'server', names: Object.keys(server), section: 'serverExports' },
+];
+const ALL_LIVE = [...Object.keys(core), ...Object.keys(server)];
+
+test('every agent-facing export is documented (or exempt)', () => {
+  const exempt = new Set(Object.keys(EXEMPT.docsExempt || {}));
+  const errors = staleExemptions(exempt, ALL_LIVE);
+  for (const s of SURFACES) {
+    const e = reconcileCoverage(s.names, (n) => isInternal(n, s.section), exempt, (n) => hasWord(DOCS, n), 'docs');
+    const agentFacing = s.names.filter((n) => !isInternal(n, s.section));
+    console.log(`[api-coverage] ${s.label}: ${agentFacing.length} agent-facing exports, ${e.length} undocumented.`);
+    errors.push(...e);
+  }
+  assert.deepEqual(errors, [], `docs coverage gaps:\n  - ${errors.join('\n  - ')}`);
+});
+
+test('every agent-facing export is referenced by a test (or exempt)', () => {
+  const exempt = new Set(Object.keys(EXEMPT.testExempt || {}));
+  const errors = staleExemptions(exempt, ALL_LIVE);
+  for (const s of SURFACES) {
+    const e = reconcileCoverage(s.names, (n) => isInternal(n, s.section), exempt, (n) => hasWord(TESTS, n), 'test');
+    const agentFacing = s.names.filter((n) => !isInternal(n, s.section));
+    console.log(`[api-coverage] ${s.label}: ${agentFacing.length} agent-facing exports, ${e.length} untested.`);
+    errors.push(...e);
+  }
+  assert.deepEqual(errors, [], `test coverage gaps:\n  - ${errors.join('\n  - ')}`);
+});
+
+// --- counterfactuals ---
+
+test('reconcileCoverage FAILS on an undocumented agent-facing export', () => {
+  const errors = reconcileCoverage(['newThing'], () => false, new Set(), () => false, 'docs');
+  assert.equal(errors.length, 1);
+  assert.match(errors[0], /agent-facing export "newThing" is not referenced/);
+});
+
+test('reconcileCoverage SKIPS an internal export', () => {
+  const errors = reconcileCoverage(['plumbing'], (n) => n === 'plumbing', new Set(), () => false, 'docs');
+  assert.deepEqual(errors, []);
+});
+
+test('reconcileCoverage SKIPS an exempted export', () => {
+  const errors = reconcileCoverage(['aliasFn'], () => false, new Set(['aliasFn']), () => false, 'docs');
+  assert.deepEqual(errors, []);
+});
+
+test('staleExemptions FAILS on an exemption that is not a live export', () => {
+  const errors = staleExemptions(new Set(['gone']), ['html', 'signal']);
+  assert.equal(errors.length, 1);
+  assert.match(errors[0], /stale exemption "gone"/);
+});
+
+test('staleExemptions passes a server-only alias while auditing across surfaces', () => {
+  // cookieSession is a live server export; it must NOT be flagged stale.
+  const errors = staleExemptions(new Set(['cookieSession']), ['html', 'cookieSession']);
+  assert.deepEqual(errors, []);
+});

test/knowledge/knowledge-coverage.json

@@ -0,0 +1,10 @@
+{
+  "$comment": "Source of truth for agent-knowledge coverage. knowledge-coverage.test.js reconciles the LIVE webjs check RULES against the troubleshooting page + muscle-memory gotcha docs, and asserts the AGENTS.md headings the MCP init primer sources still exist. A new check rule that is neither explained in a symptom-keyed surface nor exempted here FAILS CI, and a renamed AGENTS.md anchor fails CI. Explained = the rule name appears in docs/app/docs/troubleshooting/page.ts or agent-docs/{nextjs,lit}-muscle-memory-gotchas.md. See agent-docs/framework-dev.md.",
+  "checkRulesExempt": {
+    "components-have-register": "basic setup requirement (call Class.register('tag')); framework invariant 3, not a distinctive error signature",
+    "tag-name-has-hyphen": "HTML-spec basic; framework invariant 3, no muscle-memory divergence to counter",
+    "no-duplicate-tag": "two components claim one tag name, a build-time collision surfaced by the check, not a runtime error signature",
+    "array-prop-uses-array-type": "shape hint (declare an array prop with Array not Object); the Array-vs-Object prop concept is taught in agent-docs/lit-muscle-memory-gotchas.md",
+    "no-scaffold-placeholder": "scaffold sentinel that fails check until example content is kept-or-pruned, not a code error signature"
+  }
+}