atest runners unusable from published npm package: devDeps not shipped, codegen-compile not registered

[rfxlamia]

Problem

atest_run cannot successfully run either registered runner when agentpair is installed as a real consumer would (npx agentpair@x.y.z), because both runners depend on packages that only live in devDependencies:

• payload-size (packages/mcp-server/src/runners/payload-size.ts) requires json-schema-faker
• spectral (packages/mcp-server/src/runners/spectral.ts) requires @stoplight/spectral-cli

Both are listed only under devDependencies in packages/mcp-server/package.json — npm does not install devDependencies for consumers, so a fresh install has neither. Each runner fails with an explicit but user-facing error ("... runner unavailable: install X (dev dependency)") instead of working out of the box.

This was found by manually publishing 0.1.17 and inspecting the tarball / trying an atest scenario for a hackathon demo — no existing test exercises a runner against a fresh (non-monorepo) install, so CI didn't catch it.

Second, separate issue found in the same investigation

packages/mcp-server/src/runners/codegen-compile.ts — the ESP32 OpenAPI→C compile runner (quicktype + Docker + agentpair/runner-esp32 image) — is fully implemented with its own tests (codegen-compile.test.ts) but is never registered in packages/mcp-server/src/runners/registry.ts. Only payload-size and spectral are in the RUNNERS map. atest_run cannot reach codegen-compile at all, registered or not — its own dependency (quicktype) has the same devDependency-only problem as above, plus it requires Docker + a pre-built agentpair/runner-esp32 image at runtime, which is a much heavier install-time requirement than the other two.

codegen-compile is arguably the project's most representative acceptance-test scenario (matches ROADMAP's description "OpenAPI→C on ESP32" and the dedicated packages/runner-esp32 package) but is currently dead code from atest_run's perspective.

Suggested fix

• Move json-schema-faker to dependencies (lightweight, pure JS, safe default-on runner).
• Decide deliberately whether spectral and codegen-compile should also ship as real dependencies (heavier — CLI binary / Docker + image), or stay opt-in with a clear runtime error and installation docs. If opt-in, document it (M4.2 docs pass) rather than leaving the error message as the only explanation.
• Register codegen-compile in registry.ts once its dependency story is decided.
• Add a test/CI step that exercises atest_run against a runner using only dependencies (no monorepo devDependencies) to prevent this regressing again.

Not a SPEC gap

SPEC.md §9 deliberately treats runners as pluggable "named, versioned executables" and OPEN-1's resolution explicitly notes negotiation works with zero runner infrastructure. This is purely an mcp-server packaging bug, not a protocol design issue.

[rfxlamia]

v1.0 partial fix (hotfix, 2026-07-23)

Per ROADMAP cut line + judges review: payload-size slice only for v1.0.

Done in repo:

• json-schema-faker → production dependencies (ships with npm agentpair)
• scripts/verify-npm-consumer.sh + CI step (staged publish-shaped pack + consumer install smoke test)
• Runner error messages no longer say "dev dependency"; spectral/quicktype document opt-in project install
• User guide EN/ID: atest/1 runner deps paragraph

Remaining (deferred v1.1):

• Extract @agentpair/runners package + npx resolution design
• Register codegen-compile
• Spectral as shipped/opt-in dependency story

Tracked: #66

Leaving #62 open until v1.1 closes the full scope.