[lucasbraum]

UPT-114, following ADR 0017: the behavior-record format as shared code, and the first record, for zod 3 → 4 behind the MCP SDK.

What changes

• Behavior records (packs/records.ts). A record is a silent change with a cited source and a trigger that finds sites with the type checker (ADR §2). UPT-113 reuses the format for Stripe.
  • sources. Each has a URL, a read date, a verbatim quote, and a checked-in snapshot of the page with its sha256. recordProblems() refuses a quote that isn't in its snapshot, a snapshot whose hash changed, or a non-exact keepOld with no alsoChanges. The pack contract test and pack test both run it.
  • trigger. flows-to-consumer (packs/flows-to-consumer.ts): a value of the package's type passed to another package's function. Receiver type, method, argument and property are data. The consumer's installed version must be in the record's range.
  • effect, keepOld (edit, exact, alsoChanges), adopt.
  • Findings. A match is a finding of its own rule, the record id. It is unverified, never breaking, and lists every quote and URL under the site.
• The zod record zod/mcp-input-schema-additional-properties (packs/zod/records.json, packs/zod/sources/).
  • What it matches. A zod 3 raw shape, or a plain (strip) z.object, passed as inputSchema to McpServer#registerTool, or as a schema argument to #tool, with @modelcontextprotocol/sdk >=1.23.0 <2.
  • Not a site. Strict, passthrough and catchall objects (they advertise the same thing on both majors), anything the checker sees as any, another type's registerTool, and a workspace that imports zod/v3.
  • Sources.
    • zod's JSON Schema docs at cc4cd4e: "When converting to JSON Schema in "input" mode, additionalProperties is not set."
    • The SDK 1.30.1 converter: io: opts?.pipeStrategy ?? 'input'.
    • zod's docs again, for the edit: "z.strictObject() will always set additionalProperties: false".
  • SDK range checked. I read the SDK tarballs: 1.22.0 has no zod 4 support, 1.23.0 is the first release with the zod-4 branch, and the converter is byte-identical in 1.30.1 and 1.32.1.
  • Real repository. On calmcp at 741de4e it matches exactly the five lines the human changed: src/tools/index.ts:69, 83, 102, 119, 148.
• fix.
  • Agent prompt. The prompt gets a Behavior changes at these sites block with each site, the quotes, the edit and what else it changes. It asks the agent to apply the edit, or to say in decisions why not, naming the record id.
  • After the session. The record is re-matched on the branch at the target to see which sites the edit reached.
  • PR body.
    • "Decisions for you" gets one line per record: the quotes and links, where the edit was applied, where it was not, and what else it changes.
    • Risk reads "behavior changes", not "unverified sites".
    • An agent decision that restates the record is not repeated.
• CHANGELOG.md as context (ADR §2, the generic tier).
  • Where it's read. From the target copy the install just put in node_modules, so there's no new network use (only the target is extracted, and its changelog covers the range).
  • What the agent gets. Only lines from releases after the installed version up to the target that do both:
    • name something the files at the reported sites import from the package;
    • say default, no longer, now, instead of or silently.
  • Never findings. These lines are context only.
  • No effect on calmcp. zod 3.25.76 and 4.6.5 ship no CHANGELOG.md, so this adds nothing here.
• Measurement.
  • Ground truth. It gains silent[{ file, line, record, kind: 'required' | 'product-choice' }]. zod's ground truth gains consetto/calmcp: 9 breaking/deprecated findings from bdaef39 and tsc, and the 5 required silent sites.
  • pack test. It scores silent changes caught apart from breaking, overall and behavior notes. A record site on a product-choice line, or on any line the truth doesn't name, is a false positive. Any silent FP fails pack test (the zero-FP gate).
  • verification.json. It gets silent only for a pack with records or silent truth, so no other pack's record goes stale.
• Bench scorer. The columns are on bench: UpgradeBench v0 pilot harness (UPT-24) #169's branch, appended as 339e306 (fast-forward, nothing rewritten).
  • Silent truth. selection.json gains calmcp's five silent sites.
  • Caught. score.mjs counts a site as caught when the diff touches its line and the head file has the keep edit, or when the PR body names the record.
  • Silent FP. A site that a Behavior change (<record>) line names and the selection doesn't is a silent false positive.

Decision for you: .strict() also rejects unknown keys

The edit that keeps zod 3's advertised schema is .strict() (z.object(shape).strict() for a raw shape). It isn't exact. On both majors calmcp's server silently dropped an unknown key; with .strict() the SDK rejects the call with InvalidParams, naming the key. The human chose that too (their test names an invented parameter instead of dropping it). It's also a behavior change neither version had, so the record is exact: false. Its PR line always says what else the edit changes, and offers dropping it, which keeps additionalProperties unset and unknown keys silently dropped.

Should the record keep recommending .strict(), or describe both options without recommending one?

Measured

pack test zod, all 7 repositories, run fresh:

	main	this branch
breaking	126 TP · 0 FP · 33 FN	131 TP · 0 FP · 33 FN
silent changes caught	—	5 TP · 0 FP · 0 FN
status	verified	verified

calmcp alone: predicted 9, expected 9, 0 FP, 0 FN. Every other repository's numbers are unchanged.

Other verified packs. pack test on all 12 others: ai, express, lucide-react, next, react-router, react, redis, @sentry/browser, @sentry/nextjs, stripe, typescript, vitest.

• Result. All pass and stay verified, with 0 breaking FP each.
• Behavior-note FP. Equal to their records: redis 2, typescript 8.
• Stripe. It reports stale only because two keys under mechanical come out in a different order. The values are identical, and stripe is untouched here.

Tests. core: 2321 passed. cli: 536 passed. biome: clean apart from one existing warning.

UpgradeBench calmcp rerun. One run, with the #169 harness (bd96018 plus the scorer commit).

• Setup.
  • Uptide arm with this branch's packed CLI (pnpm pack of e727cb4, run as npx -y file:uptide-0.16.1.tgz).
  • Model claude-sonnet-5-5, effort medium, default --max-cost of $1, an empty cache.
  • Human reference: round 3's run.

	round 4 (0.16.0)	this run
.strict() applied	no	yes, all 5 sites (z.object(shape).strict() at each inputSchema)
silent changes caught	0/5	5/5 (by edit) · silent FP 0
coverage of the human's hunks	6/27	13/27 · 0 unrelated · no regressions
cost	$0.084	$0.135 (the agent estimated $0.60 against the $1 cap)
wall time	62 s	91 s, including the first-run Agent SDK download
local checks	all ok	install, typecheck, lint, test, build all ok
fork CI	3 pass, 3 fail	3 pass, 3 fail, the same three

• Fork PR. uptide-bench/consetto--calmcp#13.
• The three CI failures match round 4's PR fix: align package versions with npm and guard releases #10, and none comes from the upgrade:
  • build (22) and build (24) fail at npm audit --omit=dev --audit-level=high on an MCP SDK advisory (GHSA-6qxp-vccf-f47h) in the parent's lockfile.
  • dependency-review can't run because the fork has no dependency graph.
• The run's PR body has the record's line under "Decisions for you", with the three quotes and links and the five applied sites. It also says the edit makes the server reject unknown keys.
• Fixed after the run. It showed three presentation problems, fixed in 53ca020, which changes only PR wording and was not rerun:
  • risk read "High: unverified sites";
  • the sites were listed bottom-up;
  • the agent's own decision restated the record.

Not in this PR

• Check output is UPT-113's. check still prints record sites as "+5 possible impact … not counted as breaking", and --details lists them under UNVERIFIED with the quotes. A Behavior cell and a headline that count matched records are UPT-113 item 5.
• The optional probe (UPT-114 item 3) is skipped: serializing sliced schemas through the SDK's converter on both versions.
• Kept is judged by file and snippet. An edit that rewrites a site's line without making the object strict, while another site in the same file still matches, would read as kept. It's marked ponytail: in fix/run.ts.
• The bench's run.sh got a local, uncommitted UPTIDE_SPEC hook, as in the UPT-108 round, to run the packed tarball through npx.

🤖 Generated with Claude Code