fix: reject unknown tool parameters instead of silently dropping them (#19) - #20
Conversation
…#19) zod strips unknown keys by default, so an undeclared parameter vanished with no error and the tool returned unfiltered results — real, correctly formatted, correctly sourced data answering a different question, with nothing in the response for a calling model to detect. search_contracts(vendor="Community League of the Heights", page_size=3) returned 5,755,099 unrelated contract records. `vendor_name` is the declared parameter and passing THAT correctly already throws VENDOR_NAME_UNSUPPORTED_MESSAGE with three alternatives — but `vendor` is undeclared, so zod stripped it and the guard never fired. A one-word typo defeated a deliberate guard. Each tool's inputSchema becomes a .strict() ZodObject rather than a raw shape. The SDK's getZodSchemaObject returns a schema instance unchanged and only wraps bare raw shapes, so this is supported; verified against @modelcontextprotocol/sdk 1.29.0. search_contracts additionally maps the observed guess `vendor` to `vendor_name` via a per-schema errorMap, so the caller lands on the same three alternatives instead of 5.7M rows. Note: tools/list already emitted additionalProperties:false under SDK 1.29.0 — the advertised contract was honest and the server contradicted it. A test now pins it so an SDK change cannot drop it silently. Tests: 40 pass / 0 fail, fixture-backed, zero network. Refs #19 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ests Minor rather than patch, matching the fleet-wide decision: tool-call behavior visibly changes for any caller that was passing an undeclared parameter. No declared parameter was renamed or removed. Also unquotes the `node --test` glob. Quoted, node does the matching itself, and Node 20 does not support glob patterns there at all (added in 22) — it treats the pattern as a literal path and runs ZERO tests. Unquoted, the shell expands it and node receives literal file paths on every supported version. Confirmed on Node 20 in nyc-record-mcp CI, which is where this was caught. Note this repo's CI is still build-only, so these 40 tests do not yet run there. Tracked separately. Verified locally: 40 tests, 40 passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Version collision to watch with #18. This PR is now versioned 1.4.0 per the fleet-wide minor-bump decision (2026-07-21). #18 targets 1.3.2. If this merges first, #18 needs re-versioning to 1.4.1 or 1.5.0 — shipping 1.3.2 after 1.4.0 would be a downgrade. If #18 merges first, this is unaffected. Also folded in here: the
|
…/ changes The Test step here was not build-only — correcting an earlier claim of mine. Bare `node --test` discovers test files by walking the directory, which has worked since Node 18, so all 40 tests were already running on both Node 20 and 22. The gap is subtler: CI ran `node --test` directly while developers run `npm test`. Those are different commands, and that divergence is precisely why CI stayed green while `npm test` was broken on Node 20 — its quoted glob is not a supported `node --test` argument before Node 22, so it matched nothing and ran zero tests. CI cannot vouch for a command it does not run. Now it runs the same one. The separate build step goes away because `npm test` compiles first; keeping it would only run tsc twice. Also adds `test/**` to the path filters. The tests and fixtures are .mjs/.xml, so `**/*.ts` does not match them and a test-only PR would have skipped this gate without saying so. Job renamed Build -> Build & test so the check name stops understating what it does; main has no branch protection, so no required-check name is affected. Verified locally: 40 tests, 40 passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Correction to my earlier comment on this PR. I said this repo's CI was build-only and that the 40 tests did not run. That was wrong. The workflow already had a Bare The real gap, now closed: CI ran Apologies for the noise. Reading a check name instead of the workflow file is the same mistake as trusting a plausible response instead of reading the schema, which is the bug this PR exists to fix. |
….5.0 main moved to 1.4.0 when #20 (strict tool schemas) merged, so this branch's 1.3.2 would have shipped as a downgrade. Re-versioned to 1.5.0 — minor, since this adds filters — and the CHANGELOG entry moved above 1.4.0 with its "do not tag/publish until operator live-verification" warning intact. src/tools.ts auto-merged, but textual success is not semantic success here, so the seam was checked rather than assumed: #20 wrapped every inputSchema in `strictSchema(...)` and this branch adds seven keys to search_contracts. All seven landed INSIDE the strict object, verified against a real tools/list response — 27 declared params, additionalProperties: false. That interaction matters more than it looks. Under #20 an undeclared parameter is rejected outright, so a filter added outside the strict wrapper would be unusable the moment it shipped and its fail-fast gate would never be reached. Added a test asserting every CONTRACT_UNVERIFIED_FILTER_KEYS entry is declared, so the seam cannot regress silently. Also updated stale 1.3.2 references in the CHANGELOG compare link and a test section header. 50 tests pass. No live API call was made — the gate keeps all five filters disabled by default, and the operator live-verification requirement is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DO NOT MERGE BEFORE 2026-07-27 — live demos run on this server until then.
Gonzalez (Wed) and Maloney (Mon) run against these MCPs. Strict schemas turn silently-succeeding calls into throwing ones, which is the point, but it is not a change to land underneath a live meeting. Merge window opens Monday 2026-07-27.
The bug
zod strips unknown keys by default. An undeclared parameter vanished with no error and the tool returned unfiltered results — real, correctly formatted, correctly sourced data answering a different question, with nothing in the response for a calling model to detect.
vendor_nameis the declared parameter, and passing that correctly already throwsVENDOR_NAME_UNSUPPORTED_MESSAGEnaming three alternatives. Butvendoris undeclared, so zod stripped it and the guard never fired. A one-word typo defeated a deliberate guard.The fix — two layers
(a) Advertised contract. Worth stating plainly, because it corrects the issue's premise for this repo:
tools/listalready emittedadditionalProperties: falsefor all ten tools under@modelcontextprotocol/sdk1.29.0 —zod-to-json-schemaemits it for a plain strip-modez.objecttoo. So the advertised contract was already honest and the server was contradicting it, which is the worse of the two failure shapes..strict()preserves that output, andtest/strict-schema.test.mjsnow pins it so an SDK bump inside the^1.0.0range cannot drop it silently.(b) Server-side enforcement. This is the layer that was actually missing. An unknown key now raises
Input validation errorbefore the handler runs, upstream request included — the API is never reached.(c) Alias hint.
search_contractsmaps the observed guessvendor→vendor_nameand surfacesVENDOR_NAME_UNSUPPORTED_MESSAGE, so a caller who typos into the undeclared name lands on the same three alternatives rather than 5.7M rows:Only
search_contractscarries an alias. Every other tool gets plain.strict()— zod's default already names the offending key loudly, and speculative alias entries are not worth the surface.The raw-shape → ZodObject change
This repo is the odd one out in the fleet: it uses
server.registerTool(...)fromMcpServerwith a zod raw shape asinputSchema, and you cannot call.strict()on a raw shape. So eachinputSchema: { ... }becomesinputSchema: strictSchema({ ... }), wherestrictSchemareturns a fullZodObjectwith.strict()and an optional per-schemaerrorMapfor aliases.Passing a
ZodObjectdirectly is supported:registerToolroutesinputSchemathroughgetZodSchemaObject(node_modules/@modelcontextprotocol/sdk/dist/esm/server/mcp.js:861), which returns a schema instance unchanged and only wraps bare raw shapes.tools/listseparately routes throughnormalizeObjectSchema, which likewise passes a_def-bearing schema through untouched. Verified empirically against SDK 1.29.0, not just read..default()values still apply through.strict()— a regression test assertsstatus/categorystill reach the request XML.Tests
New
test/strict-schema.test.mjs(5 tests), fixture-backed via a stubbedglobalThis.fetch, zero network calls to Checkbook NYC. New fixturetest/fixtures/contracts-response.xml.Before the fix (the two behavioral tests fail, which is the point):
After the fix, full suite (
npm test=npm run build && node --test "test/*.test.mjs", local build, not the npx cache):Not in this PR
[Unreleased].^3.23.0). No parameter renamed or removed./mcp-update-ed.Follow-up when this lands
Presenter notes in
BetaNYC/grounding-ai-with-ny-open-datadocument the current, broken behavior as verified. When this merges those notes go stale, and a stale presenter note is worse than none because someone will trust it in a meeting. Update the Reinvent Albany script's worked example in the same pass.Refs #19
🤖 Generated with Claude Code