Skip to content

fix: reject unknown tool parameters instead of silently dropping them - #11

Merged
noneck merged 3 commits into
mainfrom
fix/strict-tool-schemas
Jul 22, 2026
Merged

fix: reject unknown tool parameters instead of silently dropping them#11
noneck merged 3 commits into
mainfrom
fix/strict-tool-schemas

Conversation

@noneck

@noneck noneck commented Jul 21, 2026

Copy link
Copy Markdown
Member

DO NOT MERGE BEFORE 2026-07-27 — live demos run on this server until then.

The defect

zod strips unknown keys by default. A tool call carrying a parameter the tool does not declare parsed cleanly with that parameter removed, and the handler returned real, correctly formatted, correctly sourced data — just not the data that was asked for. Nothing in the response signalled the drop, so a consuming model could not detect it.

get_open_solicitations(limit=1, bogus_unknown_param="SHOULD_REJECT")
  → a normal solicitation record

Agents guess parameter names. That is normal. The correct response is to reject the guess loudly, not to answer a different question.

The fix — two layers

1. additionalProperties: false on every advertised inputSchema. This is the layer that actually prevents the bad call: it tells the calling model, before it calls, that an invented parameter is invalid. Verified end to end over stdio — tools/list returns additionalProperties: false on all seven tools after SDK serialization.

2. .strict() on every argument schema. Server-side enforcement, so anything slipping past the advertised contract raises rather than being silently stripped.

The error names the right parameter. zod's default strict message names the bad key but offers no alternative. parseToolArgs catches the unrecognized_keys issue and reports the accepted names, read back off that tool's advertised schema rather than a hand-maintained table:

get_open_solicitations received unrecognized parameter(s): bogus_unknown_param.
It accepts only: limit. The call was rejected rather than run without that
filter, because dropping it silently would return results for a different question.

search_notices received unrecognized parameter(s): council_district.
It accepts only: query, limit. ...

This follows the spirit of VENDOR_NAME_UNSUPPORTED_MESSAGE in nyc-checkbook-mcp: name the limitation, then name the supported alternatives, in plain prose.

Structural change

src/index.ts connects a stdio transport at import time, so nothing in it can be imported by a test without starting a server. The tool array and the per-tool zod schemas move to src/tools.ts (first commit, no behavior change) so both are testable offline. index.ts shrinks: each dispatch case is now a single parseToolArgs(...) call with type inference preserved.

Test plan

New test/strict-schema.test.mjs, run offline — it exercises the argument-validation layer where the drop happened and never reaches the Socrata City Record API.

  • Schema-level, durable across tools added later: every tool sets additionalProperties: false, and TOOLS and SCHEMAS cover the same tool names.
  • Behavioral: a call carrying bogus_unknown_param raises, for every one of the seven tools.
  • Message quality: the rejection names both the bad key and the accepted parameters.
  • Regression guards: valid calls still parse (including undefined args), and an ordinary bad value for a declared parameter still reports its own message rather than the unknown-key one.

Before the fix (extraction commit built and run, so the failure is real, not a missing import):

✔ every advertised tool has an argument schema, and vice versa
✔ valid calls still parse
✔ a bad value for a declared parameter still reports normally
✖ every advertised tool sets additionalProperties: false
  actual: undefined  expected: false
✖ a call carrying an undeclared parameter is rejected
  Missing expected exception: an undeclared parameter must raise, not be silently dropped
✖ the rejection names the bad key and the accepted parameters
✖ every tool rejects an undeclared parameter
  Missing expected exception: search_notices accepted an undeclared parameter

Direct reproduction against that same build:

BEFORE — silent drop reproduction:
{"limit":1}                       ← bogus_unknown_param gone, no error
advertised additionalProperties: undefined

After the fix, clean build (rm -rf dist && npm test), Node 26.5.0:

ℹ tests 15
ℹ pass 15
ℹ fail 0

All eight pre-existing SoQL and encoding tests still pass. No new dependencies; no parameter renamed or removed.

Refs #10

noneck and others added 2 commits July 21, 2026 12:19
…s.ts

No behavior change. src/index.ts connects a stdio transport at import time,
so nothing in it can be imported by a test without starting a server. Moving
the tool array and the per-tool zod schemas into their own module makes both
testable offline, which the strict-parameter fix (#10) needs.

Refs #10

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zod strips unknown keys by default, so a tool call carrying an invented
filter parsed cleanly with the filter removed and returned real, correctly
formatted, unfiltered data. Nothing in the response signalled the drop, so a
consuming model could not detect it. In the sibling budget repo the same
defect returned $47.5M of citywide awards for a question scoped to one
council district.

Two layers:

- `additionalProperties: false` on every advertised inputSchema, so the
  calling model knows an invented parameter is invalid before it calls.
- `.strict()` on every argument schema, so anything that slips past the
  advertised contract raises server-side.

The error names the offending key and the parameters that tool does accept,
read back off the advertised schema rather than a hand-maintained table.

Refs #10

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Minor rather than patch, matching the fleet-wide decision: tool-call behavior
visibly changes for any caller that was passing an undeclared parameter. No
declared parameter was renamed or removed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@noneck
noneck merged commit 6e79eff into main Jul 22, 2026
2 checks passed
@noneck
noneck deleted the fix/strict-tool-schemas branch July 22, 2026 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant