Repository navigation
test(mcp): Lint the metadata clients receive from the server - #2039
Merged
Merged
Conversation
Add a vitest linter for the initialize and tools/list surface in mcp-core, a structured-output guard for the mcp-core suite, and server.json checks in @sentry/mcp-server, so the MCP metadata we just added does not drift back. The guard mirrors the SDK's tools/call validation: a tool with an outputSchema must return structuredContent that matches it. Tools whose outputSchema PR is still open are listed as pending, and their entry fails once the schema lands, so the PR that lands it deletes the entry. Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
The metadata linter only sees listed tools, so deprecated aliases and the DSN-gated feedback tool lost their title-uniqueness check when it moved out of tools.test.ts. Check those definitions directly. Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
Drop the pending entries whose outputSchema PRs have merged, move the search_metrics, search_profiles and search_replays tests to structured search stubs, and mark the guard probe's deliberately invalid result as a type error now that handler results are typed from outputSchema.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dcramer
approved these changes
Nov 6, 2026
sentry-junior Bot
added a commit
that referenced
this pull request
Nov 6, 2026
Resolve server.metadata.test.ts add/add conflict after #2039 squash-merged: keep this branch's removal of search_docs from OUTPUT_SCHEMA_EXEMPT.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a linter so the MCP metadata added in the recent PRs can't drift back: server instructions, serverInfo fields, tool titles,
_meta,server.json, andoutputSchemaplus structured output. It runs as ordinary vitest and node tests in the existing per-package CI, so there's no new workflow.packages/mcp-core/src/server.metadata.test.tsbuilds the server with every skill and connects an in-memory client. It then checks whatinitializeandtools/listactually return, in both stable and experimental modes.nameis exactly "Sentry MCP"title,description,websiteUrlandicons(withmimeType) are presentinstructionsare non-empty and at mostMAX_SERVER_INSTRUCTIONS_LENGTH(1,500)^[A-Za-z0-9_.-]{1,128}$inputSchema.type === "object"execution.taskSupporttitlereadOnlyHint,destructiveHintandopenWorldHintare all set explicitlyMAX_TOOL_DESCRIPTION_LENGTH(2,048)_metakeys appear only on an allowlist, which today is justget_sentry_resource→anthropic/alwaysLoadOUTPUT_SCHEMA_EXEMPTis permanent. It covers tools that are markdown-only by policy, return doc text, images or attachments, pass through likeget_sentry_resource, or poll Seer.OUTPUT_SCHEMA_PENDINGlists tools still waiting on a schema. An entry fails once its tool gains a schema, so the PR that adds the schema also deletes the entry.outputSchemahas a colocated<name>*.test.tsthat reads itsstructuredContent.test-utils/structured-output-guard.tsis a vitest setup file that wrapsdefineToolfor the whole mcp-core suite. It mirrors the SDK'stools/callvalidation. If a tool has anoutputSchemaand its result isn'tisError, it must returnstructuredContentthat parses against that schema. That turns every existing fixture and mock test into a structured-output check.test-utils/search-events-stub.tsholds schema-valid results that dataset search tests return from their mockedrunSearchEvents:SEARCH_EVENTS_STUB, plusSEARCH_REPLAYS_STUBfor search_replays.tools.test.tskeeps the definition-level checks the wire linter can't do:submit_feedback_to_sentry_teamThe description cap is now one shared constant.
packages/mcp-server/test/server-json.test.mjschecksserver.json:repository.urlis getsentry/toolkittitle,websiteUrl, andiconswithmimeTypeare presentSENTRY_*vars the bundledmcp --helplists.SENTRY_AUTH_TOKENmust be optional and secret, andSENTRY_HOSTandSENTRY_URLoptional and not secret.It doesn't validate against the registry schema, because that schema isn't vendored and the tests make no network requests.
Docs: there's a new "Metadata Linter" section in
docs/contributing/tool-responses.md, which also explains how to add a skip entry with a reason.adding-tools.mdanddocs/releases/stdio.mdare updated to match.Pending
OUTPUT_SCHEMA_PENDINGholds onlysearch_attributes. It's experimental-only, and nooutputSchemaPR exists for it yet.Verification
Every check ran under
env -u SENTRY_AUTH_TOKEN, scoped to its package:@sentry/mcp-server-mocksbuild@sentry/mcp-coregenerate-definitionsgit statusclean@sentry/mcp-coretsc@sentry/mcp-corevitest@sentry/mcp-serverbundle andtest:bundleI also broke things on purpose to confirm the linter catches them, reverting each change afterward. Each case failed with a message naming the tool and the problem:
_metakey addedoutputSchemadropped from a tool that isn't exemptserverInfo.titleblankedvia @dcramer.
--
View Session [Sentry]