Add MCP gateway + authority introspection endpoint (0.4.0)#4
Open
Azdaroth wants to merge 8 commits into
Open
Conversation
Add the generic, SDK-independent gateway/upstream layer, the session
data payload, and the authority introspection endpoint to mcp_toolkit
(Part A of the extraction). Everything app-specific is injected via
McpToolkit::Configuration; the layer names no deployment.
- Gateway::UpstreamRegistry — per-config upstream registry (config.upstreams
/ config.register_upstream); Upstream Data with #name_for + split_tool_name.
- Gateway::Client — Streamable-HTTP MCP client with single-shot session-loss
recovery, SSE data: unwrap, content negotiation; Client::Error < McpToolkit::Error
carrying jsonrpc_error/http_status; clientInfo + protocol version from config
(DEFAULT_PROTOCOL_VERSION falls back to the wrapped mcp SDK's latest).
- Gateway::Aggregator — concurrent (concurrent-ruby) namespaced tool-list
aggregation, cached per-upstream (non-empty-only / stale-empty self-heal),
degrade-on-failure; Rails executor guarded for a booted app only.
- Gateway::Proxy — forwards resolved account_id as _meta[config.account_meta_key];
raises Gateway::UnknownUpstream / Gateway::UpstreamCallError (no protocol coupling).
- Session#data — opaque, back-compatible payload (legacy rows default {}).
- TokensController + engine route POST /mcp/tokens/introspect (authority).
- Configuration: upstreams, register_upstream, upstream_timeout (10),
upstream_list_ttl (900), logger (nil); concurrent-ruby direct dep.
- Specs (WebMock/subprocess-Rails), README + CHANGELOG [0.4.0], version 0.4.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase-2 extraction: relocate the hand-rolled JSON-RPC dispatcher + protocol into the gem (vendor-neutralized) as a second dispatch front-end alongside the SDK-backed satellite path (which is left untouched). - Protocol: JSON-RPC envelope + error codes + 3-version negotiation constants. - Dispatcher(context:, config:): initialize/tools/list/tools/call/ping + upstream list_changed; host tools via config.tool_provider (scope-gated centrally), namespaced calls via Gateway::Proxy with verbatim error relay. - Authority::ControllerMethods: authority transport concern with every billing/tenancy step an overridable hook (auth/rate-limit/usage/account/ session/dispatch/health); per-request account loop preserved across batches. - Authority::ServerController: subclassable base (lazily-parented). - Authority::Context: per-request account/principal/bearer_token/superuser?. - Tools::AuthorityBase: optional tool base over the api-agnostic tool_provider seam; the gem never references a host's API/serializers/catalog. - Config: tool_provider, supported_protocol_versions, rate_limiter/ usage_recorder/usage_flusher, gateway_client_name/version (identity split so the gateway handshake stays byte-identical while server_name is the host's). - Lazy parent_controller (Constraint B): delete the eager-loadable engine controllers; build them (+ the authority base) from current config via build_engine_controllers!, triggered by const_missing and reset on reload by the engine's to_prepare, so the parent is read after the host's to_prepare. Specs: dispatcher, dispatcher_gateway, protocol, authority/controller_methods (hooks + mixed-account batch loop), authority_controller_lazy_parent (real-app subprocess boot), tool_provider_contract, tools/authority_base, and a Gateway::Client identity-split spec. 284 examples, 0 failures; rubocop clean; public-gem diff is fingerprint-free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…into 0.4.0 0.4.0 is not yet released (PR #4 open, untagged), so the Phase-1 gateway extraction and the Phase-2 authority dispatch path are one unreleased 0.4.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eams (v0.4.0)
GEM-side of MCP full tool unification: the authority dispatch path can now serve
the four generic read tools (resources/resource_schema/get/list) over
config.registry, reusing the existing executors + serializer + schema builder
unchanged, so a host (core) drops its own generic-tool engine and plugs in.
- Resource: superusers_only!/superusers_only?, note(text)+reader, and
filter(name, type:, description:, &applier) + custom_filters — a resource-specific
filter block, all api-agnostic.
- ListExecutor: applies matching custom_filters (top-level request keys) BEFORE the
allowlist filterable filters (its only change).
- Authority::RegistryToolProvider.new(config:) + the four thin
Authority::Tools::{Resources,ResourceSchema,Get,List}: resolve descriptor,
gate superusers_only? (refuse in get/list/resource_schema, hide in resources),
gate per-resource scope, require account for get/list. Return a raw Hash for the
dispatcher to wrap — added ALONGSIDE the untouched satellite SDK tool path.
- Authority::CompositeToolProvider.new(*providers): concatenate definitions, find
tries each in order (compose generic + bespoke tools).
- ResourceSchema: per-attribute filter `operators` (from Filtering::OPERATORS_BY_TYPE)
+ `note` passthrough.
Version stays 0.4.0 (CHANGELOG bullet + README authority section). 323 specs green
(284 prior + 39 new), RuboCop clean, zero fingerprints.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A host that derives its filterable map from the DB (Model.column_names) must not run that at registration time — registration typically happens in an initializer's to_prepare, before the database exists (e.g. CI db:create), which aborts boot. filterable now accepts a callable resolved once on first read (a tool call), memoized; a plain Hash still merges eagerly (backward compatible). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- McpToolkit::RateLimiter: fixed-window per-principal counter over config.cache_store - config.rate_limit_max_requests (nil=off) / rate_limit_window (3600) - Authority::ControllerMethods#mcp_rate_limit! built-in: X-RateLimit-* headers, -32029 JSON-RPC 429 + Retry-After over limit; mcp_rate_limit_max_requests + mcp_rate_limit_key hooks; config.rate_limiter kept as escape hatch - config.superuser_resolver + Authority::Context#superuser? (resolver else duck-type) - specs: RateLimiter unit, Context#superuser?, built-in rate-limit controller-methods - CHANGELOG [0.4.0] + README; version stays 0.4.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a generic gateway + authority layer to the toolkit, so a central app can aggregate and proxy the tools of downstream ("satellite") MCP servers and answer token introspection — all configured through
McpToolkit::Configuration, with no app-, company-, or deployment-specific code in the gem.What's new (
McpToolkit::Gateway::*)UpstreamRegistry(per-config,config.upstreams) — register/lookup downstream servers;<key>__<tool>namespacing.Client— minimal Streamable-HTTP MCP client (initialize handshake,tools/list,tools/call) with single-shot session-loss recovery and SSE unwrapping.Aggregator— concurrent, cached, namespaced aggregation of upstream tool lists; only non-empty pulls are cached (self-healing), failing upstreams degrade gracefully.Proxy— proxies a namespacedtools/callto its upstream, forwarding the bearer and the resolved account viaconfig.account_meta_key.UnknownUpstream/UpstreamCallError— transport-agnostic errors the consumer maps to its own JSON-RPC shape.Also:
POST /mcp/tokens/introspect), driven byconfig.token_authenticator— consumers no longer hand-write it.Sessiongains an opaquedata:payload (back-compatible) so an authority can bind a session to a token.upstreams/register_upstream,upstream_timeout,upstream_list_ttl,logger.This reverses the earlier "gateway/upstream aggregation is intentionally out of scope" note — it is now a first-class, fully generic part of the toolkit.
Verification
🤖 Generated with Claude Code