From 854ea2eeb95493eda71403b251c45ce096015a68 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 11:29:53 +0200 Subject: [PATCH 01/10] Extract MCP gateway + authority endpoint into the gem (0.4.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add the generic, SDK-independent gateway/upstream layer, the session data payload, and the authority introspection endpoint to mcp_toolkit (Part A of the extraction). Everything app-specific is injected via McpToolkit::Configuration; the layer names no deployment. - Gateway::UpstreamRegistry — per-config upstream registry (config.upstreams / config.register_upstream); Upstream Data with #name_for + split_tool_name. - Gateway::Client — Streamable-HTTP MCP client with single-shot session-loss recovery, SSE data: unwrap, content negotiation; Client::Error < McpToolkit::Error carrying jsonrpc_error/http_status; clientInfo + protocol version from config (DEFAULT_PROTOCOL_VERSION falls back to the wrapped mcp SDK's latest). - Gateway::Aggregator — concurrent (concurrent-ruby) namespaced tool-list aggregation, cached per-upstream (non-empty-only / stale-empty self-heal), degrade-on-failure; Rails executor guarded for a booted app only. - Gateway::Proxy — forwards resolved account_id as _meta[config.account_meta_key]; raises Gateway::UnknownUpstream / Gateway::UpstreamCallError (no protocol coupling). - Session#data — opaque, back-compatible payload (legacy rows default {}). - TokensController + engine route POST /mcp/tokens/introspect (authority). - Configuration: upstreams, register_upstream, upstream_timeout (10), upstream_list_ttl (900), logger (nil); concurrent-ruby direct dep. - Specs (WebMock/subprocess-Rails), README + CHANGELOG [0.4.0], version 0.4.0. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 53 +++- README.md | 88 +++++- .../mcp_toolkit/tokens_controller.rb | 47 +++ config/routes.rb | 10 +- lib/mcp_toolkit.rb | 9 +- lib/mcp_toolkit/configuration.rb | 40 +++ lib/mcp_toolkit/engine.rb | 16 +- lib/mcp_toolkit/gateway/aggregator.rb | 113 +++++++ lib/mcp_toolkit/gateway/client.rb | 298 ++++++++++++++++++ lib/mcp_toolkit/gateway/proxy.rb | 70 ++++ lib/mcp_toolkit/gateway/unknown_upstream.rb | 8 + .../gateway/upstream_call_error.rb | 23 ++ lib/mcp_toolkit/gateway/upstream_registry.rb | 65 ++++ lib/mcp_toolkit/session.rb | 26 +- lib/mcp_toolkit/version.rb | 2 +- mcp_toolkit.gemspec | 6 +- spec/mcp_toolkit/engine_route_reload_spec.rb | 24 +- spec/mcp_toolkit/engine_spec.rb | 7 +- spec/mcp_toolkit/gateway/aggregator_spec.rb | 149 +++++++++ spec/mcp_toolkit/gateway/client_spec.rb | 268 ++++++++++++++++ .../gateway/introspect_endpoint_spec.rb | 152 +++++++++ spec/mcp_toolkit/gateway/proxy_spec.rb | 131 ++++++++ .../gateway/upstream_registry_spec.rb | 99 ++++++ spec/mcp_toolkit/session_spec.rb | 30 ++ spec/mcp_toolkit_spec.rb | 8 + spec/spec_helper.rb | 2 + 26 files changed, 1698 insertions(+), 46 deletions(-) create mode 100644 app/controllers/mcp_toolkit/tokens_controller.rb create mode 100644 lib/mcp_toolkit/gateway/aggregator.rb create mode 100644 lib/mcp_toolkit/gateway/client.rb create mode 100644 lib/mcp_toolkit/gateway/proxy.rb create mode 100644 lib/mcp_toolkit/gateway/unknown_upstream.rb create mode 100644 lib/mcp_toolkit/gateway/upstream_call_error.rb create mode 100644 lib/mcp_toolkit/gateway/upstream_registry.rb create mode 100644 spec/mcp_toolkit/gateway/aggregator_spec.rb create mode 100644 spec/mcp_toolkit/gateway/client_spec.rb create mode 100644 spec/mcp_toolkit/gateway/introspect_endpoint_spec.rb create mode 100644 spec/mcp_toolkit/gateway/proxy_spec.rb create mode 100644 spec/mcp_toolkit/gateway/upstream_registry_spec.rb diff --git a/CHANGELOG.md b/CHANGELOG.md index 1cea5c6..2241ce2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,51 @@ +## [0.4.0] - 2026-07-06 + +### Added + +- **Gateway / upstream layer** (`McpToolkit::Gateway::*`) — the generic, + SDK-independent machinery a central app uses to aggregate *other* MCP servers + and proxy calls to them, previously an app-only concern. All app-specific values + (upstream URLs, account-selector meta key, logger, timeouts) are injected via + `McpToolkit::Configuration`; nothing in the layer names a deployment. + - `McpToolkit::Gateway::UpstreamRegistry` — a PER-CONFIG registry of upstream + servers (`Upstream = Data.define(:key, :url)` with `#name_for`), exposed as + `config.upstreams` and reset with a fresh config (test isolation for free). + API: `#register(key:, url:)` (blank url ignored), `#reset!`, `#all`, `#find`, + `#split_tool_name`. Config sugar: `config.register_upstream(key:, url:)`. + - `McpToolkit::Gateway::Client` — a minimal Streamable-HTTP MCP client + (`#tools_list`, `#tools_call`) with single-shot session-loss recovery (HTTP + 404 / JSON-RPC `-32001`), SSE `data:` unwrapping, and content negotiation. Its + `Client::Error` (< `McpToolkit::Error`) carries `jsonrpc_error` / `http_status` + and references NO transport/protocol-error class — the consumer maps it. The + handshake `clientInfo` and protocol version come from config + (`DEFAULT_PROTOCOL_VERSION` falls back to the wrapped `mcp` SDK's latest). + - `McpToolkit::Gateway::Aggregator` — namespaces + caches (`config.cache_store`, + `config.upstream_list_ttl`) each upstream's tool list, pulled CONCURRENTLY via + concurrent-ruby (wrapped in `Rails.application.executor` when a booted Rails app + is present, plain futures otherwise). Only a non-empty pull is cached; a stale + empty is a miss (poisoned-cache self-heal); a failing upstream degrades (omit + + log) rather than breaking the list. + - `McpToolkit::Gateway::Proxy` — proxies a namespaced call, forwarding the + resolved `account_id` as `_meta[config.account_meta_key]`. An unknown key raises + `McpToolkit::Gateway::UnknownUpstream`; an upstream failure raises + `McpToolkit::Gateway::UpstreamCallError` (carrying `jsonrpc_error` / + `http_status`). Neither is mapped to a protocol-error class here. +- **Authority introspection endpoint** — `McpToolkit::TokensController#introspect`, + drawn by the engine at `POST /mcp/tokens/introspect`, so a central app answers + introspection with no controller of its own. Its parent class is configurable via + `parent_controller` (like `ServerController`). Safe to draw unconditionally: with + no `token_authenticator` it answers `{ valid: false }`. +- **`McpToolkit::Session#data`** — an opaque payload attachable at + `create!(data:)` and round-tripped through `find`, so an authority can bind a + session to a token id (letting a revoked token kill an in-flight session). The + gem does not interpret it; legacy rows default to `{}`. +- `McpToolkit::Configuration` gains `upstreams` (a `Gateway::UpstreamRegistry`), + `register_upstream`, `upstream_timeout` (default `10`), `upstream_list_ttl` + (default `900`), and `logger` (default `nil`; all gateway/session call sites + guard with `logger&.`). +- `concurrent-ruby` is now a direct dependency (already transitive via + activesupport) — the aggregator's parallel upstream pulls require it. + ## [0.3.0] - 2026-07-03 ### Added @@ -119,5 +167,6 @@ hand-rolled controller wiring. ### Notes -- The gateway / upstream-aggregation layer (`Mcp::Upstreams*`) is intentionally - out of scope (core-only). +- The gateway / upstream-aggregation layer was intentionally out of scope for this + initial extraction. It was later extracted into the gem in 0.4.0 (see above), as + `McpToolkit::Gateway::*` — fully config-injected and app-agnostic. diff --git a/README.md b/README.md index 31f1a8d..f9e6e45 100644 --- a/README.md +++ b/README.md @@ -263,6 +263,10 @@ discovery tool, a custom serializer may also expose `declared_attributes` / | `parent_controller` | `"ActionController::Base"` | superclass of the engine's `ServerController` (set to `"ApplicationController"` for `helper_method` compat) | | `account_meta_key` | `"mcp-toolkit/account-id"` | `_meta` key a superuser uses to pin the account | | `account_id_header` | `"X-MCP-Account-ID"` | header fallback for the account selector | +| `upstreams` | empty registry | gateway: registered upstream MCP servers (register via `register_upstream`) | +| `upstream_timeout` | `10` | gateway: HTTP timeout (s) for calls to an upstream | +| `upstream_list_ttl` | `900` | gateway: TTL (s) for an upstream's cached tool list | +| `logger` | `nil` | optional logger for gateway/session diagnostics (`Rails.logger`) | ## Public API surface @@ -278,16 +282,90 @@ discovery tool, a custom serializer may also expose `declared_attributes` / `McpToolkit::ServerController` (its controller; parent via `parent_controller`) - `McpToolkit::Transport::ControllerMethods` (standalone controller concern; override `mcp_config` / `mcp_extra_tools`) -- `McpToolkit::Session` +- `McpToolkit::Session` (opaque `#data` payload, e.g. to bind a session to a token id) - `McpToolkit::Auth::Introspection` / `Authenticator` (satellite), `McpToolkit::Auth::Authority` (authority) - `McpToolkit::Errors::{InvalidParams, Unauthorized, ConfigurationError}` +- Gateway layer (a central app aggregating/proxying other MCP servers): + `McpToolkit::Gateway::UpstreamRegistry` (via `config.upstreams` / + `config.register_upstream`), `McpToolkit::Gateway::{Client, Aggregator, Proxy}`, + and its errors `McpToolkit::Gateway::{UnknownUpstream, UpstreamCallError}` + + `McpToolkit::Gateway::Client::Error` +- `McpToolkit::TokensController` — the authority introspection endpoint drawn by + the engine at `POST /mcp/tokens/introspect` -## Scope (what's intentionally NOT here) +## Gateway / authority endpoint -The gateway / upstream-aggregation layer (a central app's `Mcp::Upstreams*`) is -**out of scope** — it's core-only and ships to no satellite. This toolkit is for -servers that expose tools, not for aggregating other servers. +Beyond exposing a single server's own tools, the toolkit also ships the generic +**gateway** layer a central app uses to aggregate *other* MCP servers and proxy +calls to them, plus the **authority** introspection endpoint satellites call. +Every app-specific value (the upstream URLs, the account-selector meta key, the +logger, timeouts) is injected via config — nothing here names a deployment. + +### Register upstreams + +Each upstream has a `key` (the tool-name namespace prefix — its tools surface as +`__`) and a `url` (its MCP HTTP endpoint). A blank url is ignored, so +an ENV lookup can be passed directly: + +```ruby +McpToolkit.configure do |c| + c.cache_store = Rails.cache # share the upstream tool-list cache across workers + c.logger = Rails.logger # optional; degrade/recovery diagnostics + c.register_upstream(key: "notifications", url: ENV["NOTIFICATIONS_SERVER_URL"]) + c.register_upstream(key: "billing", url: ENV["BILLING_SERVER_URL"]) +end +``` + +### Aggregate upstream tool lists + +`Aggregator#tool_definitions` returns every upstream's tools, namespaced, pulled +concurrently. Each upstream's list is cached (`config.upstream_list_ttl`, default +15 min); only a **non-empty** pull is cached, and a failing upstream is omitted +(and logged via `config.logger`) rather than breaking the whole list. + +```ruby +definitions = McpToolkit::Gateway::Aggregator.new.tool_definitions(bearer_token: token) +# => [{ "name" => "notifications__list_items", "description" => ..., "inputSchema" => ... }, ...] + +McpToolkit::Gateway::Aggregator.new.flush! # bust every upstream's cache +McpToolkit::Gateway::Aggregator.new.flush!("notifications") # or just one +``` + +### Proxy a namespaced call + +Split a namespaced tool name via the registry, then proxy it. The caller passes +the already-resolved account id (a scalar); it is forwarded as +`_meta[config.account_meta_key]`. + +```ruby +key, bare = McpToolkit.config.upstreams.split_tool_name("notifications__list_items") +proxy = McpToolkit::Gateway::Proxy.new( + app_key: key, tool_name: bare, account_id: current_account_id, bearer_token: token +) +result = proxy.call({ "since" => "2026-01-01" }) # the upstream's `result` hash, verbatim +``` + +The proxy is transport-agnostic: an unregistered key raises +`McpToolkit::Gateway::UnknownUpstream`, and an upstream call failure raises +`McpToolkit::Gateway::UpstreamCallError` (carrying the upstream's `jsonrpc_error` +/ `http_status`). Your dispatcher maps those to whatever error shape its transport +speaks — the gem never welds the gateway to a protocol-error class. + +### Authority introspection endpoint (built in) + +Mounting `McpToolkit::Engine` also draws `POST /mcp/tokens/introspect`, backed by +the gem-provided `McpToolkit::TokensController`. A central app configured with a +`token_authenticator` answers introspection with **no controller of its own** — +the Quickstart 2 hand-rolled controller becomes optional: + +```ruby +# config/routes.rb +mount McpToolkit::Engine => "/mcp" # POST /mcp/tokens/introspect now works +``` + +Drawing it is safe even on an app that is not an authority: with no +`token_authenticator`, it simply answers `{ "valid": false }`. ## Development diff --git a/app/controllers/mcp_toolkit/tokens_controller.rb b/app/controllers/mcp_toolkit/tokens_controller.rb new file mode 100644 index 0000000..b72564f --- /dev/null +++ b/app/controllers/mcp_toolkit/tokens_controller.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +# The AUTHORITY-side introspection endpoint, provided BY the gem so a central app +# answers the introspection requests satellites send without writing a controller +# of its own. Mounted by McpToolkit::Engine at `POST /mcp/tokens/introspect`. +# +# Like McpToolkit::ServerController, its parent class is configurable +# (Doorkeeper-style) via `McpToolkit.config.parent_controller`. It authenticates +# the bearer against `config.token_authenticator` (touching last-used) and renders +# the exact payload McpToolkit::Auth::Authority builds — the contract the +# satellite's Auth::Introspection parses. +# +# Drawing the route unconditionally is safe: an app that never configured a +# `token_authenticator` (i.e. is not an authority) answers `{ valid: false }` +# rather than erroring, so mounting the engine on a pure satellite is harmless. +# +# Lives under the gem's app/controllers (an engine path), so it is loaded by +# Rails' autoloader via the engine — never by the gem's own Zeitwerk loader, which +# only manages lib/. Non-Rails consumers never see it. +class McpToolkit::TokensController < McpToolkit.config.parent_controller.constantize + def introspect + token = McpToolkit::Auth::Authority.authenticate(mcp_extract_token, config: mcp_config) + return render(json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized) if token.nil? + + render json: McpToolkit::Auth::Authority.introspection_payload(token) + rescue McpToolkit::Errors::ConfigurationError + # Not configured as an authority (no token_authenticator): behave as if the + # token were invalid instead of surfacing a 500. + render json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized + end + + private + + # Overridable, mirroring the transport concern's `mcp_config` hook. + def mcp_config + McpToolkit.config + end + + # Bearer extraction order: `Authorization: Bearer`, then `X-MCP-Token`, then + # `?token=`. + def mcp_extract_token + auth_header = request.headers["Authorization"] + return auth_header.sub("Bearer ", "") if auth_header&.start_with?("Bearer ") + + request.headers["X-MCP-Token"].presence || params[:token].presence + end +end diff --git a/config/routes.rb b/config/routes.rb index 0eb4805..6c17b13 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -6,13 +6,15 @@ # # mount McpToolkit::Engine => "/mcp" # -# POST /mcp -> create (JSON-RPC requests/responses) -# GET /mcp -> stream (405; no server-initiated SSE) -# DELETE /mcp -> destroy (terminate the session) -# GET /mcp/health -> health (unauthenticated probe) +# POST /mcp -> create (JSON-RPC requests/responses) +# GET /mcp -> stream (405; no server-initiated SSE) +# DELETE /mcp -> destroy (terminate the session) +# GET /mcp/health -> health (unauthenticated probe) +# POST /mcp/tokens/introspect -> introspect (authority token introspection) McpToolkit::Engine.routes.draw do post "/", to: "server#create" get "/", to: "server#stream" delete "/", to: "server#destroy" get "health", to: "server#health" + post "tokens/introspect", to: "tokens#introspect" end diff --git a/lib/mcp_toolkit.rb b/lib/mcp_toolkit.rb index 5e9528d..0eb3f01 100644 --- a/lib/mcp_toolkit.rb +++ b/lib/mcp_toolkit.rb @@ -15,9 +15,12 @@ # active_support/concern - Transport::ControllerMethods is an includable concern # active_support/cache - the default MemoryStore cache_store # -# `faraday` is the one exception: it is required alongside its sole owner, -# Auth::AuthorityServerClient, which is the only object that builds an HTTP -# connection. +# Two third-party libs are the exception to the centralize-here rule: each is +# required alongside its owner file rather than up front. +# faraday - the HTTP client, required by the objects that build a connection +# (Auth::AuthorityServerClient and Gateway::Client). +# concurrent - concurrent-ruby's futures, required by its sole owner +# Gateway::Aggregator (which pulls upstreams in parallel). require "json" require "digest" require "time" diff --git a/lib/mcp_toolkit/configuration.rb b/lib/mcp_toolkit/configuration.rb index f995428..9afbbd4 100644 --- a/lib/mcp_toolkit/configuration.rb +++ b/lib/mcp_toolkit/configuration.rb @@ -151,6 +151,33 @@ class McpToolkit::Configuration # @return [McpToolkit::Registry] attr_accessor :registry + # --- gateway / upstreams --------------------------------------------------- + + # @return [Integer] HTTP open/read timeout (s) for a gateway's calls to an + # upstream MCP server (McpToolkit::Gateway::Client). + attr_accessor :upstream_timeout + # @return [Integer] TTL (s) for an upstream's cached, namespaced tool list in + # McpToolkit::Gateway::Aggregator. + attr_accessor :upstream_list_ttl + + # The registry of upstream MCP servers this gateway aggregates + proxies to. + # Each config carries its own (like `registry`), so it resets with a fresh + # config. Register via the `register_upstream` sugar below or directly on this + # instance. Empty unless the app registers upstreams, so a non-gateway app is + # unaffected. + # + # @return [McpToolkit::Gateway::UpstreamRegistry] + attr_reader :upstreams + + # --- diagnostics ----------------------------------------------------------- + + # Optional logger for gateway/session diagnostics. All call sites guard with + # `logger&.warn` / `logger&.error`, so nil (the default) silences them. A Rails + # host typically sets this to `Rails.logger`. + # + # @return [#warn, #error, nil] + attr_accessor :logger + # Vendor-neutral defaults; apps override the auth wiring + identity as needed. def initialize @server_name = "mcp-server" @@ -178,7 +205,20 @@ def initialize @account_meta_key = "mcp-toolkit/account-id" @account_id_header = "X-MCP-Account-ID" + @upstream_timeout = 10 + @upstream_list_ttl = 900 # 15 minutes + @logger = nil + @registry = McpToolkit::Registry.new + @upstreams = McpToolkit::Gateway::UpstreamRegistry.new + end + + # Config sugar: register a gateway upstream. Delegates to `upstreams.register`, + # so a blank url is ignored (an unconfigured upstream is simply absent). + # + # c.register_upstream(key: "notifications", url: ENV["NOTIFICATIONS_SERVER_URL"]) + def register_upstream(key:, url:) + upstreams.register(key:, url:) end # The serializer base, lazily defaulting to the gem's bundled DSL base. Lazy so diff --git a/lib/mcp_toolkit/engine.rb b/lib/mcp_toolkit/engine.rb index 099a6a5..e8a76cd 100644 --- a/lib/mcp_toolkit/engine.rb +++ b/lib/mcp_toolkit/engine.rb @@ -1,18 +1,20 @@ # frozen_string_literal: true -# Mountable Rails engine that draws the four MCP transport routes (defined in the -# engine's config/routes.rb so they survive Rails' route reloads) against the -# gem-provided McpToolkit::ServerController. A satellite mounts it in one line: +# Mountable Rails engine that draws the MCP transport routes plus the authority +# introspection route (defined in the engine's config/routes.rb so they survive +# Rails' route reloads) against the gem-provided McpToolkit::ServerController / +# McpToolkit::TokensController. A satellite mounts it in one line: # # # config/routes.rb # mount McpToolkit::Engine => "/mcp" # # yielding exactly the endpoints a hand-rolled satellite declared: # -# POST /mcp -> create (JSON-RPC requests/responses) -# GET /mcp -> stream (405; no server-initiated SSE) -# DELETE /mcp -> destroy (terminate the session) -# GET /mcp/health -> health (unauthenticated probe) +# POST /mcp -> create (JSON-RPC requests/responses) +# GET /mcp -> stream (405; no server-initiated SSE) +# DELETE /mcp -> destroy (terminate the session) +# GET /mcp/health -> health (unauthenticated probe) +# POST /mcp/tokens/introspect -> introspect (authority token introspection) # # Loaded ONLY when Rails::Engine is available (see lib/mcp_toolkit.rb); the gem's # non-Rails consumers and its own unit suite never reference it. diff --git a/lib/mcp_toolkit/gateway/aggregator.rb b/lib/mcp_toolkit/gateway/aggregator.rb new file mode 100644 index 0000000..49c9f17 --- /dev/null +++ b/lib/mcp_toolkit/gateway/aggregator.rb @@ -0,0 +1,113 @@ +# frozen_string_literal: true + +require "concurrent" + +# Aggregates the tool lists of all configured upstream MCP servers for a +# gateway's `tools/list`, namespacing each tool as `__`. +# +# Per upstream the namespaced list is cached in `config.cache_store` +# (`config.upstream_list_ttl`, default 15 min) and bustable via `flush!` / +# `flush!(key)`. On a cache miss the live pull runs per-upstream (one HTTP call +# each); a failing/timeout upstream is omitted + logged, never breaking the list. +# +# Why the cache is safe to share globally: an upstream's `tools/list` is +# token-INDEPENDENT — it returns the same public tool definitions to every valid +# caller and enforces scope only when a tool is CALLED (per-call authorization is +# the upstream's job). So one caller's pull is a correct answer for all callers. +# +# Only a NON-EMPTY pull is ever cached. An empty or failed pull is almost always a +# transient upstream hiccup (timeout, a session/handshake blip, a degenerate 200); +# caching it would freeze a whole app's tools out of `tools/list` for the full TTL +# for EVERY caller (a poisoned global cache), which is exactly the failure this +# guards against. A stale empty already in the cache is treated as a miss and +# re-pulled, so the aggregate self-heals as soon as the upstream returns tools. +# +# Concurrency: upstreams are pulled CONCURRENTLY via concurrent-ruby futures. When +# running inside Rails, each future is wrapped in `Rails.application.executor` so +# it participates in the framework's per-request lifecycle (reloading, query +# cache, connection checkout); a non-Rails host runs plain futures. Output order +# follows the registry order. +class McpToolkit::Gateway::Aggregator + CACHE_KEY_PREFIX = "mcp_toolkit:gateway:tools:" + + def initialize(config: McpToolkit.config) + @config = config + end + + # Namespaced tool definitions across all configured upstreams. `bearer_token` + # is used only on a cache miss, so the upstream can authenticate the list + # request the same way it would a call. Upstreams are fetched concurrently; + # output order follows the registry order. + def tool_definitions(bearer_token: nil) + futures = config.upstreams.all.map do |upstream| + Concurrent::Promises.future do + within_executor { cached_or_live_definitions(upstream, bearer_token:) } + end + end + + futures.flat_map(&:value!) + end + + def flush!(key = nil) + if key + cache.delete(cache_key(key)) + else + config.upstreams.all.each { |upstream| cache.delete(cache_key(upstream.key)) } + end + end + + private + + attr_reader :config + + # Runs the block inside the Rails executor when a BOOTED Rails app is present + # (so a future participates in the request lifecycle: reloading, query cache, + # connection checkout), or plainly otherwise. Guards for `Rails` being defined + # but not booted (e.g. `rails/version` required without an initialized app), in + # which case `Rails.application` is nil and we run the block directly. + def within_executor(&) + if defined?(Rails) && Rails.respond_to?(:application) && Rails.application + Rails.application.executor.wrap(&) + else + yield + end + end + + def cached_or_live_definitions(upstream, bearer_token:) + cached = cache.read(cache_key(upstream.key)) + # `present?` treats both a nil miss AND a stale empty list as "no cache", so a + # previously poisoned empty entry is re-pulled instead of served. + return cached if cached.present? + + definitions = live_definitions(upstream, bearer_token:) + # Only persist a real, non-empty list; never cache an empty/degraded pull. + cache.write(cache_key(upstream.key), definitions, expires_in: config.upstream_list_ttl) if definitions.present? + definitions + rescue McpToolkit::Gateway::Client::Error => e + # Degrade gracefully: omit this upstream's tools, don't cache the failure. + config.logger&.error("MCP upstream #{upstream.key} tools/list failed, omitting: #{e.message}") + [] + end + + def live_definitions(upstream, bearer_token:) + client = McpToolkit::Gateway::Client.new(upstream:, bearer_token:, config:) + client.tools_list.map do |definition| + namespaced(upstream, definition) + end + end + + # Re-keys an upstream tool definition into the gateway's aggregate namespace. + def namespaced(upstream, definition) + definition = definition.dup + definition["name"] = upstream.name_for(definition["name"]) + definition + end + + def cache + config.cache_store + end + + def cache_key(key) + "#{CACHE_KEY_PREFIX}#{key}" + end +end diff --git a/lib/mcp_toolkit/gateway/client.rb b/lib/mcp_toolkit/gateway/client.rb new file mode 100644 index 0000000..f023436 --- /dev/null +++ b/lib/mcp_toolkit/gateway/client.rb @@ -0,0 +1,298 @@ +# frozen_string_literal: true + +require "faraday" + +# Minimal MCP client over Streamable HTTP, used by a GATEWAY to talk to an +# upstream MCP server when aggregating its tool list and proxying tool calls. +# +# It speaks the same Streamable-HTTP MCP that McpToolkit::Transport serves: +# 1. POST `initialize` -> capture the `Mcp-Session-Id` response header +# 2. POST `notifications/initialized` (a notification; no response expected) +# 3. POST `tools/list` / `tools/call`, echoing the session header +# +# Auth & account selection are pass-through: the caller's bearer token and the +# account selector (`_meta`) are forwarded so the upstream can introspect the +# same token against its authority and resolve the same account. +# +# Transport notes +# - Content negotiation: we POST application/json and Accept both +# application/json and text/event-stream. If the upstream answers with SSE +# (one message + EOF, as the toolkit's own transport does) we extract the +# single JSON payload from the `data:` line. +# - Every public method may raise McpToolkit::Gateway::Client::Error (timeouts, +# non-2xx, unparseable bodies, JSON-RPC errors). Callers decide whether to +# degrade (omit from list) or surface the error (proxied call). +# +# Everything app-specific (server identity, protocol version, timeout, logger) is +# injected via McpToolkit::Configuration; nothing here names a deployment. +class McpToolkit::Gateway::Client + SESSION_HEADER = "Mcp-Session-Id" + JSONRPC_VERSION = "2.0" + + # The protocol version offered on the handshake when the config does not pin + # one. Sourced from the wrapped `mcp` SDK's latest-supported constant when + # available, with a literal fallback so the gem still loads if the SDK moves the + # constant. `config.protocol_version` overrides it. + DEFAULT_PROTOCOL_VERSION = + if defined?(MCP::Configuration) && MCP::Configuration.const_defined?(:LATEST_STABLE_PROTOCOL_VERSION) + MCP::Configuration::LATEST_STABLE_PROTOCOL_VERSION + else + "2025-06-18" + end + + # An upstream whose session store is not shared across its pods can answer a + # NON-initialize request with HTTP 404 (the pod fielding it never saw our + # initialize), or with a JSON-RPC -32001 "Session not found or expired". Both + # mean the same thing: our session is gone and a fresh handshake recovers it. + SESSION_LOSS_HTTP_STATUS = 404 + SESSION_NOT_FOUND_CODE = -32_001 + + # Raised for any upstream failure. `jsonrpc_error` carries an upstream JSON-RPC + # error hash when the failure was a protocol-level error response, so a proxy + # can relay it verbatim; nil for transport/HTTP failures. `http_status` carries + # the HTTP status for a non-2xx response (nil otherwise), so callers can + # distinguish a session-loss 404 from other failures. This class references NO + # transport/protocol-error type — the consumer maps it. + class Error < McpToolkit::Error + attr_reader :jsonrpc_error, :http_status + + def initialize(message, jsonrpc_error: nil, http_status: nil) + super(message) + @jsonrpc_error = jsonrpc_error + @http_status = http_status + end + end + + attr_reader :upstream, :bearer_token, :timeout, :config + + def initialize(upstream:, bearer_token: nil, config: McpToolkit.config) + @upstream = upstream + @bearer_token = bearer_token + @config = config + @timeout = config.upstream_timeout + @session_id = nil + @initialized = false + end + + # Returns the upstream's raw tools array (each a tool definition hash with + # string keys: "name", "description", "inputSchema"). Bare names — the caller + # namespaces them. + def tools_list + with_session_recovery("tools/list") do + result = rpc!("tools/list") + Array(result["tools"]) + end + end + + # Proxies a tools/call to the upstream. `arguments` and `meta` are forwarded + # as-is. Returns the upstream's `result` hash verbatim (typically + # `{ "content" => [...] }`). A JSON-RPC error from the upstream is raised as an + # Error carrying `jsonrpc_error` so a proxy can relay it. + def tools_call(name:, arguments: {}, meta: nil) + params = { "name" => name, "arguments" => arguments } + params["_meta"] = meta if meta.present? + with_session_recovery("tools/call #{name}") do + rpc!("tools/call", params) + end + end + + private + + # The protocol version to offer on the handshake: the config's pin, else the + # gem's default. + def protocol_version + config.protocol_version || DEFAULT_PROTOCOL_VERSION + end + + # Runs a request-bearing RPC against an initialized session, transparently + # recovering from a SINGLE session-loss response: if the upstream reports our + # session is gone (see SESSION_LOSS_HTTP_STATUS / SESSION_NOT_FOUND_CODE) we + # drop the dead session, re-handshake, and retry the block exactly ONCE. Any + # other error — a genuine JSON-RPC tool error, a timeout, a non-404 HTTP + # failure — propagates verbatim, unchanged. An upstream tool that returns an + # `isError` content result is a normal return value here (not an exception), so + # it passes straight through. + def with_session_recovery(operation, &) + ensure_initialized! + yield + rescue Error => e + raise unless session_loss?(e) + + recover_and_retry(operation, e, &) + end + + # Second (and final) attempt after a session-loss signal: re-establish the + # session and run the block once more. Bounded to one retry — this method does + # not recurse into `with_session_recovery`, so it cannot loop. If re-init or the + # retry itself fails, the error is clarified before it propagates. + def recover_and_retry(operation, original_error) + log_session_recovery(operation, original_error) + reset_session! + ensure_initialized! + yield + rescue Error => e + raise clarified_session_error(e, operation) + end + + # Performs the initialize handshake once per client instance, capturing the + # session id, then sends the `notifications/initialized` notification. + def ensure_initialized! + return if @initialized + + response = post_jsonrpc( + jsonrpc_request("initialize", { + "protocolVersion" => protocol_version, + "capabilities" => {}, + "clientInfo" => { "name" => config.server_name, "version" => config.server_version } + }) + ) + @session_id = response.headers[SESSION_HEADER].presence + parse_rpc_result!(response) + + # Best-effort lifecycle notification; upstreams may ignore it. A notification + # returns no body (202), so we don't parse a result. + post_jsonrpc(jsonrpc_notification("notifications/initialized")) + + @initialized = true + end + + # Drops the current session so the next `ensure_initialized!` re-handshakes. + def reset_session! + @initialized = false + @session_id = nil + end + + # True when an upstream error signals our session is gone and a fresh handshake + # could recover it: a bare HTTP 404, or a JSON-RPC -32001 session error. + def session_loss?(error) + error.http_status == SESSION_LOSS_HTTP_STATUS || + error.jsonrpc_error&.dig("code") == SESSION_NOT_FOUND_CODE + end + + # A session-loss failure we could NOT recover from (already retried once). + # Preserve verbatim relay for a genuine JSON-RPC error and pass through any + # unrelated failure; only rewrite the bare "returned HTTP 404", whose message + # hides the real cause. + def clarified_session_error(error, operation) + return error if error.jsonrpc_error + return error unless session_loss?(error) + + Error.new( + "upstream #{upstream.key} #{operation} failed: the MCP session was lost and could not " \ + "be re-established after re-initializing the handshake", + http_status: error.http_status + ) + end + + def log_session_recovery(operation, error) + config.logger&.warn( + "MCP upstream #{upstream.key} #{operation}: session lost (#{session_loss_reason(error)}), " \ + "re-initializing and retrying once" + ) + end + + def session_loss_reason(error) + return "JSON-RPC #{error.jsonrpc_error["code"]}" if error.jsonrpc_error + return "HTTP #{error.http_status}" if error.http_status + + "unknown" + end + + # Sends a request-bearing JSON-RPC call and returns its `result`. + def rpc!(method, params = {}) + response = post_jsonrpc(jsonrpc_request(method, params)) + parse_rpc_result!(response) + end + + def jsonrpc_request(method, params = {}) + { "jsonrpc" => JSONRPC_VERSION, "id" => SecureRandom.uuid, "method" => method, "params" => params } + end + + def jsonrpc_notification(method, params = {}) + { "jsonrpc" => JSONRPC_VERSION, "method" => method, "params" => params } + end + + def post_jsonrpc(body) + connection.post(upstream.url) do |request| + apply_request_headers(request.headers) + request.body = JSON.generate(body) + end + rescue Faraday::TimeoutError => e + raise upstream_error("timed out after #{timeout}s", e) + rescue Faraday::Error => e + raise upstream_error("request failed", e) + end + + def upstream_error(reason, cause) + Error.new("upstream #{upstream.key} #{reason}: #{cause.message}") + end + + # Sets the content-negotiation, auth, and session headers on an outgoing + # request. Auth and session headers are added only when present. + def apply_request_headers(headers) + headers["Content-Type"] = "application/json" + headers["Accept"] = "application/json, text/event-stream" + headers["Authorization"] = "Bearer #{bearer_token}" if bearer_token.present? + headers[SESSION_HEADER] = @session_id if @session_id.present? + end + + # Validates the HTTP response and extracts the JSON-RPC `result`, raising on a + # JSON-RPC error (with the error hash attached for verbatim relay). + def parse_rpc_result!(response) + raise_http_error!(response) unless response.success? + + payload = decode_body(response) + return {} if payload.nil? # e.g. 202 Accepted for a notification + + if payload["error"] + raise Error.new( + "upstream #{upstream.key} JSON-RPC error: #{payload.dig("error", "message")}", + jsonrpc_error: payload["error"] + ) + end + + payload["result"] || {} + end + + # The `http_status` is carried on the Error so a session-loss 404 can be told + # apart from other non-2xx failures (see `session_loss?`). + def raise_http_error!(response) + raise Error.new("upstream #{upstream.key} returned HTTP #{response.status}", http_status: response.status) + end + + # Decodes either a plain JSON body or a single SSE `data:` frame. + def decode_body(response) + json = json_payload(response) + return nil if json.nil? + + JSON.parse(json) + rescue JSON::ParserError => e + raise Error, "upstream #{upstream.key} returned unparseable body: #{e.message}" + end + + # Extracts the JSON text to parse, unwrapping an SSE frame when the response is + # an event stream. Returns nil for an empty body or an empty/absent frame. + def json_payload(response) + body = response.body.to_s + return nil if body.strip.empty? + + content_type = response.headers["Content-Type"].to_s + json = content_type.include?("text/event-stream") ? extract_sse_data(body) : body + json unless json.nil? || json.strip.empty? + end + + # Pulls the JSON payload from the first `data:` line of an SSE stream. + def extract_sse_data(body) + body.each_line.filter_map do |line| + line.start_with?("data:") ? line.sub(/\Adata:\s?/, "").chomp : nil + end.first + end + + def connection + @connection ||= Faraday.new do |conn| + conn.options.timeout = timeout + conn.options.open_timeout = timeout + conn.adapter Faraday.default_adapter + end + end +end diff --git a/lib/mcp_toolkit/gateway/proxy.rb b/lib/mcp_toolkit/gateway/proxy.rb new file mode 100644 index 0000000..19ef244 --- /dev/null +++ b/lib/mcp_toolkit/gateway/proxy.rb @@ -0,0 +1,70 @@ +# frozen_string_literal: true + +# Proxies a namespaced upstream tool call (`__`) to its upstream MCP +# server. +# +# The account selector is forwarded as `_meta[config.account_meta_key]` when an +# `account_id` is supplied. The caller passes the already-resolved account id (a +# scalar), not a domain object — resolving the tenant is the consumer's job. +# +# Error mapping is deliberately thin and transport-agnostic: +# * an unregistered `app_key` raises McpToolkit::Gateway::UnknownUpstream; +# * an upstream call failure is re-raised as McpToolkit::Gateway::UpstreamCallError +# carrying the upstream's `jsonrpc_error` / `http_status`. +# Neither is mapped to a JSON-RPC/protocol error class here — the consuming +# dispatcher does that at its call site. +class McpToolkit::Gateway::Proxy + attr_reader :app_key, :tool_name, :account_id, :bearer_token, :config + + def initialize(app_key:, tool_name:, account_id: nil, bearer_token: nil, config: McpToolkit.config) + @app_key = app_key + @tool_name = tool_name + @account_id = account_id + @bearer_token = bearer_token + @config = config + end + + def call(arguments) + upstream = config.upstreams.find(app_key) + raise McpToolkit::Gateway::UnknownUpstream, "Unknown application: #{app_key}" if upstream.nil? + + client = McpToolkit::Gateway::Client.new(upstream:, bearer_token:, config:) + client.tools_call(name: tool_name, arguments:, meta:) + rescue McpToolkit::Gateway::Client::Error => e + relay_upstream_error(e) + end + + private + + def relay_upstream_error(error) + log_proxied_failure(error) + + raise McpToolkit::Gateway::UpstreamCallError.new( + error.message, + jsonrpc_error: error.jsonrpc_error, + http_status: error.http_status + ) + end + + # Emit one concise, greppable ERROR line per failed proxied call — never a + # bearer token or a full response body. + def log_proxied_failure(error) + config.logger&.error( + "MCP upstream #{app_key} tools/call #{tool_name} failed#{failure_detail(error)}: #{error.message}" + ) + end + + def failure_detail(error) + code = error.jsonrpc_error&.dig("code") + return " (jsonrpc_code=#{code})" if code + return " (http_status=#{error.http_status})" if error.http_status + + "" + end + + def meta + return nil if account_id.nil? + + { config.account_meta_key => account_id } + end +end diff --git a/lib/mcp_toolkit/gateway/unknown_upstream.rb b/lib/mcp_toolkit/gateway/unknown_upstream.rb new file mode 100644 index 0000000..f78b9c0 --- /dev/null +++ b/lib/mcp_toolkit/gateway/unknown_upstream.rb @@ -0,0 +1,8 @@ +# frozen_string_literal: true + +# Raised by McpToolkit::Gateway::Proxy when a namespaced tool call targets an +# upstream key that is not registered. The GEM does NOT translate this into any +# JSON-RPC / protocol error class — the consuming dispatcher maps it to whatever +# error shape its transport speaks (e.g. a "method not found"). Kept a plain +# McpToolkit::Error so the gem stays transport-agnostic. +class McpToolkit::Gateway::UnknownUpstream < McpToolkit::Error; end diff --git a/lib/mcp_toolkit/gateway/upstream_call_error.rb b/lib/mcp_toolkit/gateway/upstream_call_error.rb new file mode 100644 index 0000000..1cb2d32 --- /dev/null +++ b/lib/mcp_toolkit/gateway/upstream_call_error.rb @@ -0,0 +1,23 @@ +# frozen_string_literal: true + +# Raised by McpToolkit::Gateway::Proxy when a proxied `tools/call` fails at the +# upstream. It carries the upstream failure detail so the CONSUMER can decide how +# to surface it: +# +# * `jsonrpc_error` — the upstream's JSON-RPC error hash when the failure was a +# protocol-level error response (so the consumer can relay it verbatim); nil +# for transport/HTTP failures. +# * `http_status` — the HTTP status for a non-2xx response (nil otherwise). +# +# The gem deliberately does NOT map this to a protocol/transport error class: +# that mapping lives in the consuming dispatcher, keeping the gateway +# transport-agnostic. +class McpToolkit::Gateway::UpstreamCallError < McpToolkit::Error + attr_reader :jsonrpc_error, :http_status + + def initialize(message, jsonrpc_error: nil, http_status: nil) + super(message) + @jsonrpc_error = jsonrpc_error + @http_status = http_status + end +end diff --git a/lib/mcp_toolkit/gateway/upstream_registry.rb b/lib/mcp_toolkit/gateway/upstream_registry.rb new file mode 100644 index 0000000..74089e7 --- /dev/null +++ b/lib/mcp_toolkit/gateway/upstream_registry.rb @@ -0,0 +1,65 @@ +# frozen_string_literal: true + +# Registry of upstream MCP servers a GATEWAY app aggregates and proxies to. +# +# Each upstream has a `key` (the tool-name namespace prefix — its tools are +# surfaced as `__`) and a `url` (the upstream's MCP HTTP endpoint). +# Upstreams are registered at boot (typically from ENV); an upstream whose url is +# blank is never registered, so an unconfigured environment behaves exactly like +# a gateway with no upstreams. +# +# Unlike a global module singleton, this is a PER-CONFIG instance (like +# `config.registry`): each `McpToolkit::Configuration` carries its own, exposed as +# `config.upstreams`. That gives test isolation for free (a fresh config resets +# it) and matches the gem's per-config convention. Register via the config sugar +# `config.register_upstream(key:, url:)` or directly on `config.upstreams`. +class McpToolkit::Gateway::UpstreamRegistry + # Separates an app key from a tool name in an aggregated tool name. A double + # underscore so it doesn't collide with single underscores in bare tool names + # (e.g. "list_items"); it also matches the gem's `__` scope + # separator. + NAMESPACE_SEPARATOR = "__" + + Upstream = Data.define(:key, :url) do + def name_for(tool_name) + "#{key}#{NAMESPACE_SEPARATOR}#{tool_name}" + end + end + + def initialize + @registered = {} + end + + # Registers an upstream by key. A blank url is ignored, so callers can pass an + # ENV lookup directly without guarding it. + def register(key:, url:) + return if url.blank? + + @registered[key.to_s] = Upstream.new(key: key.to_s, url:) + end + + # Clears every registered upstream. + def reset! + @registered = {} + end + + # All registered upstreams (insertion order). + def all + @registered.values + end + + # The registered upstream for a key, or nil. + def find(key) + @registered[key.to_s] + end + + # `__` -> [app_key, bare_tool_name] for a registered upstream; nil + # for an un-namespaced name or an unknown/unregistered key. + def split_tool_name(tool_name) + prefix, separator, rest = tool_name.to_s.partition(NAMESPACE_SEPARATOR) + return nil if separator.empty? || rest.empty? + return nil unless find(prefix) + + [prefix, rest] + end +end diff --git a/lib/mcp_toolkit/session.rb b/lib/mcp_toolkit/session.rb index ec6e582..4b67fc6 100644 --- a/lib/mcp_toolkit/session.rb +++ b/lib/mcp_toolkit/session.rb @@ -7,24 +7,31 @@ # Cache-backed (rather than the gem's in-process StreamableHTTPTransport) so # sessions survive across Puma workers and interoperate with a gateway's client. # The cache store + TTL come from McpToolkit.config. +# +# A session carries an opaque `data` hash the transport can attach at creation +# (e.g. `{ token_id: ... }`) so an AUTHORITY can bind a session to a token id — +# the property that lets a revoked token kill an in-flight session. The gem does +# NOT interpret `data` (it never re-resolves a token; that's the consumer's auth +# concern): it stores it, round-trips it, and exposes it via `#data`. class McpToolkit::Session CACHE_KEY_PREFIX = "mcp_toolkit:session:" - def self.create!(config: McpToolkit.config) + def self.create!(data: {}, config: McpToolkit.config) id = SecureRandom.uuid - config.cache_store.write(cache_key(id), { created_at: Time.now.to_i }, expires_in: config.session_ttl) - new(id:) + config.cache_store.write(cache_key(id), { created_at: Time.now.to_i, data: }, expires_in: config.session_ttl) + new(id:, data:) end def self.find(id, config: McpToolkit.config) return nil if id.to_s.empty? - data = config.cache_store.read(cache_key(id)) - return nil unless data + stored = config.cache_store.read(cache_key(id)) + return nil unless stored # Sliding expiry: bump TTL on every successful lookup. - config.cache_store.write(cache_key(id), data, expires_in: config.session_ttl) - new(id:) + config.cache_store.write(cache_key(id), stored, expires_in: config.session_ttl) + # `data` defaults to {} for legacy rows written before the payload existed. + new(id:, data: stored[:data] || {}) end def self.delete(id, config: McpToolkit.config) @@ -38,9 +45,10 @@ def self.cache_key(id) end private_class_method :cache_key - attr_reader :id + attr_reader :id, :data - def initialize(id:) + def initialize(id:, data: {}) @id = id + @data = data || {} end end diff --git a/lib/mcp_toolkit/version.rb b/lib/mcp_toolkit/version.rb index 7bb52ea..3b017eb 100644 --- a/lib/mcp_toolkit/version.rb +++ b/lib/mcp_toolkit/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module McpToolkit - VERSION = "0.3.0" + VERSION = "0.4.0" end diff --git a/mcp_toolkit.gemspec b/mcp_toolkit.gemspec index 06b4dd4..29703d7 100644 --- a/mcp_toolkit.gemspec +++ b/mcp_toolkit.gemspec @@ -49,6 +49,10 @@ Gem::Specification.new do |spec| # depend on it, not on full Rails, so non-Rails hosts can consume the gem. spec.add_dependency "activesupport", ">= 6.1" # faraday is the HTTP client the satellite uses to introspect tokens against - # the central app. + # the central app, and a gateway uses to reach its upstream MCP servers. spec.add_dependency "faraday", ">= 1.0" + # concurrent-ruby powers the gateway aggregator's parallel upstream pulls. It is + # already an activesupport transitive dependency; depended on directly here + # because the gem's own code (Gateway::Aggregator) requires it. + spec.add_dependency "concurrent-ruby", ">= 1.1" end diff --git a/spec/mcp_toolkit/engine_route_reload_spec.rb b/spec/mcp_toolkit/engine_route_reload_spec.rb index 31a5c14..e3875e1 100644 --- a/spec/mcp_toolkit/engine_route_reload_spec.rb +++ b/spec/mcp_toolkit/engine_route_reload_spec.rb @@ -4,7 +4,7 @@ # Regression spec for the route-reloader wipe. # -# The engine draws its four MCP routes in the engine's `config/routes.rb` rather +# The engine draws its MCP routes in the engine's `config/routes.rb` rather # than in a class-body `McpToolkit::Engine.routes.draw` block. The reason is # Rails' routes_reloader: a host application builds its route set lazily and runs # the reloader on boot (and again on every reload). The reloader re-evaluates each @@ -14,7 +14,7 @@ # 404'ing. Moving the draw into config/routes.rb is what makes the routes # materialize and survive subsequent reloads. # -# The sibling engine_spec.rb stubs Rails and asserts the four routes are *drawn*, +# The sibling engine_spec.rb stubs Rails and asserts the routes are *drawn*, # but a stub cannot reproduce the reloader: only a REAL Rails::Application driving # its routes_reloader does, which is why the original bug slipped through. # @@ -38,15 +38,17 @@ end RSpec.describe "Engine survives a route reload", if: rails_available do - # The four endpoints as [verb, engine-relative path, controller#action]. Paths - # are relative to the engine's own route set (the `/mcp` mount supplies the - # prefix), so the roots are "/" and the probe is "/health". `mcp_toolkit/server` - # is the isolated-namespace path of the gem-provided McpToolkit::ServerController. + # The endpoints as [verb, engine-relative path, controller#action]. Paths are + # relative to the engine's own route set (the `/mcp` mount supplies the prefix), + # so the roots are "/" and the probe is "/health". `mcp_toolkit/server` / + # `mcp_toolkit/tokens` are the isolated-namespace paths of the gem-provided + # McpToolkit::ServerController / McpToolkit::TokensController. EXPECTED_ENGINE_ROUTES = [ - ["POST", "/", "mcp_toolkit/server#create"], - ["GET", "/", "mcp_toolkit/server#stream"], - ["DELETE", "/", "mcp_toolkit/server#destroy"], - ["GET", "/health", "mcp_toolkit/server#health"] + ["POST", "/", "mcp_toolkit/server#create"], + ["GET", "/", "mcp_toolkit/server#stream"], + ["DELETE", "/", "mcp_toolkit/server#destroy"], + ["GET", "/health", "mcp_toolkit/server#health"], + ["POST", "/tokens/introspect", "mcp_toolkit/tokens#introspect"] ].freeze # The driver script: boots a real, minimal Rails app in a child process, mounts @@ -136,7 +138,7 @@ expect(@result.fetch("mount_path")).to eq("/mcp") end - it "draws the four MCP endpoints in the engine route set once booted" do + it "draws the MCP endpoints in the engine route set once booted" do # JSON parses each route back to a [verb, path, action] array, matching the # shape of EXPECTED_ENGINE_ROUTES. expect(@result.fetch("before_reload")).to contain_exactly(*EXPECTED_ENGINE_ROUTES) diff --git a/spec/mcp_toolkit/engine_spec.rb b/spec/mcp_toolkit/engine_spec.rb index df412fb..fbd9939 100644 --- a/spec/mcp_toolkit/engine_spec.rb +++ b/spec/mcp_toolkit/engine_spec.rb @@ -50,7 +50,7 @@ def self.before_action(*); end describe "McpToolkit::Engine routes" do # A recorder standing in for the engine's route set: captures each verb call - # as [verb, path, to] so we can assert the four endpoints are drawn. The routes + # as [verb, path, to] so we can assert the endpoints are drawn. The routes # live in the engine's config/routes.rb (drawn through the routes_reloader so # they survive route reloads), so we load THAT file against the recorder. let(:route_recorder) do @@ -88,12 +88,13 @@ def draw(&block) after { McpToolkit.send(:remove_const, :Engine) if McpToolkit.const_defined?(:Engine, false) } - it "draws the four MCP endpoints mapping to the server controller actions" do + it "draws the MCP endpoints mapping to the server + tokens controller actions" do expect(route_recorder.drawn).to contain_exactly( [:post, "/", "server#create"], [:get, "/", "server#stream"], [:delete, "/", "server#destroy"], - [:get, "health", "server#health"] + [:get, "health", "server#health"], + [:post, "tokens/introspect", "tokens#introspect"] ) end diff --git a/spec/mcp_toolkit/gateway/aggregator_spec.rb b/spec/mcp_toolkit/gateway/aggregator_spec.rb new file mode 100644 index 0000000..38a3db6 --- /dev/null +++ b/spec/mcp_toolkit/gateway/aggregator_spec.rb @@ -0,0 +1,149 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe McpToolkit::Gateway::Aggregator do + subject(:aggregator) { described_class.new } + + let(:client) { instance_double(McpToolkit::Gateway::Client) } + let(:tools) do + [{ "name" => "send_email", "description" => "Send", "inputSchema" => { "type" => "object" } }] + end + + before do + McpToolkit.config.register_upstream(key: "notifications", url: "https://notif.test/mcp") + allow(McpToolkit::Gateway::Client).to receive(:new).and_return(client) + allow(client).to receive(:tools_list).and_return(tools) + end + + describe "#tool_definitions" do + it "namespaces each upstream tool as __" do + expect(aggregator.tool_definitions).to eq([ + { "name" => "notifications__send_email", "description" => "Send", "inputSchema" => { "type" => "object" } } + ]) + end + + it "forwards the bearer token to the client (used on a cache miss)" do + aggregator.tool_definitions(bearer_token: "tok-9") + + expect(McpToolkit::Gateway::Client) + .to have_received(:new).with(hash_including(bearer_token: "tok-9")) + end + + it "caches a non-empty pull (a second call does not hit the upstream)" do + aggregator.tool_definitions + aggregator.tool_definitions + + expect(client).to have_received(:tools_list).once + end + + it "writes the namespaced list to config.cache_store under the gateway key" do + aggregator.tool_definitions + + cached = McpToolkit.config.cache_store.read("#{described_class::CACHE_KEY_PREFIX}notifications") + expect(cached).to eq([ + { "name" => "notifications__send_email", "description" => "Send", "inputSchema" => { "type" => "object" } } + ]) + end + + context "when .flush! has been called" do + it "pulls live again" do + aggregator.tool_definitions + aggregator.flush! + aggregator.tool_definitions + + expect(client).to have_received(:tools_list).twice + end + + it "flushes only the named upstream when a key is given" do + aggregator.tool_definitions + aggregator.flush!("notifications") + aggregator.tool_definitions + + expect(client).to have_received(:tools_list).twice + end + end + + context "when the upstream list pull fails" do + before do + allow(client).to receive(:tools_list).and_raise(McpToolkit::Gateway::Client::Error.new("down")) + end + + it "omits that upstream's tools instead of erroring the whole list" do + expect(aggregator.tool_definitions).to eq([]) + end + + it "does not cache the failure (retries on the next call)" do + aggregator.tool_definitions + aggregator.tool_definitions + + expect(client).to have_received(:tools_list).twice + end + + it "logs the degrade via config.logger when one is set" do + McpToolkit.config.logger = instance_double(Logger, error: nil) + + aggregator.tool_definitions + + expect(McpToolkit.config.logger) + .to have_received(:error).with(/upstream notifications tools\/list failed, omitting/) + end + + it "does not require a logger (config.logger defaults to nil)" do + expect { aggregator.tool_definitions }.not_to raise_error + end + end + + context "when the upstream list pull returns empty (a transient blip, not an error)" do + before { allow(client).to receive(:tools_list).and_return([]) } + + it "omits that upstream's tools" do + expect(aggregator.tool_definitions).to eq([]) + end + + it "does not cache the empty list, so a poisoned global cache cannot persist" do + aggregator.tool_definitions + aggregator.tool_definitions + + expect(client).to have_received(:tools_list).twice + end + + it "self-heals: once the upstream returns tools they are served (and cached)" do + aggregator.tool_definitions # empty pull, not cached + + allow(client).to receive(:tools_list).and_return(tools) + + expect(aggregator.tool_definitions).to eq([ + { "name" => "notifications__send_email", "description" => "Send", "inputSchema" => { "type" => "object" } } + ]) + end + end + + context "with a stale EMPTY entry already in the cache" do + it "treats it as a miss and re-pulls (self-heal from a previously poisoned entry)" do + McpToolkit.config.cache_store.write("#{described_class::CACHE_KEY_PREFIX}notifications", []) + + expect(aggregator.tool_definitions).to eq([ + { "name" => "notifications__send_email", "description" => "Send", "inputSchema" => { "type" => "object" } } + ]) + expect(client).to have_received(:tools_list).once + end + end + + context "across multiple upstreams" do + it "aggregates all upstreams' tools in registry order" do + McpToolkit.config.register_upstream(key: "billing", url: "https://billing.test/mcp") + allow(McpToolkit::Gateway::Client).to receive(:new) do |upstream:, **| + instance_double(McpToolkit::Gateway::Client).tap do |c| + allow(c).to receive(:tools_list).and_return( + [{ "name" => "#{upstream.key}_tool", "description" => "d", "inputSchema" => {} }] + ) + end + end + + names = aggregator.tool_definitions.map { |d| d["name"] } + expect(names).to eq(%w[notifications__notifications_tool billing__billing_tool]) + end + end + end +end diff --git a/spec/mcp_toolkit/gateway/client_spec.rb b/spec/mcp_toolkit/gateway/client_spec.rb new file mode 100644 index 0000000..f20db05 --- /dev/null +++ b/spec/mcp_toolkit/gateway/client_spec.rb @@ -0,0 +1,268 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe McpToolkit::Gateway::Client do + subject(:client) { described_class.new(upstream:, bearer_token: "tok-123") } + + let(:upstream_url) { "http://notifications.test/mcp" } + let(:upstream) do + McpToolkit::Gateway::UpstreamRegistry::Upstream.new(key: "notifications", url: upstream_url) + end + + # --- request stubs, routed by the JSON-RPC method in the POST body --------- + + def stub_initialize(session_id: "sess-1") + stub_request(:post, upstream_url) + .with(body: hash_including("method" => "initialize")) + .to_return( + status: 200, + headers: { "Content-Type" => "application/json", described_class::SESSION_HEADER => session_id }, + body: JSON.generate("jsonrpc" => "2.0", "id" => "x", "result" => { "protocolVersion" => "2025-06-18" }) + ) + end + + def stub_initialized + stub_request(:post, upstream_url) + .with(body: hash_including("method" => "notifications/initialized")) + .to_return(status: 202, body: "") + end + + def stub_method(method, *responses) + stub_request(:post, upstream_url).with(body: hash_including("method" => method)).to_return(*responses) + end + + def json_response(result) + { status: 200, headers: { "Content-Type" => "application/json" }, + body: JSON.generate("jsonrpc" => "2.0", "id" => "x", "result" => result) } + end + + def tools_payload + [{ "name" => "list_items", "description" => "List items.", "inputSchema" => { "type" => "object" } }] + end + + describe "#tools_list" do + before do + stub_initialize + stub_initialized + end + + it "handshakes, then returns the upstream's bare tools array" do + stub_method("tools/list", json_response("tools" => tools_payload)) + + expect(client.tools_list).to eq(tools_payload) + end + + it "sends the genericized clientInfo (config identity, not a hardcoded app name)" do + McpToolkit.config.server_name = "gateway-under-test" + McpToolkit.config.server_version = "9.9.9" + stub_method("tools/list", json_response("tools" => [])) + + client.tools_list + + expect(a_request(:post, upstream_url).with { |req| + JSON.parse(req.body)["params"]["clientInfo"] == { "name" => "gateway-under-test", "version" => "9.9.9" } + }).to have_been_made + end + + it "applies content-negotiation, auth and session headers on the list request" do + stub_method("tools/list", json_response("tools" => tools_payload)) + + client.tools_list + + expect( + a_request(:post, upstream_url).with( + body: hash_including("method" => "tools/list"), + headers: { + "Content-Type" => "application/json", + "Accept" => "application/json, text/event-stream", + "Authorization" => "Bearer tok-123", + described_class::SESSION_HEADER => "sess-1" + } + ) + ).to have_been_made + end + + it "unwraps a single SSE `data:` frame when the upstream answers text/event-stream" do + sse_body = "event: message\ndata: #{JSON.generate("jsonrpc" => "2.0", "id" => "x", + "result" => { "tools" => tools_payload })}\n\n" + stub_method( + "tools/list", + status: 200, headers: { "Content-Type" => "text/event-stream" }, body: sse_body + ) + + expect(client.tools_list).to eq(tools_payload) + end + end + + describe "#tools_call" do + before do + stub_initialize + stub_initialized + end + + it "forwards arguments and returns the upstream result verbatim" do + stub_method("tools/call", json_response("content" => [{ "type" => "text", "text" => "ok" }])) + + result = client.tools_call(name: "list_items", arguments: { "q" => "hi" }) + + expect(result).to eq("content" => [{ "type" => "text", "text" => "ok" }]) + end + + it "includes `_meta` in the params only when a meta hash is present" do + stub_method("tools/call", json_response("content" => [])) + + client.tools_call(name: "list_items", arguments: {}, meta: { "mcp-toolkit/account-id" => 42 }) + + expect(a_request(:post, upstream_url).with { |req| + JSON.parse(req.body)["params"]["_meta"] == { "mcp-toolkit/account-id" => 42 } + }).to have_been_made + end + + it "omits `_meta` when no meta is given" do + stub_method("tools/call", json_response("content" => [])) + + client.tools_call(name: "list_items", arguments: {}) + + expect(a_request(:post, upstream_url).with { |req| + parsed = JSON.parse(req.body) + parsed["method"] == "tools/call" && !parsed["params"].key?("_meta") + }).to have_been_made + end + end + + describe "error handling" do + before do + stub_initialize + stub_initialized + end + + it "raises Error carrying the upstream JSON-RPC error hash for verbatim relay" do + stub_method( + "tools/call", + json_response(nil).merge( + body: JSON.generate("jsonrpc" => "2.0", "id" => "x", + "error" => { "code" => -32_603, "message" => "boom", "data" => { "x" => 1 } }) + ) + ) + + expect { client.tools_call(name: "list_items", arguments: {}) } + .to raise_error(described_class::Error) do |e| + expect(e.jsonrpc_error).to eq("code" => -32_603, "message" => "boom", "data" => { "x" => 1 }) + expect(e.http_status).to be_nil + end + end + + it "raises Error carrying http_status for a non-2xx (non-session-loss) response" do + stub_method("tools/list", status: 500, body: "upstream exploded") + + expect { client.tools_list }.to raise_error(described_class::Error) do |e| + expect(e.http_status).to eq(500) + expect(e.jsonrpc_error).to be_nil + end + end + + it "raises Error on an unparseable body" do + stub_method("tools/list", status: 200, headers: { "Content-Type" => "application/json" }, body: "not json{") + + expect { client.tools_list }.to raise_error(described_class::Error, /unparseable body/) + end + + it "is an McpToolkit::Error (no transport/protocol coupling)" do + expect(described_class::Error.ancestors).to include(McpToolkit::Error) + end + end + + describe "single-shot session-loss recovery" do + it "recovers from a bare HTTP 404 by re-handshaking and retrying once" do + stub_initialize + stub_initialized + stub_method( + "tools/list", + { status: 404, body: "gone" }, + json_response("tools" => tools_payload) + ) + + expect(client.tools_list).to eq(tools_payload) + # initialize was performed twice: the original + the recovery handshake. + expect(a_request(:post, upstream_url).with(body: hash_including("method" => "initialize"))) + .to have_been_made.twice + end + + it "recovers from a JSON-RPC -32001 session error and retries once" do + stub_initialize + stub_initialized + stub_method( + "tools/call", + json_response(nil).merge( + body: JSON.generate("jsonrpc" => "2.0", "id" => "x", + "error" => { "code" => -32_001, "message" => "Session not found or expired" }) + ), + json_response("content" => [{ "type" => "text", "text" => "recovered" }]) + ) + + result = client.tools_call(name: "list_items", arguments: {}) + + expect(result).to eq("content" => [{ "type" => "text", "text" => "recovered" }]) + end + + it "logs a warning via config.logger on recovery" do + McpToolkit.config.logger = instance_double(Logger, warn: nil) + stub_initialize + stub_initialized + stub_method("tools/list", { status: 404, body: "gone" }, json_response("tools" => tools_payload)) + + client.tools_list + + expect(McpToolkit.config.logger).to have_received(:warn).with(/session lost.*re-initializing/) + end + + it "clarifies the error when recovery also fails (second attempt still 404)" do + stub_initialize + stub_initialized + stub_method("tools/list", { status: 404, body: "gone" }, { status: 404, body: "still gone" }) + + expect { client.tools_list }.to raise_error(described_class::Error) do |e| + expect(e.message).to match(/session was lost and could not be re-established/) + expect(e.http_status).to eq(404) + end + end + + it "does NOT retry a genuine (non-session-loss) JSON-RPC error" do + stub_initialize + stub_initialized + stub_method( + "tools/list", + json_response(nil).merge( + body: JSON.generate("jsonrpc" => "2.0", "id" => "x", + "error" => { "code" => -32_603, "message" => "boom" }) + ) + ) + + expect { client.tools_list }.to raise_error(described_class::Error, /boom/) + # A single initialize: no recovery handshake for a non-session-loss error. + expect(a_request(:post, upstream_url).with(body: hash_including("method" => "initialize"))) + .to have_been_made.once + end + end + + describe "DEFAULT_PROTOCOL_VERSION" do + it "sources the wrapped mcp SDK's latest-supported version" do + expect(described_class::DEFAULT_PROTOCOL_VERSION) + .to eq(MCP::Configuration::LATEST_STABLE_PROTOCOL_VERSION) + end + + it "offers config.protocol_version on the handshake when pinned" do + McpToolkit.config.protocol_version = "2025-03-26" + stub_initialize + stub_initialized + stub_method("tools/list", json_response("tools" => [])) + + client.tools_list + + expect(a_request(:post, upstream_url).with { |req| + JSON.parse(req.body)["params"]["protocolVersion"] == "2025-03-26" + }).to have_been_made + end + end +end diff --git a/spec/mcp_toolkit/gateway/introspect_endpoint_spec.rb b/spec/mcp_toolkit/gateway/introspect_endpoint_spec.rb new file mode 100644 index 0000000..c840a5b --- /dev/null +++ b/spec/mcp_toolkit/gateway/introspect_endpoint_spec.rb @@ -0,0 +1,152 @@ +# frozen_string_literal: true + +require "spec_helper" + +# Request spec for the gem-provided authority introspection endpoint +# (McpToolkit::TokensController#introspect), routed through McpToolkit::Engine at +# POST /mcp/tokens/introspect. +# +# The controller lives under the gem's app/controllers (an engine path) and calls +# ActionController methods (`request.headers`, `params`, `render`), so — like the +# engine route-reload regression — it can only be exercised against a REAL Rails +# app. Booting Rails in-process irreversibly mutates global state and would +# contaminate this otherwise Rails-absent suite across random orderings, so the +# boot runs in an ISOLATED CHILD Ruby process: it boots a minimal app configured +# as an authority, mounts the engine, dispatches a handful of requests through the +# full Rack stack, and prints each response's [status, parsed-body] as one JSON +# line for this parent to assert on. +# +# Rails-only: railties/actionpack are a TEST dependency (see Gemfile), skipped +# when Rails is unavailable. +rails_available = + begin + require "rails/version" + true + rescue LoadError + false + end + +RSpec.describe "Authority introspection endpoint (via the engine)", if: rails_available do + INTROSPECT_DRIVER = <<~'RUBY' + require "bundler/setup" + require "json" + require "tmpdir" + require "logger" + require "mcp_toolkit" + require "rails" + require "action_controller/railtie" + require "rack/mock" + + # A token object matching the gem's duck-typed authority contract. + token_class = Struct.new(:kind, :account_id, :account_ids, :expires_at, :scopes, keyword_init: true) do + def touch_last_used! = nil + end + valid_token = token_class.new( + kind: "accounts_user", account_id: 42, account_ids: [42], + expires_at: Time.utc(2026, 12, 31), scopes: ["notifications__read"] + ) + + McpToolkit.configure do |c| + c.parent_controller = "ActionController::Base" + c.auth_role = :authority + c.token_authenticator = ->(plaintext) { plaintext == "good" ? valid_token : nil } + end + + # mcp_toolkit was required before Rails, so load the (Zeitwerk-ignored) engine + # by path. + unless McpToolkit.const_defined?(:Engine, false) + load File.expand_path("lib/mcp_toolkit/engine.rb", Dir.pwd) + end + + app = Class.new(Rails::Application) do + config.eager_load = false + config.consider_all_requests_local = true + config.secret_key_base = "introspect-spec-secret-key-base" + config.logger = Logger.new(IO::NULL) + config.root = Dir.mktmpdir("mcp_toolkit_introspect_spec") + # Rack::MockRequest posts from host "example.org"; disable Rails 8's host + # authorization so the request isn't blocked with a 403. + config.hosts.clear + end + app.initialize! + app.routes.draw { mount McpToolkit::Engine => "/mcp" } + + rack = Rack::MockRequest.new(app) + path = "/mcp/tokens/introspect" + + call = lambda do |opts| + res = rack.post(opts.fetch(:path, path), opts.fetch(:env, {})) + body = res.body.to_s.empty? ? nil : JSON.parse(res.body) + { "status" => res.status, "body" => body } + end + + result = { + "valid_bearer" => call.call(env: { "HTTP_AUTHORIZATION" => "Bearer good" }), + "invalid_bearer" => call.call(env: { "HTTP_AUTHORIZATION" => "Bearer nope" }), + "missing_token" => call.call(env: {}), + "x_mcp_token" => call.call(env: { "HTTP_X_MCP_TOKEN" => "good" }), + "query_token" => call.call(path: "#{path}?token=good"), + } + + # Unconfigured-authority case: drawing the route is safe even with no + # token_authenticator -> {valid:false} rather than a 500. + McpToolkit.config.token_authenticator = nil + result["unconfigured"] = call.call(env: { "HTTP_AUTHORIZATION" => "Bearer good" }) + + puts JSON.generate(result) + RUBY + + before(:all) do + require "json" + require "open3" + + gem_root = File.expand_path("../../..", __dir__) + stdout, stderr, status = Open3.capture3(Gem.ruby, "-e", INTROSPECT_DRIVER, chdir: gem_root) + raise "introspect endpoint driver failed (#{status.exitstatus}):\n#{stderr}" unless status.success? + + @result = JSON.parse(stdout.lines.last) + end + + it "returns the exact introspection payload for a valid Bearer token (200)" do + valid = @result.fetch("valid_bearer") + + expect(valid.fetch("status")).to eq(200) + expect(valid.fetch("body")).to eq( + "valid" => true, + "kind" => "accounts_user", + "account_id" => 42, + "account_ids" => [42], + "expires_at" => "2026-12-31T00:00:00Z", + "scopes" => ["notifications__read"] + ) + end + + it "returns { valid: false } with 401 for an invalid token" do + invalid = @result.fetch("invalid_bearer") + + expect(invalid.fetch("status")).to eq(401) + expect(invalid.fetch("body")).to eq("valid" => false) + end + + it "returns { valid: false } with 401 when no token is presented" do + missing = @result.fetch("missing_token") + + expect(missing.fetch("status")).to eq(401) + expect(missing.fetch("body")).to eq("valid" => false) + end + + it "accepts the token via the X-MCP-Token header" do + expect(@result.fetch("x_mcp_token")).to match("status" => 200, "body" => a_hash_including("valid" => true)) + end + + it "accepts the token via the ?token= query param" do + expect(@result.fetch("query_token")).to match("status" => 200, "body" => a_hash_including("valid" => true)) + end + + it "answers { valid: false } (not a 500) when no token_authenticator is configured" do + unconfigured = @result.fetch("unconfigured") + + expect(unconfigured.fetch("status")).to eq(401) + expect(unconfigured.fetch("body")).to eq("valid" => false) + end +end diff --git a/spec/mcp_toolkit/gateway/proxy_spec.rb b/spec/mcp_toolkit/gateway/proxy_spec.rb new file mode 100644 index 0000000..8da6806 --- /dev/null +++ b/spec/mcp_toolkit/gateway/proxy_spec.rb @@ -0,0 +1,131 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe McpToolkit::Gateway::Proxy do + subject(:proxy) do + described_class.new(app_key: "notifications", tool_name: "send_email", account_id: 42, bearer_token: "caller_tok") + end + + let(:client) { instance_double(McpToolkit::Gateway::Client) } + let(:upstream) { McpToolkit.config.upstreams.find("notifications") } + + before do + McpToolkit.config.register_upstream(key: "notifications", url: "https://notif.test/mcp") + allow(McpToolkit::Gateway::Client).to receive(:new).and_return(client) + end + + describe "#call" do + it "forwards bearer, resolved account_id (as _meta) and arguments, returning the result verbatim" do + allow(client).to receive(:tools_call).and_return("content" => [{ "type" => "text", "text" => "ok" }]) + + result = proxy.call({ "to" => "a@b.c" }) + + expect(result).to eq("content" => [{ "type" => "text", "text" => "ok" }]) + expect(McpToolkit::Gateway::Client) + .to have_received(:new).with(upstream:, bearer_token: "caller_tok", config: McpToolkit.config) + expect(client).to have_received(:tools_call).with( + name: "send_email", + arguments: { "to" => "a@b.c" }, + meta: { "mcp-toolkit/account-id" => 42 } + ) + end + + it "uses the configured account_meta_key for the _meta selector" do + McpToolkit.config.account_meta_key = "example.com/account-id" + allow(client).to receive(:tools_call).and_return("content" => []) + + proxy.call({}) + + expect(client).to have_received(:tools_call) + .with(hash_including(meta: { "example.com/account-id" => 42 })) + end + + it "omits _meta entirely when no account_id is supplied" do + no_account = described_class.new(app_key: "notifications", tool_name: "send_email", bearer_token: "t") + allow(client).to receive(:tools_call).and_return("content" => []) + + no_account.call({}) + + expect(client).to have_received(:tools_call).with(hash_including(meta: nil)) + end + + it "raises UnknownUpstream for an unregistered app_key (no protocol coupling)" do + unknown = described_class.new(app_key: "ghost", tool_name: "x") + + expect { unknown.call({}) } + .to raise_error(McpToolkit::Gateway::UnknownUpstream, /Unknown application: ghost/) + end + end + + describe "error relay" do + it "re-raises a JSON-RPC upstream error as UpstreamCallError carrying jsonrpc_error" do + allow(client).to receive(:tools_call).and_raise( + McpToolkit::Gateway::Client::Error.new("upstream error", jsonrpc_error: { "code" => -32_602, "message" => "bad" }) + ) + + expect { proxy.call({}) }.to raise_error(McpToolkit::Gateway::UpstreamCallError) do |e| + expect(e.jsonrpc_error).to eq("code" => -32_602, "message" => "bad") + expect(e.http_status).to be_nil + end + end + + it "re-raises a transport failure as UpstreamCallError carrying http_status" do + allow(client).to receive(:tools_call).and_raise( + McpToolkit::Gateway::Client::Error.new("upstream notifications timed out", http_status: 404) + ) + + expect { proxy.call({}) }.to raise_error(McpToolkit::Gateway::UpstreamCallError) do |e| + expect(e.http_status).to eq(404) + expect(e.jsonrpc_error).to be_nil + end + end + + it "does NOT translate to any JSON-RPC / protocol error class (consumer maps it)" do + expect(McpToolkit::Gateway::UpstreamCallError.ancestors).to include(McpToolkit::Error) + expect(McpToolkit::Gateway::UnknownUpstream.ancestors).to include(McpToolkit::Error) + end + end + + describe "failure logging (greppable, never a token or a body)" do + let(:logger) { instance_double(Logger, error: nil) } + + before { McpToolkit.config.logger = logger } + + it "logs a proxied-call failure at ERROR with the upstream key, tool, and HTTP status" do + allow(client).to receive(:tools_call).and_raise( + McpToolkit::Gateway::Client::Error.new("upstream notifications tools/call send_email failed", http_status: 404) + ) + + expect { proxy.call({}) }.to raise_error(McpToolkit::Gateway::UpstreamCallError) + + expect(logger).to have_received(:error) + .with(a_string_including("MCP upstream notifications tools/call send_email failed", "http_status=404")) + end + + it "logs a relayed JSON-RPC error at ERROR with its code" do + allow(client).to receive(:tools_call).and_raise( + McpToolkit::Gateway::Client::Error.new("upstream error", jsonrpc_error: { "code" => -32_602, "message" => "bad" }) + ) + + expect { proxy.call({}) }.to raise_error(McpToolkit::Gateway::UpstreamCallError) + + expect(logger).to have_received(:error).with(a_string_including("jsonrpc_code=-32602")) + end + + it "never includes the bearer token in the log line" do + allow(client).to receive(:tools_call).and_raise(McpToolkit::Gateway::Client::Error.new("boom")) + + expect { proxy.call({}) }.to raise_error(McpToolkit::Gateway::UpstreamCallError) + + expect(logger).to have_received(:error).with(satisfy { |line| !line.include?("caller_tok") }) + end + + it "does not require a logger (config.logger defaults to nil)" do + McpToolkit.config.logger = nil + allow(client).to receive(:tools_call).and_raise(McpToolkit::Gateway::Client::Error.new("boom")) + + expect { proxy.call({}) }.to raise_error(McpToolkit::Gateway::UpstreamCallError) + end + end +end diff --git a/spec/mcp_toolkit/gateway/upstream_registry_spec.rb b/spec/mcp_toolkit/gateway/upstream_registry_spec.rb new file mode 100644 index 0000000..8840ea2 --- /dev/null +++ b/spec/mcp_toolkit/gateway/upstream_registry_spec.rb @@ -0,0 +1,99 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe McpToolkit::Gateway::UpstreamRegistry do + subject(:registry) { described_class.new } + + describe "#register" do + it "registers an upstream by key, exposed via #find and #all" do + registry.register(key: "notifications", url: "http://notifications.test/mcp") + + upstream = registry.find("notifications") + expect(upstream.key).to eq("notifications") + expect(upstream.url).to eq("http://notifications.test/mcp") + expect(registry.all).to eq([upstream]) + end + + it "normalizes a symbol key to a string" do + registry.register(key: :notifications, url: "http://notifications.test/mcp") + + expect(registry.find(:notifications)).to eq(registry.find("notifications")) + expect(registry.find("notifications").key).to eq("notifications") + end + + it "ignores a blank url so an unconfigured ENV lookup is a no-op" do + registry.register(key: "notifications", url: nil) + registry.register(key: "billing", url: "") + + expect(registry.all).to be_empty + end + + it "preserves insertion order in #all" do + registry.register(key: "a", url: "http://a.test") + registry.register(key: "b", url: "http://b.test") + + expect(registry.all.map(&:key)).to eq(%w[a b]) + end + end + + describe "#reset!" do + it "clears every registered upstream" do + registry.register(key: "notifications", url: "http://notifications.test/mcp") + + registry.reset! + + expect(registry.all).to be_empty + expect(registry.find("notifications")).to be_nil + end + end + + describe "#split_tool_name" do + before { registry.register(key: "notifications", url: "http://notifications.test/mcp") } + + it "splits a namespaced name into [key, bare] for a registered upstream" do + expect(registry.split_tool_name("notifications__list_items")).to eq(["notifications", "list_items"]) + end + + it "returns nil for an un-namespaced name" do + expect(registry.split_tool_name("list_items")).to be_nil + end + + it "returns nil when the key is not registered" do + expect(registry.split_tool_name("billing__list_items")).to be_nil + end + + it "returns nil when the bare tool name is empty" do + expect(registry.split_tool_name("notifications__")).to be_nil + end + end + + describe "Upstream#name_for" do + it "namespaces a bare tool name as `__`" do + upstream = described_class::Upstream.new(key: "notifications", url: "http://notifications.test/mcp") + + expect(upstream.name_for("list_items")).to eq("notifications__list_items") + end + end + + describe "the per-config instance + sugar" do + it "exposes a fresh registry per config as `config.upstreams`" do + expect(McpToolkit.config.upstreams).to be_a(described_class) + expect(McpToolkit.config.upstreams.all).to be_empty + end + + it "delegates `config.register_upstream` to `config.upstreams.register`" do + McpToolkit.config.register_upstream(key: "notifications", url: "http://notifications.test/mcp") + + expect(McpToolkit.config.upstreams.find("notifications").url).to eq("http://notifications.test/mcp") + end + + it "is reset by reset_config! (test isolation)" do + McpToolkit.config.register_upstream(key: "notifications", url: "http://notifications.test/mcp") + + McpToolkit.reset_config! + + expect(McpToolkit.config.upstreams.all).to be_empty + end + end +end diff --git a/spec/mcp_toolkit/session_spec.rb b/spec/mcp_toolkit/session_spec.rb index 44369aa..2f7f570 100644 --- a/spec/mcp_toolkit/session_spec.rb +++ b/spec/mcp_toolkit/session_spec.rb @@ -51,4 +51,34 @@ expect(described_class.delete(nil)).to be(false) end end + + describe "opaque data payload" do + it "defaults #data to {} when none is supplied on create" do + expect(described_class.create!.data).to eq({}) + end + + it "round-trips an opaque data payload through create -> find" do + created = described_class.create!(data: { token_id: 99 }) + + expect(created.data).to eq(token_id: 99) + expect(described_class.find(created.id).data).to eq(token_id: 99) + end + + it "defaults #data to {} for a legacy row written without the payload" do + # Simulate a pre-`data` cache row: only created_at, no :data key. + id = SecureRandom.uuid + McpToolkit.config.cache_store.write( + "#{described_class::CACHE_KEY_PREFIX}#{id}", { created_at: Time.now.to_i } + ) + + expect(described_class.find(id).data).to eq({}) + end + + it "preserves the data payload across the sliding-TTL rewrite on find" do + created = described_class.create!(data: { token_id: 7 }) + + described_class.find(created.id) # slides TTL, rewrites the row + expect(described_class.find(created.id).data).to eq(token_id: 7) + end + end end diff --git a/spec/mcp_toolkit_spec.rb b/spec/mcp_toolkit_spec.rb index e34a9bf..d68bfc7 100644 --- a/spec/mcp_toolkit_spec.rb +++ b/spec/mcp_toolkit_spec.rb @@ -37,6 +37,14 @@ expect(config.serializer_base).to eq(McpToolkit::Serializer::Base) end + it "ships vendor-neutral gateway + diagnostics defaults" do + expect(config.upstream_timeout).to eq(10) + expect(config.upstream_list_ttl).to eq(900) + expect(config.logger).to be_nil + expect(config.upstreams).to be_a(McpToolkit::Gateway::UpstreamRegistry) + expect(config.upstreams.all).to be_empty + end + it "defaults the account_resolver to identity" do expect(config.account_resolver.call(42)).to eq(42) end diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index 0795935..8a2447b 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -2,6 +2,8 @@ require "mcp_toolkit" require "webmock/rspec" +# stdlib Logger, so gateway specs can `instance_double(Logger, ...)` a config.logger. +require "logger" # Block all real HTTP; auth specs stub the central app's introspection endpoint. WebMock.disable_net_connect! From ffc5f0f6eb4dac38b5c0c3147651e7f71f1dc64d Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 15:18:20 +0200 Subject: [PATCH 02/10] Add authority + gateway dispatch path (0.5.0) Phase-2 extraction: relocate the hand-rolled JSON-RPC dispatcher + protocol into the gem (vendor-neutralized) as a second dispatch front-end alongside the SDK-backed satellite path (which is left untouched). - Protocol: JSON-RPC envelope + error codes + 3-version negotiation constants. - Dispatcher(context:, config:): initialize/tools/list/tools/call/ping + upstream list_changed; host tools via config.tool_provider (scope-gated centrally), namespaced calls via Gateway::Proxy with verbatim error relay. - Authority::ControllerMethods: authority transport concern with every billing/tenancy step an overridable hook (auth/rate-limit/usage/account/ session/dispatch/health); per-request account loop preserved across batches. - Authority::ServerController: subclassable base (lazily-parented). - Authority::Context: per-request account/principal/bearer_token/superuser?. - Tools::AuthorityBase: optional tool base over the api-agnostic tool_provider seam; the gem never references a host's API/serializers/catalog. - Config: tool_provider, supported_protocol_versions, rate_limiter/ usage_recorder/usage_flusher, gateway_client_name/version (identity split so the gateway handshake stays byte-identical while server_name is the host's). - Lazy parent_controller (Constraint B): delete the eager-loadable engine controllers; build them (+ the authority base) from current config via build_engine_controllers!, triggered by const_missing and reset on reload by the engine's to_prepare, so the parent is read after the host's to_prepare. Specs: dispatcher, dispatcher_gateway, protocol, authority/controller_methods (hooks + mixed-account batch loop), authority_controller_lazy_parent (real-app subprocess boot), tool_provider_contract, tools/authority_base, and a Gateway::Client identity-split spec. 284 examples, 0 failures; rubocop clean; public-gem diff is fingerprint-free. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 74 ++++ README.md | 120 ++++++- .../mcp_toolkit/server_controller.rb | 19 - .../mcp_toolkit/tokens_controller.rb | 47 --- lib/mcp_toolkit.rb | 10 + lib/mcp_toolkit/authority.rb | 24 ++ lib/mcp_toolkit/authority/context.rb | 35 ++ .../authority/controller_methods.rb | 318 +++++++++++++++++ lib/mcp_toolkit/configuration.rb | 95 +++++ lib/mcp_toolkit/dispatcher.rb | 228 ++++++++++++ lib/mcp_toolkit/engine.rb | 9 + lib/mcp_toolkit/engine_controllers.rb | 116 ++++++ lib/mcp_toolkit/gateway/client.rb | 9 +- lib/mcp_toolkit/protocol.rb | 106 ++++++ lib/mcp_toolkit/tools/authority_base.rb | 127 +++++++ lib/mcp_toolkit/version.rb | 2 +- .../authority/controller_methods_spec.rb | 334 ++++++++++++++++++ .../authority_controller_lazy_parent_spec.rb | 94 +++++ spec/mcp_toolkit/dispatcher_gateway_spec.rb | 144 ++++++++ spec/mcp_toolkit/dispatcher_spec.rb | 179 ++++++++++ spec/mcp_toolkit/engine_spec.rb | 17 +- spec/mcp_toolkit/gateway/client_spec.rb | 40 +++ spec/mcp_toolkit/protocol_spec.rb | 102 ++++++ .../tool_provider_contract_spec.rb | 100 ++++++ spec/mcp_toolkit/tools/authority_base_spec.rb | 132 +++++++ spec/support/fake_authority.rb | 92 +++++ 26 files changed, 2493 insertions(+), 80 deletions(-) delete mode 100644 app/controllers/mcp_toolkit/server_controller.rb delete mode 100644 app/controllers/mcp_toolkit/tokens_controller.rb create mode 100644 lib/mcp_toolkit/authority.rb create mode 100644 lib/mcp_toolkit/authority/context.rb create mode 100644 lib/mcp_toolkit/authority/controller_methods.rb create mode 100644 lib/mcp_toolkit/dispatcher.rb create mode 100644 lib/mcp_toolkit/engine_controllers.rb create mode 100644 lib/mcp_toolkit/protocol.rb create mode 100644 lib/mcp_toolkit/tools/authority_base.rb create mode 100644 spec/mcp_toolkit/authority/controller_methods_spec.rb create mode 100644 spec/mcp_toolkit/authority_controller_lazy_parent_spec.rb create mode 100644 spec/mcp_toolkit/dispatcher_gateway_spec.rb create mode 100644 spec/mcp_toolkit/dispatcher_spec.rb create mode 100644 spec/mcp_toolkit/protocol_spec.rb create mode 100644 spec/mcp_toolkit/tool_provider_contract_spec.rb create mode 100644 spec/mcp_toolkit/tools/authority_base_spec.rb create mode 100644 spec/support/fake_authority.rb diff --git a/CHANGELOG.md b/CHANGELOG.md index 2241ce2..36bc626 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,77 @@ +## [0.5.0] - 2026-07-06 + +### Added + +- **Authority dispatch path** — a hand-rolled JSON-RPC front-end for a first-party + server that authenticates tokens locally and serves its OWN tools (and, as a + gateway, aggregates + proxies upstreams) WITHOUT the official `mcp` SDK in the + request path. It coexists with the SDK-backed satellite path + (`McpToolkit::Server.build`) by design — the gem now carries two dispatch + front-ends, each for its role. The satellite path is unchanged. + - `McpToolkit::Protocol` — JSON-RPC constants + envelope helpers + (`SUPPORTED_VERSIONS`, `LATEST_VERSION`, `JSONRPC_VERSION`, `ErrorCodes`, + `Error` + subclasses with `#code`/`#data`/`#to_h`, `success_response` / + `error_response`). The byte contract of a first-party endpoint's error + envelope + version negotiation. + - `McpToolkit::Dispatcher` — `new(context:, config:)` + `#handle_request(request)`. + Dispatches `initialize` / `initialized` / `tools/list` / `tools/call` / `ping` + and the custom `notifications//tools/list_changed` cache-bust. `tools/list` + merges the host's own tool definitions with the gateway's namespaced upstream + tools; `tools/call` routes a namespaced name to `Gateway::Proxy` (translating + `UnknownUpstream` → method-not-found and relaying an upstream JSON-RPC error + verbatim) or a host tool (scope-gated) otherwise. Server identity + negotiated + versions come from config; NO SDK touchpoint. + - `McpToolkit::Authority::ControllerMethods` — the authority transport as an + includable concern. Every billing/tenancy step is an overridable hook + (`mcp_authenticate!`, `mcp_rate_limit!`, `mcp_track_usage`, `mcp_flush_usage`, + `mcp_resolve_account`, `mcp_session_data`, `mcp_dispatch`, `mcp_health_payload`, + `mcp_config`), each defaulting to a config callable so a PURE host needs no + subclass. The per-request loop RE-RESOLVES the account for every JSON-RPC call + — including each element of a batch — so a mixed-account batch still meters one + usage event per call against the right account (the batch is never delegated to + a bulk handler that couldn't re-resolve per element). + - `McpToolkit::Authority::ServerController` — a base controller (concern wired in, + lazily-parented) a host subclasses when its hooks touch app models (the + recommended path). + - `McpToolkit::Authority::Context` — the per-request context threaded into the + dispatcher + tools: `account`, `principal`, `bearer_token`, and a derived + `superuser?` (duck-typed off the principal). + - `McpToolkit::Tools::AuthorityBase` — an optional base for a host's own tools + (class DSL `tool_name` / `description` / `input_schema` / + `required_permissions_scope` / `definition`; `.call(context:, **arguments)` + entry; context accessors; `ensure_resource_accessible!`; ArgumentError → + InvalidParams / StandardError → InternalError mapping). Host tools plug in + through the api-agnostic `config.tool_provider` seam — the gem never references + a host's API layer, serializers, or resource catalog. +- **`config.tool_provider`** — the api-agnostic tool seam. Duck-typed: + `provider.tool_definitions(context) -> [{ name:, description:, inputSchema: }]` + (context lets the host hide superuser-only tools) and `provider.find(name) -> a + tool object` (responding to `#required_permissions_scope` + `#call(context:, + **arguments)`). The dispatcher enforces the per-tool scope gate CENTRALLY. +- **Server-vs-gateway identity split** — `config.gateway_client_name` / + `gateway_client_version` (each defaulting to `server_name` / `server_version`). + `Gateway::Client`'s handshake `clientInfo` now reads the GATEWAY identity, so an + authority can advertise its own `server_name` to its callers while keeping its + upstream handshake byte-identical. +- **`config.supported_protocol_versions`** (default + `McpToolkit::Protocol::SUPPORTED_VERSIONS`) — the version set the authority + dispatcher negotiates. +- **`config.rate_limiter` / `usage_recorder` / `usage_flusher`** — the authority + transport's billing hooks as config callables (all default `nil` / no-op). +- **Lazy `parent_controller` (Constraint B)** — the engine's `ServerController` / + `TokensController` and the authority `ServerController` are no longer eager- + loadable files; they are built from the CURRENT config by + `McpToolkit.build_engine_controllers!`, triggered lazily via `const_missing` and + reset on each reload by the engine's `config.to_prepare`. The parent is therefore + read only at build time — after the host's initializers/to_prepare — so a host's + whole MCP initializer can live in `to_prepare`. `TokensController#introspect` + behavior is preserved exactly. + +### Removed + +- The engine's `app/controllers/mcp_toolkit/{server_controller,tokens_controller}.rb` + files, replaced by the lazy builder above (their routes + behavior are unchanged). + ## [0.4.0] - 2026-07-06 ### Added diff --git a/README.md b/README.md index f9e6e45..685f7b3 100644 --- a/README.md +++ b/README.md @@ -249,6 +249,7 @@ discovery tool, a custom serializer may also expose `declared_attributes` / | Setting | Default | Purpose | |---|---|---| | `server_name` / `server_version` / `server_instructions` | `"mcp-server"` / `"1.0.0"` / `nil` | advertised on `initialize` | +| `gateway_client_name` / `gateway_client_version` | `server_name` / `server_version` | `clientInfo` a gateway presents to its upstreams (identity split) | | `serializer_base` | `McpToolkit::Serializer::Base` | the default base to subclass | | `auth_role` | `:satellite` | `:satellite` or `:authority` | | `central_app_url` | `nil` | satellite: base URL of the auth authority | @@ -259,8 +260,11 @@ discovery tool, a custom serializer may also expose `declared_attributes` / | `token_authenticator` | `nil` | authority: `->(plaintext) { token_or_nil }` | | `cache_store` | `MemoryStore` | sessions + introspection cache (set to `Rails.cache`) | | `session_ttl` | `3600` | session sliding TTL (s) | -| `protocol_version` | `nil` (negotiate) | pin an MCP protocol version | -| `parent_controller` | `"ActionController::Base"` | superclass of the engine's `ServerController` (set to `"ApplicationController"` for `helper_method` compat) | +| `protocol_version` | `nil` (negotiate) | pin an MCP protocol version (satellite/upstream client) | +| `supported_protocol_versions` | `Protocol::SUPPORTED_VERSIONS` | version set the authority dispatcher negotiates | +| `tool_provider` | `nil` | authority: the host's api-agnostic tool catalog (see below) | +| `rate_limiter` / `usage_recorder` / `usage_flusher` | `nil` | authority transport billing hooks (config callables) | +| `parent_controller` | `"ActionController::Base"` | superclass of the engine's controllers, read lazily (set to `"ActionController::API"` for the authority, or `"ApplicationController"` for `helper_method` compat) | | `account_meta_key` | `"mcp-toolkit/account-id"` | `_meta` key a superuser uses to pin the account | | `account_id_header` | `"X-MCP-Account-ID"` | header fallback for the account selector | | `upstreams` | empty registry | gateway: registered upstream MCP servers (register via `register_upstream`) | @@ -277,11 +281,22 @@ discovery tool, a custom serializer may also expose `declared_attributes` / `#default_required_permissions_scope` - `McpToolkit::Serializer::Base` (DSL: `attributes`, `has_one`, `has_many`, `translates`) -- `McpToolkit::Server.build(server_context:, config:, extra_tools:)` +- `McpToolkit::Server.build(server_context:, config:, extra_tools:)` (satellite, + SDK-backed) - `McpToolkit::Engine` (mountable; `mount McpToolkit::Engine => "/mcp"`) + - `McpToolkit::ServerController` (its controller; parent via `parent_controller`) -- `McpToolkit::Transport::ControllerMethods` (standalone controller concern; - override `mcp_config` / `mcp_extra_tools`) + `McpToolkit::ServerController` (its controller; parent via `parent_controller`, + built lazily) +- `McpToolkit::Transport::ControllerMethods` (standalone satellite controller + concern; override `mcp_config` / `mcp_extra_tools`) +- **Authority dispatch path** (a first-party server serving its own tools + + upstreams, no SDK): `McpToolkit::Protocol`, + `McpToolkit::Dispatcher.new(context:, config:)#handle_request`, + `McpToolkit::Authority::ControllerMethods` (the transport concern, all + billing/tenancy steps overridable hooks), + `McpToolkit::Authority::ServerController` (subclassable base), + `McpToolkit::Authority::Context` (`account` / `principal` / `bearer_token` / + `superuser?`), `McpToolkit::Tools::AuthorityBase` (optional tool base), and + `config.tool_provider` (the api-agnostic tool seam) - `McpToolkit::Session` (opaque `#data` payload, e.g. to bind a session to a token id) - `McpToolkit::Auth::Introspection` / `Authenticator` (satellite), `McpToolkit::Auth::Authority` (authority) @@ -367,6 +382,99 @@ mount McpToolkit::Engine => "/mcp" # POST /mcp/tokens/introspect now works Drawing it is safe even on an app that is not an authority: with no `token_authenticator`, it simply answers `{ "valid": false }`. +## Authority + gateway server (own tools + upstreams, no SDK) + +Beyond the SDK-backed satellite path, the toolkit also ships a **hand-rolled +dispatch path** for a first-party server that authenticates tokens LOCALLY and +serves its OWN tools — and, as a gateway, aggregates + proxies upstreams — with +the official `mcp` SDK out of the request path. The gem carries the two dispatch +front-ends side by side: `McpToolkit::Server.build` (satellite) and +`McpToolkit::Dispatcher` (authority). The wire behavior of the authority path — +top-level JSON-RPC tool-error codes, `initialize` capabilities, 3-version +negotiation, verbatim upstream error relay, the custom `list_changed` cache-bust — +is fixed, so a monetized endpoint keeps its byte contract. + +### 1. Expose your tools through a provider (the api-agnostic seam) + +The gem never sees your API layer. It serves your tools only through a duck-typed +`tool_provider` you register: + +```ruby +# provider.tool_definitions(context) -> [{ name:, description:, inputSchema: }] +# provider.find(name) -> a tool object, or nil +# +# a tool object responds to: +# #required_permissions_scope -> String | nil (the gem's per-tool scope gate) +# #call(context:, **arguments) -> Hash | String (wrapped into { content: [...] }) +McpToolkit.configure do |c| + c.tool_provider = MyToolProvider.new # your glue over your own tool classes +end +``` + +A tool MAY subclass the bundled base (or be any object satisfying the contract): + +```ruby +class ListWidgets < McpToolkit::Tools::AuthorityBase + tool_name "list_widgets" + description "List widgets for the active account." + required_permissions_scope "widgets__read" # gem gates this centrally + input_schema { { type: "object", properties: { limit: { type: "integer" } } } } + + # `account` / `principal` / `bearer_token` / `superuser?` come from the context. + def call(limit: 25) + Widget.for(account).limit(limit).map(&:as_json) # your domain, behind #call + end +end +``` + +`context` is an `McpToolkit::Authority::Context` (`account`, `principal`, +`bearer_token`, `superuser?`). It is re-created for EVERY JSON-RPC call — including +each element of a batch — so a mixed-account batch resolves the right account per +call. + +### 2. Serve it through the authority transport + +A **pure host** mounts the engine's authority base and drives everything from +config callables. A host whose rate-limit / usage / account logic touches its own +models **subclasses** the base and overrides the hook methods: + +```ruby +class ServerController < McpToolkit::Authority::ServerController + # Local token auth, session binding, and account resolution have working + # defaults (duck-typed on your token via config.token_authenticator). Override + # only what touches your models: + def mcp_rate_limit! + # ...your limiter; render + halt when over the limit... + end + + def mcp_track_usage(request_data, account) + # ...accumulate one usage row for this call (fires per batch element)... + end + + def mcp_flush_usage = # ...bulk-insert the accumulated rows... + def mcp_session_data = { token_id: mcp_principal.id } # revoked token kills the session +end +``` + +Every billing/tenancy step is an overridable hook: `mcp_authenticate!`, +`mcp_rate_limit!`, `mcp_track_usage`, `mcp_flush_usage`, `mcp_resolve_account`, +`mcp_session_data`, `mcp_dispatch`, `mcp_health_payload`, `mcp_config`. The +per-request loop (`resolve account → track usage → dispatch`) is preserved across +batches, so usage metering survives a mixed-account batch. + +Point your `POST /mcp` route at the subclass (or mount the engine for a pure host); +keep `POST /mcp/tokens/introspect` on the gem's `TokensController`. + +### Lazy `parent_controller` + +The gem's controllers subclass `config.parent_controller`. That parent is read +only at **build** time — the controllers are built by +`McpToolkit.build_engine_controllers!`, triggered lazily on first reference and +reset on each reload by the engine's `to_prepare` — so it is always resolved AFTER +your app's initializers/to_prepare. Your whole MCP initializer can therefore live +in `to_prepare`. Set `c.parent_controller = "ActionController::API"` for the +authority. + ## Development ```bash diff --git a/app/controllers/mcp_toolkit/server_controller.rb b/app/controllers/mcp_toolkit/server_controller.rb deleted file mode 100644 index dc80cba..0000000 --- a/app/controllers/mcp_toolkit/server_controller.rb +++ /dev/null @@ -1,19 +0,0 @@ -# frozen_string_literal: true - -# The MCP transport controller, provided BY the gem so a satellite mounting -# McpToolkit::Engine writes no controller of its own. It is the standalone -# McpToolkit::Transport::ControllerMethods concern wired into a controller. -# -# Its parent class is configurable (Doorkeeper-style) via -# `McpToolkit.config.parent_controller` (default "ActionController::Base"), so a -# satellite can keep ActionController::Base (NOT ::API) for its logstasher -# `helper_method` hook: -# -# c.parent_controller = "ApplicationController" -# -# Lives under the gem's app/controllers (an engine path), so it's loaded by Rails' -# autoloader via the engine — never by the gem's own Zeitwerk loader, which only -# manages lib/. Non-Rails consumers never see it. -class McpToolkit::ServerController < McpToolkit.config.parent_controller.constantize - include McpToolkit::Transport::ControllerMethods -end diff --git a/app/controllers/mcp_toolkit/tokens_controller.rb b/app/controllers/mcp_toolkit/tokens_controller.rb deleted file mode 100644 index b72564f..0000000 --- a/app/controllers/mcp_toolkit/tokens_controller.rb +++ /dev/null @@ -1,47 +0,0 @@ -# frozen_string_literal: true - -# The AUTHORITY-side introspection endpoint, provided BY the gem so a central app -# answers the introspection requests satellites send without writing a controller -# of its own. Mounted by McpToolkit::Engine at `POST /mcp/tokens/introspect`. -# -# Like McpToolkit::ServerController, its parent class is configurable -# (Doorkeeper-style) via `McpToolkit.config.parent_controller`. It authenticates -# the bearer against `config.token_authenticator` (touching last-used) and renders -# the exact payload McpToolkit::Auth::Authority builds — the contract the -# satellite's Auth::Introspection parses. -# -# Drawing the route unconditionally is safe: an app that never configured a -# `token_authenticator` (i.e. is not an authority) answers `{ valid: false }` -# rather than erroring, so mounting the engine on a pure satellite is harmless. -# -# Lives under the gem's app/controllers (an engine path), so it is loaded by -# Rails' autoloader via the engine — never by the gem's own Zeitwerk loader, which -# only manages lib/. Non-Rails consumers never see it. -class McpToolkit::TokensController < McpToolkit.config.parent_controller.constantize - def introspect - token = McpToolkit::Auth::Authority.authenticate(mcp_extract_token, config: mcp_config) - return render(json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized) if token.nil? - - render json: McpToolkit::Auth::Authority.introspection_payload(token) - rescue McpToolkit::Errors::ConfigurationError - # Not configured as an authority (no token_authenticator): behave as if the - # token were invalid instead of surfacing a 500. - render json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized - end - - private - - # Overridable, mirroring the transport concern's `mcp_config` hook. - def mcp_config - McpToolkit.config - end - - # Bearer extraction order: `Authorization: Bearer`, then `X-MCP-Token`, then - # `?token=`. - def mcp_extract_token - auth_header = request.headers["Authorization"] - return auth_header.sub("Bearer ", "") if auth_header&.start_with?("Bearer ") - - request.headers["X-MCP-Token"].presence || params[:token].presence - end -end diff --git a/lib/mcp_toolkit.rb b/lib/mcp_toolkit.rb index 0eb3f01..55ea3df 100644 --- a/lib/mcp_toolkit.rb +++ b/lib/mcp_toolkit.rb @@ -66,8 +66,18 @@ # under `app/controllers` (outside this lib-rooted loader), so it needs no ignore; # it is loaded by Rails' autoloader via the engine when mounted. loader.ignore("#{__dir__}/mcp_toolkit/engine.rb") +# `engine_controllers.rb` reopens `McpToolkit` to add the lazy `parent_controller` +# builder + `const_missing` (it defines module methods, not a file-named +# constant), so it is required explicitly below rather than autoloaded. +loader.ignore("#{__dir__}/mcp_toolkit/engine_controllers.rb") loader.setup +# The lazy `parent_controller` builder (McpToolkit.build_engine_controllers! / +# reset_engine_controllers! / const_missing). Required unconditionally (it names no +# Rails constant at load time); it only builds controllers when one is referenced, +# which happens only under Rails. +require_relative "mcp_toolkit/engine_controllers" + # The toolkit for building account-scoped, read-only MCP servers on top of the # official `mcp` gem. See README.md for the satellite + authority quickstarts. # diff --git a/lib/mcp_toolkit/authority.rb b/lib/mcp_toolkit/authority.rb new file mode 100644 index 0000000..0098c2d --- /dev/null +++ b/lib/mcp_toolkit/authority.rb @@ -0,0 +1,24 @@ +# frozen_string_literal: true + +# Namespace for the AUTHORITY-side building blocks: the per-request Context, the +# transport concern (Authority::ControllerMethods), and the lazily-parented base +# controller (Authority::ServerController). +# +# `ServerController` is NOT a file under this directory: like the engine's +# controllers, it subclasses a host controller (`config.parent_controller`) that +# is absent from the gem's own unit suite, so it cannot be a Zeitwerk-managed +# file (eager-loading it would raise). It is built on demand by +# `McpToolkit.build_engine_controllers!`, so this `const_missing` triggers that +# build the first time `McpToolkit::Authority::ServerController` is referenced +# (which, in a host, is after the app's initializers/to_prepare have run — so the +# configured parent is read at build time, not at autoload time). +module McpToolkit::Authority + def self.const_missing(name) + if name == :ServerController + McpToolkit.build_engine_controllers! + return const_get(name) if const_defined?(name, false) + end + + super + end +end diff --git a/lib/mcp_toolkit/authority/context.rb b/lib/mcp_toolkit/authority/context.rb new file mode 100644 index 0000000..5408f7f --- /dev/null +++ b/lib/mcp_toolkit/authority/context.rb @@ -0,0 +1,35 @@ +# frozen_string_literal: true + +# The per-JSON-RPC-request context the authority transport threads into the +# dispatcher and, through it, into each host tool. Re-created for EVERY element of +# a batch so each call carries its own resolved account (the property usage +# metering relies on). +# +# It carries three host-supplied values and derives one: +# * `account` — the tenant resolved for THIS call (nil when none applies). +# The object need only respond to `#id` (forwarded upstream +# as the account selector). +# * `principal` — the authenticated caller (the token object). Duck-typed; +# the gem calls `#authorized_for_scope?(scope)` (tool scope +# gate) and, optionally, `#superuser?` (see `#superuser?`). +# The transport's account resolution additionally uses +# `#default_account` / `#authorize_account(id)`. +# * `bearer_token` — the raw bearer, forwarded to upstream MCP servers so they +# introspect the same token and resolve the same account. +# * `superuser?` — derived: true only when the principal responds to +# `#superuser?` truthily. Lets a host tool base +# (McpToolkit::Tools::AuthorityBase) gate superuser-only +# resources without the gem naming any app concept. +class McpToolkit::Authority::Context + attr_reader :account, :principal, :bearer_token + + def initialize(account:, principal:, bearer_token: nil) + @account = account + @principal = principal + @bearer_token = bearer_token + end + + def superuser? + principal.respond_to?(:superuser?) && !!principal.superuser? + end +end diff --git a/lib/mcp_toolkit/authority/controller_methods.rb b/lib/mcp_toolkit/authority/controller_methods.rb new file mode 100644 index 0000000..bf92b1f --- /dev/null +++ b/lib/mcp_toolkit/authority/controller_methods.rb @@ -0,0 +1,318 @@ +# frozen_string_literal: true + +# The AUTHORITY-side MCP Streamable-HTTP transport, provided as an includable +# concern. Unlike the satellite transport (McpToolkit::Transport::ControllerMethods, +# which forwards a token to a central app for per-tool introspection), the +# authority AUTHENTICATES the token locally and dispatches through the hand-rolled +# McpToolkit::Dispatcher, serving its own tools and (as a gateway) proxying +# upstreams. +# +# Because the POST endpoint is the billing/tenancy boundary of a first-party +# server, EVERY billing/tenancy step is an overridable hook. A pure host can drive +# the whole thing from config callables (`rate_limiter`, `usage_recorder`, +# `usage_flusher`, `tool_provider`, `token_authenticator`); a host whose metering +# touches its own models subclasses McpToolkit::Authority::ServerController and +# overrides the hook methods directly (the recommended path). +# +# Endpoints +# POST /mcp - JSON-RPC requests/responses (single or batch) +# GET /mcp - server-initiated SSE stream (none emitted; 405) +# DELETE /mcp - terminate the current session +# GET /mcp/health - unauthenticated health probe +# +# Per-request loop (the metering-critical invariant) +# Each JSON-RPC call — including every element of a batch — RE-RESOLVES its +# account from its own `_meta` / `account_id` argument, then tracks usage, then +# dispatches with a fresh Authority::Context. The batch is deliberately NOT +# delegated to a bulk handler that can't re-resolve per element, so a mixed- +# account batch still meters one usage event per call against the right account. +# +# Overridable hooks (defaults in parentheses) +# mcp_config -> the McpToolkit::Configuration (McpToolkit.config) +# mcp_authenticate! -> set @mcp_principal or render 401 (local token auth via +# config.token_authenticator, through Auth::Authority) +# mcp_rate_limit! -> throttle (config.rate_limiter&.call) +# mcp_track_usage -> record one usage event (config.usage_recorder&.call) +# mcp_flush_usage -> persist accumulated usage (config.usage_flusher&.call) +# mcp_resolve_account -> the account for one call (principal#default_account / +# principal#authorize_account(id)) +# mcp_session_data -> opaque payload bound to the session ({}; a host binds +# e.g. { token_id: principal.id } so a revoked token kills +# the session) +# mcp_dispatch -> run one JSON-RPC call (Dispatcher + Authority::Context) +# mcp_health_payload -> the GET /mcp/health body +# +# CSRF: the concern disables forgery protection (this is a token-authenticated +# JSON API). Its host controller should inherit from ActionController::API (or +# ::Base with null_session) via `config.parent_controller`. +module McpToolkit::Authority::ControllerMethods + extend ActiveSupport::Concern + + SESSION_HEADER = "Mcp-Session-Id" + + included do + protect_from_forgery with: :null_session if respond_to?(:protect_from_forgery) + + before_action :mcp_authenticate!, except: [:health] + before_action :mcp_rate_limit!, except: [:health] + before_action :mcp_resolve_session!, only: [:create] + before_action :mcp_require_session!, only: [:destroy] + after_action :mcp_flush_usage + end + + def create + request_body = mcp_parse_body + + if request_body.is_a?(Array) + mcp_handle_batch(request_body) + else + mcp_handle_single(request_body) + end + end + + # GET /mcp — opens an SSE stream for server-initiated messages. None are emitted + # (no sampling, progress, or notifications today), so we reply 405 as the MCP + # spec explicitly allows. + def stream + head :method_not_allowed + end + + # DELETE /mcp — terminate the current session. + def destroy + McpToolkit::Session.delete(request.headers[SESSION_HEADER], config: mcp_config) + head :no_content + end + + # GET /mcp/health — unauthenticated health probe. + def health + render json: mcp_health_payload + end + + private + + # ---- overridable hooks ---------------------------------------------- + + def mcp_config + McpToolkit.config + end + + # The authenticated principal for this request (the token object), set by + # mcp_authenticate!. Read by the other hooks. + def mcp_principal + @mcp_principal + end + + # Authenticate the bearer LOCALLY (the authority's job). The default resolves it + # through Auth::Authority (config.token_authenticator), which also touches + # last-used. Renders a JSON-RPC 401 and halts on a missing/invalid token. + def mcp_authenticate! + token = mcp_extract_token + return mcp_render_unauthorized("Missing authorization token") if token.blank? + + @mcp_principal = McpToolkit::Auth::Authority.authenticate(token, config: mcp_config) + mcp_render_unauthorized("Invalid or expired token") unless @mcp_principal + end + + # Throttle the request. The default delegates to `config.rate_limiter` (a + # `->(controller:, principal:)` that renders + halts when over the limit, or + # sets headers when under). A host whose limiter touches app models overrides + # this method wholesale. + def mcp_rate_limit! + mcp_config.rate_limiter&.call(controller: self, principal: mcp_principal) + end + + # Record one usage event for a single JSON-RPC call. The default delegates to + # `config.usage_recorder`; a host that accumulates into its own ledger overrides + # this. MUST never affect the MCP response. + def mcp_track_usage(request_data, account) + mcp_config.usage_recorder&.call( + request_data:, account:, principal: mcp_principal, controller: self + ) + end + + # Persist accumulated usage (after_action). The default delegates to + # `config.usage_flusher`. MUST never affect the MCP response. + def mcp_flush_usage + mcp_config.usage_flusher&.call(controller: self) + end + + # Pick the active account for a SINGLE JSON-RPC call. Duck-typed on the + # principal: no candidate -> its default account (nil for a multi-account + # token); a candidate -> the authorized account or a JSON-RPC InvalidParams. + def mcp_resolve_account(request_data) + candidate = mcp_candidate_account_id(request_data) + return mcp_principal.default_account if candidate.blank? + + account = mcp_principal.authorize_account(candidate) + return account if account + + raise McpToolkit::Protocol::InvalidParams, "account_id #{candidate.inspect} is not authorized for this token" + end + + # Opaque payload bound to the session on `initialize`. Default: none. A host + # binds e.g. `{ token_id: mcp_principal.id }` so a revoked token can kill an + # in-flight session. + def mcp_session_data + {} + end + + # Run one JSON-RPC call through the hand-rolled dispatcher with a fresh + # per-request context. + def mcp_dispatch(request_data, account) + context = McpToolkit::Authority::Context.new( + account:, principal: mcp_principal, bearer_token: mcp_extract_token + ) + McpToolkit::Dispatcher.new(context:, config: mcp_config).handle_request(request_data) + end + + def mcp_health_payload + { + status: "ok", + server: mcp_config.server_name, + version: mcp_config.server_version, + protocol_version: McpToolkit::Protocol::LATEST_VERSION + } + end + + # ---- request handling ----------------------------------------------- + + def mcp_handle_batch(requests) + responses = requests.filter_map { |req| mcp_process_single_request(req) } + return head :accepted if responses.empty? + + mcp_render_response(responses) + end + + def mcp_handle_single(request_data) + response_data = mcp_process_single_request(request_data) + return head :accepted if response_data.nil? + + mcp_render_response(response_data) + end + + # The per-request loop body: resolve THIS call's account, meter it, dispatch. + # A protocol error raised while resolving the account (e.g. an unauthorized + # account id) becomes this call's JSON-RPC error, leaving sibling batch elements + # untouched. + def mcp_process_single_request(request_data) + account = mcp_resolve_account(request_data) + mcp_track_usage(request_data, account) + mcp_dispatch(request_data, account) + rescue McpToolkit::Protocol::Error => e + return nil unless request_data.is_a?(Hash) && request_data.key?("id") + + McpToolkit::Protocol.error_response(id: request_data["id"], error: e) + end + + # Renders the JSON-RPC payload as application/json (default) or text/event-stream + # when the client's Accept header includes "text/event-stream". We never actually + # stream — one message then EOF — but emitting SSE on demand keeps strict MCP + # clients happy. + def mcp_render_response(payload) + if mcp_event_stream_requested? + mcp_render_sse_stream(payload) + else + render json: payload + end + end + + def mcp_render_sse_stream(payload) + response.headers["Cache-Control"] = "no-cache" + messages = payload.is_a?(Array) ? payload : [payload] + body = messages.map { |message| mcp_format_sse_event(message) }.join + + render body:, content_type: Mime::Type.lookup("text/event-stream").to_s + end + + def mcp_format_sse_event(message) + "event: message\ndata: #{message.to_json}\n\n" + end + + def mcp_event_stream_requested? + request.headers["Accept"].to_s.include?("text/event-stream") + end + + # ---- session lifecycle ---------------------------------------------- + + # POST: create a session on `initialize` (binding mcp_session_data), otherwise + # require an existing one. + def mcp_resolve_session! + methods = mcp_methods_from(mcp_parse_body) + + if methods.include?("initialize") + @mcp_session = McpToolkit::Session.create!(data: mcp_session_data, config: mcp_config) + else + @mcp_session = McpToolkit::Session.find(request.headers[SESSION_HEADER], config: mcp_config) + return mcp_render_session_not_found if @mcp_session.nil? + end + + response.headers[SESSION_HEADER] = @mcp_session.id + end + + def mcp_require_session! + @mcp_session = McpToolkit::Session.find(request.headers[SESSION_HEADER], config: mcp_config) + mcp_render_session_not_found if @mcp_session.nil? + end + + # ---- token + account extraction ------------------------------------- + + # Token sources, highest priority first: Bearer header, legacy X-MCP-Token + # header, then the `token` query param (the header-less client fallback). + def mcp_extract_token + auth_header = request.headers["Authorization"] + return auth_header.sub("Bearer ", "") if auth_header&.start_with?("Bearer ") + + request.headers["X-MCP-Token"].presence || params[:token].presence + end + + # Account selector for a single call, highest priority first: params._meta key, + # tools/call `account_id` argument, then the account-id header (request-wide + # fallback). Key/header names come from config so a host on a specific authority + # can match that authority's convention. + def mcp_candidate_account_id(request_data) + params = request_data.to_h["params"].to_h + meta = params["_meta"].to_h + arguments = params["arguments"].to_h + + meta[mcp_config.account_meta_key] || + arguments["account_id"] || + request.headers[mcp_config.account_id_header] + end + + # ---- error renders -------------------------------------------------- + + def mcp_render_unauthorized(message) + render json: { + jsonrpc: McpToolkit::Protocol::JSONRPC_VERSION, + id: nil, + error: { + code: -32_000, + message: "Unauthorized: #{message}" + } + }, status: :unauthorized + end + + def mcp_render_session_not_found + render json: { + jsonrpc: McpToolkit::Protocol::JSONRPC_VERSION, + id: nil, + error: { code: -32_001, message: "Session not found or expired" } + }, status: :not_found + end + + # ---- body parsing --------------------------------------------------- + + def mcp_parse_body + return @mcp_parsed_body if defined?(@mcp_parsed_body) + + @mcp_parsed_body = JSON.parse(request.body.read) + rescue JSON::ParserError => e + raise McpToolkit::Protocol::ParseError, "Invalid JSON: #{e.message}" + end + + def mcp_methods_from(request_body) + # Array.wrap (not Kernel#Array): a single JSON-RPC Hash must wrap to [hash], + # whereas Kernel#Array(hash) would explode it into [[k, v], ...]. + Array.wrap(request_body).filter_map { |req| req.is_a?(Hash) ? req["method"] : nil } + end +end diff --git a/lib/mcp_toolkit/configuration.rb b/lib/mcp_toolkit/configuration.rb index 9afbbd4..e7877b5 100644 --- a/lib/mcp_toolkit/configuration.rb +++ b/lib/mcp_toolkit/configuration.rb @@ -25,6 +25,24 @@ class McpToolkit::Configuration # @return [String, nil] human-readable `instructions` returned on `initialize`. attr_accessor :server_instructions + # --- gateway client identity (identity split) ------------------------------ + + # The `clientInfo` an app presents when it acts as a GATEWAY talking to its + # upstream MCP servers (McpToolkit::Gateway::Client's handshake). Split from the + # SERVER identity (`server_name`/`server_version`, advertised to the app's OWN + # callers) so an authority can present its real server identity downstream while + # keeping its upstream handshake byte-identical to a prior deployment. Both + # default to the server identity, so a satellite/gateway that doesn't care sets + # nothing. + # + # c.server_name = "acme-mcp" # advertised to our callers + # c.gateway_client_name = "acme-mcp-gateway" # presented to our upstreams + # + # @return [String] gateway handshake client name (defaults to `server_name`). + attr_writer :gateway_client_name + # @return [String] gateway handshake client version (defaults to `server_version`). + attr_writer :gateway_client_version + # --- serialization --------------------------------------------------------- # The DEFAULT serializer base class. A `Resource` registration that does not @@ -124,6 +142,14 @@ class McpToolkit::Configuration # nil lets the gem negotiate (recommended). Set only to force an older spec. attr_accessor :protocol_version + # The protocol versions the hand-rolled AUTHORITY dispatcher + # (McpToolkit::Dispatcher) negotiates, newest first. `initialize` echoes the + # requested version when it is in this set, else the first (latest). Defaults to + # McpToolkit::Protocol::SUPPORTED_VERSIONS; override to pin a host's own set. + # + # @return [Array] + attr_accessor :supported_protocol_versions + # The parent class (as a String, resolved via `constantize`) of the # gem-provided McpToolkit::ServerController that McpToolkit::Engine mounts. # Doorkeeper-style indirection so a satellite mounting the engine can keep @@ -169,6 +195,48 @@ class McpToolkit::Configuration # @return [McpToolkit::Gateway::UpstreamRegistry] attr_reader :upstreams + # --- authority hooks ------------------------------------------------------- + # + # Injection points for the AUTHORITY transport (McpToolkit::Authority:: + # ControllerMethods) so a PURE host drives billing/tenancy from config without + # subclassing. A host whose logic touches its own models overrides the matching + # hook METHOD on its McpToolkit::Authority::ServerController subclass instead; + # then these stay nil. All default to nil (a no-op). + + # Throttles a request. `->(controller:, principal:)`; renders + halts when over + # the limit (or sets rate-limit headers when under). nil = no throttling. + # + # @return [#call, nil] + attr_accessor :rate_limiter + + # Records ONE usage event for a single JSON-RPC call (called per batch element). + # `->(request_data:, account:, principal:, controller:)`. MUST never affect the + # MCP response. nil = no metering. + # + # @return [#call, nil] + attr_accessor :usage_recorder + + # Persists accumulated usage after the response (an after_action). + # `->(controller:)`. MUST never affect the MCP response. nil = no flush. + # + # @return [#call, nil] + attr_accessor :usage_flusher + + # The host's tool catalog — the api-agnostic seam. Duck-typed; the dispatcher + # calls: + # + # provider.tool_definitions(context) -> [{ name:, description:, inputSchema: }] + # provider.find(name) -> a tool object, or nil + # + # where a tool object responds to `#required_permissions_scope` (String|nil, the + # gem's scope gate) and `#call(context:, **arguments)` (returns Hash|String, + # which the gem wraps into `{ content: [{ type: "text", text: }] }`). See + # McpToolkit::Tools::AuthorityBase for a base that satisfies this. nil = the host + # contributes no own tools (a pure gateway). + # + # @return [#tool_definitions, #find, nil] + attr_accessor :tool_provider + # --- diagnostics ----------------------------------------------------------- # Optional logger for gateway/session diagnostics. All call sites guard with @@ -184,6 +252,9 @@ def initialize @server_version = "1.0.0" @server_instructions = nil + @gateway_client_name = nil + @gateway_client_version = nil + @serializer_base = nil # set lazily in #serializer_base to avoid load-order issues @auth_role = :satellite @@ -201,10 +272,13 @@ def initialize @sql_sanitizer = McpToolkit::SqlSanitizer.new @protocol_version = nil + @supported_protocol_versions = McpToolkit::Protocol::SUPPORTED_VERSIONS @parent_controller = "ActionController::Base" @account_meta_key = "mcp-toolkit/account-id" @account_id_header = "X-MCP-Account-ID" + initialize_authority_hook_defaults + @upstream_timeout = 10 @upstream_list_ttl = 900 # 15 minutes @logger = nil @@ -213,6 +287,15 @@ def initialize @upstreams = McpToolkit::Gateway::UpstreamRegistry.new end + # The authority transport's injection points all default to nil (a no-op): a + # pure satellite/gateway never touches them. + def initialize_authority_hook_defaults + @rate_limiter = nil + @usage_recorder = nil + @usage_flusher = nil + @tool_provider = nil + end + # Config sugar: register a gateway upstream. Delegates to `upstreams.register`, # so a blank url is ignored (an unconfigured upstream is simply absent). # @@ -221,6 +304,18 @@ def register_upstream(key:, url:) upstreams.register(key:, url:) end + # The gateway handshake client name, defaulting to the server identity when the + # host hasn't split it. Read (not stored) so a `server_name` change before the + # split is set still flows through. + def gateway_client_name + @gateway_client_name || server_name + end + + # The gateway handshake client version, defaulting to the server version. + def gateway_client_version + @gateway_client_version || server_version + end + # The serializer base, lazily defaulting to the gem's bundled DSL base. Lazy so # `McpToolkit::Serializer::Base` is referenced after it has been required, # regardless of file load order. diff --git a/lib/mcp_toolkit/dispatcher.rb b/lib/mcp_toolkit/dispatcher.rb new file mode 100644 index 0000000..39bf4ee --- /dev/null +++ b/lib/mcp_toolkit/dispatcher.rb @@ -0,0 +1,228 @@ +# frozen_string_literal: true + +# Hand-rolled JSON-RPC dispatcher for the AUTHORITY + gateway path: it handles a +# single JSON-RPC request (one element of a batch), serving the host's own tools +# and, as a gateway, aggregating + proxying upstream MCP servers — WITHOUT the +# official `mcp` SDK in the request path. +# +# The gem carries two dispatch front-ends by design (see McpToolkit::Protocol): +# this Dispatcher for the authority/gateway endpoint, and McpToolkit::Server.build +# (the SDK wrapper) for satellites. They are independent; nothing here touches the +# satellite path. +# +# Everything host-specific is injected: +# * `context` — an McpToolkit::Authority::Context (the resolved account, the +# authenticated principal, and the bearer token to forward +# upstream). Re-created PER JSON-RPC request by the transport, so +# each batch element carries its own account. +# * `config` — server identity (`server_name`/`server_version`), the +# negotiable protocol versions, the registered `upstreams`, and +# the `tool_provider` (the host's api-agnostic tool catalog). +# +# The wire behavior — top-level JSON-RPC tool-error codes, `initialize` +# capabilities `{ tools: { listChanged: true } }`, 3-version negotiation, verbatim +# upstream error relay, and the custom `notifications//tools/list_changed` +# cache-bust — is the byte contract of a first-party endpoint and is preserved +# exactly. +class McpToolkit::Dispatcher + attr_reader :context, :config + + def initialize(context:, config: McpToolkit.config) + @context = context + @config = config + end + + def handle_request(request) + dispatch_request(request) + rescue McpToolkit::Protocol::Error => e + return nil unless request.key?("id") + + McpToolkit::Protocol.error_response(id: request["id"], error: e) + rescue StandardError => e + config.logger&.error("MCP dispatcher error: #{e.message}\n#{e.backtrace&.join("\n")}") + return nil unless request.key?("id") + + McpToolkit::Protocol.error_response( + id: request["id"], + error: McpToolkit::Protocol::InternalError.new(e.message) + ) + end + + private + + # Happy path; raises here are turned into JSON-RPC errors by handle_request. + def dispatch_request(request) + validate_request!(request) + + result = dispatch_method(request["method"], request["params"] || {}) + + # JSON-RPC 2.0: notifications (requests without `id`) MUST NOT receive a response. + return nil unless request.key?("id") + + McpToolkit::Protocol.success_response(id: request["id"], result:) + end + + def validate_request!(request) + unless request["jsonrpc"] == McpToolkit::Protocol::JSONRPC_VERSION + raise McpToolkit::Protocol::InvalidRequest, "Missing jsonrpc version" + end + raise McpToolkit::Protocol::InvalidRequest, "Missing method" if request["method"].blank? + end + + def dispatch_method(method, params) + case method + when "initialize" + handle_initialize(params) + when "initialized", "notifications/initialized" + handle_initialized + when "tools/list" + handle_tools_list + when "tools/call" + handle_tools_call(params) + when "ping" + handle_ping + else + return handle_upstream_list_changed(method) if upstream_list_changed_notification?(method) + + raise McpToolkit::Protocol::MethodNotFound, method + end + end + + # Satellites can tell the authority their tool list changed via a + # `notifications//tools/list_changed` notification, busting that upstream's + # cached aggregate. Matches the configured upstream keys only. + def upstream_list_changed_notification?(method) + upstream_key_from_notification(method).present? + end + + def handle_upstream_list_changed(method) + McpToolkit::Gateway::Aggregator.new(config:).flush!(upstream_key_from_notification(method)) + {} + end + + def upstream_key_from_notification(method) + match = method.to_s.match(%r{\Anotifications/(?.+)/tools/list_changed\z}) + return nil unless match + + config.upstreams.find(match[:key]) ? match[:key] : nil + end + + def handle_initialize(params) + requested = params["protocolVersion"].to_s + versions = config.supported_protocol_versions + negotiated = versions.include?(requested) ? requested : versions.first + + { + protocolVersion: negotiated, + capabilities: { + # listChanged: true — the aggregated list includes upstream tools, which + # can change when an upstream is reconfigured or sends a list_changed + # notification that busts the cached aggregate. + tools: { listChanged: true } + }, + serverInfo: { + name: config.server_name, + version: config.server_version + } + } + end + + def handle_initialized + {} + end + + def handle_ping + {} + end + + def handle_tools_list + { + tools: host_tool_definitions + + McpToolkit::Gateway::Aggregator.new(config:).tool_definitions(bearer_token: context.bearer_token) + } + end + + # The host's own tool definitions, sourced from the injected tool_provider (the + # api-agnostic seam). `context` lets the provider hide superuser-only tools from + # a non-superuser caller. A host that registered no provider contributes none. + def host_tool_definitions + provider = config.tool_provider + return [] unless provider + + provider.tool_definitions(context) + end + + def handle_tools_call(params) + tool_name = params["name"] + arguments = params["arguments"] || {} + + upstream = config.upstreams.split_tool_name(tool_name) + return handle_upstream_tools_call(upstream, arguments) if upstream + + handle_host_tools_call(tool_name, arguments) + end + + def handle_host_tools_call(tool_name, arguments) + tool = config.tool_provider&.find(tool_name) + raise McpToolkit::Protocol::MethodNotFound, "Tool not found: #{tool_name}" unless tool + + ensure_tool_scope!(tool) + + result = tool.call(context:, **symbolized_arguments(arguments)) + + { + content: [ + { + type: "text", + text: result.is_a?(String) ? result : result.to_json + } + ] + } + end + + # JSON gives string keys; a tool's `call(context:, **arguments)` needs symbol + # keys for the keyword splat. Deep-symbolized so nested argument hashes reach + # the tool in the same shape a symbol-keyed caller would pass. + def symbolized_arguments(arguments) + arguments.to_h.deep_symbolize_keys + end + + def ensure_tool_scope!(tool) + required_scope = tool.required_permissions_scope + return if required_scope.blank? + return if context.principal&.authorized_for_scope?(required_scope) + + raise McpToolkit::Protocol::InvalidRequest, "This token lacks the #{required_scope.inspect} scope" + end + + def handle_upstream_tools_call((app_key, bare_tool_name), arguments) + McpToolkit::Gateway::Proxy.new( + app_key:, + tool_name: bare_tool_name, + account_id: context.account&.id, + bearer_token: context.bearer_token, + config: + ).call(arguments) + rescue McpToolkit::Gateway::UnknownUpstream => e + # The gateway stays transport-agnostic; the dispatcher maps an unknown + # upstream to its own "method not found". + raise McpToolkit::Protocol::MethodNotFound, e.message + rescue McpToolkit::Gateway::UpstreamCallError => e + raise translate_upstream_call_error(e) + end + + # Translates the gateway's transport-agnostic upstream failure into the JSON-RPC + # error shape — verbatim relay of a satellite JSON-RPC error, else a generic + # internal error. + def translate_upstream_call_error(error) + if error.jsonrpc_error + McpToolkit::Protocol::Error.new( + error.jsonrpc_error["message"].to_s, + code: error.jsonrpc_error["code"] || McpToolkit::Protocol::ErrorCodes::INTERNAL_ERROR, + data: error.jsonrpc_error["data"] + ) + else + McpToolkit::Protocol::InternalError.new(error.message) + end + end +end diff --git a/lib/mcp_toolkit/engine.rb b/lib/mcp_toolkit/engine.rb index e8a76cd..2abea4f 100644 --- a/lib/mcp_toolkit/engine.rb +++ b/lib/mcp_toolkit/engine.rb @@ -20,4 +20,13 @@ # non-Rails consumers and its own unit suite never reference it. class McpToolkit::Engine < Rails::Engine isolate_namespace McpToolkit + + # The gem-provided controllers subclass `config.parent_controller`, which the + # host sets in an initializer/to_prepare that must be READ AFTER it runs + # (Constraint B). They are therefore built lazily by + # `McpToolkit.build_engine_controllers!` (triggered via const_missing on first + # reference — at eager-load or first request). This resets them on every code + # reload so a changed parent (or a reloaded app parent class) takes effect on the + # next reference. Runs before `:eager_load!`, so the fresh classes exist for it. + config.to_prepare { McpToolkit.reset_engine_controllers! } end diff --git a/lib/mcp_toolkit/engine_controllers.rb b/lib/mcp_toolkit/engine_controllers.rb new file mode 100644 index 0000000..5f97115 --- /dev/null +++ b/lib/mcp_toolkit/engine_controllers.rb @@ -0,0 +1,116 @@ +# frozen_string_literal: true + +# Lazy `parent_controller` builder (Constraint B). +# +# The engine's controllers (McpToolkit::ServerController, +# McpToolkit::TokensController) and the authority base +# (McpToolkit::Authority::ServerController) all subclass +# `config.parent_controller`. If that superclass were resolved in a class body of +# an autoloaded/eager-loaded file, it could be read BEFORE the host's +# initializer/to_prepare had set it — defaulting to ActionController::Base and, in +# turn, breaking CSRF handling on the introspection endpoint. +# +# Instead, none of these controllers is a Zeitwerk-managed file. They are built +# here from the CURRENT config, and the build is triggered LAZILY: +# * `const_missing` (below, and on McpToolkit::Authority) builds them the first +# time they are referenced — which, in a host, is at eager-load or first- +# request time, i.e. AFTER the app's initializers/to_prepare have run; +# * the engine's `config.to_prepare` RESETS them on every code reload so a +# changed `parent_controller` (or a reloaded app parent class) takes effect on +# the next reference. +# +# The whole `config/initializers/mcp_toolkit.rb` of a host can therefore live in +# `to_prepare`: the parent is only ever read at build time. +# +# This file reopens `McpToolkit` to add module methods, so it is Zeitwerk-IGNORED +# (like engine.rb) and required explicitly from the gem entry point. +module McpToolkit + # The controllers built directly under McpToolkit (the engine's routes point at + # these). McpToolkit::Authority::ServerController is built alongside them but is + # fetched through McpToolkit::Authority's own const_missing. + ENGINE_CONTROLLER_NAMES = %i[ServerController TokensController].freeze + + # (Re)builds the engine controllers + the authority base from the current + # config. Idempotent: an existing constant is replaced so a rebuild reflects a + # changed `parent_controller`. Reads `config.parent_controller` at call time. + def self.build_engine_controllers! + parent = config.parent_controller.constantize + define_controller(self, :ServerController, build_server_controller(parent)) + define_controller(self, :TokensController, build_tokens_controller(parent)) + define_controller(Authority, :ServerController, build_authority_server_controller(parent)) + ServerController + end + + # Undefines the built controllers so the next reference rebuilds them from the + # then-current config. Called from the engine's `to_prepare` on every reload. + def self.reset_engine_controllers! + [[self, :ServerController], [self, :TokensController], [Authority, :ServerController]].each do |mod, name| + mod.send(:remove_const, name) if mod.const_defined?(name, false) + end + end + + # The SATELLITE transport controller the engine mounts (unchanged behavior; the + # SDK-backed path). Built lazily only so its parent is read after config. + def self.build_server_controller(parent) + Class.new(parent) { include McpToolkit::Transport::ControllerMethods } + end + + # The AUTHORITY introspection endpoint the engine mounts at + # `POST /mcp/tokens/introspect`. Behavior is preserved exactly: it authenticates + # the bearer against `config.token_authenticator` (via Auth::Authority) and + # renders the introspection payload; a non-authority app answers `{ valid: + # false }` rather than erroring. + def self.build_tokens_controller(parent) + Class.new(parent) do + def introspect + token = McpToolkit::Auth::Authority.authenticate(mcp_extract_token, config: mcp_config) + return render(json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized) if token.nil? + + render json: McpToolkit::Auth::Authority.introspection_payload(token) + rescue McpToolkit::Errors::ConfigurationError + # Not configured as an authority (no token_authenticator): behave as if the + # token were invalid instead of surfacing a 500. + render json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized + end + + private + + def mcp_config + McpToolkit.config + end + + def mcp_extract_token + auth_header = request.headers["Authorization"] + return auth_header.sub("Bearer ", "") if auth_header&.start_with?("Bearer ") + + request.headers["X-MCP-Token"].presence || params[:token].presence + end + end + end + + # The AUTHORITY base controller a host subclasses (the recommended path for a + # host whose rate-limit/usage/account hooks touch app models). + def self.build_authority_server_controller(parent) + Class.new(parent) { include McpToolkit::Authority::ControllerMethods } + end + + # Removes an existing same-named constant (avoiding a redefinition warning on a + # rebuild) before setting the freshly-built class. + def self.define_controller(mod, name, klass) + mod.send(:remove_const, name) if mod.const_defined?(name, false) + mod.const_set(name, klass) + end + + # Backstop: build the engine controllers the first time one is referenced before + # any `to_prepare`/eager-load pass has built them (e.g. a bespoke route that + # names McpToolkit::ServerController directly). McpToolkit::Authority defines the + # sibling backstop for its ServerController. + def self.const_missing(name) + if ENGINE_CONTROLLER_NAMES.include?(name) + build_engine_controllers! + return const_get(name) if const_defined?(name, false) + end + + super + end +end diff --git a/lib/mcp_toolkit/gateway/client.rb b/lib/mcp_toolkit/gateway/client.rb index f023436..0d9a2ae 100644 --- a/lib/mcp_toolkit/gateway/client.rb +++ b/lib/mcp_toolkit/gateway/client.rb @@ -143,7 +143,14 @@ def ensure_initialized! jsonrpc_request("initialize", { "protocolVersion" => protocol_version, "capabilities" => {}, - "clientInfo" => { "name" => config.server_name, "version" => config.server_version } + # The GATEWAY-client identity (falls back to the server + # identity), split so an authority can advertise its own + # server_name to callers while keeping this upstream + # handshake byte-identical. + "clientInfo" => { + "name" => config.gateway_client_name, + "version" => config.gateway_client_version + } }) ) @session_id = response.headers[SESSION_HEADER].presence diff --git a/lib/mcp_toolkit/protocol.rb b/lib/mcp_toolkit/protocol.rb new file mode 100644 index 0000000..22bef3a --- /dev/null +++ b/lib/mcp_toolkit/protocol.rb @@ -0,0 +1,106 @@ +# frozen_string_literal: true + +# MCP JSON-RPC protocol constants + helpers for the hand-rolled AUTHORITY +# dispatcher (McpToolkit::Dispatcher). Based on the Model Context Protocol +# specification. +# +# This is the wire vocabulary of a first-party MCP endpoint that serves its own +# tools (and, as a gateway, aggregates upstream ones) WITHOUT the official `mcp` +# SDK in the request path. The SDK-backed satellite path +# (McpToolkit::Server.build) does not use this module; the two dispatch +# front-ends coexist by design. +# +# The error codes, the success/error response envelopes, and the +# version-negotiation constants here are the BYTE contract a monetized authority +# endpoint depends on, so they are kept fixed. `SUPPORTED_VERSIONS` is the module +# default; a host negotiates against `config.supported_protocol_versions` +# (defaulting to the same list) so the set is overridable without editing this +# file. +module McpToolkit::Protocol + # Protocol versions the server supports, newest first. The version returned in + # the `initialize` response is the requested version (if supported) or the + # latest the server supports, per the MCP spec's version-negotiation rules. + SUPPORTED_VERSIONS = %w[2025-06-18 2025-03-26 2024-11-05].freeze + LATEST_VERSION = SUPPORTED_VERSIONS.first + # Kept for backwards compatibility; prefer LATEST_VERSION going forward. + VERSION = LATEST_VERSION + + JSONRPC_VERSION = "2.0" + + # Error codes per JSON-RPC 2.0 spec. + module ErrorCodes + PARSE_ERROR = -32_700 + INVALID_REQUEST = -32_600 + METHOD_NOT_FOUND = -32_601 + INVALID_PARAMS = -32_602 + INTERNAL_ERROR = -32_603 + end + + # Base protocol error. `code`/`data` land verbatim in the JSON-RPC `error` + # object via `#to_h`; the dispatcher turns a raised Error into a top-level + # JSON-RPC error response (the envelope a client sees for a bad tool arg or an + # unknown method). + class Error < StandardError + attr_reader :code, :data + + def initialize(message, code:, data: nil) + super(message) + @code = code + @data = data + end + + def to_h + error = { code:, message: } + error[:data] = data if data + error + end + end + + class ParseError < Error + def initialize(message = "Parse error", data: nil) + super(message, code: ErrorCodes::PARSE_ERROR, data:) + end + end + + class InvalidRequest < Error + def initialize(message = "Invalid request", data: nil) + super(message, code: ErrorCodes::INVALID_REQUEST, data:) + end + end + + class MethodNotFound < Error + def initialize(method_name, data: nil) + super("Method not found: #{method_name}", code: ErrorCodes::METHOD_NOT_FOUND, data:) + end + end + + class InvalidParams < Error + def initialize(message = "Invalid params", data: nil) + super(message, code: ErrorCodes::INVALID_PARAMS, data:) + end + end + + class InternalError < Error + def initialize(message = "Internal error", data: nil) + super(message, code: ErrorCodes::INTERNAL_ERROR, data:) + end + end + + module_function + + def success_response(id:, result:) + { + jsonrpc: JSONRPC_VERSION, + id:, + result: + } + end + + def error_response(id:, error:) + { + jsonrpc: JSONRPC_VERSION, + id:, + error: error.is_a?(Error) ? error.to_h : error + } + end +end diff --git a/lib/mcp_toolkit/tools/authority_base.rb b/lib/mcp_toolkit/tools/authority_base.rb new file mode 100644 index 0000000..e45e036 --- /dev/null +++ b/lib/mcp_toolkit/tools/authority_base.rb @@ -0,0 +1,127 @@ +# frozen_string_literal: true + +# Optional base class for a HOST's own tools served by the authority dispatcher +# (McpToolkit::Dispatcher). A host tool MAY subclass this, or may be any object +# satisfying the duck-typed tool contract the dispatcher calls: +# +# tool.required_permissions_scope -> String | nil (gem's scope gate) +# tool.call(context:, **arguments) -> Hash | String (gem wraps into +# { content: [...] }) +# +# The dispatcher treats the object returned by `provider.find(name)` as the tool; +# an AuthorityBase SUBCLASS satisfies the contract as CLASS methods (the class +# `.call` instantiates, runs, and error-maps a single invocation). +# +# What this base adds over hand-rolling the contract: +# * a class DSL (`tool_name` / `description` / `input_schema` / +# `required_permissions_scope` / `definition`) mirroring the tool-definition +# shape `tools/list` returns; +# * per-request accessors (`account` / `principal` / `bearer_token` / +# `superuser?`) read from the injected Authority::Context; +# * `ensure_resource_accessible!` to gate a superuser-only resource; +# * error mapping — an ArgumentError (e.g. a missing required kwarg) becomes an +# InvalidParams, any other StandardError an InternalError, while a +# deliberately-raised McpToolkit::Protocol::Error passes through with its own +# code. +# +# The gem NEVER references a host's API layer, serializers, or resource catalog — +# all of that lives behind the host's `#call`. +class McpToolkit::Tools::AuthorityBase + class << self + attr_reader :_tool_name, :_description, :_input_schema + + def tool_name(name = nil) + if name + @_tool_name = name.to_s + else + @_tool_name || self.name.to_s.demodulize.underscore.gsub(/_tool$/, "") + end + end + + def description(desc = nil) + @_description = desc if desc + @_description + end + + def input_schema(&block) + @_input_schema = yield if block + @_input_schema || { type: "object", properties: {} } + end + + # OAuth-style scope (`__`) a token must carry to call this tool, + # enforced by the dispatcher before the tool runs. Defaults to nil (no scope + # required). NOT inherited — a subclass that doesn't declare its own scope is + # unscoped, even if an ancestor declared one. + def required_permissions_scope(scope = nil) + @_required_permissions_scope = scope.to_s if scope + @_required_permissions_scope + end + + def definition + { + name: tool_name, + description: _description, + inputSchema: _input_schema || { type: "object", properties: {} } + } + end + + # The dispatcher's entry point: build an instance bound to this request's + # context and run it, mapping tool-level errors to protocol errors. + def call(context:, **arguments) + new(context:).execute(**arguments) + end + end + + attr_reader :context + + def initialize(context:) + @context = context + end + + def account + context.account + end + + def principal + context.principal + end + + def bearer_token + context.bearer_token + end + + # Whether the caller is a superuser, per the Context (which duck-types it off + # the principal). Used to gate resources/tools that expose cross-tenant data. + def superuser? + context.superuser? + end + + # Guards a resource flagged `superusers_only?`: a non-superuser caller is + # refused. No-op for unrestricted resources. + def ensure_resource_accessible!(resource) + return unless resource.superusers_only? + return if superuser? + + raise McpToolkit::Protocol::InvalidRequest, "#{resource.name} is restricted to superuser (user-scoped) MCP tokens" + end + + # Runs the tool's business logic (the subclass's `#call`) with error mapping. + # Arrives with symbol-keyed arguments from the dispatcher. + def execute(**arguments) + call(**arguments) + rescue McpToolkit::Protocol::Error + # A deliberately-raised protocol error carries its own JSON-RPC code + # (e.g. InvalidParams); let it bubble untouched so the client sees it. + raise + rescue ArgumentError => e + raise McpToolkit::Protocol::InvalidParams, e.message + rescue StandardError => e + raise McpToolkit::Protocol::InternalError, e.message + end + + # The subclass implements its business logic here, receiving the tool arguments + # as keywords and returning a Hash or String. + def call(**_arguments) + raise NotImplementedError, "#{self.class} must implement #call" + end +end diff --git a/lib/mcp_toolkit/version.rb b/lib/mcp_toolkit/version.rb index 3b017eb..b0c7d2f 100644 --- a/lib/mcp_toolkit/version.rb +++ b/lib/mcp_toolkit/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module McpToolkit - VERSION = "0.4.0" + VERSION = "0.5.0" end diff --git a/spec/mcp_toolkit/authority/controller_methods_spec.rb b/spec/mcp_toolkit/authority/controller_methods_spec.rb new file mode 100644 index 0000000..2507ce0 --- /dev/null +++ b/spec/mcp_toolkit/authority/controller_methods_spec.rb @@ -0,0 +1,334 @@ +# frozen_string_literal: true + +require "spec_helper" +require "stringio" + +# The authority transport concern runs WITHOUT Rails in the gem's suite, so this +# drives it against a minimal host class providing only the surface the concern +# touches: the class hooks its `included do` invokes (before_action/after_action; +# protect_from_forgery is behind respond_to? and thus skipped), plus request / +# response / render / head / params. +# +# before_actions are NOT auto-run here (there is no filter runner), so each example +# invokes the relevant hook/action directly — which is exactly what lets us pin the +# per-request account loop and the hook seams in isolation. +RSpec.describe McpToolkit::Authority::ControllerMethods do + let(:controller_class) do + Class.new do + def self.before_action(*); end + def self.after_action(*); end + include McpToolkit::Authority::ControllerMethods + + attr_accessor :request_body, :request_headers, :params + attr_reader :rendered, :head_status + + def initialize + @request_headers = {} + @params = {} + @response_headers = {} + end + + def request + @request ||= Struct.new(:headers, :body).new(request_headers, StringIO.new(request_body.to_s)) + end + + def response + @response ||= Struct.new(:headers).new(@response_headers) + end + + def render(payload) + @rendered = payload + end + + def head(status) + @head_status = status + end + end + end + + let(:controller) { controller_class.new } + + let(:account_one) { FakeAccount.new(1) } + let(:account_two) { FakeAccount.new(2) } + let(:principal) { FakePrincipal.new(id: 55, default_account: account_one, accounts: [account_one, account_two]) } + + # Bypass the before_action chain: set the authenticated principal directly for + # examples that exercise dispatch/loop rather than authentication. + def authenticate_as(a_principal) + controller.instance_variable_set(:@mcp_principal, a_principal) + end + + def body_for(*requests) + JSON.generate(requests.size == 1 ? requests.first : requests) + end + + def rpc(method, params = {}, id: 1) + { "jsonrpc" => "2.0", "id" => id, "method" => method, "params" => params } + end + + # ---- hook defaults + override seams -------------------------------------- + + describe "hook defaults" do + it "mcp_config defaults to McpToolkit.config" do + expect(controller.send(:mcp_config)).to be(McpToolkit.config) + end + + it "mcp_session_data defaults to an empty hash (a host binds e.g. { token_id: })" do + expect(controller.send(:mcp_session_data)).to eq({}) + end + + it "mcp_rate_limit! delegates to config.rate_limiter with controller + principal" do + authenticate_as(principal) + seen = nil + McpToolkit.config.rate_limiter = ->(controller:, principal:) { seen = { controller:, principal: } } + + controller.send(:mcp_rate_limit!) + + expect(seen).to eq(controller:, principal:) + end + + it "mcp_rate_limit! is a no-op when no rate_limiter is configured" do + expect { controller.send(:mcp_rate_limit!) }.not_to raise_error + end + + it "mcp_track_usage delegates to config.usage_recorder with the call, account, principal, controller" do + authenticate_as(principal) + seen = nil + McpToolkit.config.usage_recorder = lambda { |request_data:, account:, principal:, controller:| + seen = { request_data:, account:, principal:, controller: } + } + + request_data = rpc("tools/call") + controller.send(:mcp_track_usage, request_data, account_one) + + expect(seen).to eq(request_data:, account: account_one, principal:, controller:) + end + + it "mcp_flush_usage delegates to config.usage_flusher with the controller" do + seen = nil + McpToolkit.config.usage_flusher = ->(controller:) { seen = controller } + + controller.send(:mcp_flush_usage) + + expect(seen).to be(controller) + end + end + + describe "#mcp_resolve_account (duck-typed on the principal)" do + before { authenticate_as(principal) } + + it "returns the principal's default account when no candidate is supplied" do + expect(controller.send(:mcp_resolve_account, rpc("tools/call"))).to eq(account_one) + end + + it "resolves a candidate from params._meta" do + request_data = rpc("tools/call", { "_meta" => { McpToolkit.config.account_meta_key => 2 } }) + + expect(controller.send(:mcp_resolve_account, request_data)).to eq(account_two) + end + + it "resolves a candidate from the tools/call account_id argument" do + request_data = rpc("tools/call", { "arguments" => { "account_id" => 2 } }) + + expect(controller.send(:mcp_resolve_account, request_data)).to eq(account_two) + end + + it "resolves a candidate from the account-id header (request-wide fallback)" do + controller.request_headers[McpToolkit.config.account_id_header] = "2" + + expect(controller.send(:mcp_resolve_account, rpc("tools/call"))).to eq(account_two) + end + + it "raises InvalidParams for an account the token is not authorized for" do + request_data = rpc("tools/call", { "arguments" => { "account_id" => 999 } }) + + expect { controller.send(:mcp_resolve_account, request_data) } + .to raise_error(McpToolkit::Protocol::InvalidParams, /not authorized/) + end + end + + # ---- authentication ------------------------------------------------------- + + describe "#mcp_authenticate!" do + it "renders a JSON-RPC 401 when no token is present" do + controller.send(:mcp_authenticate!) + + expect(controller.rendered[:status]).to eq(:unauthorized) + expect(controller.rendered[:json][:error][:message]).to include("Missing authorization token") + end + + it "renders a JSON-RPC 401 when the token is invalid" do + controller.request_headers["Authorization"] = "Bearer bad" + McpToolkit.config.token_authenticator = ->(_plaintext) { nil } + + controller.send(:mcp_authenticate!) + + expect(controller.rendered[:status]).to eq(:unauthorized) + expect(controller.rendered[:json][:error][:message]).to include("Invalid or expired token") + end + + it "sets the principal (and touches last-used) for a valid token" do + controller.request_headers["Authorization"] = "Bearer good" + token = principal + allow(token).to receive(:touch_last_used!) + McpToolkit.config.token_authenticator = ->(_plaintext) { token } + + controller.send(:mcp_authenticate!) + + expect(controller.send(:mcp_principal)).to be(token) + expect(token).to have_received(:touch_last_used!) + end + end + + # ---- session lifecycle ---------------------------------------------------- + + describe "#mcp_resolve_session!" do + before { authenticate_as(principal) } + + it "creates a session on initialize and echoes its id in the response header" do + controller.request_body = body_for(rpc("initialize")) + + controller.send(:mcp_resolve_session!) + + session_id = controller.response.headers[described_class::SESSION_HEADER] + expect(session_id).to be_present + expect(McpToolkit::Session.find(session_id)).not_to be_nil + end + + it "binds mcp_session_data to the created session (host override point)" do + def controller.mcp_session_data = { token_id: mcp_principal.id } + controller.request_body = body_for(rpc("initialize")) + + controller.send(:mcp_resolve_session!) + + session_id = controller.response.headers[described_class::SESSION_HEADER] + expect(McpToolkit::Session.find(session_id).data).to eq(token_id: 55) + end + + it "renders a 404 for a non-initialize POST with no valid session" do + controller.request_body = body_for(rpc("ping")) + + controller.send(:mcp_resolve_session!) + + expect(controller.rendered[:status]).to eq(:not_found) + expect(controller.rendered[:json][:error][:code]).to eq(-32_001) + end + end + + # ---- dispatch through #create -------------------------------------------- + + describe "#create dispatch" do + let(:whoami) { FakeTool.new { |ctx, _args| { account_id: ctx.account&.id, bearer: ctx.bearer_token } } } + + before do + authenticate_as(principal) + controller.request_headers["Authorization"] = "Bearer caller-token" + McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "whoami" => whoami }) + end + + it "echoes the negotiated protocol version on initialize" do + controller.request_body = body_for(rpc("initialize", { "protocolVersion" => "2025-03-26" })) + + controller.create + + expect(controller.rendered[:json][:result][:protocolVersion]).to eq("2025-03-26") + end + + it "merges host tools with namespaced upstream tools on tools/list" do + aggregator = instance_double(McpToolkit::Gateway::Aggregator) + allow(McpToolkit::Gateway::Aggregator).to receive(:new).and_return(aggregator) + allow(aggregator).to receive(:tool_definitions).and_return( + [{ "name" => "billing__charge", "description" => "Charge", "inputSchema" => { "type" => "object" } }] + ) + controller.request_body = body_for(rpc("tools/list")) + + controller.create + + names = controller.rendered[:json][:result][:tools].map { |t| t[:name] || t["name"] } + expect(names).to include("whoami", "billing__charge") + end + + it "routes a host tools/call through the dispatcher with the resolved account + bearer" do + controller.request_body = body_for(rpc("tools/call", { "name" => "whoami", "arguments" => {} })) + + controller.create + + content = JSON.parse(controller.rendered[:json][:result][:content].first[:text]) + expect(content).to eq("account_id" => 1, "bearer" => "caller-token") + end + + it "refuses a scoped tool the principal cannot reach (scope gate)" do + McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "whoami" => FakeTool.new(scope: "billing__read") }) + controller.request_body = body_for(rpc("tools/call", { "name" => "whoami", "arguments" => {} })) + + controller.create + + expect(controller.rendered[:json][:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + end + + it "returns 202 Accepted with no body for a notification (no id)" do + controller.request_body = body_for(rpc("notifications/initialized", {}, id: nil).except("id")) + + controller.create + + expect(controller.head_status).to eq(:accepted) + end + end + + # ---- the per-request account loop across a mixed-account BATCH ------------ + + describe "per-request account loop (batch with mixed accounts)" do + let(:whoami) { FakeTool.new { |ctx, _args| { account_id: ctx.account&.id } } } + let(:recorded) { [] } + + before do + authenticate_as(principal) + events = recorded + McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "whoami" => whoami }) + McpToolkit.config.usage_recorder = lambda { |request_data:, account:, principal:, **| + events << { method: request_data["method"], account_id: account&.id, principal_id: principal.id } + } + + controller.request_body = body_for( + rpc("tools/call", { "name" => "whoami", "arguments" => {}, + "_meta" => { McpToolkit.config.account_meta_key => 1 } }, id: 1), + rpc("tools/call", { "name" => "whoami", "arguments" => {}, + "_meta" => { McpToolkit.config.account_meta_key => 2 } }, id: 2) + ) + end + + it "re-resolves the account per element and dispatches each with its own account" do + controller.create + + results = controller.rendered[:json].map { |r| JSON.parse(r[:result][:content].first[:text])["account_id"] } + expect(results).to eq([1, 2]) + end + + it "meters exactly one usage event per call, each against its own resolved account" do + controller.create + + expect(recorded).to eq( + [ + { method: "tools/call", account_id: 1, principal_id: 55 }, + { method: "tools/call", account_id: 2, principal_id: 55 } + ] + ) + end + + it "isolates a per-element account failure to that element's JSON-RPC error" do + controller.request_body = body_for( + rpc("tools/call", { "name" => "whoami", "arguments" => {}, + "_meta" => { McpToolkit.config.account_meta_key => 1 } }, id: 1), + rpc("tools/call", { "name" => "whoami", "arguments" => { "account_id" => 999 } }, id: 2) + ) + + controller.create + + responses = controller.rendered[:json] + ok = responses.find { |r| r[:id] == 1 || r["id"] == 1 } + failed = responses.find { |r| r[:id] == 2 || r["id"] == 2 } + expect(JSON.parse(ok[:result][:content].first[:text])["account_id"]).to eq(1) + expect(failed[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_PARAMS) + end + end +end diff --git a/spec/mcp_toolkit/authority_controller_lazy_parent_spec.rb b/spec/mcp_toolkit/authority_controller_lazy_parent_spec.rb new file mode 100644 index 0000000..c944380 --- /dev/null +++ b/spec/mcp_toolkit/authority_controller_lazy_parent_spec.rb @@ -0,0 +1,94 @@ +# frozen_string_literal: true + +require "spec_helper" + +# Constraint B regression spec: the gem-provided controllers subclass +# `config.parent_controller`, and that parent MUST be read at BUILD time — after +# the host's initializer/to_prepare has run — not at autoload/eager-load time. If +# it were read early it would default to ActionController::Base, breaking CSRF +# handling on the introspection endpoint. +# +# Only a REAL booted Rails::Application driving its initializers + prepare +# callbacks reproduces the timing; a stub cannot. Booting one in-process mutates +# irreversible global state (Rails.application, the Zeitwerk loader set the +# eager_load_spec inspects, the const-set controllers), so — like +# engine_route_reload_spec — the boot runs in an ISOLATED CHILD process that sets +# the parent ENTIRELY inside `config.to_prepare` (the worst case), references the +# lazily-built controllers, and prints their superclasses as JSON for this parent +# process to assert on. +rails_available = + begin + require "rails/version" + true + rescue LoadError + false + end + +RSpec.describe "Authority controllers resolve their parent lazily (Constraint B)", if: rails_available do + LAZY_PARENT_CONTROLLER_DRIVER = <<~'RUBY' + require "bundler/setup" + require "json" + require "tmpdir" + require "logger" + require "mcp_toolkit" + require "rails" + require "action_controller/railtie" + + # mcp_toolkit is required before Rails here, so load the (Zeitwerk-ignored) + # engine explicitly by path. + unless McpToolkit.const_defined?(:Engine, false) + load File.expand_path("lib/mcp_toolkit/engine.rb", Dir.pwd) + end + + app = Class.new(Rails::Application) do + config.eager_load = false + config.consider_all_requests_local = true + config.secret_key_base = "lazy-parent-spec-secret-key-base" + config.logger = Logger.new(IO::NULL) + config.root = Dir.mktmpdir("mcp_toolkit_lazy_parent_spec") + + # The whole MCP config lives in to_prepare (the to_prepare-safe pattern): + # the parent is set AFTER the engine's own to_prepare in the same cycle, so a + # correct result proves the parent is read at build (reference) time, not at + # autoload time. + config.to_prepare { McpToolkit.config.parent_controller = "ActionController::API" } + end + + app.initialize! + app.routes.draw { mount McpToolkit::Engine => "/mcp" } + + # Referencing each controller AFTER boot triggers the lazy build (const_missing). + puts JSON.generate( + "server_parent" => McpToolkit::ServerController.superclass.name, + "tokens_parent" => McpToolkit::TokensController.superclass.name, + "authority_parent" => McpToolkit::Authority::ServerController.superclass.name, + "tokens_has_introspect" => McpToolkit::TokensController.instance_methods.include?(:introspect), + "authority_has_create" => McpToolkit::Authority::ServerController.instance_methods.include?(:create) + ) + RUBY + + before(:all) do + require "json" + require "open3" + + gem_root = File.expand_path("../..", __dir__) + stdout, stderr, status = Open3.capture3(Gem.ruby, "-e", LAZY_PARENT_CONTROLLER_DRIVER, chdir: gem_root) + raise "lazy-parent driver failed (#{status.exitstatus}):\n#{stderr}" unless status.success? + + @result = JSON.parse(stdout.lines.last) + end + + it "builds the engine ServerController against the configured parent" do + expect(@result.fetch("server_parent")).to eq("ActionController::API") + end + + it "builds the engine TokensController against the configured parent, preserving #introspect" do + expect(@result.fetch("tokens_parent")).to eq("ActionController::API") + expect(@result.fetch("tokens_has_introspect")).to be(true) + end + + it "builds the Authority::ServerController base against the configured parent, with the transport wired" do + expect(@result.fetch("authority_parent")).to eq("ActionController::API") + expect(@result.fetch("authority_has_create")).to be(true) + end +end diff --git a/spec/mcp_toolkit/dispatcher_gateway_spec.rb b/spec/mcp_toolkit/dispatcher_gateway_spec.rb new file mode 100644 index 0000000..81d57a6 --- /dev/null +++ b/spec/mcp_toolkit/dispatcher_gateway_spec.rb @@ -0,0 +1,144 @@ +# frozen_string_literal: true + +require "spec_helper" + +# Gateway behaviour of the authority dispatcher: aggregating + proxying upstream +# MCP tools. The gateway machinery itself is unit-tested under gateway/*; this +# pins how the DISPATCHER wires to it — list merge, account-id + bearer +# forwarding, upstream error translation, and the list_changed cache flush — +# mocking only the gem's gateway boundary. Ported from core's server_gateway_spec. +RSpec.describe McpToolkit::Dispatcher do + subject(:dispatcher) { described_class.new(context:, config: McpToolkit.config) } + + let(:account) { FakeAccount.new(42) } + let(:principal) { FakePrincipal.new(scopes: []) } + let(:context) { McpToolkit::Authority::Context.new(account:, principal:, bearer_token: "mcp_caller_token") } + + let(:host_tool) { FakeTool.new(result: { ok: true }) } + let(:provider) { FakeToolProvider.new(tools: { "host_list" => host_tool }) } + + let(:upstream) { McpToolkit::Gateway::UpstreamRegistry::Upstream.new(key: "notifications", url: "https://notif.test/mcp") } + + before { McpToolkit.config.tool_provider = provider } + + def request(method, params = {}, id: 1) + body = { "jsonrpc" => "2.0", "method" => method, "params" => params } + id.nil? ? body : body.merge("id" => id) + end + + describe "tools/list aggregation" do + let(:aggregator) { instance_double(McpToolkit::Gateway::Aggregator) } + + before do + allow(McpToolkit::Gateway::Aggregator).to receive(:new).and_return(aggregator) + allow(aggregator).to receive(:tool_definitions).with(bearer_token: "mcp_caller_token").and_return( + [ + { "name" => "notifications__send_email", "description" => "Send", "inputSchema" => { "type" => "object" } }, + { "name" => "owners__list", "description" => "List", "inputSchema" => { "type" => "object" } } + ] + ) + end + + it "returns the host's own tools plus the namespaced upstream tools" do + names = dispatcher.handle_request(request("tools/list"))[:result][:tools].map { |t| t[:name] || t["name"] } + + expect(names).to include("host_list") # the host's own tool + expect(names).to include("notifications__send_email", "owners__list") + end + end + + describe "tools/list when the upstream is down" do + let(:aggregator) { instance_double(McpToolkit::Gateway::Aggregator, tool_definitions: []) } + + before { allow(McpToolkit::Gateway::Aggregator).to receive(:new).and_return(aggregator) } + + it "still returns the host's own tools (the aggregator degrades to [])" do + names = dispatcher.handle_request(request("tools/list"))[:result][:tools].map { |t| t[:name] || t["name"] } + + expect(names).to include("host_list") + expect(names.none? { |n| n.start_with?("notifications__") }).to be(true) + end + end + + describe "tools/call routing" do + let(:proxy) { instance_double(McpToolkit::Gateway::Proxy) } + + before do + McpToolkit.config.upstreams.register(key: upstream.key, url: upstream.url) + allow(McpToolkit::Gateway::Proxy).to receive(:new).and_return(proxy) + end + + it "proxies a namespaced tool, forwarding the resolved account id + bearer" do + allow(proxy).to receive(:call).and_return({ "content" => [{ "type" => "text", "text" => "sent" }] }) + + response = dispatcher.handle_request( + request("tools/call", { "name" => "notifications__send_email", "arguments" => { "to" => "a@b.c" } }) + ) + + expect(response[:result]).to eq({ "content" => [{ "type" => "text", "text" => "sent" }] }) + expect(McpToolkit::Gateway::Proxy).to have_received(:new).with( + app_key: "notifications", + tool_name: "send_email", + account_id: 42, + bearer_token: "mcp_caller_token", + config: McpToolkit.config + ) + expect(proxy).to have_received(:call).with({ "to" => "a@b.c" }) + end + + it "falls through to a host tool-not-found for an unknown, non-upstream name" do + response = dispatcher.handle_request(request("tools/call", { "name" => "does_not_exist", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::METHOD_NOT_FOUND) + end + end + + describe "upstream error translation" do + let(:proxy) { instance_double(McpToolkit::Gateway::Proxy) } + + before do + McpToolkit.config.upstreams.register(key: upstream.key, url: upstream.url) + allow(McpToolkit::Gateway::Proxy).to receive(:new).and_return(proxy) + end + + it "relays an upstream JSON-RPC error verbatim (code + message + data)" do + allow(proxy).to receive(:call).and_raise( + McpToolkit::Gateway::UpstreamCallError.new("boom", jsonrpc_error: { "code" => -32_050, "message" => "nope", "data" => { "x" => 1 } }) + ) + + response = dispatcher.handle_request(request("tools/call", { "name" => "notifications__send_email", "arguments" => {} })) + + expect(response[:error]).to eq(code: -32_050, message: "nope", data: { "x" => 1 }) + end + + it "maps a transport-level upstream failure (no jsonrpc_error) to internal_error" do + allow(proxy).to receive(:call).and_raise(McpToolkit::Gateway::UpstreamCallError.new("timed out")) + + response = dispatcher.handle_request(request("tools/call", { "name" => "notifications__send_email", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INTERNAL_ERROR) + end + + it "maps an unknown upstream to method-not-found" do + allow(proxy).to receive(:call).and_raise(McpToolkit::Gateway::UnknownUpstream, "Unknown application: notifications") + + response = dispatcher.handle_request(request("tools/call", { "name" => "notifications__send_email", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::METHOD_NOT_FOUND) + end + end + + describe "upstream list_changed notification" do + it "flushes the named upstream's cached aggregate" do + McpToolkit.config.upstreams.register(key: upstream.key, url: upstream.url) + aggregator = instance_double(McpToolkit::Gateway::Aggregator, flush!: nil) + allow(McpToolkit::Gateway::Aggregator).to receive(:new).and_return(aggregator) + + # A notification (no id) returns nil from the dispatcher but must still flush. + result = dispatcher.handle_request(request("notifications/notifications/tools/list_changed", {}, id: nil)) + + expect(result).to be_nil + expect(aggregator).to have_received(:flush!).with("notifications") + end + end +end diff --git a/spec/mcp_toolkit/dispatcher_spec.rb b/spec/mcp_toolkit/dispatcher_spec.rb new file mode 100644 index 0000000..3187326 --- /dev/null +++ b/spec/mcp_toolkit/dispatcher_spec.rb @@ -0,0 +1,179 @@ +# frozen_string_literal: true + +require "spec_helper" + +# The hand-rolled authority dispatcher: request validation, method dispatch, +# version negotiation, host tool serving + scope gate, and the JSON-RPC error +# envelope. Gateway routing lives in dispatcher_gateway_spec. Ported from core's +# server_spec, rebased on the injected context + tool_provider. +RSpec.describe McpToolkit::Dispatcher do + subject(:dispatcher) { described_class.new(context:, config: McpToolkit.config) } + + let(:account) { FakeAccount.new(42) } + let(:principal) { FakePrincipal.new(scopes:) } + let(:scopes) { [] } + let(:context) { McpToolkit::Authority::Context.new(account:, principal:, bearer_token: "tok") } + + let(:echo_tool) { FakeTool.new { |ctx, args| { echoed: args, account_id: ctx.account&.id } } } + let(:provider) { FakeToolProvider.new(tools: { "echo" => echo_tool }) } + + before { McpToolkit.config.tool_provider = provider } + + def request(method, params = {}, id: 1) + body = { "jsonrpc" => "2.0", "method" => method, "params" => params } + id.nil? ? body : body.merge("id" => id) + end + + describe "request validation" do + it "rejects a request missing the jsonrpc version" do + response = dispatcher.handle_request({ "method" => "ping", "id" => 1 }) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + expect(response[:error][:message]).to include("Missing jsonrpc version") + end + + it "rejects a request with the wrong jsonrpc version" do + response = dispatcher.handle_request({ "jsonrpc" => "1.0", "method" => "ping", "id" => 1 }) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + end + + it "rejects a request without a method" do + response = dispatcher.handle_request({ "jsonrpc" => "2.0", "id" => 1 }) + + expect(response[:error][:message]).to include("Missing method") + end + end + + describe "initialize" do + it "echoes the requested protocol version when supported, else the latest" do + supported = dispatcher.handle_request(request("initialize", { "protocolVersion" => "2025-03-26" })) + unsupported = dispatcher.handle_request(request("initialize", { "protocolVersion" => "1999-01-01" })) + + expect(supported[:result][:protocolVersion]).to eq("2025-03-26") + expect(unsupported[:result][:protocolVersion]).to eq(McpToolkit::Protocol::LATEST_VERSION) + end + + it "negotiates against config.supported_protocol_versions when the host narrows the set" do + McpToolkit.config.supported_protocol_versions = %w[2025-06-18] + + response = dispatcher.handle_request(request("initialize", { "protocolVersion" => "2025-03-26" })) + + expect(response[:result][:protocolVersion]).to eq("2025-06-18") + end + + it "returns the configured server identity and listChanged capability" do + McpToolkit.config.server_name = "acme-mcp" + McpToolkit.config.server_version = "3.1.4" + + response = dispatcher.handle_request(request("initialize")) + + expect(response[:result][:serverInfo]).to eq(name: "acme-mcp", version: "3.1.4") + expect(response[:result][:capabilities][:tools]).to eq(listChanged: true) + end + end + + describe "notifications (no id)" do + it "recognizes notifications/initialized and returns no response" do + expect(dispatcher.handle_request(request("notifications/initialized", {}, id: nil))).to be_nil + end + end + + describe "ping" do + it "returns an empty result" do + expect(dispatcher.handle_request(request("ping"))[:result]).to eq({}) + end + end + + describe "tools/list" do + it "exposes the host provider's tool definitions" do + names = dispatcher.handle_request(request("tools/list"))[:result][:tools].pluck(:name) + + expect(names).to eq(["echo"]) + end + + it "contributes nothing when no tool_provider is configured (a pure gateway)" do + McpToolkit.config.tool_provider = nil + + expect(dispatcher.handle_request(request("tools/list"))[:result][:tools]).to eq([]) + end + end + + describe "tools/call" do + it "runs the host tool and wraps a Hash result as JSON text content" do + response = dispatcher.handle_request( + request("tools/call", { "name" => "echo", "arguments" => { "q" => "hi" } }) + ) + content = JSON.parse(response[:result][:content].first[:text]) + + expect(content).to eq("echoed" => { "q" => "hi" }, "account_id" => 42) + end + + it "passes the request context and deep-symbolized arguments to the tool" do + dispatcher.handle_request( + request("tools/call", { "name" => "echo", "arguments" => { "nested" => { "a" => 1 } } }) + ) + + call = echo_tool.calls.first + expect(call[:arguments]).to eq(nested: { a: 1 }) + expect(call[:context]).to be(context) + end + + it "wraps a String result as text content verbatim" do + McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "echo" => FakeTool.new(result: "plain text") }) + + response = dispatcher.handle_request(request("tools/call", { "name" => "echo", "arguments" => {} })) + + expect(response[:result][:content]).to eq([{ type: "text", text: "plain text" }]) + end + + it "returns method-not-found for an unknown tool" do + response = dispatcher.handle_request(request("tools/call", { "name" => "nope", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::METHOD_NOT_FOUND) + expect(response[:error][:message]).to include("Tool not found") + end + + it "maps an unexpected tool error to internal_error" do + exploding = FakeTool.new { |_ctx, _args| raise "boom" } + McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "echo" => exploding }) + + response = dispatcher.handle_request(request("tools/call", { "name" => "echo", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INTERNAL_ERROR) + end + end + + describe "per-tool scope enforcement (centralized in the dispatcher)" do + let(:scoped_tool) { FakeTool.new(scope: "acme__read", result: { ok: true }) } + let(:provider) { FakeToolProvider.new(tools: { "echo" => scoped_tool }) } + + it "refuses a call when the principal lacks the tool's required scope" do + response = dispatcher.handle_request(request("tools/call", { "name" => "echo", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + expect(response[:error][:message]).to include("acme__read") + expect(scoped_tool.calls).to be_empty + end + + context "when the principal carries the required scope" do + let(:scopes) { ["acme__read"] } + + it "passes the gate and runs the tool" do + response = dispatcher.handle_request(request("tools/call", { "name" => "echo", "arguments" => {} })) + + expect(response).to have_key(:result) + expect(scoped_tool.calls.size).to eq(1) + end + end + end + + describe "unknown method" do + it "returns method-not-found" do + response = dispatcher.handle_request(request("frobnicate")) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::METHOD_NOT_FOUND) + expect(response[:error][:message]).to include("Method not found: frobnicate") + end + end +end diff --git a/spec/mcp_toolkit/engine_spec.rb b/spec/mcp_toolkit/engine_spec.rb index fbd9939..08b0bbc 100644 --- a/spec/mcp_toolkit/engine_spec.rb +++ b/spec/mcp_toolkit/engine_spec.rb @@ -18,22 +18,22 @@ RSpec.describe "Mountable engine + gem controller" do describe "McpToolkit::ServerController (engine controller)" do # A fake ActionController::Base: provides only the class-level hooks the - # transport concern's `included do` block invokes. + # transport concerns' `included do` blocks invoke. before do stub_const("FakeActionControllerBase", Class.new do def self.before_action(*); end + def self.after_action(*); end # The concern guards protect_from_forgery behind respond_to?; omit it so # that branch is exercised (no CSRF machinery needed in a unit test). end) McpToolkit.config.parent_controller = "FakeActionControllerBase" - # Load the gem-provided controller against the configured parent. It lives - # under app/controllers (outside the lib loader), so require it by path. - path = File.expand_path("../../app/controllers/mcp_toolkit/server_controller.rb", __dir__) - load path + # The engine controllers are no longer files: they are built lazily from the + # configured parent by the builder (Constraint B). Build them explicitly. + McpToolkit.build_engine_controllers! end - after { McpToolkit.send(:remove_const, :ServerController) if McpToolkit.const_defined?(:ServerController, false) } + after { McpToolkit.reset_engine_controllers! } it "inherits the configured parent_controller" do expect(McpToolkit::ServerController.superclass).to eq(FakeActionControllerBase) @@ -74,9 +74,14 @@ def draw(&block) before do recorder = route_recorder stub_const("Rails", Module.new) + # The engine class body now also calls `config.to_prepare { ... }` (lazy + # controller reset), so the fake base must expose a `config` responding to + # `to_prepare` (a no-op here — nothing is reloaded in this unit test). + config_double = Class.new { def to_prepare(*); end }.new engine_base = Class.new do define_singleton_method(:isolate_namespace) { |_mod| } end + engine_base.define_singleton_method(:config) { config_double } stub_const("Rails::Engine", engine_base) load File.expand_path("../../lib/mcp_toolkit/engine.rb", __dir__) diff --git a/spec/mcp_toolkit/gateway/client_spec.rb b/spec/mcp_toolkit/gateway/client_spec.rb index f20db05..c36a043 100644 --- a/spec/mcp_toolkit/gateway/client_spec.rb +++ b/spec/mcp_toolkit/gateway/client_spec.rb @@ -246,6 +246,46 @@ def tools_payload end end + describe "handshake clientInfo identity split" do + before do + stub_initialize + stub_initialized + stub_method("tools/list", json_response("tools" => [])) + end + + def sent_client_info + request = nil + expect(a_request(:post, upstream_url).with { |req| + parsed = JSON.parse(req.body) + next false unless parsed["method"] == "initialize" + + request = parsed["params"]["clientInfo"] + true + }).to have_been_made + request + end + + it "presents gateway_client_name/version (NOT server_name/version) when the host split them" do + McpToolkit.config.server_name = "acme-mcp" # advertised to OUR callers + McpToolkit.config.server_version = "3.0.0" + McpToolkit.config.gateway_client_name = "acme-mcp-gateway" # presented to upstreams + McpToolkit.config.gateway_client_version = "9.9.9" + + client.tools_list + + expect(sent_client_info).to eq("name" => "acme-mcp-gateway", "version" => "9.9.9") + end + + it "falls back to the server identity when the gateway identity is unset" do + McpToolkit.config.server_name = "acme-mcp" + McpToolkit.config.server_version = "3.0.0" + + client.tools_list + + expect(sent_client_info).to eq("name" => "acme-mcp", "version" => "3.0.0") + end + end + describe "DEFAULT_PROTOCOL_VERSION" do it "sources the wrapped mcp SDK's latest-supported version" do expect(described_class::DEFAULT_PROTOCOL_VERSION) diff --git a/spec/mcp_toolkit/protocol_spec.rb b/spec/mcp_toolkit/protocol_spec.rb new file mode 100644 index 0000000..4f02fb7 --- /dev/null +++ b/spec/mcp_toolkit/protocol_spec.rb @@ -0,0 +1,102 @@ +# frozen_string_literal: true + +require "spec_helper" + +# Ported from core's protocol_spec: the byte contract of the JSON-RPC envelope + +# error codes the authority dispatcher emits. +RSpec.describe McpToolkit::Protocol do + describe "SUPPORTED_VERSIONS" do + it "lists the protocol versions the server can negotiate, newest first" do + expect(described_class::SUPPORTED_VERSIONS).to eq(%w[2025-06-18 2025-03-26 2024-11-05]) + expect(described_class::LATEST_VERSION).to eq("2025-06-18") + expect(described_class::VERSION).to eq(described_class::LATEST_VERSION) + end + end + + describe "JSONRPC_VERSION" do + it "returns the JSON-RPC version" do + expect(described_class::JSONRPC_VERSION).to eq("2.0") + end + end + + describe "ErrorCodes" do + it "pins the JSON-RPC 2.0 codes" do + expect(described_class::ErrorCodes::PARSE_ERROR).to eq(-32_700) + expect(described_class::ErrorCodes::INVALID_REQUEST).to eq(-32_600) + expect(described_class::ErrorCodes::METHOD_NOT_FOUND).to eq(-32_601) + expect(described_class::ErrorCodes::INVALID_PARAMS).to eq(-32_602) + expect(described_class::ErrorCodes::INTERNAL_ERROR).to eq(-32_603) + end + end + + describe ".success_response" do + it "returns a properly formatted success response" do + expect(described_class.success_response(id: 1, result: { foo: "bar" })).to eq( + jsonrpc: "2.0", id: 1, result: { foo: "bar" } + ) + end + end + + describe ".error_response" do + it "formats an Error object via #to_h" do + error = described_class::InternalError.new("Something went wrong") + + expect(described_class.error_response(id: 1, error:)).to eq( + jsonrpc: "2.0", id: 1, error: { code: -32_603, message: "Something went wrong" } + ) + end + + it "passes a raw hash error through as-is" do + error = { code: -32_000, message: "Custom error" } + + expect(described_class.error_response(id: 1, error:)).to eq( + jsonrpc: "2.0", id: 1, error: { code: -32_000, message: "Custom error" } + ) + end + end + + describe McpToolkit::Protocol::Error do + subject(:error) { described_class.new("Test error", code: -32_000, data: { extra: "info" }) } + + it "exposes message, code and data" do + expect(error.message).to eq("Test error") + expect(error.code).to eq(-32_000) + expect(error.data).to eq(extra: "info") + end + + it "serializes to a hash, including data when present" do + expect(error.to_h).to eq(code: -32_000, message: "Test error", data: { extra: "info" }) + end + + it "omits the data key when absent" do + expect(described_class.new("Test error", code: -32_000).to_h).to eq(code: -32_000, message: "Test error") + end + end + + describe "the error subclasses carry their own code + default message" do + it "ParseError" do + error = McpToolkit::Protocol::ParseError.new + expect([error.code, error.message]).to eq([-32_700, "Parse error"]) + end + + it "InvalidRequest" do + error = McpToolkit::Protocol::InvalidRequest.new + expect([error.code, error.message]).to eq([-32_600, "Invalid request"]) + end + + it "MethodNotFound embeds the method name" do + error = McpToolkit::Protocol::MethodNotFound.new("unknown_method") + expect([error.code, error.message]).to eq([-32_601, "Method not found: unknown_method"]) + end + + it "InvalidParams" do + error = McpToolkit::Protocol::InvalidParams.new + expect([error.code, error.message]).to eq([-32_602, "Invalid params"]) + end + + it "InternalError" do + error = McpToolkit::Protocol::InternalError.new + expect([error.code, error.message]).to eq([-32_603, "Internal error"]) + end + end +end diff --git a/spec/mcp_toolkit/tool_provider_contract_spec.rb b/spec/mcp_toolkit/tool_provider_contract_spec.rb new file mode 100644 index 0000000..7faf819 --- /dev/null +++ b/spec/mcp_toolkit/tool_provider_contract_spec.rb @@ -0,0 +1,100 @@ +# frozen_string_literal: true + +require "spec_helper" + +# The api-agnostic seam (Req 1): the gem serves a host's own tools ONLY through the +# injected `config.tool_provider`, and enforces the per-tool scope gate CENTRALLY +# (in the dispatcher) rather than trusting each tool. This exercises the full +# contract through the dispatcher (the gem's real caller) with a provider + tools +# that have ZERO app/api knowledge, proving the gem never needs to. +RSpec.describe "tool_provider contract" do + subject(:dispatcher) { McpToolkit::Dispatcher.new(context:, config: McpToolkit.config) } + + let(:account) { FakeAccount.new(7) } + let(:principal) { FakePrincipal.new(scopes:, superuser:) } + let(:scopes) { [] } + let(:superuser) { false } + let(:context) { McpToolkit::Authority::Context.new(account:, principal:, bearer_token: "tok") } + + def request(method, params = {}) + { "jsonrpc" => "2.0", "id" => 1, "method" => method, "params" => params } + end + + describe "provider.tool_definitions(context)" do + let(:provider) do + FakeToolProvider.new do |ctx| + base = [{ name: "widgets_list", description: "List widgets", inputSchema: { type: "object" } }] + # The context lets the host hide a superuser-only tool from other callers. + ctx.superuser? ? base + [{ name: "audit_dump", description: "Dump", inputSchema: { type: "object" } }] : base + end + end + + before { McpToolkit.config.tool_provider = provider } + + it "is passed the request context so the provider can hide superuser-only tools" do + names = dispatcher.handle_request(request("tools/list"))[:result][:tools].pluck(:name) + + expect(names).to eq(["widgets_list"]) + end + + context "when the principal is a superuser" do + let(:superuser) { true } + + it "reveals the superuser-only tool" do + names = dispatcher.handle_request(request("tools/list"))[:result][:tools].pluck(:name) + + expect(names).to contain_exactly("widgets_list", "audit_dump") + end + end + end + + describe "provider.find(name) + tool.call(context:, **arguments)" do + let(:tool) { FakeTool.new { |ctx, args| { seen_account: ctx.account.id, args: } } } + + before { McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "widgets_list" => tool }) } + + it "resolves the tool by name and invokes it with the context + arguments" do + response = dispatcher.handle_request( + request("tools/call", { "name" => "widgets_list", "arguments" => { "limit" => 5 } }) + ) + content = JSON.parse(response[:result][:content].first[:text]) + + expect(content).to eq("seen_account" => 7, "args" => { "limit" => 5 }) + end + end + + describe "central scope enforcement (the gem gates, not the tool)" do + let(:tool) { FakeTool.new(scope: "widgets_app__read") { |_ctx, _args| { ok: true } } } + + before { McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "widgets_list" => tool }) } + + it "refuses a scoped tool for a principal without the scope, WITHOUT invoking it" do + response = dispatcher.handle_request(request("tools/call", { "name" => "widgets_list", "arguments" => {} })) + + expect(response[:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + expect(tool.calls).to be_empty + end + + context "with the scope granted" do + let(:scopes) { ["widgets_app__read"] } + + it "invokes the tool" do + response = dispatcher.handle_request(request("tools/call", { "name" => "widgets_list", "arguments" => {} })) + + expect(response).to have_key(:result) + expect(tool.calls.size).to eq(1) + end + end + end + + describe "the gem carries no host/api-layer coupling" do + it "has no api_v3 / serializer / catalog references in the authority + dispatch code" do + root = File.expand_path("../../lib/mcp_toolkit", __dir__) + sources = Dir[File.join(root, "{dispatcher,protocol}.rb"), File.join(root, "authority", "**", "*.rb"), + File.join(root, "authority.rb"), File.join(root, "tools", "authority_base.rb")] + blob = sources.map { |f| File.read(f) }.join("\n").downcase + + expect(blob).not_to match(/api_v3|api::v3/) + end + end +end diff --git a/spec/mcp_toolkit/tools/authority_base_spec.rb b/spec/mcp_toolkit/tools/authority_base_spec.rb new file mode 100644 index 0000000..a37055c --- /dev/null +++ b/spec/mcp_toolkit/tools/authority_base_spec.rb @@ -0,0 +1,132 @@ +# frozen_string_literal: true + +require "spec_helper" + +# McpToolkit::Tools::AuthorityBase — the optional base a host tool subclasses. It +# satisfies the dispatcher's duck-typed tool contract as CLASS methods +# (`.required_permissions_scope` + `.call(context:, **arguments)`), exposes the +# context accessors, gates superuser-only resources, and maps tool errors to +# protocol errors. +RSpec.describe McpToolkit::Tools::AuthorityBase do + let(:account) { FakeAccount.new(99) } + let(:principal) { FakePrincipal.new(superuser:) } + let(:superuser) { false } + let(:context) { McpToolkit::Authority::Context.new(account:, principal:, bearer_token: "tok") } + + let(:tool_class) do + Class.new(described_class) do + tool_name "widget_get" + description "Fetch a widget." + required_permissions_scope "widgets__read" + input_schema { { type: "object", properties: { id: { type: "string" } }, required: ["id"] } } + + def call(id:) + { id:, account_id: account.id, principal_present: !principal.nil?, bearer: bearer_token } + end + end + end + + describe "the class-level tool contract" do + it "exposes required_permissions_scope for the dispatcher's gate" do + expect(tool_class.required_permissions_scope).to eq("widgets__read") + end + + it "builds a tool definition mirroring the tools/list shape" do + expect(tool_class.definition).to eq( + name: "widget_get", + description: "Fetch a widget.", + inputSchema: { type: "object", properties: { id: { type: "string" } }, required: ["id"] } + ) + end + + it "does NOT inherit a scope declared on an ancestor" do + subclass = Class.new(tool_class) { tool_name "sub" } + + expect(subclass.required_permissions_scope).to be_nil + end + + it "defaults tool_name to the demodulized/underscored class name" do + stub_const("Acme::FooBarTool", Class.new(described_class)) + + expect(Acme::FooBarTool.tool_name).to eq("foo_bar") + end + end + + describe ".call(context:, **arguments)" do + it "runs the business logic with the context accessors bound" do + result = tool_class.call(context:, id: "w1") + + expect(result).to eq(id: "w1", account_id: 99, principal_present: true, bearer: "tok") + end + + it "maps a missing required keyword (ArgumentError) to InvalidParams" do + expect { tool_class.call(context:) } + .to raise_error(McpToolkit::Protocol::InvalidParams) + end + + it "maps an unexpected StandardError to InternalError" do + exploding = Class.new(described_class) { def call(**) = raise("kaboom") } + + expect { exploding.call(context:) }.to raise_error(McpToolkit::Protocol::InternalError, /kaboom/) + end + + it "lets a deliberately-raised protocol error pass through with its own code" do + raising = Class.new(described_class) do + def call(**) + raise McpToolkit::Protocol::InvalidParams.new("id is required") + end + end + + expect { raising.call(context:) }.to raise_error(McpToolkit::Protocol::InvalidParams, /id is required/) + end + end + + describe "#ensure_resource_accessible!" do + let(:restricted) { double("resource", superusers_only?: true, name: "AuditLog") } + let(:open_resource) { double("resource", superusers_only?: false, name: "Widget") } + + let(:gating_tool) do + Class.new(described_class) do + def call(resource:) + ensure_resource_accessible!(resource) + { ok: true } + end + end + end + + it "refuses a superuser-only resource for a non-superuser caller" do + expect { gating_tool.call(context:, resource: restricted) } + .to raise_error(McpToolkit::Protocol::InvalidRequest, /restricted to superuser/) + end + + it "allows an unrestricted resource for any caller" do + expect(gating_tool.call(context:, resource: open_resource)).to eq(ok: true) + end + + context "with a superuser caller" do + let(:superuser) { true } + + it "allows the superuser-only resource" do + expect(gating_tool.call(context:, resource: restricted)).to eq(ok: true) + end + end + end + + describe "end-to-end through the dispatcher" do + let(:principal) { FakePrincipal.new(scopes: ["widgets__read"]) } + + it "plugs into the provider seam and is served + scope-gated by the dispatcher" do + klass = tool_class + McpToolkit.config.tool_provider = FakeToolProvider.new(tools: { "widget_get" => klass }) + + response = McpToolkit::Dispatcher.new(context:, config: McpToolkit.config).handle_request( + { "jsonrpc" => "2.0", "id" => 1, "method" => "tools/call", + "params" => { "name" => "widget_get", "arguments" => { "id" => "w9" } } } + ) + content = JSON.parse(response[:result][:content].first[:text]) + + expect(content["id"]).to eq("w9") + expect(content["account_id"]).to eq(99) + end + end +end diff --git a/spec/support/fake_authority.rb b/spec/support/fake_authority.rb new file mode 100644 index 0000000..3f58457 --- /dev/null +++ b/spec/support/fake_authority.rb @@ -0,0 +1,92 @@ +# frozen_string_literal: true + +# Reusable, vendor-neutral doubles for the AUTHORITY path specs. They stand in for +# a host's token model, account, tool and tool-provider WITHOUT any api_v3 / app +# knowledge — the whole point of the api-agnostic seam. + +# A resolved tenant. The gem only reads `#id` (forwarded upstream as the account +# selector; embedded in a usage event). +FakeAccount = Struct.new(:id) + +# A duck-typed principal (the "token"): the methods the authority path calls. +# #authorized_for_scope?(scope) — the dispatcher's per-tool scope gate +# #default_account — the account when a caller pins none +# #authorize_account(id) — resolve a pinned account id (nil = unauthorized) +# #superuser? — surfaced via Authority::Context#superuser? +class FakePrincipal + attr_reader :id, :scopes + + def initialize(id: 1, scopes: [], default_account: nil, accounts: [], superuser: false) + @id = id + @scopes = Array(scopes).map(&:to_s) + @default_account = default_account + # Keyed by String so a header/meta/arg candidate (any of which may arrive as a + # string) resolves regardless of the caller's type. + @accounts = Array(accounts).to_h { |account| [account.id.to_s, account] } + @superuser = superuser + end + + def authorized_for_scope?(scope) + return true if scope.to_s.empty? + + scopes.include?(scope.to_s) + end + + attr_reader :default_account + + def authorize_account(candidate) + return nil if candidate.to_s.empty? + + @accounts[candidate.to_s] + end + + def superuser? + @superuser + end +end + +# A tool satisfying the gem's duck-typed contract as an INSTANCE (the dispatcher +# calls `#required_permissions_scope` + `#call(context:, **arguments)` on whatever +# `provider.find` returns). Records every call for assertions. `result` may be a +# proc (receiving the context + arguments), a String, or a Hash. +class FakeTool + attr_reader :calls + + def initialize(scope: nil, result: { ok: true }, &block) + @scope = scope + @result = block || result + @calls = [] + end + + def required_permissions_scope + @scope + end + + def call(context:, **arguments) + @calls << { context:, arguments: } + @result.respond_to?(:call) ? @result.call(context, arguments) : @result + end +end + +# A host tool catalog satisfying the provider contract: +# #tool_definitions(context) -> [{ name:, description:, inputSchema: }] +# #find(name) -> a tool, or nil +# `definitions_for` may filter by context (e.g. hide superuser-only tools). +class FakeToolProvider + def initialize(tools: {}, definitions: nil, &definitions_for) + @tools = tools # name => FakeTool + @definitions = definitions + @definitions_for = definitions_for + end + + def tool_definitions(context) + return @definitions_for.call(context) if @definitions_for + return @definitions if @definitions + + @tools.keys.map { |name| { name:, description: "#{name} tool", inputSchema: { type: "object" } } } + end + + def find(name) + @tools[name] + end +end From 77d44d1bc79b265dddb00038a933720b9ef7ea02 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 15:36:29 +0200 Subject: [PATCH 03/10] Scrub residual consuming-app fingerprints from docs/comments --- README.md | 44 +++++++++---------- lib/mcp_toolkit/auth/authority.rb | 4 +- lib/mcp_toolkit/configuration.rb | 4 +- lib/mcp_toolkit/serializer/base.rb | 4 +- .../transport/controller_methods.rb | 4 +- 5 files changed, 30 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 685f7b3..78dd8a5 100644 --- a/README.md +++ b/README.md @@ -61,8 +61,8 @@ introspects each forwarded bearer token against the central app. ```ruby McpToolkit.configure do |c| - c.server_name = "bsa-notifications-mcp" - c.server_instructions = "Read-only access to this account's notifications domain." + c.server_name = "acme-mcp" + c.server_instructions = "Read-only access to this account's widgets domain." # --- satellite auth --- c.auth_role = :satellite @@ -72,7 +72,7 @@ McpToolkit.configure do |c| # can override it per-resource (see below). Omit entirely for "no scope # required". Whether a scope is required is PER TOOL — there is no app-wide # permission flag. - c.registry.default_required_permissions_scope "notifications__read" + c.registry.default_required_permissions_scope "widgets__read" # Map the central account id to this app's LOCAL scope root (an Account here). c.account_resolver = ->(synced_account_id) { Account.find_by(synced_id: synced_account_id) } @@ -94,22 +94,22 @@ the resolved scope root — this is the single tenancy chokepoint: Rails.application.config.to_prepare do McpToolkit.registry.reset! - McpToolkit.registry.register(:notifications) do - model Notification - serializer Mcp::NotificationSerializer # your serializer (see below) - description "Email notification templates + their scheduling rules." - scope(&:notifications) # account.notifications + McpToolkit.registry.register(:widgets) do + model Widget + serializer WidgetSerializer # your serializer (see below) + description "Widget templates + their scheduling rules." + scope(&:widgets) # account.widgets end - McpToolkit.registry.register(:scheduled_notifications) do - model ScheduledNotification - serializer Mcp::ScheduledNotificationSerializer - description "Scheduled mailings." + McpToolkit.registry.register(:scheduled_widgets) do + model ScheduledWidget + serializer ScheduledWidgetSerializer + description "Scheduled widget deliveries." # Expose a public filter key that maps to a synced storage column: filterable booking_id: :synced_booking_id # Override the registry default scope for just this resource (optional): - required_permissions_scope "notifications__read" - scope { |account| ScheduledNotification.where(synced_account_id: account.synced_id) } + required_permissions_scope "widgets__read" + scope { |account| ScheduledWidget.where(synced_account_id: account.synced_id) } end end ``` @@ -141,12 +141,12 @@ introspecting the forwarded token and scoped to the resolved account. The authority authenticates plaintext tokens locally and answers the introspection requests satellites send. -**1. Configure** the local token lookup (your `McpToken.authenticate` equivalent): +**1. Configure** the local token lookup (your `AccessToken.authenticate` equivalent): ```ruby McpToolkit.configure do |c| c.auth_role = :authority - c.token_authenticator = ->(plaintext) { McpToken.authenticate(plaintext) } + c.token_authenticator = ->(plaintext) { AccessToken.authenticate(plaintext) } c.cache_store = Rails.cache end ``` @@ -155,12 +155,12 @@ The token object your authenticator returns must respond to: `kind` (`:accounts_user` | `:user`), `account_id`, `account_ids`, `expires_at` (an `#iso8601`-able time or nil), and `scopes` (an array of `__` scopes; `[]` = no scopes). Optionally `touch_last_used!`. A typical app token -model (e.g. `McpToken`) fits. +model (e.g. `AccessToken`) fits. **2. Expose the introspection endpoint** the satellites call: ```ruby -class Mcp::TokensController < ActionController::API +class TokensController < ActionController::API def introspect token = McpToolkit::Auth::Authority.authenticate(extract_token) return render(json: McpToolkit::Auth::Authority.invalid_payload, status: :unauthorized) unless token @@ -181,7 +181,7 @@ end ```ruby # config/routes.rb -post "mcp/tokens/introspect", to: "mcp/tokens#introspect" +post "mcp/tokens/introspect", to: "tokens#introspect" ``` The payload `introspection_payload` emits is exactly the contract the satellite's @@ -226,13 +226,13 @@ end ### Using the bundled base ```ruby -class Mcp::NotificationSerializer < McpToolkit::Serializer::Base +class WidgetSerializer < McpToolkit::Serializer::Base attributes :id, :name, :active, :created_at, :updated_at translates :subject, :template_html # Globalize-backed { locale => value } has_one :account, foreign_key: :synced_account_id - has_one :mail_layout - has_many :scheduled_notifications + has_one :layout + has_many :scheduled_widgets end ``` diff --git a/lib/mcp_toolkit/auth/authority.rb b/lib/mcp_toolkit/auth/authority.rb index befef53..55296d0 100644 --- a/lib/mcp_toolkit/auth/authority.rb +++ b/lib/mcp_toolkit/auth/authority.rb @@ -5,7 +5,7 @@ # introspection request satellites send. # # Both are thin and config-driven: the actual token lookup is the app's -# `config.token_authenticator` callable (its `McpToken.authenticate` +# `config.token_authenticator` callable (its `AccessToken.authenticate` # equivalent), and the introspection payload is derived from the duck-typed # token object that callable returns. # @@ -23,7 +23,7 @@ # The sole authorization source on the satellite side. # # Optionally `#touch_last_used!` (called after a successful authenticate if -# present). A typical app token model (e.g. `McpToken`) satisfies this. +# present). A typical app token model (e.g. `AccessToken`) satisfies this. module McpToolkit::Auth::Authority # Authenticate a plaintext bearer locally. Returns the token object or nil. # Calls `touch_last_used!` on the token if it responds to it (throttled diff --git a/lib/mcp_toolkit/configuration.rb b/lib/mcp_toolkit/configuration.rb index e7877b5..dd59bc3 100644 --- a/lib/mcp_toolkit/configuration.rb +++ b/lib/mcp_toolkit/configuration.rb @@ -99,10 +99,10 @@ class McpToolkit::Configuration # Looks up + verifies a plaintext bearer token locally, returning a token # object (duck-typed, see below) or nil. This is the authority's - # `McpToken.authenticate(plaintext)` equivalent. Required for the :authority + # `AccessToken.authenticate(plaintext)` equivalent. Required for the :authority # role; unused by a pure satellite. # - # c.token_authenticator = ->(plaintext) { McpToken.authenticate(plaintext) } + # c.token_authenticator = ->(plaintext) { AccessToken.authenticate(plaintext) } # # The returned token object must respond to the methods # `McpToolkit::Auth::Authority#introspection_payload` reads (see that module for diff --git a/lib/mcp_toolkit/serializer/base.rb b/lib/mcp_toolkit/serializer/base.rb index 29e15aa..040c2c8 100644 --- a/lib/mcp_toolkit/serializer/base.rb +++ b/lib/mcp_toolkit/serializer/base.rb @@ -144,8 +144,8 @@ def self.root_key # Infer the serialized model from the serializer class name by stripping a # trailing "Serializer" and the host namespace, e.g. - # Mcp::NotificationSerializer -> Notification - # Mcp::PushNotifications::FilterSerializer -> PushNotifications::Filter + # Api::WidgetSerializer -> Widget + # Api::ScheduledWidgets::FilterSerializer -> ScheduledWidgets::Filter # Subclasses whose name doesn't follow the convention set `model_class`. def self.model_class @model_class ||= begin diff --git a/lib/mcp_toolkit/transport/controller_methods.rb b/lib/mcp_toolkit/transport/controller_methods.rb index f5e004f..f838ec6 100644 --- a/lib/mcp_toolkit/transport/controller_methods.rb +++ b/lib/mcp_toolkit/transport/controller_methods.rb @@ -5,7 +5,7 @@ # The MCP Streamable-HTTP transport, provided as an includable concern. An # app's controller includes this to get the full transport with no per-app code: # -# class Mcp::ServerController < ApplicationController +# class McpController < ApplicationController # include McpToolkit::Transport::ControllerMethods # end # @@ -44,7 +44,7 @@ # # CSRF: the concern disables forgery protection (this is a token-authenticated # JSON API). Inherit from ActionController::Base (not ::API) if your app's -# controller stack needs helper_method, as bsa-notifications does. +# controller stack needs helper_method (as some host controller stacks require). module McpToolkit::Transport::ControllerMethods extend ActiveSupport::Concern From 2dd497aa2021d3068c9d05d8be6a93f9e5be9912 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 15:48:51 +0200 Subject: [PATCH 04/10] Keep version at 0.4.0 (unreleased); fold authority/gateway changelog into 0.4.0 0.4.0 is not yet released (PR #4 open, untagged), so the Phase-1 gateway extraction and the Phase-2 authority dispatch path are one unreleased 0.4.0. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 6 +----- lib/mcp_toolkit/version.rb | 2 +- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 36bc626..36dd8aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -## [0.5.0] - 2026-07-06 +## [0.4.0] - 2026-07-06 ### Added @@ -72,10 +72,6 @@ - The engine's `app/controllers/mcp_toolkit/{server_controller,tokens_controller}.rb` files, replaced by the lazy builder above (their routes + behavior are unchanged). -## [0.4.0] - 2026-07-06 - -### Added - - **Gateway / upstream layer** (`McpToolkit::Gateway::*`) — the generic, SDK-independent machinery a central app uses to aggregate *other* MCP servers and proxy calls to them, previously an app-only concern. All app-specific values diff --git a/lib/mcp_toolkit/version.rb b/lib/mcp_toolkit/version.rb index b0c7d2f..3b017eb 100644 --- a/lib/mcp_toolkit/version.rb +++ b/lib/mcp_toolkit/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module McpToolkit - VERSION = "0.5.0" + VERSION = "0.4.0" end From 70bdc39f7f7f3eb3c1fb8ea35c9b499f02e97711 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 16:21:01 +0200 Subject: [PATCH 05/10] Add Registry-backed authority tools + generic filter/note/superuser seams (v0.4.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GEM-side of MCP full tool unification: the authority dispatch path can now serve the four generic read tools (resources/resource_schema/get/list) over config.registry, reusing the existing executors + serializer + schema builder unchanged, so a host (core) drops its own generic-tool engine and plugs in. - Resource: superusers_only!/superusers_only?, note(text)+reader, and filter(name, type:, description:, &applier) + custom_filters — a resource-specific filter block, all api-agnostic. - ListExecutor: applies matching custom_filters (top-level request keys) BEFORE the allowlist filterable filters (its only change). - Authority::RegistryToolProvider.new(config:) + the four thin Authority::Tools::{Resources,ResourceSchema,Get,List}: resolve descriptor, gate superusers_only? (refuse in get/list/resource_schema, hide in resources), gate per-resource scope, require account for get/list. Return a raw Hash for the dispatcher to wrap — added ALONGSIDE the untouched satellite SDK tool path. - Authority::CompositeToolProvider.new(*providers): concatenate definitions, find tries each in order (compose generic + bespoke tools). - ResourceSchema: per-attribute filter `operators` (from Filtering::OPERATORS_BY_TYPE) + `note` passthrough. Version stays 0.4.0 (CHANGELOG bullet + README authority section). 323 specs green (284 prior + 39 new), RuboCop clean, zero fingerprints. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 33 ++ README.md | 57 +++- .../authority/composite_tool_provider.rb | 32 ++ .../authority/registry_tool_provider.rb | 41 +++ lib/mcp_toolkit/authority/tools/base.rb | 122 ++++++++ lib/mcp_toolkit/authority/tools/get.rb | 58 ++++ lib/mcp_toolkit/authority/tools/list.rb | 84 +++++ .../authority/tools/resource_schema.rb | 44 +++ lib/mcp_toolkit/authority/tools/resources.rb | 33 ++ lib/mcp_toolkit/list_executor.rb | 17 + lib/mcp_toolkit/resource.rb | 66 ++++ lib/mcp_toolkit/resource_schema.rb | 15 +- .../authority/composite_tool_provider_spec.rb | 44 +++ .../authority/registry_tool_provider_spec.rb | 290 ++++++++++++++++++ .../registry_and_executors_spec.rb | 91 ++++++ 15 files changed, 1023 insertions(+), 4 deletions(-) create mode 100644 lib/mcp_toolkit/authority/composite_tool_provider.rb create mode 100644 lib/mcp_toolkit/authority/registry_tool_provider.rb create mode 100644 lib/mcp_toolkit/authority/tools/base.rb create mode 100644 lib/mcp_toolkit/authority/tools/get.rb create mode 100644 lib/mcp_toolkit/authority/tools/list.rb create mode 100644 lib/mcp_toolkit/authority/tools/resource_schema.rb create mode 100644 lib/mcp_toolkit/authority/tools/resources.rb create mode 100644 spec/mcp_toolkit/authority/composite_tool_provider_spec.rb create mode 100644 spec/mcp_toolkit/authority/registry_tool_provider_spec.rb diff --git a/CHANGELOG.md b/CHANGELOG.md index 36dd8aa..816a906 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,39 @@ (context lets the host hide superuser-only tools) and `provider.find(name) -> a tool object` (responding to `#required_permissions_scope` + `#call(context:, **arguments)`). The dispatcher enforces the per-tool scope gate CENTRALLY. +- **Registry-backed authority tools** — the authority-path counterpart to the + satellite's SDK tools, so a first-party server can serve the SAME four generic + read tools (`resources` / `resource_schema` / `get` / `list`) over + `config.registry` through the hand-rolled dispatcher, reusing the existing + `ListExecutor` / `GetExecutor` / `ResourceSchema` / `Serialization` / + `FieldSelection` / `Filtering` UNCHANGED. + - `McpToolkit::Authority::RegistryToolProvider.new(config:)` — a `tool_provider` + serving the four generic tools; `find(name)` returns a tool instance, and each + tool declares NO static scope (the per-resource scope is enforced dynamically + at call time). The satellite SDK tool path (`McpToolkit::Tools::*`, + `McpToolkit::Server`) is untouched — this is added alongside it. + - `McpToolkit::Authority::Tools::{Resources,ResourceSchema,Get,List}` — the four + thin tools. Each resolves the `resource` argument against the registry + (InvalidParams for unknown), gates a `superusers_only?` resource against + `context.superuser?` (REFUSE in get/list/resource_schema, HIDE in resources), + gates the resource's `required_scope_for` against the principal, and (get/list) + requires a resolved `context.account`. Returns a raw Hash for the dispatcher to + wrap — distinct by design from the satellite tools' `MCP::Tool::Response`. + - `McpToolkit::Authority::CompositeToolProvider.new(*providers)` — composes + several providers (e.g. the RegistryToolProvider + a host's bespoke tools) + behind one `config.tool_provider`: `tool_definitions` concatenates in order, + `find` returns the first match. +- **`McpToolkit::Resource` generic seams** (all api-agnostic) — `superusers_only!` + / `superusers_only?` (authority tools honor it), `note(text)` + reader (surfaced + by `resource_schema`), and `filter(name, type:, description:, &applier)` + + `custom_filters` — a resource-specific filter whose block narrows the scoped + relation from a TOP-LEVEL request param, so a host can express a relational + filter the generic equality/operator allowlist can't. `ListExecutor` applies the + matching custom filters BEFORE the allowlist filters (its only change). +- **`McpToolkit::ResourceSchema` enrichment** — each attribute now advertises the + filter `operators` it accepts (derived from `Filtering::OPERATORS_BY_TYPE`), and + the resource `note` is passed through, so a client can discover exactly which + `{ op:, value: }` conditions `list` will accept. - **Server-vs-gateway identity split** — `config.gateway_client_name` / `gateway_client_version` (each defaulting to `server_name` / `server_version`). `Gateway::Client`'s handshake `clientInfo` now reads the GATEWAY identity, so an diff --git a/README.md b/README.md index 78dd8a5..28b54e8 100644 --- a/README.md +++ b/README.md @@ -277,7 +277,8 @@ discovery tool, a custom serializer may also expose `declared_attributes` / - `McpToolkit.configure { |c| ... }`, `McpToolkit.config`, `McpToolkit.registry`, `McpToolkit.reset_config!` - `McpToolkit::Registry#register(name) { ... }` (DSL: `model`, `serializer`, - `scope`, `description`, `filterable`, `required_permissions_scope`) + + `scope`, `description`, `note`, `filterable`, `filter(name, type:, description:, + &applier)`, `superusers_only!`, `required_permissions_scope`) + `#default_required_permissions_scope` - `McpToolkit::Serializer::Base` (DSL: `attributes`, `has_one`, `has_many`, `translates`) @@ -295,8 +296,13 @@ discovery tool, a custom serializer may also expose `declared_attributes` / billing/tenancy steps overridable hooks), `McpToolkit::Authority::ServerController` (subclassable base), `McpToolkit::Authority::Context` (`account` / `principal` / `bearer_token` / - `superuser?`), `McpToolkit::Tools::AuthorityBase` (optional tool base), and - `config.tool_provider` (the api-agnostic tool seam) + `superuser?`), `McpToolkit::Tools::AuthorityBase` (optional tool base), + `config.tool_provider` (the api-agnostic tool seam), + `McpToolkit::Authority::RegistryToolProvider.new(config:)` (serves the four + generic Registry-backed tools `resources` / `resource_schema` / `get` / `list`, + reusing the executors + schema builder) + + `McpToolkit::Authority::CompositeToolProvider.new(*providers)` (compose it with + bespoke tools) - `McpToolkit::Session` (opaque `#data` payload, e.g. to bind a session to a token id) - `McpToolkit::Auth::Introspection` / `Authenticator` (satellite), `McpToolkit::Auth::Authority` (authority) @@ -432,6 +438,51 @@ end each element of a batch — so a mixed-account batch resolves the right account per call. +#### Or serve the generic Registry-backed tools + +If your tools are just account-scoped, read-only views over models, you don't need +to hand-write them. Register each as a resource (exactly as on the satellite side) +and let the bundled provider serve the same four generic tools — `resources`, +`resource_schema`, `get`, `list` — over the authority dispatcher: + +```ruby +McpToolkit.configure do |c| + c.registry.register(:widgets) do + model Widget + serializer WidgetSerializer # any class satisfying the serializer contract + description "Widgets for the active account." + filterable status: :status, owner_id: :owner_id + # A resource-specific ("custom") filter: an arbitrary block, keyed by a + # top-level request param, that the generic equality allowlist can't express. + filter :for_project, type: :integer, description: "Only widgets in this project" do |relation, id| + relation.joins(:board).where(boards: { project_id: id }) + end + superusers_only! # optional: refuse/hide for non-superuser tokens + note "Read-only projection; do not interpret status codes without domain context." + scope { |account| Widget.where(account_id: account.id) } + end + + # The generic tools, served over config.registry: + c.tool_provider = McpToolkit::Authority::RegistryToolProvider.new(config: c) +end +``` + +Each generic tool resolves the `resource` argument against the registry, refuses a +`superusers_only!` resource for a non-superuser (and hides it from `resources`), +enforces the resource's `required_permissions_scope`, and requires a resolved +account for `get` / `list`. `resource_schema` advertises each attribute's filter +`operators` and the resource `note`. + +To serve the generic tools **and** your own bespoke tools behind one provider, +compose them: + +```ruby +c.tool_provider = McpToolkit::Authority::CompositeToolProvider.new( + McpToolkit::Authority::RegistryToolProvider.new(config: c), + MyBespokeToolProvider.new # e.g. an audit/versions tool +) +``` + ### 2. Serve it through the authority transport A **pure host** mounts the engine's authority base and drives everything from diff --git a/lib/mcp_toolkit/authority/composite_tool_provider.rb b/lib/mcp_toolkit/authority/composite_tool_provider.rb new file mode 100644 index 0000000..0f675aa --- /dev/null +++ b/lib/mcp_toolkit/authority/composite_tool_provider.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +# Composes several `tool_provider`s into one, so a host can serve the generic +# Registry-backed tools (McpToolkit::Authority::RegistryToolProvider) ALONGSIDE +# its own bespoke tools (e.g. a paper-trail/versions tool that doesn't fit the +# generic resource model) behind a single `config.tool_provider`. +# +# It satisfies the same duck-typed provider contract the dispatcher calls: +# tool_definitions(context) -> the concatenation of every provider's definitions, +# in registration order +# find(name) -> the first provider (in order) that resolves the +# name, else nil +# +# Ordering is significant only if two providers advertise the same tool name; the +# first registered wins. A host controls precedence by argument order. +class McpToolkit::Authority::CompositeToolProvider + def initialize(*providers) + @providers = providers + end + + def tool_definitions(context) + @providers.flat_map { |provider| provider.tool_definitions(context) } + end + + def find(name) + @providers.each do |provider| + tool = provider.find(name) + return tool if tool + end + nil + end +end diff --git a/lib/mcp_toolkit/authority/registry_tool_provider.rb b/lib/mcp_toolkit/authority/registry_tool_provider.rb new file mode 100644 index 0000000..9b543a7 --- /dev/null +++ b/lib/mcp_toolkit/authority/registry_tool_provider.rb @@ -0,0 +1,41 @@ +# frozen_string_literal: true + +# A `tool_provider` (the dispatcher's api-agnostic seam) that serves the four +# GENERIC, Registry-backed tools — `resources`, `resource_schema`, `get`, `list` — +# over the resources a host registered in `config.registry`. It is the authority- +# path counterpart to the satellite's SDK tools (McpToolkit::Tools::*): same +# generic contract (discover resources, learn a shape, read one/many rows, all +# account-scoped and read-only), but plugged into the hand-rolled dispatcher. +# +# Satisfies the provider contract the dispatcher calls: +# tool_definitions(context) -> the four static generic tool definitions +# find(name) -> a tool instance bound to this config, or nil +# +# The top-level definitions are context-independent (per-resource visibility and +# scope are enforced inside each tool at call time), so `tool_definitions` ignores +# the context. Compose this with bespoke host tools via CompositeToolProvider. +class McpToolkit::Authority::RegistryToolProvider + # Tool name (as advertised in `tools/list` and matched in `tools/call`) => the + # tool class. The four generic read tools; nothing here names an app concept. + TOOLS = { + "resources" => McpToolkit::Authority::Tools::Resources, + "resource_schema" => McpToolkit::Authority::Tools::ResourceSchema, + "get" => McpToolkit::Authority::Tools::Get, + "list" => McpToolkit::Authority::Tools::List + }.freeze + + def initialize(config:) + @config = config + end + + # The four static generic tool definitions (context-independent). + def tool_definitions(_context) + TOOLS.each_value.map(&:definition) + end + + # A tool instance bound to this provider's config, or nil for an unknown name. + def find(name) + klass = TOOLS[name.to_s] + klass&.new(config: @config) + end +end diff --git a/lib/mcp_toolkit/authority/tools/base.rb b/lib/mcp_toolkit/authority/tools/base.rb new file mode 100644 index 0000000..e2cc588 --- /dev/null +++ b/lib/mcp_toolkit/authority/tools/base.rb @@ -0,0 +1,122 @@ +# frozen_string_literal: true + +# Shared base for the four GENERIC, Registry-backed authority tools +# (McpToolkit::Authority::Tools::{Resources,ResourceSchema,Get,List}) served +# through McpToolkit::Authority::RegistryToolProvider on the hand-rolled +# authority dispatch path. +# +# Unlike the satellite tools (McpToolkit::Tools::*, which subclass the SDK's +# MCP::Tool, self-authenticate, and return an MCP::Tool::Response), these are +# plain objects satisfying the dispatcher's duck-typed tool contract: +# +# tool.required_permissions_scope -> nil (see below — no STATIC scope) +# tool.call(context:, **arguments) -> Hash (the dispatcher wraps it into +# { content: [{ type: "text", ... }] }) +# +# The scope a caller needs is DYNAMIC — it depends on which `resource` argument +# was passed — so these tools declare no static scope and instead enforce the +# resolved resource's `required_scope_for` INSIDE #call (see #ensure_scope!). +# The `context` (McpToolkit::Authority::Context) supplies the resolved account, +# the principal, and the derived superuser flag. +# +# The tools reuse the existing executors / schema builder UNCHANGED; this base +# only holds the resolution + gating every one of them repeats: resolve the +# resource descriptor, gate a superuser-only resource, gate the per-resource +# scope, and (for get/list) require a selected account. +class McpToolkit::Authority::Tools::Base + class << self + attr_reader :_description, :_input_schema + + def tool_name(name = nil) + @_tool_name = name.to_s if name + @_tool_name + end + + def description(text = nil) + @_description = text if text + @_description + end + + def input_schema(schema = nil) + @_input_schema = schema if schema + @_input_schema || { type: "object", properties: {} } + end + + # The static tool definition returned by the provider's `tool_definitions` + # (part of `tools/list`). Generic and context-independent. + def definition + { name: tool_name, description: _description, inputSchema: input_schema } + end + end + + attr_reader :config + + def initialize(config:) + @config = config + end + + # The dispatcher's central scope gate reads this; nil = no STATIC scope. The + # real, per-resource scope is enforced dynamically in #ensure_scope! (the scope + # depends on the `resource` argument, unknown until #call). + def required_permissions_scope + nil + end + + private + + def registry + config.registry + end + + # Resolves the `resource` argument to a registered descriptor, raising the + # protocol InvalidParams (=> JSON-RPC -32602) for a blank or unknown name so the + # dispatcher renders a clean top-level error. + def resolve_descriptor(name) + raise McpToolkit::Protocol::InvalidParams, "resource is required" if name.to_s.strip.empty? + + registry.fetch(name) + rescue McpToolkit::Registry::UnknownResource => e + raise McpToolkit::Protocol::InvalidParams, e.message + end + + # Refuses a superuser-only resource for a non-superuser caller (get / list / + # resource_schema). `resources` HIDES such resources instead — see that tool. + def ensure_resource_accessible!(descriptor, context) + return unless descriptor.superusers_only? + return if context.superuser? + + raise McpToolkit::Protocol::InvalidRequest, + "#{descriptor.name} is restricted to superuser (user-scoped) MCP tokens" + end + + # Enforces the resource's effective required scope against the principal. Blank + # scope => no check. Mirrors the dispatcher's central gate error shape + # (InvalidRequest), keeping scope refusals byte-consistent across host tools. + def ensure_scope!(descriptor, context) + required = registry.required_scope_for(descriptor) + return if required.to_s.empty? + return if context.principal&.authorized_for_scope?(required) + + raise McpToolkit::Protocol::InvalidRequest, "This token lacks the #{required.inspect} scope" + end + + # get / list read tenant data, so they REQUIRE a resolved account: a superuser + # token that selected none would otherwise reach `scope.call(nil)` and leak + # across tenants. resource_schema / resources (shape only) do not call this. + def ensure_account!(context) + return if context.account + + raise McpToolkit::Protocol::InvalidParams, + "an account must be selected (pass account_id) to read this resource" + end + + # Runs an executor, translating a data-layer McpToolkit::Errors::InvalidParams + # (bad id, unknown filter/field key, ...) into the protocol InvalidParams the + # dispatcher renders as JSON-RPC -32602 (rather than letting it fall through to + # the dispatcher's generic -32603 internal-error mapping). + def run_executor + yield + rescue McpToolkit::Errors::InvalidParams => e + raise McpToolkit::Protocol::InvalidParams, e.message + end +end diff --git a/lib/mcp_toolkit/authority/tools/get.rb b/lib/mcp_toolkit/authority/tools/get.rb new file mode 100644 index 0000000..3ba0cb6 --- /dev/null +++ b/lib/mcp_toolkit/authority/tools/get.rb @@ -0,0 +1,58 @@ +# frozen_string_literal: true + +# Authority-path tool: fetch a single record by id from a registered resource, +# scoped to the caller's resolved account. Gates a superuser-only resource, the +# resource's required scope, and requires a selected account before reading. +class McpToolkit::Authority::Tools::Get < McpToolkit::Authority::Tools::Base + tool_name "get" + description <<~DESC.strip + Fetch a single record by id from a read-only resource. Pass the resource name as `resource` + and the record id as `id`. Use the `resources` tool to discover available resources. + + For tokens that span multiple accounts (superuser), pass `account_id` to pin the active + account; account-scoped tokens may omit it. The response mirrors the resource's record + shape (attributes + a `links` block). + + Pass `fields` to return a sparse fieldset — the attributes and/or relationships you name + (as an array or comma-separated string), omitting everything else. Include "id" if you need + it. Valid names come from the resource's `resource_schema`; unknown names are rejected. + DESC + + input_schema( + { + type: "object", + properties: { + resource: { + type: "string", + description: "Resource name (use the `resources` tool to discover valid values)" + }, + # The id type is left open so a string/UUID primary key works as well as an + # integer one; the record is looked up by the value as given, uncoerced. + id: { type: %w[string integer], description: "The record ID (integer or string/UUID)" }, + account_id: { + type: "integer", + description: "Account to operate on. Required for superuser tokens; ignored otherwise." + }, + fields: { + type: %w[array string], + items: { type: "string" }, + description: "Sparse fieldset — names of attributes and/or relationships to include, as " \ + "an array or a comma-separated string. Omit to return every field. Include " \ + "\"id\" if you need it. Unknown names are rejected." + } + }, + required: %w[resource id] + } + ) + + def call(context:, resource: nil, id: nil, fields: nil, **_args) + descriptor = resolve_descriptor(resource) + ensure_resource_accessible!(descriptor, context) + ensure_scope!(descriptor, context) + ensure_account!(context) + + run_executor do + McpToolkit::GetExecutor.call(resource: descriptor, scope_root: context.account, id:, fields:) + end + end +end diff --git a/lib/mcp_toolkit/authority/tools/list.rb b/lib/mcp_toolkit/authority/tools/list.rb new file mode 100644 index 0000000..8dcce11 --- /dev/null +++ b/lib/mcp_toolkit/authority/tools/list.rb @@ -0,0 +1,84 @@ +# frozen_string_literal: true + +# Authority-path tool: fetch a paginated list of records from a registered +# resource, scoped to the caller's resolved account. Gates a superuser-only +# resource, the resource's required scope, and requires a selected account before +# reading. Standard filters, per-attribute equality/operator filters, resource +# custom filters, pagination, and sparse fieldsets are all handled by the reused +# McpToolkit::ListExecutor. +class McpToolkit::Authority::Tools::List < McpToolkit::Authority::Tools::Base + tool_name "list" + description <<~DESC.strip + Fetch a paginated list of records from a read-only resource. Pass the resource name as + `resource`. Use the `resources` tool to discover resources and `resource_schema` to learn a + resource's shape. + + Standard filters: + - ids: comma-separated list of IDs to fetch + - updated_since: ISO 8601 timestamp; only records updated after this time + - limit: page size (default 25, max 100) + - offset: pagination offset (default 0) + + Per-attribute filters: + - filter: an object of { : } filters, applied ON TOP of the account scope + (they can only narrow, never widen). Each resource advertises its available filter keys + and operators via `resource_schema`. Unknown keys are rejected. + + Sparse fieldset: + - fields: names of the attributes and/or relationships to include in each record, as an + array or a comma-separated string. Omit to return every field. Include "id" if you need + it. Valid names come from a resource's `resource_schema`; unknown names are rejected. + + For tokens that span multiple accounts (superuser), pass `account_id` to pin the active + account; account-scoped tokens may omit it. The response shape is + { "": [...], "meta": { total_count, limit, offset } }. + DESC + + input_schema( + { + type: "object", + properties: { + resource: { + type: "string", + description: "Resource name (use the `resources` tool to discover valid values)" + }, + account_id: { + type: "integer", + description: "Account to operate on. Required for superuser tokens; ignored otherwise." + }, + ids: { type: "string", description: "Comma-separated list of IDs to fetch" }, + updated_since: { + type: "string", + description: "ISO 8601 timestamp; only records updated after this time" + }, + filter: { + type: "object", + description: "Per-attribute filters, e.g. { \"booking_id\": 42 }. See a resource's " \ + "`resource_schema` `filters` for the keys and operators it accepts.", + additionalProperties: true + }, + limit: { type: "integer", description: "Page size (default 25, max 100)" }, + offset: { type: "integer", description: "Pagination offset (default 0)" }, + fields: { + type: %w[array string], + items: { type: "string" }, + description: "Sparse fieldset — names of attributes and/or relationships to include in " \ + "each record, as an array or a comma-separated string. Omit to return every " \ + "field. Include \"id\" if you need it. Unknown names are rejected." + } + }, + required: ["resource"] + } + ) + + def call(context:, resource: nil, **params) + descriptor = resolve_descriptor(resource) + ensure_resource_accessible!(descriptor, context) + ensure_scope!(descriptor, context) + ensure_account!(context) + + run_executor do + McpToolkit::ListExecutor.call(resource: descriptor, scope_root: context.account, params:) + end + end +end diff --git a/lib/mcp_toolkit/authority/tools/resource_schema.rb b/lib/mcp_toolkit/authority/tools/resource_schema.rb new file mode 100644 index 0000000..f4d4548 --- /dev/null +++ b/lib/mcp_toolkit/authority/tools/resource_schema.rb @@ -0,0 +1,44 @@ +# frozen_string_literal: true + +# Authority-path discovery tool: the detailed schema (attributes with types + +# filter operators, relationships, filters, note) of one registered resource. +# +# Reveals shape, not tenant data, so it does NOT require a selected account — but +# it still gates a superuser-only resource (refuse) and the resource's required +# scope, so a caller can't discover the shape of something it can't read. +class McpToolkit::Authority::Tools::ResourceSchema < McpToolkit::Authority::Tools::Base + tool_name "resource_schema" + description <<~DESC.strip + Describe a single read-only resource in detail. Pass the resource name as `resource` (use + the `resources` tool to discover names). Returns: + - attributes: every field in the response, each with its `type`, a value `format` hint, + whether it is `filterable`, and the filter `operators` it accepts + - relationships: associated resources emitted in the record's `links`; each names the + `target_resource` it resolves to (callable via `list`/`get`) + - standard_filters: ids, updated_since, limit, offset (accepted by the `list` tool) + - filters: the per-attribute equality filter keys the `list` tool accepts + The `attributes` and `relationships` names are also the valid values for the `fields` sparse + fieldset argument on `get` / `list`. Call this before `list` to learn a resource's shape. + DESC + + input_schema( + { + type: "object", + properties: { + resource: { + type: "string", + description: "Resource name (use the `resources` tool to discover valid values)" + } + }, + required: ["resource"] + } + ) + + def call(context:, resource: nil, **_args) + descriptor = resolve_descriptor(resource) + ensure_resource_accessible!(descriptor, context) + ensure_scope!(descriptor, context) + + McpToolkit::ResourceSchema.call(descriptor, registry:) + end +end diff --git a/lib/mcp_toolkit/authority/tools/resources.rb b/lib/mcp_toolkit/authority/tools/resources.rb new file mode 100644 index 0000000..cfeae58 --- /dev/null +++ b/lib/mcp_toolkit/authority/tools/resources.rb @@ -0,0 +1,33 @@ +# frozen_string_literal: true + +# Authority-path discovery tool: lists every read-only resource registered in the +# config's Registry, HIDING superuser-only resources from a non-superuser caller +# (so they are neither advertised nor discoverable without a superuser token). +# +# The top-level list is context-independent except for that visibility filter, so +# the tool takes no arguments and does no per-resource scope check (per-resource +# scopes are enforced by `get` / `list` / `resource_schema`). +class McpToolkit::Authority::Tools::Resources < McpToolkit::Authority::Tools::Base + tool_name "resources" + description <<~DESC.strip + List all read-only resources available via the `list` and `get` tools. Returns each + resource's name and a short description. Call this once at the start of a session to learn + what exists, then use `resource_schema` for a specific resource's attributes and + relationships. + DESC + + def call(context:, **_args) + { + resources: visible_resources(context).map do |resource| + { name: resource.name, description: resource.description } + end + } + end + + private + + # Superuser-only resources are hidden from a non-superuser caller. + def visible_resources(context) + registry.resources.reject { |resource| resource.superusers_only? && !context.superuser? } + end +end diff --git a/lib/mcp_toolkit/list_executor.rb b/lib/mcp_toolkit/list_executor.rb index 3de1a28..1598eda 100644 --- a/lib/mcp_toolkit/list_executor.rb +++ b/lib/mcp_toolkit/list_executor.rb @@ -45,10 +45,27 @@ def build_relation relation = resource.resolve_relation(scope_root) relation = apply_ids(relation) relation = apply_updated_since(relation) + relation = apply_custom_filters(relation) relation = apply_attribute_filters(relation) apply_order(relation) end + # Applies the resource's declared custom (resource-specific) filters — each an + # arbitrary host-supplied block — for the keys actually present as TOP-LEVEL + # request params, BEFORE the generic allowlist `filter` attributes. Each block + # receives the already-scoped relation and the raw value and returns a narrowed + # relation, so a host can express a relational filter the generic path can't + # derive (see Resource#filter). A resource with no custom filters is a no-op. + def apply_custom_filters(relation) + resource.custom_filters.each_value do |custom_filter| + value = params[custom_filter.name] + next if value.nil? || value == "" + + relation = custom_filter.applier.call(relation, value) + end + relation + end + # Order by `id` when the primary key is numeric; otherwise by `created_at` # (a non-numeric PK does not sort meaningfully). def apply_order(relation) diff --git a/lib/mcp_toolkit/resource.rb b/lib/mcp_toolkit/resource.rb index 1275b25..fbf2584 100644 --- a/lib/mcp_toolkit/resource.rb +++ b/lib/mcp_toolkit/resource.rb @@ -16,6 +16,16 @@ class McpToolkit::Resource class NotConfigured < StandardError; end + # A resource-specific ("custom") filter: a request-facing key whose value is + # applied to the relation by an arbitrary host-supplied block, rather than the + # generic equality/operator allowlist. The block is api-agnostic — it receives + # the already-scoped relation and the raw request value and returns a narrowed + # relation — so a host can express a relational or otherwise non-column filter + # (e.g. "only rows whose associated booking is in this rental") without the gem + # knowing anything about the query. `type`/`description` are surfaced by + # resource_schema so a client can discover the filter. + CustomFilter = Struct.new(:name, :type, :description, :applier, keyword_init: true) + attr_reader :name def initialize(name) @@ -24,7 +34,10 @@ def initialize(name) @serializer = nil @scope_block = nil @description = nil + @note = nil + @superusers_only = false @filterable = {} + @custom_filters = {} @required_permissions_scope = nil end @@ -48,6 +61,59 @@ def description(text = nil) @description end + # Free-form usage caveat surfaced by the `resources` / `resource_schema` tools, + # e.g. to flag a resource as internal-debugging-only and not to be interpreted + # without domain knowledge. Read with no arg. api-agnostic passthrough string. + def note(text = nil) + @note = text if text + @note + end + + # Restricts this resource to superuser (cross-tenant) callers on the AUTHORITY + # path: an authority tool refuses `get` / `list` / `resource_schema` for a + # non-superuser and HIDES the resource from `resources` discovery. Declared in a + # resource's registration block: + # + # McpToolkit.registry.register(:audit_events) do + # superusers_only! + # ... + # end + # + # Generic and api-agnostic — the gem never names an app concept; the caller's + # superuser-ness is derived by the Authority::Context off the principal. + def superusers_only! + @superusers_only = true + end + + # Whether this resource is restricted to superuser callers (default false). + def superusers_only? + @superusers_only + end + + # Declares a resource-specific ("custom") filter: a request-facing `name` whose + # value is applied to the already-scoped relation by the given block. Unlike the + # `filterable` allowlist (generic equality/operator filters on a declared + # column), a custom filter runs ARBITRARY host logic, so a host can express a + # relational filter the gem could not derive: + # + # filter :rental_id, type: :integer, description: "Only rows for this rental" do |relation, value| + # relation.joins(:booking).where(bookings: { rental_id: value }) + # end + # + # The block receives `(relation, value)` and MUST return a relation (narrowing + # only). `type` / `description` are metadata surfaced by `resource_schema`. The + # value arrives from a TOP-LEVEL request param keyed by `name` (see ListExecutor), + # applied BEFORE the allowlist `filterable` filters. api-agnostic: the gem stores + # and calls the block without inspecting it. + def filter(name, type:, description:, &applier) + @custom_filters[name.to_sym] = CustomFilter.new(name: name.to_sym, type:, description:, applier:) + end + + # Request-facing custom-filter key (symbol) => CustomFilter. Consumed by the list + # executor (which applies each block whose key is present in the request params) + # and by resource_schema (which surfaces each filter's type/description). + attr_reader :custom_filters + # The OAuth-style scope a token MUST carry to reach this resource via the # generic tools (e.g. "notifications__read"). Declared explicitly per resource: # diff --git a/lib/mcp_toolkit/resource_schema.rb b/lib/mcp_toolkit/resource_schema.rb index 8eb880e..2c3bff6 100644 --- a/lib/mcp_toolkit/resource_schema.rb +++ b/lib/mcp_toolkit/resource_schema.rb @@ -38,6 +38,7 @@ def call { name: resource.name, description: resource.description, + note: resource.note, attributes:, relationships:, standard_filters: STANDARD_FILTERS, @@ -59,10 +60,22 @@ def attribute_schema(name) name:, type: type ? type.to_s : COMPUTED_TYPE, format: type ? TYPE_FORMATS[type] : nil, - filterable: filterable_column_for(name).present? + filterable: filterable_column_for(name).present?, + operators: operators_for(name) }.compact end + # The filter operators an attribute accepts, derived from the backing column's + # type via McpToolkit::Filtering::OPERATORS_BY_TYPE. `[]` for a non-filterable + # attribute (or one whose column type has no operator set) — self-describing so + # a client knows exactly which `{ op:, value: }` conditions `list` will accept. + def operators_for(attribute_name) + pair = filterable_column_for(attribute_name) + return [] unless pair + + McpToolkit::Filtering::OPERATORS_BY_TYPE.fetch(column_type(pair.last), []) + end + # Per-attribute equality filters this resource accepts on the `list` tool's # `filter` argument. Each entry is the request-facing key, the backing column # it matches against, and the column's type — self-describing so an MCP client diff --git a/spec/mcp_toolkit/authority/composite_tool_provider_spec.rb b/spec/mcp_toolkit/authority/composite_tool_provider_spec.rb new file mode 100644 index 0000000..0ee949b --- /dev/null +++ b/spec/mcp_toolkit/authority/composite_tool_provider_spec.rb @@ -0,0 +1,44 @@ +# frozen_string_literal: true + +require "spec_helper" + +# Composing the generic Registry-backed provider with a host's bespoke tools +# behind a single `config.tool_provider`, so both are served by the one +# dispatcher. Uses the reusable FakeToolProvider double for the bespoke side. +RSpec.describe McpToolkit::Authority::CompositeToolProvider do + let(:registry_provider) { McpToolkit::Authority::RegistryToolProvider.new(config: McpToolkit.config) } + let(:bespoke_tool) { FakeTool.new { |_ctx, _args| { versions: [] } } } + let(:bespoke_provider) { FakeToolProvider.new(tools: { "paper_trail_versions" => bespoke_tool }) } + let(:context) { McpToolkit::Authority::Context.new(account: FakeAccount.new(1), principal: FakePrincipal.new) } + + subject(:composite) { described_class.new(registry_provider, bespoke_provider) } + + describe "#tool_definitions" do + it "concatenates every provider's definitions in registration order" do + names = composite.tool_definitions(context).map { |definition| definition[:name] } + + expect(names).to eq(%w[resources resource_schema get list paper_trail_versions]) + end + end + + describe "#find" do + it "resolves a name owned by the first provider" do + expect(composite.find("get")).to be_a(McpToolkit::Authority::Tools::Get) + end + + it "falls through to a later provider for a name it owns" do + expect(composite.find("paper_trail_versions")).to eq(bespoke_tool) + end + + it "returns nil when no provider resolves the name" do + expect(composite.find("nope")).to be_nil + end + + it "prefers the first provider that resolves a shared name" do + shadow = FakeToolProvider.new(tools: { "get" => FakeTool.new }) + first_wins = described_class.new(registry_provider, shadow) + + expect(first_wins.find("get")).to be_a(McpToolkit::Authority::Tools::Get) + end + end +end diff --git a/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb b/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb new file mode 100644 index 0000000..24218de --- /dev/null +++ b/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb @@ -0,0 +1,290 @@ +# frozen_string_literal: true + +require "spec_helper" + +# The authority-path counterpart to the satellite tools: the four GENERIC, +# Registry-backed tools (`resources` / `resource_schema` / `get` / `list`) served +# through McpToolkit::Authority::RegistryToolProvider over resources registered in +# `config.registry`, using a fake AR model + a fake serializer (no database). This +# proves the gem serves get/list/resources/resource_schema on the authority path +# by reusing the existing executors + schema builder UNCHANGED, and enforces the +# superuser / scope / account gates the design locked in. +RSpec.describe McpToolkit::Authority::RegistryToolProvider do + subject(:provider) { described_class.new(config: McpToolkit.config) } + + let(:account) { FakeAccount.new(7) } + let(:principal) { FakePrincipal.new(scopes:, superuser:) } + let(:scopes) { [] } + let(:superuser) { false } + let(:context) { McpToolkit::Authority::Context.new(account:, principal:, bearer_token: "tok") } + + # A fake model exposing the column metadata resource_schema / ordering read. + let(:widget_model) do + Class.new do + def self.columns_hash + { + "id" => FakeRelation::Column.new(:integer), + "name" => FakeRelation::Column.new(:string), + "booking_id" => FakeRelation::Column.new(:integer) + } + end + def self.primary_key = "id" + def self.model_name = FakeModelName.new("widgets") + end + end + + let(:widget_serializer) do + model = widget_model + Class.new(McpToolkit::Serializer::Base) do + attributes :id, :name, :booking_id + self.model_class = model + def self.name = "WidgetSerializer" + end + end + + let(:rows) do + [ + FakeRecord.new(id: 1, name: "alpha", booking_id: 10), + FakeRecord.new(id: 2, name: "beta", booking_id: 20) + ] + end + let(:relation) { FakeRelation.new(rows, table_name: "widgets", model: widget_model) } + + before do + serializer = widget_serializer + model = widget_model + rel = relation + McpToolkit.configure do |c| + c.sql_sanitizer = FakeSqlSanitizer.new + c.registry.register(:widgets) do + model model + serializer serializer + description "Test widgets." + filterable booking_id: :booking_id, name: :name + scope { |_root| rel } + end + end + end + + describe "#tool_definitions" do + it "returns the four static generic tool definitions" do + names = provider.tool_definitions(context).map { |definition| definition[:name] } + + expect(names).to contain_exactly("resources", "resource_schema", "get", "list") + end + + it "gives every definition a name, description and inputSchema (the tools/list shape)" do + provider.tool_definitions(context).each do |definition| + expect(definition).to include(:name, :description, :inputSchema) + end + end + + it "is context-independent (superuser visibility is handled inside `resources`)" do + su = McpToolkit::Authority::Context.new(account:, principal: FakePrincipal.new(superuser: true), bearer_token: "t") + + expect(provider.tool_definitions(su)).to eq(provider.tool_definitions(context)) + end + end + + describe "#find" do + it "resolves each known tool name to its tool instance" do + expect(provider.find("get")).to be_a(McpToolkit::Authority::Tools::Get) + expect(provider.find("list")).to be_a(McpToolkit::Authority::Tools::List) + expect(provider.find("resources")).to be_a(McpToolkit::Authority::Tools::Resources) + expect(provider.find("resource_schema")).to be_a(McpToolkit::Authority::Tools::ResourceSchema) + end + + it "returns nil for an unknown tool name" do + expect(provider.find("delete")).to be_nil + end + + it "declares no STATIC scope (the per-resource scope is enforced at call time)" do + expect(provider.find("list").required_permissions_scope).to be_nil + end + end + + describe "the `get` tool" do + subject(:get) { provider.find("get") } + + it "fetches a single record by id, scoped through the account" do + result = get.call(context:, resource: "widgets", id: 1) + + expect(result).to include(id: 1, name: "alpha") + end + + it "honors a sparse fieldset" do + result = get.call(context:, resource: "widgets", id: 1, fields: "id,name") + + expect(result).to eq(id: 1, name: "alpha") + end + + it "maps a data-layer InvalidParams (unknown id) to the protocol InvalidParams (-32602)" do + expect { get.call(context:, resource: "widgets", id: 999) } + .to raise_error(McpToolkit::Protocol::InvalidParams, /not found/) + end + + it "rejects an unknown resource with the protocol InvalidParams" do + expect { get.call(context:, resource: "nope", id: 1) } + .to raise_error(McpToolkit::Protocol::InvalidParams) + end + + it "requires the resource argument" do + expect { get.call(context:, resource: "", id: 1) } + .to raise_error(McpToolkit::Protocol::InvalidParams, /resource is required/) + end + end + + describe "the `list` tool" do + subject(:list) { provider.find("list") } + + it "returns the paginated collection wrapper scoped through the account" do + result = list.call(context:, resource: "widgets") + + expect(result[:widgets].map { |widget| widget[:id] }).to eq([1, 2]) + expect(result[:meta]).to eq(total_count: 2, limit: 25, offset: 0) + end + + it "applies a declared equality filter" do + result = list.call(context:, resource: "widgets", filter: { booking_id: 10 }) + + expect(result[:widgets].map { |widget| widget[:id] }).to eq([1]) + end + + it "rejects an unknown filter key with the protocol InvalidParams" do + expect { list.call(context:, resource: "widgets", filter: { bogus: 1 }) } + .to raise_error(McpToolkit::Protocol::InvalidParams, /unknown filter attribute/) + end + + it "ignores a superuser account_id argument (it is resolved by the transport, not the tool)" do + result = list.call(context:, resource: "widgets", account_id: 99) + + expect(result[:widgets].size).to eq(2) + end + end + + describe "the `resources` tool" do + subject(:resources) { provider.find("resources") } + + it "lists every registered resource's name and description" do + result = resources.call(context:) + + expect(result[:resources]).to contain_exactly({ name: "widgets", description: "Test widgets." }) + end + end + + describe "the `resource_schema` tool" do + subject(:resource_schema) { provider.find("resource_schema") } + + it "describes the resource shape without requiring a selected account" do + accountless = McpToolkit::Authority::Context.new(account: nil, principal:, bearer_token: "t") + + schema = resource_schema.call(context: accountless, resource: "widgets") + + expect(schema[:name]).to eq("widgets") + expect(schema[:attributes].map { |attribute| attribute[:name] }).to include(:booking_id) + end + end + + describe "superusers_only gating" do + before do + s = widget_serializer + m = widget_model + rel = relation + McpToolkit.configure do |c| + c.registry.register(:secrets) do + superusers_only! + model m + serializer s + description "Superuser-only." + scope { |_root| rel } + end + end + end + + context "for a non-superuser caller" do + it "refuses `get`" do + expect { provider.find("get").call(context:, resource: "secrets", id: 1) } + .to raise_error(McpToolkit::Protocol::InvalidRequest, /superuser/) + end + + it "refuses `list`" do + expect { provider.find("list").call(context:, resource: "secrets") } + .to raise_error(McpToolkit::Protocol::InvalidRequest, /superuser/) + end + + it "refuses `resource_schema`" do + expect { provider.find("resource_schema").call(context:, resource: "secrets") } + .to raise_error(McpToolkit::Protocol::InvalidRequest, /superuser/) + end + + it "HIDES the resource from `resources` discovery" do + names = provider.find("resources").call(context:)[:resources].map { |resource| resource[:name] } + + expect(names).to eq(["widgets"]) + end + end + + context "for a superuser caller" do + let(:superuser) { true } + + it "allows `get`" do + result = provider.find("get").call(context:, resource: "secrets", id: 1) + + expect(result).to include(id: 1) + end + + it "reveals the resource in `resources` discovery" do + names = provider.find("resources").call(context:)[:resources].map { |resource| resource[:name] } + + expect(names).to contain_exactly("widgets", "secrets") + end + end + end + + describe "account requirement (get / list)" do + let(:accountless) { McpToolkit::Authority::Context.new(account: nil, principal:, bearer_token: "t") } + + it "refuses `get` with InvalidParams when no account is resolved" do + expect { provider.find("get").call(context: accountless, resource: "widgets", id: 1) } + .to raise_error(McpToolkit::Protocol::InvalidParams, /account must be selected/) + end + + it "refuses `list` with InvalidParams when no account is resolved" do + expect { provider.find("list").call(context: accountless, resource: "widgets") } + .to raise_error(McpToolkit::Protocol::InvalidParams, /account must be selected/) + end + end + + describe "per-resource scope gating" do + before do + s = widget_serializer + m = widget_model + rel = relation + McpToolkit.configure do |c| + c.registry.register(:scoped_widgets) do + required_permissions_scope "widgets__read" + model m + serializer s + description "Scoped widgets." + scope { |_root| rel } + end + end + end + + it "refuses a resource whose required scope the principal lacks" do + expect { provider.find("list").call(context:, resource: "scoped_widgets") } + .to raise_error(McpToolkit::Protocol::InvalidRequest, /widgets__read/) + end + + context "when the principal carries the scope" do + let(:scopes) { ["widgets__read"] } + + it "allows the call" do + result = provider.find("list").call(context:, resource: "scoped_widgets") + + # The collection root key derives from the shared serializer's model name. + expect(result[:widgets].size).to eq(2) + end + end + end +end diff --git a/spec/mcp_toolkit/registry_and_executors_spec.rb b/spec/mcp_toolkit/registry_and_executors_spec.rb index 5be4894..ab2f572 100644 --- a/spec/mcp_toolkit/registry_and_executors_spec.rb +++ b/spec/mcp_toolkit/registry_and_executors_spec.rb @@ -235,6 +235,57 @@ def self.name = "ThingSerializer" expect(rel.ordered_by).to eq(:created_at) end end + + describe "custom filters (the Resource#filter seam)" do + before do + serializer = widget_serializer + model = widget_model + rel = relation + McpToolkit.configure do |c| + c.registry.register(:custom_widgets) do + model model + serializer serializer + description "Widgets with a relational custom filter." + filterable price: :price + # A custom filter keyed OUTSIDE the equality allowlist: it applies an + # arbitrary block to the scoped relation, reading a TOP-LEVEL request param. + filter :for_booking, type: :integer, description: "Only widgets for this booking" do |relation, value| + relation.where(booking_id: value) + end + scope { |_root| rel } + end + end + end + + subject(:custom_resource) { McpToolkit.registry.fetch("custom_widgets") } + + it "applies the custom-filter block for a matching TOP-LEVEL request key" do + result = described_class.call(resource: custom_resource, scope_root: account, params: { for_booking: 10 }) + + expect(result[:widgets].map { |w| w[:id] }).to eq([1, 3]) + end + + it "is a no-op when the custom-filter key is absent or blank" do + result = described_class.call(resource: custom_resource, scope_root: account, params: { for_booking: "" }) + + expect(result[:widgets].map { |w| w[:id] }).to eq([1, 2, 3]) + end + + it "runs BEFORE and composes with the allowlist equality filters" do + result = described_class.call( + resource: custom_resource, scope_root: account, + params: { for_booking: 10, filter: { price: { op: "gteq", value: 300 } } } + ) + + expect(result[:widgets].map { |w| w[:id] }).to eq([3]) + end + + it "does NOT expose the custom key through the equality allowlist" do + expect do + described_class.call(resource: custom_resource, scope_root: account, params: { filter: { for_booking: 10 } }) + end.to raise_error(McpToolkit::Errors::InvalidParams, /unknown filter attribute/) + end + end end describe McpToolkit::GetExecutor do @@ -332,6 +383,46 @@ def self.serialize_collection(records, scope: nil, total_count: nil, limit: nil, ) end + it "advertises the per-attribute filter operators derived from the column type" do + schema = described_class.call(resource) + + booking = schema[:attributes].find { |a| a[:name] == :booking_id } + name = schema[:attributes].find { |a| a[:name] == :name } + id = schema[:attributes].find { |a| a[:name] == :id } + + expect(booking[:operators]).to eq(%w[eq not_eq gt gteq lt lteq]) + expect(name[:operators]).to eq(%w[eq not_eq in matches does_not_match]) + # A non-filterable attribute advertises an empty operator set. + expect(id[:operators]).to eq([]) + end + + it "passes through a nil note for a resource without one" do + expect(described_class.call(resource)).to include(note: nil) + end + + context "with a resource note" do + before do + serializer = widget_serializer + model = widget_model + rel = relation + McpToolkit.configure do |c| + c.registry.register(:noted_widgets) do + model model + serializer serializer + description "Noted widgets." + note "Internal debugging resource; do not interpret without domain knowledge." + scope { |_root| rel } + end + end + end + + it "passes the resource note through to the schema" do + schema = described_class.call(McpToolkit.registry.fetch("noted_widgets")) + + expect(schema[:note]).to eq("Internal debugging resource; do not interpret without domain knowledge.") + end + end + # The DX gap this closes: a `scheduled_notifications.notification` link is # discoverably the `notifications` resource (callable via list/get) rather than # a name the caller has to guess. From 6b9b2cfa1650805609c14343ec9ab799d90254ca Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 17:22:23 +0200 Subject: [PATCH 06/10] Add config.generic_tool_name_prefix for the Registry-backed authority tools --- CHANGELOG.md | 5 +++ README.md | 14 +++++++ .../authority/registry_tool_provider.rb | 39 ++++++++++++++++--- lib/mcp_toolkit/configuration.rb | 15 +++++++ .../authority/registry_tool_provider_spec.rb | 33 ++++++++++++++++ 5 files changed, 101 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 816a906..88f6ccb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,11 @@ several providers (e.g. the RegistryToolProvider + a host's bespoke tools) behind one `config.tool_provider`: `tool_definitions` concatenates in order, `find` returns the first match. + - `config.generic_tool_name_prefix` (default `""`) — namespaces the four generic + Registry-backed tools. When set (e.g. `"foo_"`) the provider advertises and + resolves them as `foo_resources` / `foo_resource_schema` / `foo_get` / + `foo_list`, letting a host keep stable, namespaced tool names for existing + clients; the empty default keeps the bare base names. - **`McpToolkit::Resource` generic seams** (all api-agnostic) — `superusers_only!` / `superusers_only?` (authority tools honor it), `note(text)` + reader (surfaced by `resource_schema`), and `filter(name, type:, description:, &applier)` + diff --git a/README.md b/README.md index 28b54e8..a88b943 100644 --- a/README.md +++ b/README.md @@ -263,6 +263,7 @@ discovery tool, a custom serializer may also expose `declared_attributes` / | `protocol_version` | `nil` (negotiate) | pin an MCP protocol version (satellite/upstream client) | | `supported_protocol_versions` | `Protocol::SUPPORTED_VERSIONS` | version set the authority dispatcher negotiates | | `tool_provider` | `nil` | authority: the host's api-agnostic tool catalog (see below) | +| `generic_tool_name_prefix` | `""` | authority: prefix namespacing the four generic Registry-backed tools (e.g. `"foo_"` → `foo_resources` …) | | `rate_limiter` / `usage_recorder` / `usage_flusher` | `nil` | authority transport billing hooks (config callables) | | `parent_controller` | `"ActionController::Base"` | superclass of the engine's controllers, read lazily (set to `"ActionController::API"` for the authority, or `"ApplicationController"` for `helper_method` compat) | | `account_meta_key` | `"mcp-toolkit/account-id"` | `_meta` key a superuser uses to pin the account | @@ -473,6 +474,19 @@ enforces the resource's `required_permissions_scope`, and requires a resolved account for `get` / `list`. `resource_schema` advertises each attribute's filter `operators` and the resource `note`. +By default the four tools advertise their bare names (`resources`, +`resource_schema`, `get`, `list`). To **namespace** them — e.g. to keep a stable, +host-specific name for existing clients, or to run several MCP surfaces without +name collisions — set a prefix: + +```ruby +c.generic_tool_name_prefix = "foo_" # advertised + resolved as foo_resources, + # foo_resource_schema, foo_get, foo_list +``` + +The prefix applies only to these four generic tools; a composed bespoke provider's +own tool names are unaffected. + To serve the generic tools **and** your own bespoke tools behind one provider, compose them: diff --git a/lib/mcp_toolkit/authority/registry_tool_provider.rb b/lib/mcp_toolkit/authority/registry_tool_provider.rb index 9b543a7..94da4bb 100644 --- a/lib/mcp_toolkit/authority/registry_tool_provider.rb +++ b/lib/mcp_toolkit/authority/registry_tool_provider.rb @@ -15,8 +15,11 @@ # scope are enforced inside each tool at call time), so `tool_definitions` ignores # the context. Compose this with bespoke host tools via CompositeToolProvider. class McpToolkit::Authority::RegistryToolProvider - # Tool name (as advertised in `tools/list` and matched in `tools/call`) => the - # tool class. The four generic read tools; nothing here names an app concept. + # BASE tool name (the api-agnostic identity of each generic tool) => the tool + # class. The four generic read tools; nothing here names an app concept. The name + # actually advertised in `tools/list` and matched in `tools/call` is this base + # name PREFIXED with `config.generic_tool_name_prefix` (empty by default, so the + # bare base name), letting a host namespace its generic tools. TOOLS = { "resources" => McpToolkit::Authority::Tools::Resources, "resource_schema" => McpToolkit::Authority::Tools::ResourceSchema, @@ -28,14 +31,40 @@ def initialize(config:) @config = config end - # The four static generic tool definitions (context-independent). + # The four static generic tool definitions (context-independent), each advertised + # under its PREFIXED name so `tools/list` shows the host's namespaced names. def tool_definitions(_context) - TOOLS.each_value.map(&:definition) + TOOLS.map { |base_name, klass| klass.definition.merge(name: prefixed(base_name)) } end # A tool instance bound to this provider's config, or nil for an unknown name. + # The incoming name is matched against the PREFIXED names: the prefix is stripped + # to recover the base tool, so a name that does not carry the configured prefix + # (e.g. a sibling provider's unprefixed tool) is left for another provider. def find(name) - klass = TOOLS[name.to_s] + base_name = base_name_for(name.to_s) + klass = base_name && TOOLS[base_name] klass&.new(config: @config) end + + private + + # The host's generic tool-name prefix (empty by default). + def prefix + @config.generic_tool_name_prefix.to_s + end + + # The advertised name for a base tool: the configured prefix followed by the base. + def prefixed(base_name) + "#{prefix}#{base_name}" + end + + # Recovers the base tool name from an advertised name by stripping the configured + # prefix, or nil when the name does not carry the (non-empty) prefix. + def base_name_for(name) + return name if prefix.empty? + return nil unless name.start_with?(prefix) + + name.delete_prefix(prefix) + end end diff --git a/lib/mcp_toolkit/configuration.rb b/lib/mcp_toolkit/configuration.rb index dd59bc3..be4f0d7 100644 --- a/lib/mcp_toolkit/configuration.rb +++ b/lib/mcp_toolkit/configuration.rb @@ -237,6 +237,20 @@ class McpToolkit::Configuration # @return [#tool_definitions, #find, nil] attr_accessor :tool_provider + # --- generic tool naming --------------------------------------------------- + + # A prefix prepended to the four GENERIC, Registry-backed authority tool names + # (`resources`, `resource_schema`, `get`, `list`) served by + # McpToolkit::Authority::RegistryToolProvider. Lets a host NAMESPACE its generic + # tools — e.g. set `"foo_"` and they advertise (and resolve) as `foo_resources`, + # `foo_resource_schema`, `foo_get`, `foo_list` — so distinct MCP surfaces don't + # collide and existing clients keep a stable, host-chosen name. Empty by default, + # so the tools keep their bare base names. The prefix value is the host's; the gem + # names no app concept. + # + # @return [String] + attr_accessor :generic_tool_name_prefix + # --- diagnostics ----------------------------------------------------------- # Optional logger for gateway/session diagnostics. All call sites guard with @@ -278,6 +292,7 @@ def initialize @account_id_header = "X-MCP-Account-ID" initialize_authority_hook_defaults + @generic_tool_name_prefix = "" @upstream_timeout = 10 @upstream_list_ttl = 900 # 15 minutes diff --git a/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb b/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb index 24218de..123d9b1 100644 --- a/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb +++ b/spec/mcp_toolkit/authority/registry_tool_provider_spec.rb @@ -255,6 +255,39 @@ def self.name = "WidgetSerializer" end end + describe "config.generic_tool_name_prefix" do + context "when unset (the default empty prefix)" do + it "advertises the four tools under their bare base names" do + names = provider.tool_definitions(context).map { |definition| definition[:name] } + + expect(names).to contain_exactly("resources", "resource_schema", "get", "list") + end + + it "resolves the bare base names" do + expect(provider.find("list")).to be_a(McpToolkit::Authority::Tools::List) + end + end + + context "when set to a non-empty prefix" do + before { McpToolkit.config.generic_tool_name_prefix = "foo_" } + + it "advertises the four tools under their prefixed names" do + names = provider.tool_definitions(context).map { |definition| definition[:name] } + + expect(names).to contain_exactly("foo_resources", "foo_resource_schema", "foo_get", "foo_list") + end + + it "resolves a prefixed name to its tool instance" do + expect(provider.find("foo_list")).to be_a(McpToolkit::Authority::Tools::List) + expect(provider.find("foo_resource_schema")).to be_a(McpToolkit::Authority::Tools::ResourceSchema) + end + + it "no longer resolves the bare base name (only the prefixed name is a tool)" do + expect(provider.find("list")).to be_nil + end + end + end + describe "per-resource scope gating" do before do s = widget_serializer From a920c5867ea7e8166d72ca4938d43c588d43d5e3 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 18:43:03 +0200 Subject: [PATCH 07/10] Resource#filterable accepts a lazy callable (resolved on first read) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A host that derives its filterable map from the DB (Model.column_names) must not run that at registration time — registration typically happens in an initializer's to_prepare, before the database exists (e.g. CI db:create), which aborts boot. filterable now accepts a callable resolved once on first read (a tool call), memoized; a plain Hash still merges eagerly (backward compatible). Co-Authored-By: Claude Opus 4.8 (1M context) --- lib/mcp_toolkit/resource.rb | 44 ++++++++++++++++--- .../resource_lazy_filterable_spec.rb | 40 +++++++++++++++++ 2 files changed, 77 insertions(+), 7 deletions(-) create mode 100644 spec/mcp_toolkit/resource_lazy_filterable_spec.rb diff --git a/lib/mcp_toolkit/resource.rb b/lib/mcp_toolkit/resource.rb index fbf2584..1fdbeb9 100644 --- a/lib/mcp_toolkit/resource.rb +++ b/lib/mcp_toolkit/resource.rb @@ -37,6 +37,7 @@ def initialize(name) @note = nil @superusers_only = false @filterable = {} + @filterable_source = nil @custom_filters = {} @required_permissions_scope = nil end @@ -146,24 +147,35 @@ def effective_required_permissions_scope(default = nil) # # Unmapped/unknown keys are rejected by the list executor, never silently # dropped, so a typo surfaces as actionable feedback. - def filterable(mapping = nil) - return @filterable if mapping.nil? - - mapping.each do |request_key, column| - @filterable[request_key.to_sym] = column.to_sym + # Accepts a Hash (merged now) OR a callable returning a Hash (resolved LAZILY on + # first read). The lazy form lets a host derive the map from something that must + # NOT be touched at registration/boot time — e.g. a DB-backed column list + # (`Model.column_names`): registration typically runs inside an initializer's + # `to_prepare`, before the database may exist (e.g. CI's `db:create`), so hitting + # the DB there aborts boot. A callable source is invoked at most once — on the + # first read (a tool call, when the DB is present) — then memoized. + def filterable(mapping = nil, &block) + source = block || mapping + return filterable_columns if source.nil? + + if source.respond_to?(:call) + @filterable_source = source + else + merge_filterable!(source) end - @filterable + self end # Request-facing filter keys (symbols, sorted) this resource can be filtered # by. Surfaced via the `resource_schema` tool. def filterable_keys - @filterable.keys.sort + filterable_columns.keys.sort end # Request-facing filter key (symbol) => backing column (symbol). Consumed by # the list executor to build the WHERE clause. def filterable_columns + resolve_filterable_source! @filterable end @@ -192,4 +204,22 @@ def association_descriptors serializer.declared_associations end + + private + + # Resolves a lazily-provided filterable source (a callable) exactly once — on the + # first `filterable_columns` / `filterable_keys` read — then drops it, so later + # reads are pure Hash access. This is what keeps a DB-derived map (e.g. + # `Model.column_names`) out of registration/boot time. + def resolve_filterable_source! + return unless @filterable_source + + source = @filterable_source + @filterable_source = nil + merge_filterable!(source.call || {}) + end + + def merge_filterable!(mapping) + mapping.each { |request_key, column| @filterable[request_key.to_sym] = column.to_sym } + end end diff --git a/spec/mcp_toolkit/resource_lazy_filterable_spec.rb b/spec/mcp_toolkit/resource_lazy_filterable_spec.rb new file mode 100644 index 0000000..ac6befe --- /dev/null +++ b/spec/mcp_toolkit/resource_lazy_filterable_spec.rb @@ -0,0 +1,40 @@ +# frozen_string_literal: true + +RSpec.describe McpToolkit::Resource do + describe "#filterable with a lazy (callable) source" do + it "does NOT invoke the callable at registration; resolves it once on first read, then memoizes" do + calls = 0 + resource = described_class.new("widgets") + + resource.filterable(lambda { + calls += 1 + { color: :color, size: :size_cm } + }) + + # Registration-time: the callable must not run (it may touch the DB, which is + # unavailable while an initializer's to_prepare runs — e.g. CI's db:create). + expect(calls).to eq(0) + + expect(resource.filterable_columns).to eq(color: :color, size: :size_cm) + expect(resource.filterable_keys).to eq(%i[color size]) + + # Resolved exactly once, then cached. + expect(calls).to eq(1) + end + + it "still accepts a plain Hash (merged eagerly, no source)" do + resource = described_class.new("widgets") + resource.filterable(color: :color) + + expect(resource.filterable_columns).to eq(color: :color) + expect(resource.filterable_keys).to eq(%i[color]) + end + + it "tolerates a callable returning nil (treated as an empty map)" do + resource = described_class.new("widgets") + resource.filterable(-> {}) + + expect(resource.filterable_columns).to eq({}) + end + end +end From 563367d55ff2c7b026275bb6a2820a8edf3c0d13 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Mon, 6 Jul 2026 19:05:20 +0200 Subject: [PATCH 08/10] Add gem-native rate limiting + optional superuser resolver - McpToolkit::RateLimiter: fixed-window per-principal counter over config.cache_store - config.rate_limit_max_requests (nil=off) / rate_limit_window (3600) - Authority::ControllerMethods#mcp_rate_limit! built-in: X-RateLimit-* headers, -32029 JSON-RPC 429 + Retry-After over limit; mcp_rate_limit_max_requests + mcp_rate_limit_key hooks; config.rate_limiter kept as escape hatch - config.superuser_resolver + Authority::Context#superuser? (resolver else duck-type) - specs: RateLimiter unit, Context#superuser?, built-in rate-limit controller-methods - CHANGELOG [0.4.0] + README; version stays 0.4.0 Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 20 +++++ README.md | 19 +++++ lib/mcp_toolkit/authority/context.rb | 20 +++-- .../authority/controller_methods.rb | 68 +++++++++++++++-- lib/mcp_toolkit/configuration.rb | 47 +++++++++++- lib/mcp_toolkit/rate_limiter.rb | 73 +++++++++++++++++++ spec/mcp_toolkit/authority/context_spec.rb | 28 +++++++ .../authority/controller_methods_spec.rb | 58 +++++++++++++++ spec/mcp_toolkit/rate_limiter_spec.rb | 61 ++++++++++++++++ 9 files changed, 380 insertions(+), 14 deletions(-) create mode 100644 lib/mcp_toolkit/rate_limiter.rb create mode 100644 spec/mcp_toolkit/authority/context_spec.rb create mode 100644 spec/mcp_toolkit/rate_limiter_spec.rb diff --git a/CHANGELOG.md b/CHANGELOG.md index 88f6ccb..93b09b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,26 @@ dispatcher negotiates. - **`config.rate_limiter` / `usage_recorder` / `usage_flusher`** — the authority transport's billing hooks as config callables (all default `nil` / no-op). +- **Built-in rate limiting** — `McpToolkit::RateLimiter`, a fixed-window + per-principal counter backed by `config.cache_store`, plus + `config.rate_limit_max_requests` (Integer, default `nil` = OFF) and + `config.rate_limit_window` (seconds, default `3600`). When a cap is set, the + authority transport's `mcp_rate_limit!` counts each request, sets the + `X-RateLimit-Limit` / `X-RateLimit-Remaining` / `X-RateLimit-Reset` headers on + every capped response, and over the limit renders a JSON-RPC error (code + `-32029`) at HTTP `429` with a `Retry-After` header. Two new overridable hooks — + `mcp_rate_limit_max_requests` (default `config.rate_limit_max_requests`) and + `mcp_rate_limit_key` (default `mcp_principal.id`) — let a host keep the cap in + its own constant/model or bucket the counter differently. `config.rate_limiter` + remains as an escape hatch that fully replaces the built-in when set. A pure + host that sets no cap is unaffected. +- **`config.superuser_resolver`** — an optional `->(principal) -> Boolean` making + superuser a first-class, OPTIONAL gem concept. `Authority::Context#superuser?` + calls it when set, else falls back to duck-typing `principal.superuser?` (false + when the principal doesn't respond to it). Together with the existing + `superusers_only!` resource flag and the authority tools' gating, this + formalizes superuser gating; the default (no resolver, no superuser-aware + principal) is "no superusers". - **Lazy `parent_controller` (Constraint B)** — the engine's `ServerController` / `TokensController` and the authority `ServerController` are no longer eager- loadable files; they are built from the CURRENT config by diff --git a/README.md b/README.md index a88b943..0d7c82e 100644 --- a/README.md +++ b/README.md @@ -265,6 +265,9 @@ discovery tool, a custom serializer may also expose `declared_attributes` / | `tool_provider` | `nil` | authority: the host's api-agnostic tool catalog (see below) | | `generic_tool_name_prefix` | `""` | authority: prefix namespacing the four generic Registry-backed tools (e.g. `"foo_"` → `foo_resources` …) | | `rate_limiter` / `usage_recorder` / `usage_flusher` | `nil` | authority transport billing hooks (config callables) | +| `rate_limit_max_requests` | `nil` (off) | authority: per-principal request cap for the built-in `RateLimiter`; `nil` disables rate limiting | +| `rate_limit_window` | `3600` | authority: fixed rate-limit window (s); ignored while `rate_limit_max_requests` is `nil` | +| `superuser_resolver` | `nil` | optional `->(principal) -> Boolean` for `Context#superuser?`; `nil` = duck-type `principal.superuser?` | | `parent_controller` | `"ActionController::Base"` | superclass of the engine's controllers, read lazily (set to `"ActionController::API"` for the authority, or `"ApplicationController"` for `helper_method` compat) | | `account_meta_key` | `"mcp-toolkit/account-id"` | `_meta` key a superuser uses to pin the account | | `account_id_header` | `"X-MCP-Account-ID"` | header fallback for the account selector | @@ -527,6 +530,22 @@ Every billing/tenancy step is an overridable hook: `mcp_authenticate!`, per-request loop (`resolve account → track usage → dispatch`) is preserved across batches, so usage metering survives a mixed-account batch. +**Rate limiting is built in.** Set `config.rate_limit_max_requests` (and, +optionally, `config.rate_limit_window`, default 1 hour) and the default +`mcp_rate_limit!` throttles each principal via `McpToolkit::RateLimiter` against +`config.cache_store` — no subclass needed. It sets the `X-RateLimit-*` headers on +every capped response and, over the limit, renders a JSON-RPC `-32029` error at +`429` with `Retry-After`. A host that keeps its cap in a constant/model overrides +the small `mcp_rate_limit_max_requests` hook (default `config.rate_limit_max_requests`); +`mcp_rate_limit_key` (default `mcp_principal.id`) buckets the counter. Leaving the +cap `nil` disables throttling entirely; `config.rate_limiter` stays as an escape +hatch that replaces the built-in wholesale. + +**Superuser is an optional, first-class concept.** Set +`config.superuser_resolver = ->(principal) { ... }` and `Context#superuser?` uses +it to gate `superusers_only!` resources; with no resolver it duck-types +`principal.superuser?`, and with neither, no caller is ever a superuser. + Point your `POST /mcp` route at the subclass (or mount the engine for a pure host); keep `POST /mcp/tokens/introspect` on the gem's `TokensController`. diff --git a/lib/mcp_toolkit/authority/context.rb b/lib/mcp_toolkit/authority/context.rb index 5408f7f..3f4f50f 100644 --- a/lib/mcp_toolkit/authority/context.rb +++ b/lib/mcp_toolkit/authority/context.rb @@ -16,10 +16,14 @@ # `#default_account` / `#authorize_account(id)`. # * `bearer_token` — the raw bearer, forwarded to upstream MCP servers so they # introspect the same token and resolve the same account. -# * `superuser?` — derived: true only when the principal responds to -# `#superuser?` truthily. Lets a host tool base -# (McpToolkit::Tools::AuthorityBase) gate superuser-only -# resources without the gem naming any app concept. +# * `superuser?` — derived. When `config.superuser_resolver` is set, it is the +# truth of `resolver.call(principal)`; otherwise the context +# duck-types `principal.superuser?` (false when the principal +# doesn't respond to it). Lets a host tool base +# (McpToolkit::Tools::AuthorityBase) gate `superusers_only!` +# resources without the gem naming any app concept. Superuser +# is fully OPTIONAL — with no resolver and a principal that +# isn't superuser-aware, it is always false. class McpToolkit::Authority::Context attr_reader :account, :principal, :bearer_token @@ -29,7 +33,13 @@ def initialize(account:, principal:, bearer_token: nil) @bearer_token = bearer_token end + # Superuser-ness of the caller. A configured `superuser_resolver` is the + # first-class hook; absent one, we fall back to duck-typing `principal.superuser?` + # so a host that just defines that method on its token still works. def superuser? - principal.respond_to?(:superuser?) && !!principal.superuser? + resolver = McpToolkit.config.superuser_resolver + return !!resolver.call(principal) if resolver + + principal.respond_to?(:superuser?) ? !!principal.superuser? : false end end diff --git a/lib/mcp_toolkit/authority/controller_methods.rb b/lib/mcp_toolkit/authority/controller_methods.rb index bf92b1f..d523c38 100644 --- a/lib/mcp_toolkit/authority/controller_methods.rb +++ b/lib/mcp_toolkit/authority/controller_methods.rb @@ -31,7 +31,9 @@ # mcp_config -> the McpToolkit::Configuration (McpToolkit.config) # mcp_authenticate! -> set @mcp_principal or render 401 (local token auth via # config.token_authenticator, through Auth::Authority) -# mcp_rate_limit! -> throttle (config.rate_limiter&.call) +# mcp_rate_limit! -> throttle (built-in McpToolkit::RateLimiter when +# config.rate_limit_max_requests is set; config.rate_limiter +# escape hatch takes precedence; no-op when neither) # mcp_track_usage -> record one usage event (config.usage_recorder&.call) # mcp_flush_usage -> persist accumulated usage (config.usage_flusher&.call) # mcp_resolve_account -> the account for one call (principal#default_account / @@ -113,12 +115,66 @@ def mcp_authenticate! mcp_render_unauthorized("Invalid or expired token") unless @mcp_principal end - # Throttle the request. The default delegates to `config.rate_limiter` (a - # `->(controller:, principal:)` that renders + halts when over the limit, or - # sets headers when under). A host whose limiter touches app models overrides - # this method wholesale. + # Throttle the request. Precedence: + # 1. `config.rate_limiter` escape hatch, if set (host-owned counting); + # 2. otherwise the built-in McpToolkit::RateLimiter when a cap is configured + # (via the `mcp_rate_limit_max_requests` hook, default + # `config.rate_limit_max_requests`); + # 3. otherwise a no-op (no cap => pure host unaffected). + # On every capped request it sets the X-RateLimit-* headers; over the limit it + # additionally sets Retry-After and renders the JSON-RPC error + 429 (halting + # the filter chain). def mcp_rate_limit! - mcp_config.rate_limiter&.call(controller: self, principal: mcp_principal) + return mcp_config.rate_limiter.call(controller: self, principal: mcp_principal) if mcp_config.rate_limiter + + max = mcp_rate_limit_max_requests + return if max.nil? + + result = McpToolkit::RateLimiter.new( + key: mcp_rate_limit_key, + max_requests: max, + window: mcp_config.rate_limit_window, + cache_store: mcp_config.cache_store + ).call + + mcp_set_rate_limit_headers(result) + mcp_render_rate_limited(result) unless result.allowed? + end + + # The per-window request cap the built-in limiter enforces, or nil to disable + # it. Default: `config.rate_limit_max_requests`. A host that keeps its cap in a + # constant/model overrides this (e.g. `= MyController::RATE_LIMIT`). + def mcp_rate_limit_max_requests + mcp_config.rate_limit_max_requests + end + + # The identity the built-in limiter counts against. Default: the principal id. + # Override to bucket differently (e.g. per account, or a composite key). + def mcp_rate_limit_key + mcp_principal.id + end + + # Sets the X-RateLimit-* headers from a RateLimiter result (on every capped + # response, allowed or not). + def mcp_set_rate_limit_headers(result) + response.headers["X-RateLimit-Limit"] = result.limit.to_s + response.headers["X-RateLimit-Reset"] = result.reset_at.to_s + response.headers["X-RateLimit-Remaining"] = result.remaining.to_s + end + + # Renders the over-limit response: the Retry-After header plus a JSON-RPC error + # envelope (code -32029) at HTTP 429. Called as a before_action, so the render + # halts the request. + def mcp_render_rate_limited(result) + response.headers["Retry-After"] = result.retry_after.to_s + render json: { + jsonrpc: McpToolkit::Protocol::JSONRPC_VERSION, + id: nil, + error: { + code: -32_029, + message: "Rate limit exceeded. Retry after #{result.retry_after}s." + } + }, status: :too_many_requests end # Record one usage event for a single JSON-RPC call. The default delegates to diff --git a/lib/mcp_toolkit/configuration.rb b/lib/mcp_toolkit/configuration.rb index be4f0d7..1e8ca4f 100644 --- a/lib/mcp_toolkit/configuration.rb +++ b/lib/mcp_toolkit/configuration.rb @@ -126,6 +126,39 @@ class McpToolkit::Configuration # @return [Integer] session sliding-TTL in seconds. attr_accessor :session_ttl + # --- rate limiting --------------------------------------------------------- + + # The built-in per-principal request cap enforced by the authority transport + # (McpToolkit::Authority::ControllerMethods#mcp_rate_limit!), counted against + # `cache_store` via McpToolkit::RateLimiter. nil (the default) DISABLES rate + # limiting entirely, so a pure host is unaffected until it opts in. Set an + # Integer to cap each principal to that many requests per `rate_limit_window`. + # The default `mcp_rate_limit!` reads this through the overridable + # `mcp_rate_limit_max_requests` hook, so a host that keeps the cap in its own + # constant/model overrides that hook rather than this value. + # + # @return [Integer, nil] + attr_accessor :rate_limit_max_requests + + # The fixed rate-limit window, in seconds (default 3600 = 1 hour). Ignored + # while `rate_limit_max_requests` is nil. + # + # @return [Integer] + attr_accessor :rate_limit_window + + # --- superuser (optional, first-class) ------------------------------------- + + # Optional resolver deciding whether a principal is a SUPERUSER — a cross-tenant + # caller that may reach `superusers_only!` resources. `->(principal) -> Boolean`. + # When set, McpToolkit::Authority::Context#superuser? calls it; when nil (the + # default) the context falls back to duck-typing `principal.superuser?` (false + # when the principal doesn't respond to it). Superuser is FULLY OPTIONAL: a host + # with no such concept leaves this nil and flags no `superusers_only!` resource, + # so no caller is ever a superuser. + # + # @return [#call, nil] + attr_accessor :superuser_resolver + # --- filtering ------------------------------------------------------------- # Escapes LIKE wildcards in `matches` / `does_not_match` filter values so they @@ -203,8 +236,12 @@ class McpToolkit::Configuration # hook METHOD on its McpToolkit::Authority::ServerController subclass instead; # then these stay nil. All default to nil (a no-op). - # Throttles a request. `->(controller:, principal:)`; renders + halts when over - # the limit (or sets rate-limit headers when under). nil = no throttling. + # OPTIONAL escape hatch that FULLY REPLACES the built-in limiter: a + # `->(controller:, principal:)` that renders + halts when over the limit (or + # sets rate-limit headers when under). When set, `mcp_rate_limit!` delegates to + # it and the built-in (`rate_limit_max_requests`) is skipped. nil (the default) + # means the built-in runs instead. Most hosts want the built-in; reach for this + # only when the counting itself must live in app code. # # @return [#call, nil] attr_accessor :rate_limiter @@ -303,12 +340,16 @@ def initialize end # The authority transport's injection points all default to nil (a no-op): a - # pure satellite/gateway never touches them. + # pure satellite/gateway never touches them. `rate_limit_window` is the sole + # non-nil default (the window size only matters once a cap opts in). def initialize_authority_hook_defaults @rate_limiter = nil @usage_recorder = nil @usage_flusher = nil @tool_provider = nil + @rate_limit_max_requests = nil # nil = rate limiting disabled + @rate_limit_window = 3600 # 1 hour + @superuser_resolver = nil # nil = duck-type principal.superuser? end # Config sugar: register a gateway upstream. Delegates to `upstreams.register`, diff --git a/lib/mcp_toolkit/rate_limiter.rb b/lib/mcp_toolkit/rate_limiter.rb new file mode 100644 index 0000000..acc7c95 --- /dev/null +++ b/lib/mcp_toolkit/rate_limiter.rb @@ -0,0 +1,73 @@ +# frozen_string_literal: true + +# A fixed-window request counter backing the authority transport's built-in rate +# limiting (McpToolkit::Authority::ControllerMethods#mcp_rate_limit!). It is +# storage-agnostic: it counts against the injected `cache_store` (any +# ActiveSupport::Cache::Store — a shared Rails.cache in production, a MemoryStore +# in a unit test), so a host enables per-principal throttling by setting +# `config.rate_limit_max_requests` alone, without hand-rolling a limiter. +# +# The window is FIXED, not sliding: every request whose time falls in the same +# `window`-second bucket shares one counter, keyed by that bucket's start +# (`window_start`); the entry expires after `window` seconds. The counter is +# incremented once per call, and the request is allowed while the running count is +# `<= max_requests`, blocked once it exceeds it (so exactly `max_requests` +# requests pass per window). +# +# result = McpToolkit::RateLimiter.new( +# key: principal.id, max_requests: 1_000, window: 3_600, cache_store: Rails.cache +# ).call +# result.allowed? # => false once the count exceeds max_requests +# result.limit # => 1_000 +# result.remaining # => max_requests - count, floored at 0 +# result.reset_at # => epoch seconds of the next window boundary +# result.retry_after # => seconds until reset_at (0 when already past) +# +# The cache key is namespaced (`mcp_toolkit:rate_limit::`) so a +# host's own cache entries never collide with the counter. +class McpToolkit::RateLimiter + # The outcome of one #call: the throttling decision plus the values the + # transport renders into the `X-RateLimit-*` / `Retry-After` headers. + Result = Struct.new(:allowed, :limit, :remaining, :reset_at, :retry_after, keyword_init: true) do + def allowed? + allowed + end + end + + def initialize(key:, max_requests:, cache_store:, window: 3_600, now: Time.now) + @key = key + @max_requests = max_requests + @cache_store = cache_store + @window = window + @now = now.to_i + end + + # Increments this window's counter and returns the Result. Called once per + # request by the transport's rate-limit hook. + def call + count = @cache_store.increment(cache_key, 1, expires_in: @window) || 1 + allowed = count <= @max_requests + + Result.new( + allowed:, + limit: @max_requests, + remaining: allowed ? @max_requests - count : 0, + reset_at:, + retry_after: [reset_at - @now, 0].max + ) + end + + private + + def window_start + @now - (@now % @window) + end + + def reset_at + window_start + @window + end + + def cache_key + "mcp_toolkit:rate_limit:#{@key}:#{window_start}" + end +end diff --git a/spec/mcp_toolkit/authority/context_spec.rb b/spec/mcp_toolkit/authority/context_spec.rb new file mode 100644 index 0000000..fc1867d --- /dev/null +++ b/spec/mcp_toolkit/authority/context_spec.rb @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe McpToolkit::Authority::Context do + let(:account) { FakeAccount.new(1) } + + describe "#superuser?" do + it "uses config.superuser_resolver when one is set" do + McpToolkit.config.superuser_resolver = ->(principal) { principal == :the_boss } + + expect(described_class.new(account:, principal: :the_boss).superuser?).to be(true) + expect(described_class.new(account:, principal: :nobody).superuser?).to be(false) + end + + it "falls back to duck-typing principal.superuser? when no resolver is set" do + expect(described_class.new(account:, principal: FakePrincipal.new(superuser: true)).superuser?).to be(true) + expect(described_class.new(account:, principal: FakePrincipal.new(superuser: false)).superuser?).to be(false) + end + + it "is false when neither a resolver nor a superuser-aware principal is present" do + not_superuser_aware = Object.new + + expect(described_class.new(account:, principal: not_superuser_aware).superuser?).to be(false) + expect(described_class.new(account:, principal: nil).superuser?).to be(false) + end + end +end diff --git a/spec/mcp_toolkit/authority/controller_methods_spec.rb b/spec/mcp_toolkit/authority/controller_methods_spec.rb index 2507ce0..4f787a1 100644 --- a/spec/mcp_toolkit/authority/controller_methods_spec.rb +++ b/spec/mcp_toolkit/authority/controller_methods_spec.rb @@ -114,6 +114,64 @@ def rpc(method, params = {}, id: 1) end end + # ---- built-in rate limiting (config.rate_limit_max_requests) ------------- + + describe "#mcp_rate_limit! (built-in McpToolkit::RateLimiter)" do + before do + authenticate_as(principal) + McpToolkit.config.rate_limit_max_requests = 2 + # reset_config! gives a fresh MemoryStore per example, so counts are isolated. + end + + it "sets the X-RateLimit-* headers on an allowed request and renders nothing" do + controller.send(:mcp_rate_limit!) + + headers = controller.response.headers + expect(headers["X-RateLimit-Limit"]).to eq("2") + expect(headers["X-RateLimit-Remaining"]).to eq("1") + expect(headers["X-RateLimit-Reset"]).to be_present + expect(controller.rendered).to be_nil + end + + it "renders the JSON-RPC -32029 error at 429 with Retry-After once the limit is exceeded" do + 3.times { controller.send(:mcp_rate_limit!) } # keyed on principal.id, same window + + rendered = controller.rendered + expect(rendered[:status]).to eq(:too_many_requests) + expect(rendered[:json][:error][:code]).to eq(-32_029) + expect(rendered[:json][:error][:message]).to include("Rate limit exceeded") + expect(controller.response.headers["Retry-After"]).to be_present + expect(controller.response.headers["X-RateLimit-Remaining"]).to eq("0") + end + + it "keys the counter via the overridable mcp_rate_limit_key (default principal.id)" do + expect(controller.send(:mcp_rate_limit_key)).to eq(principal.id) + end + + it "reads the cap via the overridable mcp_rate_limit_max_requests hook" do + expect(controller.send(:mcp_rate_limit_max_requests)).to eq(2) + end + + it "is a no-op when no cap and no rate_limiter are configured" do + McpToolkit.config.rate_limit_max_requests = nil + + controller.send(:mcp_rate_limit!) + + expect(controller.rendered).to be_nil + expect(controller.response.headers["X-RateLimit-Limit"]).to be_nil + end + + it "lets the config.rate_limiter escape hatch take precedence over the built-in" do + seen = nil + McpToolkit.config.rate_limiter = ->(controller:, principal:) { seen = { controller:, principal: } } + + controller.send(:mcp_rate_limit!) + + expect(seen).to eq(controller:, principal:) + expect(controller.response.headers["X-RateLimit-Limit"]).to be_nil + end + end + describe "#mcp_resolve_account (duck-typed on the principal)" do before { authenticate_as(principal) } diff --git a/spec/mcp_toolkit/rate_limiter_spec.rb b/spec/mcp_toolkit/rate_limiter_spec.rb new file mode 100644 index 0000000..9a1c43a --- /dev/null +++ b/spec/mcp_toolkit/rate_limiter_spec.rb @@ -0,0 +1,61 @@ +# frozen_string_literal: true + +require "spec_helper" + +# Drives the fixed-window counter against a REAL ActiveSupport::Cache::MemoryStore +# (the same contract a Rails.cache satisfies), pinning `now` to exercise window +# rollover deterministically. +RSpec.describe McpToolkit::RateLimiter do + let(:cache_store) { ActiveSupport::Cache::MemoryStore.new } + + def limiter(key: "p1", max_requests: 3, window: 3600, now: Time.now) + described_class.new(key:, max_requests:, window:, cache_store:, now:) + end + + describe "#call" do + it "allows requests up to the limit, reporting the decreasing remaining count" do + results = Array.new(3) { limiter.call } + + expect(results.map(&:allowed?)).to eq([true, true, true]) + expect(results.map(&:remaining)).to eq([2, 1, 0]) + expect(results.map(&:limit)).to eq([3, 3, 3]) + end + + it "blocks once the count exceeds the limit, with remaining floored at 0" do + 3.times { limiter.call } + + result = limiter.call + + expect(result).not_to be_allowed + expect(result.remaining).to eq(0) + end + + it "counts per fixed window: a fresh window resets the counter" do + base = Time.now.to_i + window_start = base - (base % 3600) + 3.times { limiter(now: window_start + 10).call } + + # Same window: still over the limit. + expect(limiter(now: window_start + 20).call).not_to be_allowed + # Next window: a brand-new counter allows again. + expect(limiter(now: window_start + 3600).call).to be_allowed + end + + it "counts each key independently" do + 3.times { limiter(key: "a").call } + + expect(limiter(key: "a").call).not_to be_allowed + expect(limiter(key: "b").call).to be_allowed + end + + it "reports reset_at at the next window boundary and retry_after until then" do + base = Time.now.to_i + window_start = base - (base % 3600) + + result = limiter(now: base).call + + expect(result.reset_at).to eq(window_start + 3600) + expect(result.retry_after).to eq(window_start + 3600 - base) + end + end +end From 97004d25aa784c086647f47e8474f93787621624 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Tue, 7 Jul 2026 16:21:36 +0200 Subject: [PATCH 09/10] Extract usage metering, token machinery, session-data + registration hooks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Further authority-side extractions so a host drives billing/tenancy from config instead of subclassing/customizing the controller or maintaining a parallel registration system: - UsageMetering::Recorder — generic accumulate-per-call / scrub / bulk-flush metering + error isolation, driven by an injected event-builder + sink; a host wires it to config.usage_recorder / config.usage_flusher. - Authority::Token — ownership-agnostic token concern (secure generation, SHA256 digest auth, OAuth-style scope helpers, expiry + throttled last-used) to include into a host's ActiveRecord token model. - config.session_data_builder + mcp_session_data delegates to it, so a host binds the session to its token id without overriding the controller. - Registry#resource_extension / #resource_finalizer + Resource#extra — hosts declare their own registration "extras" (e.g. dependencies) via a hook and derive gem-native fields in a finalizer, instead of a separate registration system. Gem suite 371 examples, 0 failures; rubocop clean. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01V1zLtLLaFLKErBdmh18feY --- .../authority/controller_methods.rb | 19 ++- lib/mcp_toolkit/authority/token.rb | 124 ++++++++++++++++ lib/mcp_toolkit/configuration.rb | 10 ++ lib/mcp_toolkit/registry.rb | 32 ++++- lib/mcp_toolkit/resource.rb | 22 +++ lib/mcp_toolkit/usage_metering/recorder.rb | 95 +++++++++++++ .../authority/controller_methods_spec.rb | 14 ++ spec/mcp_toolkit/authority/token_spec.rb | 98 +++++++++++++ spec/mcp_toolkit/registry_extension_spec.rb | 89 ++++++++++++ .../usage_metering/recorder_spec.rb | 132 ++++++++++++++++++ 10 files changed, 626 insertions(+), 9 deletions(-) create mode 100644 lib/mcp_toolkit/authority/token.rb create mode 100644 lib/mcp_toolkit/usage_metering/recorder.rb create mode 100644 spec/mcp_toolkit/authority/token_spec.rb create mode 100644 spec/mcp_toolkit/registry_extension_spec.rb create mode 100644 spec/mcp_toolkit/usage_metering/recorder_spec.rb diff --git a/lib/mcp_toolkit/authority/controller_methods.rb b/lib/mcp_toolkit/authority/controller_methods.rb index d523c38..adf3b3e 100644 --- a/lib/mcp_toolkit/authority/controller_methods.rb +++ b/lib/mcp_toolkit/authority/controller_methods.rb @@ -38,9 +38,9 @@ # mcp_flush_usage -> persist accumulated usage (config.usage_flusher&.call) # mcp_resolve_account -> the account for one call (principal#default_account / # principal#authorize_account(id)) -# mcp_session_data -> opaque payload bound to the session ({}; a host binds -# e.g. { token_id: principal.id } so a revoked token kills -# the session) +# mcp_session_data -> opaque payload bound to the session (config.session_data_builder, +# else {}; a host binds e.g. { token_id: principal.id } so a +# revoked token kills the session) # mcp_dispatch -> run one JSON-RPC call (Dispatcher + Authority::Context) # mcp_health_payload -> the GET /mcp/health body # @@ -205,11 +205,16 @@ def mcp_resolve_account(request_data) raise McpToolkit::Protocol::InvalidParams, "account_id #{candidate.inspect} is not authorized for this token" end - # Opaque payload bound to the session on `initialize`. Default: none. A host - # binds e.g. `{ token_id: mcp_principal.id }` so a revoked token can kill an - # in-flight session. + # Opaque payload bound to the session on `initialize`. Default: driven by + # `config.session_data_builder` (a host binds e.g. `{ token_id: principal.id }` + # so a revoked token can kill an in-flight session), or an empty payload when no + # builder is set. A host whose session payload needs controller state overrides + # this method instead. def mcp_session_data - {} + builder = mcp_config.session_data_builder + return {} if builder.nil? + + builder.call(principal: mcp_principal) || {} end # Run one JSON-RPC call through the hand-rolled dispatcher with a fresh diff --git a/lib/mcp_toolkit/authority/token.rb b/lib/mcp_toolkit/authority/token.rb new file mode 100644 index 0000000..a98a900 --- /dev/null +++ b/lib/mcp_toolkit/authority/token.rb @@ -0,0 +1,124 @@ +# frozen_string_literal: true + +require "active_support/core_ext/integer/time" + +# Generic, ownership-agnostic machinery for an AUTHORITY's access-token model, +# packaged as a concern to `include` into a host's ActiveRecord token model. It +# extracts everything about a token that is NOT specific to the host's tenancy +# model (how a token maps to accounts/users), leaving the host to declare only its +# ownership associations, account resolution, and any bespoke validations. +# +# Expected columns on the including model: +# token_digest :string NOT NULL, unique — SHA256(plaintext) for O(1) lookup +# token_prefix :string NOT NULL — first 11 plaintext chars, safe to display +# scopes :string[] / json — OAuth-style "__" grants (nullable) +# expires_at :datetime nullable — nil = never expires +# last_used_at :datetime nullable — throttled touch (see #touch_last_used!) +# +# What it provides: +# * Secure token generation on create (`assign_token`) + the plaintext reader +# (`#token`, populated only on the instance that generated it). +# * Lookup/verification: `.authenticate(plaintext)` / `.digest_for(plaintext)`. +# * Lifecycle scopes (`active` / `expired` / `never_used` / `used_within`) and +# `#expired?`, plus the throttled `#touch_last_used!`. +# * OAuth-style scope helpers (`#normalized_scopes` / `#authorized_for_scope?` / +# `#scope_restricted?`). +# +# Why a plain SHA256 (not bcrypt/scrypt/argon2): the token is a high-entropy random +# secret (24 bytes over a 64-char alphabet ≈ 144 bits), so rainbow tables can't +# exist and brute force is infeasible — the slow KDFs that protect low-entropy +# passwords buy nothing and would break the O(1) `find_by(token_digest:)` lookup. +module McpToolkit::Authority::Token + extend ActiveSupport::Concern + + # Plaintext token layout. "mcp_" is the MCP-generic scheme prefix (not a host + # fingerprint); the display length is that prefix plus 7 random chars. + TOKEN_PREFIX = "mcp_" + RAW_TOKEN_BYTES = 24 + TOKEN_PREFIX_DISPLAY_LENGTH = 11 + + # Skip the last_used_at UPDATE unless this much time has passed, so a burst of + # calls doesn't write on every request. + LAST_USED_AT_THROTTLE = 1.minute + + included do + # The plaintext token: only populated on the in-memory instance that just + # generated it; nil on any reloaded/queried instance (we never store plaintext). + attr_reader :token + + # Defense-in-depth presence checks (the columns are NOT NULL at the DB level); + # the host adds its own uniqueness enforcement suited to its stack. + validates :token_digest, presence: true + validates :token_prefix, presence: true + + before_validation :assign_token, on: :create + + scope :active, -> { where("expires_at IS NULL OR expires_at > ?", Time.current) } + scope :expired, -> { where("expires_at IS NOT NULL AND expires_at <= ?", Time.current) } + scope :never_used, -> { where(last_used_at: nil) } + scope :used_within, ->(duration) { where(last_used_at: duration.ago..) } + end + + module ClassMethods + # Looks up an active token by its plaintext value; nil for blank/unknown/expired. + def authenticate(plaintext) + return nil if plaintext.blank? + + active.find_by(token_digest: digest_for(plaintext)) + end + + def digest_for(plaintext) + Digest::SHA256.hexdigest(plaintext) + end + end + + def reload(...) + @token = nil + super + end + + def expired? + expires_at.present? && expires_at <= Time.current + end + + # The OAuth-style scopes granted to this token as a clean array of "__" + # strings. An unrestricted token (NULL/empty `scopes`) returns []. + def normalized_scopes + Array(scopes).compact_blank + end + + # Per-scope check. A tool requiring no scope is reachable by any token; a tool + # requiring a scope needs the token to HOLD that exact scope. An unrestricted + # token holds NO scopes, so it can reach only no-scope tools. + def authorized_for_scope?(scope) + return true if scope.blank? + + normalized_scopes.include?(scope.to_s) + end + + # True when the token carries an explicit scope set (i.e. is restricted). + def scope_restricted? + normalized_scopes.any? + end + + # Throttled last_used_at bump: persists on its own, without validations or + # bumping updated_at, and only once per LAST_USED_AT_THROTTLE window. + def touch_last_used! + return unless last_used_at.nil? || last_used_at < LAST_USED_AT_THROTTLE.ago + + self.last_used_at = Time.current + save!(validate: false, touch: false) + end + + private + + def assign_token + return if token_digest.present? + + @token = "#{TOKEN_PREFIX}#{SecureRandom.urlsafe_base64(RAW_TOKEN_BYTES)}" + self.token_digest = self.class.digest_for(@token) + # Plain-Ruby slice (not String#first) so the concern needs no ActiveSupport + # string core-ext; the token is always longer than the display length. + self.token_prefix = @token[0, TOKEN_PREFIX_DISPLAY_LENGTH] + end +end diff --git a/lib/mcp_toolkit/configuration.rb b/lib/mcp_toolkit/configuration.rb index 1e8ca4f..c59922e 100644 --- a/lib/mcp_toolkit/configuration.rb +++ b/lib/mcp_toolkit/configuration.rb @@ -259,6 +259,15 @@ class McpToolkit::Configuration # @return [#call, nil] attr_accessor :usage_flusher + # Builds the opaque payload bound to a session on `initialize`. `->(principal:)` + # returning a Hash (or nil for none). Lets a host bind e.g. + # `{ token_id: principal.id }` so a revoked token can kill an in-flight session, + # WITHOUT overriding the controller's `mcp_session_data`. nil (the default) => + # an empty session payload. + # + # @return [#call, nil] + attr_accessor :session_data_builder + # The host's tool catalog — the api-agnostic seam. Duck-typed; the dispatcher # calls: # @@ -346,6 +355,7 @@ def initialize_authority_hook_defaults @rate_limiter = nil @usage_recorder = nil @usage_flusher = nil + @session_data_builder = nil @tool_provider = nil @rate_limit_max_requests = nil # nil = rate limiting disabled @rate_limit_window = 3600 # 1 hour diff --git a/lib/mcp_toolkit/registry.rb b/lib/mcp_toolkit/registry.rb index 3a5c301..a270b93 100644 --- a/lib/mcp_toolkit/registry.rb +++ b/lib/mcp_toolkit/registry.rb @@ -13,8 +13,33 @@ class UnknownResource < StandardError; end def initialize @resources = {} @default_required_permissions_scope = nil + @resource_extension = nil + @resource_finalizer = nil end + # A Module MIXED INTO every Resource before its registration block runs, so a host + # can add its OWN declaration DSL (its "extras") on top of the gem's built-in + # `model` / `scope` / `serializer` / `filterable` / `superusers_only!` / `note` / + # `filter`. The host method typically stores into the generic `Resource#extra` + # bag; the `resource_finalizer` reads it back. nil (the default) mixes in nothing, + # so a host with no extras is unaffected. Set ONCE in `configure` (not per reload), + # since `reset!` preserves it. + # + # McpToolkit.registry.resource_extension = MyApp::ResourceExtension # adds `dependencies` + # + # @return [Module, nil] + attr_accessor :resource_extension + + # A callable run against each Resource AFTER its registration block, so a host can + # derive gem-native fields from its declared extras — e.g. build a `serializer` + # from the `model` + declared `dependencies`, or a lazy `filterable`. `->(resource)`. + # nil (the default) is a no-op. This is the hook that lets a host avoid a parallel + # registration system: it declares resources DIRECTLY against the gem registry and + # fills the derived pieces here. Set ONCE in `configure`; preserved across `reset!`. + # + # @return [#call, nil] + attr_accessor :resource_finalizer + # Registry-wide DEFAULT required scope, so a satellite declares its scope ONCE # for every resource instead of repeating it per resource: # @@ -32,7 +57,9 @@ def default_required_permissions_scope(scope = nil) def register(name, &) resource = McpToolkit::Resource.new(name) + resource.extend(@resource_extension) if @resource_extension resource.instance_eval(&) + @resource_finalizer&.call(resource) @resources[name.to_s] = resource end @@ -59,8 +86,9 @@ def resource_names end # Clears registered resources for a dev reload (the satellite re-declares them - # in `to_prepare`). The `default_required_permissions_scope` is PRESERVED, since - # it's declared once in `configure` rather than per-reload. + # in `to_prepare`). The `default_required_permissions_scope`, `resource_extension` + # and `resource_finalizer` are PRESERVED, since they're declared once in + # `configure` rather than per-reload. def reset! @resources = {} end diff --git a/lib/mcp_toolkit/resource.rb b/lib/mcp_toolkit/resource.rb index 1fdbeb9..6e9480e 100644 --- a/lib/mcp_toolkit/resource.rb +++ b/lib/mcp_toolkit/resource.rb @@ -26,6 +26,10 @@ class NotConfigured < StandardError; end # resource_schema so a client can discover the filter. CustomFilter = Struct.new(:name, :type, :description, :applier, keyword_init: true) + # Sentinel distinguishing `extra(:key)` (read) from `extra(:key, nil)` (write nil). + UNSET = Object.new + private_constant :UNSET + attr_reader :name def initialize(name) @@ -40,8 +44,26 @@ def initialize(name) @filterable_source = nil @custom_filters = {} @required_permissions_scope = nil + @extras = {} end + # A generic, api-agnostic metadata bag for host-defined "extras" — declarations + # the gem does not model itself (e.g. an app's ORM dependency list used to build a + # serializer). It is the storage behind `config.registry.resource_extension` (the + # host DSL that writes extras inside a registration block) and + # `config.registry.resource_finalizer` (which reads them back to derive gem-native + # fields such as `serializer` / `filterable`). Write with `extra(:key, value)`; + # read with `extra(:key)` (nil when unset). The gem never inspects the values. + def extra(key, value = UNSET) + return @extras[key] if value.equal?(UNSET) + + @extras[key] = value + end + + # The full host-extras bag (symbol/whatever key => value). Read-only view for a + # resource_finalizer that wants to iterate every declared extra. + attr_reader :extras + def model(klass = nil) @model = klass if klass @model diff --git a/lib/mcp_toolkit/usage_metering/recorder.rb b/lib/mcp_toolkit/usage_metering/recorder.rb new file mode 100644 index 0000000..12e3a77 --- /dev/null +++ b/lib/mcp_toolkit/usage_metering/recorder.rb @@ -0,0 +1,95 @@ +# frozen_string_literal: true + +# Generic, api-agnostic usage metering for the AUTHORITY transport. +# +# Wired to the authority controller's billing hooks purely through config, so a +# host meters MCP traffic WITHOUT subclassing or customizing the controller: +# +# meter = McpToolkit::UsageMetering::Recorder.new( +# event_builder: ->(request_data:, params:, arguments:, scrubbed_arguments:, account:, principal:) { {...} }, +# sink: ->(events) { MyLedger.insert_all(events) }, +# parameter_filter: ActiveSupport::ParameterFilter.new(Rails.application.config.filter_parameters), +# logger: Rails.logger, +# error_reporter: ->(e) { Sentry.capture_exception(e) } +# ) +# config.usage_recorder = meter.method(:record) # per JSON-RPC call +# config.usage_flusher = meter.method(:flush) # after the response +# +# One event is accumulated per BILLABLE JSON-RPC call (default: `tools/call`) and +# all of a request's events are flushed together after the response. Because the +# authority transport re-resolves the account per batch element and calls `record` +# per element, a mixed-account batch meters each call against its own account. +# +# Two invariants: +# * Metering NEVER affects the MCP response — every error here is logged (via +# `logger`) and reported (via `error_reporter`), then swallowed. +# * The raw arguments are scrubbed through `parameter_filter` BEFORE they reach +# the event, so a filtered key (e.g. a token) never leaves this object. +# +# The `event_builder` returns ONE ledger row's attributes (a Hash) for a call, or +# nil to skip it; the gem stays app-agnostic by never naming the row's columns. The +# `sink` persists the accumulated array in one shot. +# +# Per-request state (the accumulation buffer) lives on the Rack request env, so the +# Recorder itself is stateless and safe to share across requests/threads. +class McpToolkit::UsageMetering::Recorder + DEFAULT_BILLABLE_METHODS = %w[tools/call].freeze + BUFFER_ENV_KEY = "mcp_toolkit.usage_events" + + def initialize(event_builder:, sink:, parameter_filter: nil, + billable_methods: DEFAULT_BILLABLE_METHODS, logger: nil, error_reporter: nil) + @event_builder = event_builder + @sink = sink + @parameter_filter = parameter_filter + @billable_methods = billable_methods + @logger = logger + @error_reporter = error_reporter + end + + # `config.usage_recorder` target. Accumulates one event for a billable call onto + # the current request's buffer. Non-billable methods (ping, initialize, + # tools/list, ...) are ignored; a nil event from the builder is skipped. + def record(request_data:, account:, principal:, controller:) + return unless request_data.is_a?(Hash) + return unless @billable_methods.include?(request_data["method"]) + + params = request_data["params"].to_h + arguments = params["arguments"].to_h + event = @event_builder.call( + request_data:, params:, arguments:, + scrubbed_arguments: scrub(arguments), account:, principal: + ) + buffer_for(controller) << event unless event.nil? + rescue StandardError => e + report("failed to accumulate event", e) + end + + # `config.usage_flusher` target. Persists the request's accumulated events via the + # sink in one shot. No-op when nothing was accumulated. + def flush(controller:) + events = buffer_for(controller) + return if events.empty? + + @sink.call(events) + rescue StandardError => e + report("failed to flush #{events&.size} event(s)", e) + end + + private + + def scrub(arguments) + hash = arguments.to_h + @parameter_filter ? @parameter_filter.filter(hash) : hash + end + + # Per-request accumulation buffer, stored on the Rack env so the Recorder holds + # no per-request state of its own (thread-safe to share across requests). + def buffer_for(controller) + controller.request.env[BUFFER_ENV_KEY] ||= [] + end + + def report(message, error) + @logger&.warn("MCP usage tracking: #{message}: #{error.message}") + @error_reporter&.call(error) + end +end diff --git a/spec/mcp_toolkit/authority/controller_methods_spec.rb b/spec/mcp_toolkit/authority/controller_methods_spec.rb index 4f787a1..f76f775 100644 --- a/spec/mcp_toolkit/authority/controller_methods_spec.rb +++ b/spec/mcp_toolkit/authority/controller_methods_spec.rb @@ -77,6 +77,20 @@ def rpc(method, params = {}, id: 1) expect(controller.send(:mcp_session_data)).to eq({}) end + it "mcp_session_data delegates to config.session_data_builder with the principal" do + authenticate_as(principal) + McpToolkit.config.session_data_builder = ->(principal:) { { token_id: principal.id } } + + expect(controller.send(:mcp_session_data)).to eq(token_id: 55) + end + + it "mcp_session_data falls back to {} when the builder returns nil" do + authenticate_as(principal) + McpToolkit.config.session_data_builder = ->(**) {} + + expect(controller.send(:mcp_session_data)).to eq({}) + end + it "mcp_rate_limit! delegates to config.rate_limiter with controller + principal" do authenticate_as(principal) seen = nil diff --git a/spec/mcp_toolkit/authority/token_spec.rb b/spec/mcp_toolkit/authority/token_spec.rb new file mode 100644 index 0000000..db6c16f --- /dev/null +++ b/spec/mcp_toolkit/authority/token_spec.rb @@ -0,0 +1,98 @@ +# frozen_string_literal: true + +require "spec_helper" + +# The gem suite runs WITHOUT ActiveRecord, so this drives the concern's api-agnostic +# contract against a minimal host class that provides only the AR class-macro seams +# its `included do` block touches (validates / before_validation / scope as no-ops) +# plus plain attribute accessors. The AR-integrated behavior (`.authenticate`, the +# lifecycle scopes, `#touch_last_used!` persistence) is covered end-to-end by the +# host app's model spec. +RSpec.describe McpToolkit::Authority::Token do + let(:token_class) do + Class.new do + def self.validates(*); end + def self.before_validation(*); end + def self.scope(*); end + + include McpToolkit::Authority::Token + + attr_accessor :token_digest, :token_prefix, :scopes, :expires_at, :last_used_at + + def initialize(**attrs) + attrs.each { |key, value| public_send("#{key}=", value) } + end + end + end + + describe "constants" do + it "exposes the generic plaintext layout" do + expect(described_class::TOKEN_PREFIX).to eq("mcp_") + expect(described_class::RAW_TOKEN_BYTES).to eq(24) + expect(described_class::TOKEN_PREFIX_DISPLAY_LENGTH).to eq(11) + end + end + + describe ".digest_for" do + it "is a plain SHA256 of the plaintext" do + expect(token_class.digest_for("mcp_known")).to eq(Digest::SHA256.hexdigest("mcp_known")) + end + end + + describe "#assign_token (generation on create)" do + it "generates a prefixed plaintext, its digest, and the display prefix" do + token = token_class.new + token.send(:assign_token) + + expect(token.token).to start_with("mcp_") + expect(token.token_digest).to eq(token_class.digest_for(token.token)) + expect(token.token_prefix).to eq(token.token[0, 11]) + expect(token.token_prefix.length).to eq(described_class::TOKEN_PREFIX_DISPLAY_LENGTH) + end + + it "is a no-op when a digest is already present (never regenerates)" do + token = token_class.new(token_digest: "existing") + token.send(:assign_token) + + expect(token.token).to be_nil + expect(token.token_digest).to eq("existing") + end + end + + describe "#normalized_scopes" do + it "returns [] for a NULL/empty scope column" do + expect(token_class.new(scopes: nil).normalized_scopes).to eq([]) + expect(token_class.new(scopes: []).normalized_scopes).to eq([]) + end + + it "drops blank entries" do + expect(token_class.new(scopes: ["notifications__read", "", " "]).normalized_scopes) + .to eq(["notifications__read"]) + end + end + + describe "#authorized_for_scope?" do + it "lets any token reach a no-scope tool" do + expect(token_class.new(scopes: nil).authorized_for_scope?(nil)).to be(true) + expect(token_class.new(scopes: ["x"]).authorized_for_scope?("")).to be(true) + end + + it "requires the token to hold the exact scope for a scoped tool" do + token = token_class.new(scopes: ["notifications__read"]) + expect(token.authorized_for_scope?("notifications__read")).to be(true) + expect(token.authorized_for_scope?("owners__read")).to be(false) + end + + it "an unrestricted token holds no scopes, so it can reach only no-scope tools" do + unrestricted = token_class.new(scopes: nil) + expect(unrestricted.authorized_for_scope?("notifications__read")).to be(false) + end + end + + describe "#scope_restricted?" do + it "is false for an unrestricted token and true once a scope is set" do + expect(token_class.new(scopes: nil).scope_restricted?).to be(false) + expect(token_class.new(scopes: ["notifications__read"]).scope_restricted?).to be(true) + end + end +end diff --git a/spec/mcp_toolkit/registry_extension_spec.rb b/spec/mcp_toolkit/registry_extension_spec.rb new file mode 100644 index 0000000..eb54bae --- /dev/null +++ b/spec/mcp_toolkit/registry_extension_spec.rb @@ -0,0 +1,89 @@ +# frozen_string_literal: true + +require "spec_helper" + +# The host-extras seam: a `resource_extension` module adds a host DSL to each +# Resource, and a `resource_finalizer` derives gem-native fields from the extras +# after the block — so a host declares resources DIRECTLY against the gem registry +# instead of maintaining a parallel registration system. +RSpec.describe McpToolkit::Registry do + subject(:registry) { described_class.new } + + # A host DSL word that stores into the generic Resource#extra bag. + let(:extension) do + Module.new do + def dependencies(*models) + return Array(extra(:dependencies)) if models.empty? + + extra(:dependencies, models.flatten) + end + end + end + + describe "Resource#extra" do + let(:resource) { McpToolkit::Resource.new(:things) } + + it "reads nil for an unset key and round-trips a written value (including nil)" do + expect(resource.extra(:missing)).to be_nil + + resource.extra(:deps, %i[a b]) + expect(resource.extra(:deps)).to eq(%i[a b]) + + resource.extra(:explicit_nil, nil) + expect(resource.extras).to include(explicit_nil: nil) + end + end + + describe "#register with a resource_extension" do + it "makes the extension DSL callable inside the registration block" do + registry.resource_extension = extension + + registry.register(:bookings) do + model Hash + dependencies :rental, :client + end + + expect(registry.find("bookings").extra(:dependencies)).to eq(%i[rental client]) + end + end + + describe "#register with a resource_finalizer" do + it "runs the finalizer against the resource after its block, deriving gem-native fields" do + registry.resource_extension = extension + finalized = [] + registry.resource_finalizer = lambda do |resource| + finalized << resource.name + resource.description("auto: #{resource.extra(:dependencies).inspect}") + end + + registry.register(:bookings) do + model Hash + dependencies :rental + end + + expect(finalized).to eq(["bookings"]) + expect(registry.find("bookings").description).to eq("auto: [:rental]") + end + + it "leaves registration unchanged when neither hook is set" do + registry.register(:plain) { model Hash } + + expect(registry.find("plain").model).to eq(Hash) + end + end + + describe "#reset!" do + it "preserves resource_extension + resource_finalizer (declared once in configure)" do + registry.resource_extension = extension + finalizer = ->(_resource) {} + registry.resource_finalizer = finalizer + registry.register(:bookings) { model Hash } + + registry.reset! + + expect(registry.resources).to be_empty + expect(registry.resource_extension).to be(extension) + expect(registry.resource_finalizer).to be(finalizer) + end + end +end diff --git a/spec/mcp_toolkit/usage_metering/recorder_spec.rb b/spec/mcp_toolkit/usage_metering/recorder_spec.rb new file mode 100644 index 0000000..61adc83 --- /dev/null +++ b/spec/mcp_toolkit/usage_metering/recorder_spec.rb @@ -0,0 +1,132 @@ +# frozen_string_literal: true + +require "spec_helper" +require "active_support/parameter_filter" + +RSpec.describe McpToolkit::UsageMetering::Recorder do + # A minimal stand-in for the authority controller: the Recorder only touches + # `controller.request.env` (its per-request accumulation buffer). + let(:controller) do + env = {} + request = Struct.new(:env).new(env) + Struct.new(:request).new(request) + end + + # Echoes the pieces the transport hands the builder, so each example can assert + # exactly what was captured (and that scrubbing already happened upstream). + let(:event_builder) do + lambda do |request_data:, params:, arguments:, scrubbed_arguments:, account:, principal:| + { + method: request_data["method"], + tool: params["name"], + arguments: scrubbed_arguments, + account_id: account&.id, + principal_id: principal&.id + } + end + end + + let(:flushed) { [] } + let(:sink) { ->(events) { flushed.replace(events) } } + let(:principal) { Struct.new(:id).new(7) } + let(:account) { Struct.new(:id).new(42) } + + subject(:recorder) { described_class.new(event_builder:, sink:) } + + def tools_call(name: "api_v3_list", arguments: { "resource" => "bookings" }) + { "jsonrpc" => "2.0", "id" => 1, "method" => "tools/call", "params" => { "name" => name, "arguments" => arguments } } + end + + describe "#record" do + it "accumulates one event per billable call onto the request buffer" do + recorder.record(request_data: tools_call, account:, principal:, controller:) + + buffer = controller.request.env[described_class::BUFFER_ENV_KEY] + expect(buffer.size).to eq(1) + expect(buffer.first).to include(method: "tools/call", tool: "api_v3_list", account_id: 42, principal_id: 7) + end + + it "ignores non-billable methods (ping, initialize, tools/list)" do + %w[ping initialize tools/list].each do |method| + recorder.record(request_data: { "method" => method }, account:, principal:, controller:) + end + + expect(controller.request.env[described_class::BUFFER_ENV_KEY]).to be_nil + end + + it "skips a nil event returned by the builder" do + recorder = described_class.new(event_builder: ->(**) {}, sink:) + + recorder.record(request_data: tools_call, account:, principal:, controller:) + + expect(controller.request.env[described_class::BUFFER_ENV_KEY] || []).to be_empty + end + + it "accumulates one event per call across a batch (mixed accounts kept distinct)" do + account_b = Struct.new(:id).new(99) + recorder.record(request_data: tools_call(arguments: { "resource" => "bookings" }), account:, principal:, controller:) + recorder.record(request_data: tools_call(arguments: { "resource" => "rentals" }), account: account_b, principal:, + controller:) + + expect(controller.request.env[described_class::BUFFER_ENV_KEY].map { |e| e[:account_id] }).to eq([42, 99]) + end + + context "with a parameter_filter" do + subject(:recorder) { described_class.new(event_builder:, sink:, parameter_filter:) } + + let(:parameter_filter) { ActiveSupport::ParameterFilter.new([:token]) } + + it "scrubs the arguments before they reach the builder" do + recorder.record(request_data: tools_call(arguments: { "resource" => "bookings", "token" => "mcp_secret" }), + account:, principal:, controller:) + + expect(controller.request.env[described_class::BUFFER_ENV_KEY].first[:arguments]) + .to eq("resource" => "bookings", "token" => "[FILTERED]") + end + end + + it "swallows + reports a builder error without touching the buffer" do + reported = [] + logger = instance_double(Logger, warn: nil) + recorder = described_class.new( + event_builder: ->(**) { raise "boom" }, sink:, logger:, error_reporter: ->(e) { reported << e } + ) + + expect { recorder.record(request_data: tools_call, account:, principal:, controller:) }.not_to raise_error + expect(reported.map(&:message)).to eq(["boom"]) + expect(logger).to have_received(:warn).with(/MCP usage tracking/) + end + end + + describe "#flush" do + it "persists the accumulated events via the sink in one call" do + recorder.record(request_data: tools_call, account:, principal:, controller:) + recorder.flush(controller:) + + expect(flushed.size).to eq(1) + expect(flushed.first).to include(tool: "api_v3_list") + end + + it "is a no-op when nothing was accumulated" do + called = false + recorder = described_class.new(event_builder:, sink: ->(_) { called = true }) + + recorder.flush(controller:) + + expect(called).to be(false) + end + + it "swallows + reports a sink error so the response is never affected" do + reported = [] + logger = instance_double(Logger, warn: nil) + recorder = described_class.new( + event_builder:, sink: ->(_) { raise "sink down" }, logger:, error_reporter: ->(e) { reported << e } + ) + recorder.record(request_data: tools_call, account:, principal:, controller:) + + expect { recorder.flush(controller:) }.not_to raise_error + expect(reported.map(&:message)).to eq(["sink down"]) + expect(logger).to have_received(:warn).with(/MCP usage tracking/) + end + end +end From f08994c403548a345a81c454dd867fbbc16caf19 Mon Sep 17 00:00:00 2001 From: Karol Galanciak Date: Wed, 8 Jul 2026 10:24:43 +0200 Subject: [PATCH 10/10] Harden authority MCP boundary + advertise server_instructions on initialize Bad input now returns a JSON-RPC error instead of a framework 500: a malformed body maps to parse_error (-32700) via a guarded rescue_from, and a non-object request/batch element maps to invalid_request rather than raising NoMethodError in the per-call loop. Also advertise config.server_instructions on initialize, matching the SDK-backed server. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01GRbXi4JWV5H3Cz1zUXUYvV --- .../authority/controller_methods.rb | 29 +++++++++++++ lib/mcp_toolkit/dispatcher.rb | 8 +++- .../authority/controller_methods_spec.rb | 41 +++++++++++++++++++ spec/mcp_toolkit/dispatcher_spec.rb | 14 +++++++ 4 files changed, 91 insertions(+), 1 deletion(-) diff --git a/lib/mcp_toolkit/authority/controller_methods.rb b/lib/mcp_toolkit/authority/controller_methods.rb index adf3b3e..4b6b3aa 100644 --- a/lib/mcp_toolkit/authority/controller_methods.rb +++ b/lib/mcp_toolkit/authority/controller_methods.rb @@ -55,6 +55,15 @@ module McpToolkit::Authority::ControllerMethods included do protect_from_forgery with: :null_session if respond_to?(:protect_from_forgery) + # A Protocol::Error that escapes the per-call loop is, in practice, a + # parse/shape error raised at the request boundary (a malformed JSON body from + # mcp_parse_body, fired here even from the mcp_resolve_session! before_action). + # Without this it would surface as a framework 500; the MCP spec wants a + # JSON-RPC error envelope for bad input. Guarded behind respond_to? like + # protect_from_forgery: every real host (ActionController::API/::Base) carries + # ActiveSupport::Rescuable, a bare non-Rails host simply skips it. + rescue_from McpToolkit::Protocol::Error, with: :mcp_render_protocol_error if respond_to?(:rescue_from) + before_action :mcp_authenticate!, except: [:health] before_action :mcp_rate_limit!, except: [:health] before_action :mcp_resolve_session!, only: [:create] @@ -256,6 +265,11 @@ def mcp_handle_single(request_data) # account id) becomes this call's JSON-RPC error, leaving sibling batch elements # untouched. def mcp_process_single_request(request_data) + # A request element that isn't a JSON object can't carry an id, a method, or + # params, so downstream `.to_h`/`.key?` would raise a NoMethodError (a 500). + # Treat it as a JSON-RPC InvalidRequest with a null id instead. + return mcp_invalid_request_response unless request_data.is_a?(Hash) + account = mcp_resolve_account(request_data) mcp_track_usage(request_data, account) mcp_dispatch(request_data, account) @@ -265,6 +279,15 @@ def mcp_process_single_request(request_data) McpToolkit::Protocol.error_response(id: request_data["id"], error: e) end + # The response for a non-object request element: no id/method is recoverable, so + # it's an InvalidRequest carrying a null id (JSON-RPC 2.0 §4.2 / batch handling). + def mcp_invalid_request_response + McpToolkit::Protocol.error_response( + id: nil, + error: McpToolkit::Protocol::InvalidRequest.new("Request must be a JSON object") + ) + end + # Renders the JSON-RPC payload as application/json (default) or text/event-stream # when the client's Accept header includes "text/event-stream". We never actually # stream — one message then EOF — but emitting SSE on demand keeps strict MCP @@ -361,6 +384,12 @@ def mcp_render_session_not_found }, status: :not_found end + # Renders a boundary Protocol::Error (parse/shape failure) as a JSON-RPC error + # envelope with a null id at HTTP 400, rather than letting it become a 500. + def mcp_render_protocol_error(error) + render json: McpToolkit::Protocol.error_response(id: nil, error:), status: :bad_request + end + # ---- body parsing --------------------------------------------------- def mcp_parse_body diff --git a/lib/mcp_toolkit/dispatcher.rb b/lib/mcp_toolkit/dispatcher.rb index 39bf4ee..6cabc39 100644 --- a/lib/mcp_toolkit/dispatcher.rb +++ b/lib/mcp_toolkit/dispatcher.rb @@ -112,7 +112,7 @@ def handle_initialize(params) versions = config.supported_protocol_versions negotiated = versions.include?(requested) ? requested : versions.first - { + result = { protocolVersion: negotiated, capabilities: { # listChanged: true — the aggregated list includes upstream tools, which @@ -125,6 +125,12 @@ def handle_initialize(params) version: config.server_version } } + + # `instructions` is advertised on initialize only when configured, matching the + # SDK-backed satellite server (McpToolkit::Server.build) and the documented + # contract. Omitted when nil to keep the envelope clean. + result[:instructions] = config.server_instructions if config.server_instructions + result end def handle_initialized diff --git a/spec/mcp_toolkit/authority/controller_methods_spec.rb b/spec/mcp_toolkit/authority/controller_methods_spec.rb index f76f775..c6fa06a 100644 --- a/spec/mcp_toolkit/authority/controller_methods_spec.rb +++ b/spec/mcp_toolkit/authority/controller_methods_spec.rb @@ -17,6 +17,7 @@ Class.new do def self.before_action(*); end def self.after_action(*); end + def self.rescue_from(*); end include McpToolkit::Authority::ControllerMethods attr_accessor :request_body, :request_headers, :params @@ -347,6 +348,46 @@ def controller.mcp_session_data = { token_id: mcp_principal.id } end end + # ---- malformed / non-object input: a JSON-RPC error, never a framework 500 - + + describe "malformed and non-object requests" do + it "raises a JSON-RPC ParseError (not a bare JSON error) for a malformed body" do + controller.request_body = "{ not json" + + expect { controller.send(:mcp_parse_body) } + .to raise_error(McpToolkit::Protocol::ParseError, /Invalid JSON/) + end + + it "renders an escaped Protocol::Error as a JSON-RPC envelope with a null id at 400" do + controller.send(:mcp_render_protocol_error, McpToolkit::Protocol::ParseError.new) + + rendered = controller.rendered + expect(rendered[:status]).to eq(:bad_request) + expect(rendered[:json][:id]).to be_nil + expect(rendered[:json][:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::PARSE_ERROR) + end + + it "answers a non-object JSON body with InvalidRequest instead of crashing the loop" do + controller.request_body = body_for("not-an-object") + + controller.create + + expect(controller.rendered[:json][:id]).to be_nil + expect(controller.rendered[:json][:error][:code]).to eq(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + end + + it "isolates a non-object batch element to its own InvalidRequest, leaving siblings intact" do + authenticate_as(principal) + controller.request_body = body_for("bad-element", rpc("ping")) + + controller.create + + responses = controller.rendered[:json] + expect(responses.map { |r| r[:error]&.dig(:code) }).to include(McpToolkit::Protocol::ErrorCodes::INVALID_REQUEST) + expect(responses.any? { |r| r[:result] == {} }).to be(true) + end + end + # ---- the per-request account loop across a mixed-account BATCH ------------ describe "per-request account loop (batch with mixed accounts)" do diff --git a/spec/mcp_toolkit/dispatcher_spec.rb b/spec/mcp_toolkit/dispatcher_spec.rb index 3187326..8634eca 100644 --- a/spec/mcp_toolkit/dispatcher_spec.rb +++ b/spec/mcp_toolkit/dispatcher_spec.rb @@ -71,6 +71,20 @@ def request(method, params = {}, id: 1) expect(response[:result][:serverInfo]).to eq(name: "acme-mcp", version: "3.1.4") expect(response[:result][:capabilities][:tools]).to eq(listChanged: true) end + + it "advertises server_instructions when configured, matching the SDK path" do + McpToolkit.config.server_instructions = "Prefer the search tool for open-ended asks." + + response = dispatcher.handle_request(request("initialize")) + + expect(response[:result][:instructions]).to eq("Prefer the search tool for open-ended asks.") + end + + it "omits instructions when none are configured" do + response = dispatcher.handle_request(request("initialize")) + + expect(response[:result]).not_to have_key(:instructions) + end end describe "notifications (no id)" do