Add remote (Streamable HTTP + OAuth) server, search tool, and tests#13
Merged
Conversation
- src/http.ts: a sibling to the stdio entry point that speaks MCP over Streamable HTTP, so hosted clients can connect without launching a local process. It calls the SAME createServer per request, so the tool surface is identical across transports. Dependency-free (raw node http). Runs stateless: access tokens expire hourly, so binding one to a long-lived session would break mid-session; per-request binding always uses the token just presented. - OAuth resource-server role: serves /.well-known/oauth-protected- resource and answers unauthenticated or expired requests with 401 + WWW-Authenticate, which is what makes a client run (or refresh) the flow. A token is validated by one probe to the API, since an access token is just an API key; unreachable API fails closed with 503. - search tool, mirroring the API's /search. - test/: a zero-dependency (node:test) parity test asserting the stdio and HTTP entry points expose an identical tool list. CI now runs it — previously the repo had no test step at all. - railway.toml to deploy the remote server as its own service. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a shared formatInputSchema (zod enum) to the three extraction tools. `search` deliberately does not take one: it hands an agent N pages to read, while the other formats serve pipelines that already know which URL they want. MCP tools return text, so renderContent() gives each format a legible textual projection — an agent reading `[object Object]` is strictly worse off than one reading markdown. Blocks and chunks keep their structure visible (block type, token offsets), which is the reason a caller asked for them. renderMetrics() adds a one-line tokens-saved summary when the API reports it. Build and the tool-parity test are clean. NOTE: this repo's CI stays red for a reason unrelated to this change — `wellmarked@^1.1.1` is unsatisfiable (npm has only 1.1.0) and the lockfile still pins the SDK to file:../JS-TS-SDK. Locally that resolves to the sibling checkout, which is why this builds here. See PR #13. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The remote Streamable-HTTP + OAuth server is a new feature since 1.0.0, so this warrants a minor release. version.ts (reported in the MCP initialize handshake) is hand-synced to package.json — kept in sync here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Product decision: no capability to point WellMarked tooling anywhere but api.wellmarked.io. Both the stdio entrypoint and the remote Streamable-HTTP server now hardcode the API base; the env var is gone from the docs. Also tracks the SDK's removal of its baseUrl option (createServer forwards options straight into new WellMarked()). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirrors the API: search takes the same extraction parameter set as extract/ bulk/crawl. renderSearch now goes through the shared renderContent path so chunks/json/html/links results project to legible text instead of only markdown. Format description notes the Pro+ gate on json/chunks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
retry: server-side re-attempts on target_timeout (fresh connection per attempt, default 0). Shared schema across the three extraction tools; deliberately kept off search — the API's 15s per-result deadline can't absorb a retry, so the parameter doesn't exist there. max_pages narrows the plan's crawl page cap (never widens it). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a hosted/remote transport alongside the existing stdio server, plus the
searchtool — and wires up a test step CI never had.What's here
src/http.ts— a sibling to the stdio entry point speaking MCP over Streamable HTTP, so hosted clients (Claude.ai custom connectors) can connect without launching a local process. It calls the samecreateServerper request, so the tool surface is identical across transports. Dependency-free (raw Nodehttp).initializehandshake and a subsequenttools/listboth work stateless with a real client./.well-known/oauth-protected-resourceand answers unauthenticated/expired requests with401 WWW-Authenticate, which is what makes a client run (or refresh) the flow. A token is validated by one probe to the API; an unreachable API fails closed with 503.searchtool, mirroring the API's/search.test/tool-parity.test.mjs— zero-dependency (node:test) test asserting the stdio and HTTP entry points expose an identical tool list. CI now runs it; previously this repo had no test step at all.railway.tomlto deploy the remote server as its own service.🚨 CI is red for a pre-existing packaging bug — please read before merging
CI here does not just need
search(). Root-caused from the failing run:wellmarked@^1.1.1is unsatisfiable. Only 1.1.0 is published on npm, so the range can never resolve. This has been broken since the dependency was switched fromfile:../JS-TS-SDKto the published range.package-lock.jsonis stale — it still pinsnode_modules/wellmarkedto../JS-TS-SDK(the oldfile:link). In an isolated CI checkout that path doesn't exist, sonpm cisilently skipped ~29 packages (105 installed vs 134 locked), which also dropped@types/node— hence the 57TS2591 Cannot find name 'process'errors on top ofTS2307 Cannot find module 'wellmarked'.Locally everything passes only because that stale symlink resolves to the sibling checkout.
Remediation (at release time, in this order)
search()(e.g.1.2.0).wellmarkedto that version and runnpm installto regenerate the lockfile off the registry (not thefile:link).I deliberately did not pin this to
1.1.0as a stopgap — that version lackssearch(), so it would swap one red build for another while adding lockfile churn.Locally (against the linked SDK): build clean, parity test passes with all 7 tools.
🤖 Generated with Claude Code