Agent RPC MCP: merge the data and management planes, and make the branch production-ready (SHARK-3524, SHARK-3373) - #28
Agent RPC MCP: merge the data and management planes, and make the branch production-ready (SHARK-3524, SHARK-3373)#28mikhailak wants to merge 206 commits into
Conversation
Rebase of PR #6 onto main after the data-plane PR #5 merged. This branch now carries ONLY the management-plane diff; all data-plane files stay as merged in #5 (no data-plane content is reverted or duplicated). Management MCP (control plane; own Streamable HTTP server, src/mgmt-http.ts): - OAuth 2.1 shim (discovery + DCR + /authorize -> UAuth login -> /callback -> /token PKCE-S256 -> RS256 shim JWT) + accounting-gateway client. - ~28 tools: key CRUD + allowlist, usage/billing reads, notifications, payment initiators (Stripe checkout URL only, no autonomous charge), TOTP passthrough. Security fixes from the PR #6 audit (SHARK-3380 / 3381 / 3384): - 3380 (CRITICAL, account takeover): server-side redirect_uri origin allowlist, independent of client-supplied DCR data; /token binds client_id + redirect_uri to the code; S256 code_challenge format check at /authorize. - 3384 (HIGH): trust proxy = hop count (not true); legacy escape hatch requires a constant-time MGMT_LEGACY_TOKEN match AND x-ankr-api-key; RS256 pinned on jwtVerify; shim-JWT TTL treats 0/NaN UAuth expiry as expired (no 30d fallback). - 3381 (per SHARK-3392 decision): gateway is the MFA authority (shim forwards TOTP); HITL confirmToken flow added; `confirm` documented as a UX affordance, not a security boundary. Residual crypto human/agent separation = follow-up. Deploy (mgmt part of SHARK-3385): Dockerfile.mgmt (digest-pinned), deploy/mgmt/* with MGMT_ISSUER + ingress reconciled to mcp.ankr.com; DEPLOY-MGMT.md. Deps: + cors, jose (+ @types/cors). minimatch override -> pnpm audit 0 high. Local gate green: 106 tests, typecheck/lint/prettier/build, audit 0. Still DRAFT: merge is blocked on PlatEng deploying the mgmt image + one live UAuth login test (DEPLOY-MGMT.md). Data-plane deploy hardening is separate (#8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… allowlist Roman's review of PR #6 (SHARK-3373): - Secret hygiene (should-fix): add *.pem to .gitignore + add a .dockerignore. The repo ignored *.key/*.crt but not *.pem, and had no .dockerignore, so the RS256 shim signing key (gateway_rsa_private.pem) could have been committed or baked into a layer. DEPLOY-MGMT.md's claim is now accurate. - TOTP: tighten the schema to exactly 6 digits (the gateway verifies 6, not 6-8). - Alert-suppression: invert the ALERT_FLAGS denylist to a fail-safe BENIGN allowlist, so silencing deposit / withdraw / balance / credit alerts also requires the HITL confirmToken (the denylist missed them); an unknown or new flag now defaults to gated instead of slipping through. - Mgmt ingress: share one TLS secret (mcp-ankr-com-tls) with the data-plane /rpc Ingress; mgmt owns the cert-manager order for the host so cert-manager doesn't race two orders for mcp.ankr.com. NB: SHARK-3381 (a real second factor on payment/destructive writes, not just the confirmToken) is a separate posture decision, tracked on the reopened SHARK-3392 — intentionally NOT resolved in this commit. Gate green: 106 tests, typecheck/lint/prettier/build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the SHARK-3381 boundary call (Mike, 2026-07-17): mgmt_edit_api_key is dual-purpose. Changing the key's blockchain SCOPE is an access-control change and stays human-gated; editing only name/description is cosmetic and is now ungated (executes directly). Added a positive test for the name-only path; updated the "no call without approval" invariant test to use a scope-changing edit. Gate green: 107 tests, typecheck/lint/prettier/build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…/confirm (SHARK-3381)
Unblocks SHARK-3381 option A now that the auth-team answer landed: bind the HITL
confirmation to a STABLE account id instead of the per-session random sub.
- Parse `unique_id` from the UAuth access token (signed &-delimited field string,
not a JWT) and use it as the shim-JWT `sub`; fail closed (400 invalid_grant)
when absent — never sign a token with a random/blank subject (uauth.ts,
oauth-provider.ts tokenHandler).
- GET /confirm/:token now starts a FRESH interactive UAuth login (not the agent
bearer) and stores a PendingApproval; /callback branches on kind. Closes the
agent-self-approval gap: a prompt-injected agent holds no interactive UAuth
credential and cannot complete the login.
- Approval is NOT a side effect of the login (adversarial-review finding): after
login the shim renders a consent page {action, args, account} and requires a
deliberate POST /confirm/approve carrying a one-time consent ticket (the
anti-CSRF capability); a wrong-account human gets a generic error and no action
disclosure (boundSubMatches). approve() still enforces the sub match.
- Add gateway getUserProfile() -> GET /users/profile + mgmt_whoami read tool
(the account whoami Andrey pointed to).
- Tests: +parse/fail-closed, sub==unique_id at /token, consent flow (login shows
consent but does not approve; deliberate POST approves once; different account
rejected+no leak; bogus ticket rejected). Gate green: tsc/eslint/prettier + 113
node:tests. Docs: DEPLOY-MGMT approval flow + headless-hatch fail-closed note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tate flag, legacy refusal (SHARK-3381) Address the three documented SHARK-3381 follow-ups. - Browser-binding: GET /confirm/:token sets a same-site, http-only (Secure on an https issuer) cookie carrying a per-attempt nonce, stored on PendingApproval and PendingConsent; re-checked at /callback and POST /confirm/approve so the browser that completes the approval is the one that started it. A link opened or triggered in another browser context cannot complete approval. - ankrState echo can be made MANDATORY via MGMT_REQUIRE_ANKR_NONCE (default off): when on, /callback rejects a login/approval with no ankrState echo. Off by default because the one-time UAuth `state` is the primary CSRF guard; flip on only after a live prod login confirms UAuth echoes ankrState. - Legacy/headless path (MGMT_LEGACY_TOKEN) has no interactive login, so a human approver can never match the fingerprint sub. The gate now refuses HITL-gated writes UP FRONT with a clear message (no gateway call), instead of minting a token that could never be approved. Threaded via authKind -> approvalSupported. Tests: +browser-binding (no-cookie /callback rejected), +requireAnkrNonce-on rejects missing echo, +legacy refusal (no gateway call); consent-flow tests now replay the cookie. Gate green: tsc/eslint/prettier + 116 node:tests. DEPLOY-MGMT updated (approval flow, MGMT_REQUIRE_ANKR_NONCE env, legacy note). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…SHARK-3373) The one secret is a fresh RS256 key we mint for the shim (not an existing Ankr credential); generate it in-cluster (openssl genpkey PKCS#8) straight into the K8s Secret, keep it fixed, never send it through chat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brings the merged data-plane hardening (#8) onto the mgmt branch and unblocks CI: - Resolve .dockerignore add/add conflict (union: keep *.crt; keep the both-Dockerfiles / RS256-key wording). - Same audit-gate fix as #13 (advisories published post-#8): scoped pnpm overrides for brace-expansion (GHSA-3jxr-9vmj-r5cp, dev) and fast-uri (GHSA-v2hh-gcrm-f6hx + GHSA-4c8g-83qw-93j6, RUNTIME via MCP SDK > ajv). fast-uri pinned within the 3.x line ajv expects (>=3.1.4 <4) to avoid a major bump on a runtime dep; lock reconciled + deduped. Local gate green: pnpm audit 0 high, tsc, eslint, prettier, build, 116 tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rrors (SHARK-3381) Address Roman's PR #6 review (the two items before approval): - MEDIUM: suppressesAlerts() only matched value===false, so a credit_*_threshold {value, reset} change slipped through the confirm-only path an agent self-satisfies. Any threshold reset or value change is now treated as alert-suppressing and gated (HITL confirmToken). - LOW: mgmt_set_blockchain_allowlist sent reportBlockchainErrors to the gateway but hashed only {tool, token, blockchains}, so an approved token could be replayed with the flag flipped. It is now part of the argHash binding. Tests (+7, 116 -> 123): threshold reset/value gating, flag-flipped replay rejection, confirmToken 5-min TTL expiry, consentTicket single-use + TTL, and the approveHandler browser-cookie re-check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mt-mcp-poc # Conflicts: # pnpm-workspace.yaml
…ect_uris in prod (SHARK-3373) Local MCP clients (Claude Code CLI, MCP Inspector) register an ephemeral loopback OAuth callback (http://localhost:<port>), which DCR rejects in prod because loopback was hardcoded to `NODE_ENV !== "production"`. Add an explicit opt-in env so ops can enable it on mcp.ankr.com for hands-on testing without flipping NODE_ENV. Safe re SHARK-3380: loopback is not routable off-host and PKCE binds the code to the real client; isOriginAllowed still EXACT-matches the loopback hostname (look-alikes like localhost.evil.com stay rejected) and every external origin stays restricted. Default off in prod; logs a boot warning when enabled. - src/mgmt-http.ts: env toggle (OR NODE_ENV!=production) + prod warning log; the flag also adds http://localhost to the CORS default (existing behaviour). - test/mgmt-authorize.test.ts: strengthen the loopback isOriginAllowed test with look-alike + external-origin rejection (guards the prod toggle). - DEPLOY-MGMT.md: document MGMT_ALLOW_LOOPBACK_REDIRECT. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion key (SHARK-3373)
Live prod login blocked with `access_denied: "UAuth secret-code exchange
failed"` at GET /callback. Root cause is a mismatch with how prod UAuth
(uauth.ankr.com) actually behaves vs the login contract the shim was built on:
- getOauth2Params returns a CONSTANT state ("default"), not a unique per-request
one, and REFLECTS our ankrState breadcrumb into the provider `state` — so the
value UAuth echoes to /callback is the shim's own session key.
- loginUserByOauth2SecretCode (leg 2) validates `state` against the app's fixed
value and 400s "wrong state" for anything else.
The callback handler forwarded the echoed `state` straight into leg 2, so UAuth
rejected every real login. Verified live 2026-07-24: leg 2 with state="default"
-> "wrong secret code" (state accepted); with the echoed blob -> "wrong state".
Fix: send a fixed login state to leg 2 (UAUTH_LOGIN_STATE, default "default"),
while keeping the /callback session lookup keyed on the echoed value. The
shim's CSRF/one-time guard is unchanged: it is the single-use session-store key
(the reflected ankrState carrying shimNonce), never UAuth's constant state.
- src/mgmt/auth/oauth-provider.ts: AuthDeps.uauthLoginState + resolve default;
leg 2 sends it instead of the echoed state; correct the requireAnkrNonce
comment (constant UAuth state is not the CSRF guard).
- src/mgmt-http.ts: wire UAUTH_LOGIN_STATE.
- deploy/mgmt/deployment.yaml: set UAUTH_LOGIN_STATE=default (visible + tunable).
- DEPLOY-MGMT.md: document the env + the prod-UAuth state behaviour.
- test/mgmt-auth.test.ts: assert leg 2 receives the fixed login state, not the
echoed session key (124 tests pass).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… rejected (SHARK-3373) The first real end-to-end login (2026-07-24) reached /token and failed with `invalid_grant: "UAuth grant already expired"`. Root cause: the ms-only FIX 5 heuristic (`raw > 1e12 ? raw/1000 : raw`) mapped UAuth's real LoginUserByOauth2SecretCodeReply.expires_at (usermanager.proto uint64, delivered as a JSON string) into the PAST, so tokenHandler rejected the just-issued grant. This path was never exercised before because live leg-2 login was itself blocked until the loopback + login-state fixes landed. Replace it with normalizeUauthExpiryToS: classify by magnitude across the plausible units (ns/us/ms/s), accept only a value that lands in a sane FUTURE window, else treat a small value as a relative TTL in seconds, else report unknown (0). A grant the user JUST obtained is never classified as already expired — worst case it is unknown and /token uses the conservative SHIM_TTL_FALLBACK_S (~1h) instead of failing the login. Also log the raw expires_at + chosen basis once at /callback (a timestamp, not a secret) to confirm the real prod unit and guard against regressions. - src/mgmt/auth/oauth-provider.ts: normalizeUauthExpiryToS (exported) + wire into finishClientLoginLeg + diagnostic log. - test/mgmt-uauth-expiry.test.ts: unit tests (epoch s/ms/us/ns, relative TTL, unknown/huge/past -> 0). 133 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ARK-3373)
THE actual login blocker. The first real end-to-end login (2026-07-24) reached
/token and failed with `invalid_grant: "Could not derive a stable account
identity from the UAuth login"` (captured raw via a single manual exchange; the
MCP SDK then retries the same code and surfaces the misleading "Invalid or
expired authorization code" from the second, doomed request).
Root cause: the UAuth access_token is base64(StdEncoding) of the "&"-delimited
field string, per multirpc-common crypto/multiRpcMessageSigning CompileTokenDataV3:
fmt.Sprintf("signature=%s&unique_id=%s&application=%s&provider=%s&expires=%d", ...)
-> base64.StdEncoding.EncodeToString([]byte(that))
An earlier note quoted the inner Sprintf but missed the base64 wrapper, so
parseUAuthAccessToken ran URLSearchParams over the base64 blob, `unique_id` never
resolved, uauthAccountSub returned undefined, and /token failed every login. This
path only ran now because live leg-2 login was blocked until the loopback +
login-state fixes landed.
Fix: base64-decode the token first (decodeUAuthTokenBody), staying backward
compatible with raw / V1 (`address=`) tokens. Also surface the token's OWN
embedded `expires` + unique_id presence in the /callback diagnostic log, so the
next login confirms the fix and reveals the real token lifetime vs the proto
expires_at (the ~60s value seen so far may be the proto field, not the token's).
- src/mgmt/auth/uauth.ts: decodeUAuthTokenBody + corrected ACCESS-TOKEN FORMAT note.
- src/mgmt/auth/oauth-provider.ts: log token.expires + unique_id presence.
- test/mgmt-uauth-token.test.ts: base64 V3, raw backward-compat, V1, JWT-ish.
138 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…res (SHARK-3373) @roman — this REVERSES the SHARK-3384 decision to bound the shim JWT to the UAuth grant's `expires`. Please re-review; rationale below is verified against source, not assumed. Symptom: after login finally worked end-to-end, every MCP session died ~60s later (tools -> "token expired"). The UAuth V3 access token's `expires` is now+~60s (confirmed live: token.expires == proto expires_at == login+60s). SHARK-3384 set the shim JWT lifetime = min(grant_remaining, 30d) = ~60s, so the shim was the ONLY thing enforcing that 60s. Why the 60s `expires` is NOT a real deadline downstream (so decoupling is safe): - uauth-auth-service cryptoService.verifyToken: validates signature + unique_id/application/provider; parses `expires` but NEVER compares it to time.Now(). The token does not self-expire. - multirpc-accounting-gateway uauthService.ValidateAccessToken: our token is V3 (len >= 320) so it goes through getTokenUserInfo -> VerifyToken and uses `RefreshAfter`; there is NO `expires < now` check. That guard exists ONLY in the legacy/MetaMask path (tokens < 320 chars). `CreateAccessToken` sets expires = now_ms + expiresIn (ms). - The console cabinet sustains ~day-long sessions on these same tokens against the same gateway for exactly this reason. Change: shim session TTL is now MGMT_SESSION_TTL_S (default 12h, capped 30d), independent of the grant's `expires`. The held UAuth token is still used as the gateway bearer (option A) and the gateway keeps accepting it. session .uauthExpiresAt / normalizeUauthExpiryToS are retained for the /callback diagnostic log only. If the gateway ever starts enforcing `expires` for V3, the held token would need a refresh mechanism — noted inline. - src/mgmt/auth/oauth-provider.ts: MGMT_SESSION_TTL_S replaces the grant-bound TTL + SHIM_TTL_FALLBACK_S; the "UAuth grant already expired" reject is gone. - test/mgmt-auth.test.ts, test/mgmt-rate-limit.test.ts: rewrite the two grant-binding tests (old FIX 3384-5 / FIX 5) to assert the decoupled TTL and that a short/past `expires` no longer blocks login. 138 tests pass. - DEPLOY-MGMT.md, deploy/mgmt/deployment.yaml: document + set MGMT_SESSION_TTL_S. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…SHARK-3373) CI `Audit (high+)` gate failed on GHSA-mh99-v99m-4gvg (brace-expansion DoS via unbounded expansion, patched >=5.0.8; 5.0.7 now flagged). It is a dev-only transitive dep (eslint / sonarjs / typescript-eslint -> minimatch), not our code or a runtime dep, and affects all branches. Not reached before because it was published after the last install. The prior split overrides left 5.0.7 in via the unbounded `>=1.1.16` target on the 1.x line. Consolidated to a single blanket `brace-expansion: >=5.0.8` — the tree already resolves brace-expansion entirely on the 5.x line and node engine is >=23, so this is safe and deterministic. `pnpm audit --audit-level=high` now clean; tsc / eslint / prettier / build / 138 tests still green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-3373)
mgmt_whoami returned "gateway /users/profile -> HTTP 404" on the live gateway.
The profile route is registered under the /auth secured subrouter
(multirpc-accounting-gateway src/route/router.go: groupSupportedRouter GET
/users/profile, where secureRouter = insecureRouter.PathPrefix("/auth")), so the
real path is /auth/users/profile. Every other tool already prefixes /auth
(/auth/intervalUsage, /auth/jwt/all, ...) — whoami was the lone exception, hence
the 404 (vs the 401s the others return). Fixed the client path + comments.
Note: this only fixes the route; whoami still needs a valid session token like
every other data call (the ~60s UAuth-token / session-key issue is separate,
tracked in SHARK-3373).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ey (SHARK-3373)
Root cause of the ~60s session death, from the gateway/uauth source (not a guess):
- UAuth `loginUserByOauth2SecretCode` returns a ONE-TIME token: app=OneTimeToken,
TTL = OneTimeTokenValidityDuration (~60s), server-bound to the real app
(uauth-auth-service actionsProcessorService: buildAndStoreOneTimeToken).
- The console then swaps it — within that window, once — via
POST /auth/session/ui/new on the accounting-gateway (router.go: INSECURE route,
token in the BODY). CreateNewSessionKey validates the one-time token
(unexpired + unused) and mints a SESSION token whose TTL is clamped up to
AccessTokenValidityMinDuration (long) and stored server-side. That is why the
console sustains ~day-long sessions.
- The gateway ENFORCES expiry for V3 tokens (getTokenUserInfo -> VerifyToken; the
`expires < now` short-circuit is only the legacy/MetaMask path). So holding the
raw one-time token, as the shim did, means every /auth/* call 401s after ~60s —
matching the live "token is not valid" we saw. (My earlier "gateway ignores
expires" read was the legacy path — corrected.)
Fix: mirror the console. After login, swap the one-time token for the session
token and hold THAT as the gateway bearer.
- src/mgmt/gateway/client.ts: exchangeOneTimeTokenForSession() — unauthenticated
POST /auth/session/ui/new {token}, User-Agent header, parse {accessToken,
expiresAt} (camel/snake tolerant), GatewayError on failure.
- src/mgmt/auth/oauth-provider.ts: AuthDeps.exchangeSessionKey; finishClientLoginLeg
is now async and swaps one-time -> session before binding it under the MCP auth
code. Best-effort: on exchange failure it falls back to the one-time token so
login still completes (degraded) rather than breaking. Diagnostic log notes
held=session|one-time.
- src/mgmt-http.ts: wire exchangeSessionKey (GATEWAY_BASE_URL resolved in-fn).
- test/mgmt-auth.test.ts: assert the one-time token is exchanged and the shim
resolves to the SESSION token. 139 tests pass.
Follow-up (not blocking): the session token itself eventually expires; a refresh
path can be added later. Session TTL to the client stays MGMT_SESSION_TTL_S.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_meta.token_count is the number an agent budgets its context on, and it was wrong in two compounding ways. 1. It was `Math.ceil(JSON.stringify(value).length / 4)`. Measured against a real o200k_base count on live payloads, chars/4 understated by 40-60%: 5609 vs 10770, 55865 vs 107216, 57760 vs 96888. An agent trusting it overran its window. 2. Worse and previously unnamed: 13 of the 14 copies of the estimator took the OBJECT and re-stringified it MINIFIED, while the tool emitted the INDENTED text. The reported number described a string that was never sent (getBlock reported 57760 for a 275819-char body whose real count is 108612). Both collapse into one root cause: the estimator was copy-pasted into 14 files with two different signatures, so nothing forced "count the bytes you actually send". Fixed structurally rather than per-file: src/torpc/tokens.ts now owns the only serializer and the only counter, every call site binds `const text = toolText(out)` and passes THAT string to countTokens, so a payload is serialized exactly once and counted as emitted. 14 local definitions deleted. Chose the real tokenizer over renaming the field, because the measured cost is affordable: gpt-tokenizer@2.9 has ZERO transitive dependencies, adds 99 ms of one-time import at server start (eager, so it never lands inside a tool call), and encodes at ~32-45 ms/MB — under 3% against a 200-660 ms upstream RPC call. Two things the plan did not anticipate, both found by measuring: - This BPE degrades QUADRATICALLY on a long run of a character with no good vocabulary merge: "x".repeat(20_000) took 243 ms, 50_000 took 796 ms, and 1 MB extrapolates to ~5 minutes of blocked CPU. That is reachable, not theoretical: resolveContract decodes name()/symbol() out of an arbitrary caller-named contract, so a hostile token can put a degenerate run in a response body, and this pod is single-replica with a 1-CPU limit. countTokens therefore counts in 4 KB slices, which bounds the merge search and makes cost linear. Verified 0.02-0.03% deviation from a whole-string encode on real payloads (5504 vs 5503, 55018 vs 55003) while the 1 MB pathological case drops to 13 ms. - Steady-state RSS goes 42 -> 111 MB (146 MB peak), so the pod's 128Mi memory REQUEST would have sat at ~83% while idle. Raised the request to 256Mi; the 512Mi limit is untouched and keeps ~3.5x headroom over the measured peak. token_count stays honestly labelled: it is an o200k_base count of the emitted text, not a per-model count for whichever model reads it. Stated once on the listChains discovery surface rather than repeated in every response's _meta. Tests pin the encoder on fixed strings so a dependency bump that changes tokenization fails here instead of silently re-breaking the number, and pin the pathological payload's bounded cost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…scard Three defects with one root cause: the tool treated a NEGOTIATED outcome as a guarantee, and therefore never handled the case where the negotiation fails. WHY the promise was false. The proxy applies tier 2 to eth_getLogs only while the response fits its compression budget; above that the same query returns undecoded at tier 0 — raw topics/data, no `args`. Verified live on eth mainnet today: a USDC-Transfer filter over 20 blocks (2053 logs, 913 KB) came back tier 2, the same filter over 31 blocks (3414 logs, 2.17 MB) came back tier 0. The discriminator is RESPONSE SIZE, not log count. So an agent that read the description, called the tool, and looked for args found nothing and had no way to know why. Fix 1 — tell the truth in the description. getLogs/getBlock/getTransaction all made the same unconditional decode claim. All three now say the tier is requested and negotiated, that a large response comes back raw, and that the response itself reports it. Fix 2 — say it in the RESPONSE BODY, not just _meta. _meta.tier was already correct; the gap was that _meta is not where an agent looks when args are missing. A degraded response now carries tier_requested / tier_applied / tier_degraded / tier_note. Wording lives in one helper (src/torpc/tier.ts) so the three tools cannot drift. One audit assumption corrected here: degradation does NOT merely omit the header — `token-tier: 0` is explicitly present (verified live), so detection never depends on distinguishing absent from zero. Fix 3 — the real waste. The tool issued ONE eth_getLogs for the whole requested range, buffered all of it, then displayed 50 entries. Measured on the audit's own case, an unfiltered 200-block window: 62,366,519 bytes upstream, returned at token-tier 0. We paid for the largest possible transfer AND lost the decode. Replaced with a bounded ascending scan: walk the range from fromBlock upward in chunks, stop as soon as the display cap is filled. Same request now costs ONE upstream call of 1,280,714 bytes at token-tier 2 — 97.9% fewer bytes with the decode intact (verified end-to-end through the MCP against production: 594 ms, upstream_calls 1, tier 2, args present). Chose scan-and-stop over refuse-up-front because it keeps the tool usable, and over try-wide-then-halve because halving down from a wide span wastes MORE than today. Chunk sizing is adaptive rather than a tuned constant: start small (4 blocks unfiltered, 128 filtered, from measured density), grow x4 after a chunk that holds the tier, halve on one that degrades. The proxy's ~2 MB budget is deliberately NOT hardcoded as a prediction — it is Shark's, undocumented, and can move, so degradation is always detected from the response. Honesty on full_count: it is now emitted ONLY when the whole range was scanned. When the scan stops early the total is unknowable, so the response says more_available + scanned_through_block instead of asserting a number that was never computed. This deliberately changes an existing test's expectation, which had encoded the old dishonest behaviour. Fix 4 — "page via expandResult" is now true instead of impossible. getLogs had no cursor, so that advice could not be followed. Added a `logs` member to the cursor union carrying the next unscanned block, and expandResult continues it with the SAME scan helper so a continued page cannot drift from a first page. Block bounds are decimal STRINGS, never numbers, so a resume point above 2^53 stays exact. The logs cursor uses the permissive chain slug, NOT the AAPI enum — the enum would make getLogs unpageable on the ~180 non-AAPI chains it serves. For a TAG-anchored range there is no stable resume block, so instead of repeating impossible advice the note now says what actually works. Also: responses are minified (see SHARK-3525) and decoded amounts are documented as RAW BASE UNITS, so args.value "41695680" on a 6-decimal token cannot be read as 41 million. Tests cover the two risks that would corrupt an agent's accounting silently: chunk boundaries are inclusive-exclusive with no gap and no duplicate (asserted by tiling the range and checking every block appears exactly once, ascending), and the expandResult continuation resumes at exactly scanned_through_block + 1 with no overlap. Plus degraded-body reporting, early-stop-leaves-blocks-unfetched, the narrow-to-rescue-decode path, and upstream_calls reporting so the scan's cost is measurable in production. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
getBalances returned 101,918 chars / 481 asset entries and getAccountBalance
120,365 chars in a single call. Verified live for vitalik.eth: 481 assets /
213,840 raw chars on eth alone, 1056 assets / 469,328 chars cross-chain. One tool
call was spending most of an agent's context on data nobody asked for.
ROOT CAUSE the audit did not name: `pageSize` is a NO-OP on
ankr_getAccountBalance. Measured — the same 481 assets come back for pageSize 10,
50 and 300 alike, and there is no nextPageToken even at 1056 assets. So there was
nothing to fix by tuning pageSize; the bound has to be client-side, and any cursor
we emit has to be offset-based re-fetch-and-slice. Both pageSize arguments are
dropped rather than left in place implying they do something.
WHY A CAP IS SAFE HERE, measured rather than assumed: value is extremely
concentrated. The top 20 assets by USD cover 99.89% of total value on eth (98.81%
cross-chain over 1056 assets) and 50.3% of assets are worth exactly $0. So sorting
by USD and showing 20 loses ~0.1% of value for a ~96% payload cut.
Live result: getBalances 101,918 -> 4,038 chars (token_count 1496), 20 of 481
listed, full_count 481, dust {count: 242, usd_total: 0}. getAccountBalance
120,365 -> 2,733 chars.
Same honesty contract as getLogs: truncated / full_count / note. Specifically:
- Sorted by USD descending BEFORE slicing, so the cap keeps what matters.
- The dust tail is BUCKETED, not silently dropped — an agent is told the tail
exists and is worth ~nothing, instead of being left to wonder.
- minUsd filter and maxTokens override.
- syncStatus surfaced as `as_of` provenance (present on every AAPI reply, and
previously discarded).
THE SORT HAD A LANDMINE worth calling out: `balanceUsd` is a string from an
indexer and is frequently EMPTY — measured 147 of 481 assets have balanceUsd ""
(not "0"). Number(undefined) is NaN, and one NaN in a comparator makes the entire
sort order arbitrary, which would have silently broken the exact ordering the cap
depends on to keep the valuable assets. usdOf coerces explicitly and treats any
non-finite value as 0; it is unit-tested against "", undefined and garbage.
IMPLAUSIBLE BALANCES: the live reply contains exactly one, a scam token with
symbol "NOT" and balanceRawInteger == 2^256-1, rendered as a 60-digit balance
sitting next to a genuine totalBalanceUsd as though comparable. Assets at or above
2^128 base units are now flagged implausible: true and their formatted balance is
WITHHELD — a wrong number next to real holdings is worse than a missing one. The
BigInt parse is guarded because balanceRawInteger is unvalidated upstream text. In
practice this token has balanceUsd "" so it also sorts into dust, which is the
outcome the ticket wanted: it never appears beside real balances.
TAIL ACCESS: added a `balances` cursor (offset-based) and taught expandResult to
continue it. The cursor comment states plainly that this is NOT server-side
pagination — each tail page re-fetches the full ~214 KB list and the view is not
atomic — so nobody later mistakes it for real paging. A forged cursor naming a
non-AAPI chain is rejected with a clear message rather than cast into the AAPI
call, where it would surface as a confusing upstream error.
SCOPE DECISION on getAccountBalance, taken deliberately per the triage: it is the
legacy surface the README promises is "kept unchanged", so the PROSE FORMAT IS
PRESERVED and only bounded. It also gains the _meta block it previously lacked
entirely (it and getTokenPrice were the only tools with no _meta at all).
Found and fixed while verifying live: the shared note told getAccountBalance
callers to "continue with expandResult using `cursor`" even though that tool is
prose-only and emits no cursor — the same impossible-advice defect SHARK-3527
exists to remove. The tail hint is now opt-in per caller, with a regression test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine drifts between advertised behaviour and real behaviour. Each one made an
agent either believe something false or fail without being told why.
1. STRICT SCHEMAS. No inputSchema used .strict(), so an unknown argument was
silently discarded. The audit's reproducer: getWalletActivity with limit=3
returned 25 items, because the real parameter is pageSize and `limit` was
dropped without a word. All 17 tools now use z.object({...}).strict(), so a
wrong parameter name is an immediate -32602 with "unrecognized_keys". Verified
live: the reproducer now errors instead of quietly ignoring the caller.
This is a deliberate contract TIGHTENING — every advertised schema now carries
additionalProperties:false, so a client that validates locally will also start
rejecting. A test asserts no tool is left loose, so a future tool cannot
regress into silent-drop by omission.
It also immediately earned its keep: while smoke-testing the AAPI tools it
caught three of my own probe calls using the wrong argument name.
2. UNSAFE BLOCK NUMBERS. getBlock(block=1152921504606846976) answered "Block
1152921504606847000 not found" — a different block than the caller asked
about. The audit's suggested fix ("handle as bigint") IS NOT IMPLEMENTABLE:
precision is gone before our code runs. JSON.parse of that literal yields a
double whose String() is 1152921504606847000, and 1152921504606846977 parses
to the SAME double, so there is no original value left to promote. For a
non-power-of-two the upstream query itself was silently wrong. The only correct
handling is to REFUSE the JSON number and point at the string form, which is
exact — so getBlock/getLogs now require a safe integer and say "pass it as a
decimal string or 0x-hex instead". Tested both ways: the number is rejected,
the same value as a string queries exactly 0x1000000000000000. The not-found
message also now echoes the normalized parameter actually queried, so it can
never quietly describe a different block again.
3. getWalletActivity's "decoded method name" was UNCONDITIONALLY FALSE. Verified
live: the indexer's transaction keys contain no `method` at all (though the SDK
type declares one), so `t.method?.name` was always undefined and JSON.stringify
dropped the key. Every numeric was raw hex too, against a promise of decimal
values. Now: value_wei/block as exact decimal strings, time as
{ unix_seconds, iso } — the unit is in the NAME because the indexer's timestamp
is seconds and a silent ms conversion is a 1000x error — status as
"success"/"failed" matching getTransaction's tier-2 vocabulary, and `selector`
as the raw 4-byte selector, described as exactly that rather than as a resolved
name. An unconvertible value is OMITTED, never NaN or a misleading 0.
The conversion lives inside the SHARED fetchWalletActivity, not at either call
site, because expandResult continues with the same helper: formatting them
separately would leave page 2 in hex and an agent summing pages silently wrong.
A test pins that a continuation is byte-identical to a first page.
4. resolveContract emitted standard: "ERC-20?" — a question mark inside a
machine-readable field, unparseable by design, on the weak evidence of any ONE
of name/symbol/decimals decoding. Now standard: "ERC-20" with
standard_confidence ("likely" when all three probes answered, "probable"
otherwise) and detected_via naming which ones did.
5. getTokenPrice returned the bare string "Current price: $1876.35" — no asset, no
chain, no timestamp — while the upstream reply already carried all of it. Now
structured, including as_of provenance, plus the _meta it never had. Note the
honest labelling: a native-coin query is priced via the WRAPPED token, so the
wrapped address is reported as priced_via_contract rather than presented as the
asset the caller asked about, which would be a new lie.
6. searchChain's "classify a free-form query" invited exactly the ticker lookup it
cannot do. The description now names the three shapes it accepts and states up
front that tickers, contract names and ENS are NOT resolved. No behaviour
change: the classifier was already honest, the description was not.
7. getChainStats is dead on a real Premium key — reproduced today, -32075 "Method
disabled, restricted by blockchain schema" BOTH with a chain argument and
without, so there is no working call path at all. I did NOT drop the tool: that
is a product decision needing the backend answer on whether
ankr_getBlockchainStats should be enabled for Premium schemas, and silently
removing an advertised tool is the more irreversible choice. Instead the
description now opens by saying most callers cannot use it, that the failure is
permanent rather than transient, not to retry, and what to use instead. The
backend question is recorded as out-of-scope.
Smoke-tested the other AAPI tools on the same key while I was there, since
nobody had: getNFTs, getTokenHolders, getTokenPriceHistory, getInteractions,
getTokenPrice and getWalletActivity all work. getChainStats is alone.
8. getLogs' impossible "page via expandResult" advice was fixed in SHARK-3524,
which gave getLogs a real cursor.
9. Decoded amounts being unit-less is addressed by description (SHARK-3524):
getLogs/getTransaction now state that decoded amounts are RAW BASE UNITS. The
opt-in decimals ENRICHMENT is deliberately NOT in this commit — it costs N
extra eth_calls per response, which attacks the tool's whole reason to exist,
and attaching a WRONG decimals would be worse than attaching none. Left as a
scoped follow-up with the durable fix (the proxy emitting decimals in the
tier-2 decode, since it already has the ABI registry) raised as a TORPC v1.2
spec question.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… docs
Two parts: a defect found by reviewing my own SHARK-3524 change, and the docs
item, written last so it describes shipped behaviour rather than intent.
THE DEFECT (self-inflicted, same class as the audit's complaint). When getLogs
scanned the ENTIRE requested range but the DISPLAY cap still truncated the result,
it emitted a continuation cursor and told the agent to page. That cursor could only
ever point past the range end, so following it returned zero logs — reproduced:
count 50, full_count 300, then the cursor yields count 0. A block-position cursor
fundamentally cannot address "items cap+1..N of an already-scanned range". So this
case now emits NO cursor and says what actually works: raise maxLogs, or narrow the
range/filters. A regression test asserts both the absent cursor and the absent
paging advice. This is exactly the impossible-advice defect SHARK-3527 removes
elsewhere, and I had reintroduced it.
Also hardened one edge in the same function: if the upstream call budget is spent
entirely on narrowing a degrading window, no chunk ever completes and
scanned_through_block was lo - 1 (rendering as "-1" for block 0). It now reports
null plus an explanation instead of a position that was never reached.
REVIEW NOTES (step 6 of the pipeline, run on the whole diff):
- Security controls verified untouched: src/http.ts, src/net.ts, src/provider.ts,
src/torpc/client.ts and src/torpc/errors.ts have ZERO diff, so session-to-key
binding, Origin/Host checks, fail-closed production behaviour and upstream error
sanitization are unchanged. rpcCall's diff is only the schema wrapper and the
shared token helper — the default-deny read allowlist and broadcast refusal are
byte-identical and their tests still pass.
- Probed and DISPROVED a risk in the new chunked token counter: splitting a
surrogate pair at a 4096-char slice boundary does not throw and moves the count
by <=2 tokens, so an emoji or a hostile token name cannot break the response path.
- gpt-tokenizer adds zero transitive dependencies. `pnpm audit` reports the same
3 findings (1 high) as the origin/main baseline — all pre-existing, dev-only,
in eslint's own minimatch/brace-expansion chain, none introduced here.
DOCS. README asserted ABI decode as a flat fact, described _meta.tier as making
degradation unmissable (the whole point of the ticket was that it did not), and
claimed getLogs was "paged (cursor)" when it had no cursor at all. All three now
match the code, plus the o200k token_count, the strict-schema tightening and the
2^53 block rule. The proxy's compression budget is described as BEHAVIOUR ("large
results may come back undecoded, and the response tells you") and the measured
~2 MB figure is deliberately NOT published: it is Shark's internal, undocumented
threshold, and printing it in a README turns an implementation detail into a
number customers hold us to.
static/.well-known/torpc.json is a PUBLISHED discovery surface and was stale:
11 tools advertised against 17 registered, and streamable-http still marked
"planned" though src/http.ts ships. Both reconciled, and a test now pins the
manifest against the live registered tool list so it cannot drift again silently.
Final live re-verification after these changes: the audit's 200-block unfiltered
window returns tier 2 with args intact, 1 upstream call, 11361 tokens, and the
cursor continues at exactly scanned_through_block + 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rom pass 1) shapeAsset stopped copying asset.blockchain, so getAccountBalance — the MULTI-CHAIN tool, which queries every chain when `blockchains` is omitted and then sorts assets by USD — emitted a list interleaved across chains with the chain identity deleted. Three USDC holdings on eth / polygon / bsc collapsed into three entries distinguishable only by their USD figure, making "how much USDC do I hold on Polygon" unanswerable from the response. origin/main printed `• USD Coin USDC (eth): 100 ($100)`; pass 1 printed it with no chain. `blockchain` is back on ShapedAsset and in the prose line. shapeBalances is shared, so getBalances and expandResult's JSON surface carry it too and the three surfaces cannot describe the same asset differently. The prose tag is omitted when the indexer reports no chain, so the format never degrades to a literal "(undefined)". Two tests, both verified to FAIL with the `blockchain: a.blockchain` copy removed: one on shapeBalances asserting three same-symbol cross-chain assets stay distinguishable, one driving the real getAccountBalance tool over an in-memory MCP pair and asserting the prose names each chain. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…vacuous test)
The pass-1 regression guard asserted that every advertised inputSchema has
additionalProperties === false. That is VACUOUS: zod's default `strip` mode
already serializes to additionalProperties:false, so the assertion held on the
unfixed baseline. Probed directly against this SDK — a plain z.object({...})
advertises additionalProperties:false and STILL accepts an unknown key, strips
it, and runs the handler; only .strict() rejects. Removing .strict() from any
of the 17 tools would therefore have regressed with zero test signal, so the
pass-1 claim that "a test asserts no tool is left loose" was false.
The pre-existing defect is correctly named a schema/runtime MISMATCH, not a
loose advertisement: a client validating arguments locally against the
advertised schema was already rejecting unknown args.
Replaced with a behavioural loop over tools/list: each tool is called with a
single bogus key and must reject with `unrecognized_keys` naming that key, with
zero upstream calls. A strict object reports unrecognized_keys even when
required fields are also missing, while a stripping object never mentions the
key, so the loop discriminates for every tool without a per-tool fixture. The
schema-shape assertion is kept as an explicitly secondary check.
Verified: with .strict() removed from getNFTs the new test FAILS and the old
one passed. All 17 tools confirmed strict, so the behaviour is genuinely there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lost partials 1) `topics: []` and `topics: [null]` were classified as FILTERED `filtered` was `base.address !== undefined || base.topics !== undefined`, so an empty filter and an all-wildcard slot 0 — both legal under the strict schema and both semantically UNFILTERED — started the scan at START_CHUNK_FILTERED (128 blocks) instead of 4. On eth at ~386 KB per unfiltered block that first chunk asks for roughly 49 MB, precisely the waste SHARK-3524 exists to remove, and upstream now hard-rejects a result set that large with -32602. So the headline fix was defeated, and converted into a hard failure, by an input class the schema explicitly permits. Now a query is filtered only when it carries a REAL predicate: a string address, or a topics array with at least one non-null slot. Five tests pin the first-chunk width for topics:[], topics:[null], a concrete topic, [null, topic] and address. 2) a mid-scan upstream error discarded every log already collected scanLogs narrowed only on tier degradation, so an upstream failure mid-scan aborted the whole scan, threw away the logs already in hand, and left the agent with a generic error. Converting one call into up to 12 multiplied the exposure. An upstream error on a wide chunk carries the same information as a tier degradation — this window asked for too much — so both now halve and retry the same start. When the failure is irreducible the scan STOPS and returns the partial LogScan: the logs already collected, more_available, scanned_through_block and a cursor, plus upstream_error and a note that names the sanitized failure and says narrowing/filtering is the way out. The note also states that the cursor resumes AT the block that failed, so it does not promise a continuation that cannot work. Two controls deliberately preserved: - errors that a smaller window cannot fix (auth, payment, rate-limit, bad chain) are NOT narrowed, so the call budget is not burned re-failing. - the message surfaced is TorpcClient's own sanitized text, never the proxy's (which can name nodes and internal hosts). A test asserts the raw upstream hint does not appear in the response. - when NOTHING was scanned there is no partial to preserve, so the error is re-thrown rather than becoming a silent empty success. scanLogs was split into isFilteredQuery / fetchChunk / applyChunk to stay inside the sonarjs cognitive-complexity limit; behaviour is covered by the tests. Verified by mutation: reverting `filtered` to the old form fails 4 tests; replacing the narrow-and-preserve branch with a re-throw fails 4 tests. One pre-existing assertion that encoded the old abort-on-first-error call count was updated to assert bounded narrowing instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…timated token counts
MEDIUM: unpriced assets were asserted to be worth zero and hidden as dust
usdOf('') returned 0 and the default rule bucketed usd === 0 into `dust`, whose
note called them "zero/low-value assets". Measured live, 147 of 481 assets have
balanceUsd === "" — 30% of the list. That means the indexer has NO PRICE, which
is not the same as no value. A wallet whose largest holding was an unpriced token
got a response that omitted it and stated the omitted tail was worth ~nothing,
and pass 1 enshrined that in a test as intended behaviour.
Split the two cases:
- priceOf() returns null for absent/blank/non-numeric, a number otherwise. Guard
written against `unknown` on purpose: the SDK types balanceUsd as a plain
string, and trusting that declared type is how the empty case got lost.
- `dust` now holds ONLY assets the indexer actually priced at zero (or below
minUsd), so usd_total is a real sum.
- unpriced assets stay in the value-ordered list with usd: null and
unpriced: true, ranked after every priced asset via a -1 sort sentinel, so they
remain listable and reachable through the existing offset cursor. No new
unbounded bucket, so the SHARK-3526 payload bound is preserved.
- unpricedCount / unpriced_count is surfaced, the note says they are NOT in dust
and their value is UNKNOWN rather than zero, and the prose tool renders
"USD value unknown — no indexer price" instead of "($null)".
MEDIUM: token_count was advertised as EXACT while being extrapolated
Above 262,144 chars countTokens scaled the counted prefix and _meta carried no
signal, while listChains and README called the number exact. Uniform payloads
extrapolate well (-1.5% on a 3.0 MB body) but a non-uniform one measured -55.2%,
the same error band SHARK-3525 exists to eliminate, and >256 KB is reachable in
normal use (getBlock has no size cap, getLogs allows maxLogs 1000).
countTokensDetailed() now returns { tokens, exact } and tokenMeta() emits
token_count_estimated: true only on the extrapolation path. All 24 call sites go
through tokenMeta, so the signal cannot be dropped by omission. listChains now
says "exact up to 256 KB, extrapolated above that (flagged in _meta)".
Also closed here:
- the three call sites that emitted non-empty text with a hardcoded
token_count: 0 (getBlock not-found, getTransaction not-found, expandResult
errorResult) now bind the message and count it.
- the ">99% of value" parenthetical is emitted only on a full first page and is
scoped to "in the wallets we measured"; a tail page says which slice it is.
- the getLogs description no longer claims the chunked scan applies to every
large range: it states that only concrete numeric bounds (or latest) are
scanned and that a tag-anchored range is a single unbounded call.
- BLOCK_RANGE_TOO_WIDE no longer advises "page via expandResult" — the call
failed, so no cursor was ever emitted.
- the chunked-counting comment no longer cites resolveContract as a hostile-input
vector (decodeDynamicString bounds it to 255 chars); the real reason is payload
size, and it says so.
- the "40-60% understatement" figure is scoped to decode-heavy payloads
(measured -55.1% getBlock, -33.8% getBalances, but only -12.1% listChains),
and the "13 of 14" estimator-copy count corrected to 14 of 14.
Verified by mutation: reverting priceOf to zero-coercion fails 6 tests; making
tokenMeta always report exact fails 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The verifier could not reproduce the headline getLogs byte figures, so they must not survive anywhere as stable claims. Removed from the getLogs header comment: "62,366,519 -> 1,280,714 bytes (-97.9%)" and "ONE upstream call". Re-measured later the BEFORE leg does not reproduce at all — upstream now rejects the old whole-range call with -32602 "query exceeds max results" — and the scan took 2 calls, not 1, because the first chunk came back tier 0 and was narrowed. Replaced with the mechanism, an order-of-magnitude range (~94-98% fewer upstream bytes on a dense unfiltered window), an explicit statement that the number is density- and date-dependent, and a note not to turn it into a regression threshold. _meta.upstream_calls and _meta.tier are the honest per-call signals. README: token_count is no longer called "exact" full stop — it is exact up to 256 KB of emitted text and extrapolated above that, flagged with token_count_estimated. The "40-60%" understatement figure is scoped to decode-heavy payloads with the three measured points that bracket it (-55% getBlock, -34% getBalances, -12% listChains). README also no longer offers additionalProperties: false as evidence of strict inputs, since a stripping schema serializes identically; it now states the behavioural guarantee and names the test that pins it. balances.ts header: the live wallet figures are labelled as dated observations rather than a contract, with the known drift recorded (4,038 -> 4,151 chars, dust 242 -> 241, full_count 109 -> 161). Nothing in the code or tests asserts any of them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ainer-key drift toTime could THROW instead of omitting. Number.isSafeInteger is not sufficient on its own: Date spans only +/-8.64e15 ms, so a value that is a safe integer in SECONDS can still be out of range once multiplied by 1000 — a microsecond-scaled timestamp near 1.7e15 seconds does exactly that. toISOString() then raised RangeError, the throw escaped fetchWalletActivity and failed the WHOLE tool call, violating the module's own documented contract that an unusable upstream field is omitted rather than guessed. The raw seconds (the authoritative value) are now always emitted and only the ISO rendering degrades, to an explicit "out-of-range for a calendar date" rather than a fabricated date. Verified by mutation: restoring the old expression reproduces "RangeError: Invalid time value" and fails 2 tests, including one asserting a good item on the same page is not lost with the bad one. getWalletActivity page 1 returned the list under `activity` while expandResult's continuation returned it under `items`, so an agent that paged had to handle two key names for one list. The continuation now emits `activity` as well, keeping `items` as a documented deprecated alias for one release, and the tool description states which key to prefer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…fuse admin namespaces The verifier's observation, confirmed by probe: the description claimed "any method that isn't a known read is refused on EVERY chain family", but the read test is substring-based and generous, so `admin_nodeInfo` matched the "info" token, passed the local guard and was forwarded upstream — stopped only there by the proxy (-32075 Method disabled). `txpool_status` passes the same way. Two changes, neither of which weakens anything: 1) The admin/miner/personal namespaces are now refused BY NAME. This tool exists to return blockchain data; none of those namespaces contain a data read, so denying them cannot refuse a legitimate call. It TIGHTENS default-deny. txpool_* is deliberately left permitted: mempool inspection is a real read. 2) The description no longer claims more than the guard delivers. It now states plainly that broadcast/signing refusal is the guarantee (unchanged, and still covered by its own tests), that the read surface is intentionally generous rather than a curated per-method whitelist, and that the endpoint's own per-key method policy is the authoritative limit. An agent reading this will no longer conclude that a local pass means a method is a sanctioned read. Verified by mutation: removing the namespace check fails the new test. A companion test pins that the tightening refused nothing legitimate, txpool_status and server_info included. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… stop reason expandResult emitted the walletActivity list under BOTH `activity` and `items` to smooth over a key-name asymmetry, so every continuation page carried its payload twice (measured 17,572 -> 34,882 chars, 5,747 -> 11,351 token_count). Page 1 and the continuation now share getWalletActivity's own body builder, the list is emitted once under `activity`, and the alias is removed. Tests pin the container key in BOTH directions - adding an alias and deleting the key each turn the suite red, which is what the surviving mutant showed was missing. buildLogsBody never received WHY the scan stopped, so it told every unexhausted scan that the display cap had filled - including a scan that spent its 12-call upstream budget and collected nothing, which also reported truncated: true on an empty log array. scanLogs now reports its exit reason (range_scanned / cap_filled / call_budget / upstream_error) and each case is worded to what it establishes: truncated only when the display actually withheld logs, more_available only when more logs were really seen, range_fully_scanned: false plus a cursor for what the code does know. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
REVIEW-READY.md was written across sessions and had accumulated a third layer of
stale claims, in a document whose whole value is that a reviewer can trust it
without having been in those sessions. Every claim below was re-checked against
the deployment, the metrics or the manifests rather than against the previous
version of this file.
WHAT WAS WRONG, and how each was checked:
- Section 4.8 said the control-plane limiter's production behaviour was
"unexplained". It is explained and SHARK-3592 is closed: the limiter refuses,
and mcp_ankr_refusals_total{reason="bucket_empty"} stood at 150. The competing
"several replicas, several buckets" explanation is ruled out by up{} returning
exactly one series per plane.
- Section 4b's live table said GET /healthz answers 404. It answers 200, and so
does /readyz; that was the readiness gap SHARK-3607 closed. /metrics answering
404 from the public host is now stated as the intended result rather than left
looking like a missing endpoint, since it is served on its own listener and
scraped in-cluster.
- Section 4b said build identity was half solved and the missing half was ours.
Both halves shipped. Both planes answer 0.2.0+<sha> on initialize and carry
the same fact on mcp_ankr_build_info.
- Section 4b said the two planes run different source states. src/ is
byte-identical between the two RC branches and both rolled together in
infrastructure-k8s PR #2093.
- Section 5 listed SHARK-3596 as not in this branch. It is in: mgmt schemas are
strict, and getTokenPrice takes `chain` with `blockchain` as a documented
deprecated alias that refuses both-at-once.
- Section 5 listed the e2e run against a deployed build as missing. It is PR #32
and it scores 26/26 against what is serving.
- Section 6 carried seven questions for Aleksandr Balev. Two remain, and both are
decisions rather than lookups. The rest closed: some by his work, some by the
observability in SHARK-3607, and three by reading a file rather than asking
anyone, which is recorded as such because it is the least flattering way for a
question to close.
WHAT IS NEW AND IS NOT A CORRECTION:
- The Istio routing is now QUOTED in 4b instead of described, because it is
committed YAML in infrastructure-k8s. So the old summary "production matches no
manifest anywhere" is half wrong and is restated: it matches manifests, just
not the ones in this repository.
- Two VirtualServices share one host and one gateway and the management one is a
catch-all, so the data plane answers only because the specific /rpc prefix is
evaluated first. Nobody has confirmed that ordering is guaranteed. Recorded as
a live single point of failure with the cheap fix (one VirtualService, or an
explicit match set instead of a catch-all).
- The route timeout on long-lived SSE streams is genuinely unknown: neither
VirtualService sets one. A stream cut by a default looks like the server going
quiet rather than erroring, which is the worst shape this failure can take.
- The charts still carry ingress.enabled: true, a Traefik stripPrefix on /rpc and
a 128Mi request. Production is safe only because the per-cluster values
override two of the three. Restated as armed-and-stepped-around rather than
fixed.
- Production runs a chart version that says -rc.1, which is deliberate but leaves
a release decision pending.
USER-STORIES.md: row 4.5 said DONE while it fails in production on at least one
account (gateway 504 on transactionHistory, SHARK-3593). The row now carries that
where a reader meets it. The legend gained a paragraph on what DONE claims and
what it does not, plus the two facts a reader needs to hold: this branch is what
is serving, and every store is per process on one pod per plane.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SHARK-3607: metrics, structured logs and a readiness split (keeps integ current)
SHARK-3588: e2e against the deployed build, not only this checkout
docs: put the reviewer in front of what is true on 2026-08-07
Section 6 listed the route timeout on long-lived streams as open, on the grounds that neither VirtualService sets `timeout` so the mesh default applies and nobody had read it. Held GET /rpc open against production and read it a byte at a time. - The stream survived 22.2 minutes uncut, so there is no hard max-duration cap below that. - It was never idle: the wire carries `: keepalive\n\n` about every 15 seconds, so a mesh idle timeout of any value above 15s cannot fire. The second finding is a DEPENDENCY property and is written up as one, because reading it as a decision would be the mistake this file exists to prevent. The keepalive comes from the MCP SDK's WebStandardStreamableHTTPServerTransport (1.30.0), which the Node transport both planes import is a thin wrapper around. An SDK upgrade that drops or lengthens it reopens the question silently. A max-duration cap ABOVE 22 minutes remains untested and is stated as such. Section 6 now carries one item, edge rate limiting, and it is a decision rather than a lookup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… world the data plane does not produce Three defects found by walking create -> call -> restrict -> freeze against prod on 2026-08-07, all the same shape: a reply or a description asserts something the proxy contradicts. SHARK-3619. The create reply said the ready URL "works immediately". A newly minted key answers -32050 "API key not found" under HTTP 401 for roughly 60 to 90 seconds first, and every plausible reading of that error is wrong: create it again, escalate, or report MCP key creation as broken. The claim about TIME is gone; the claim about SETUP (no session, no header) survives, because that is what makes the URL the shortest path to a first call. The create path adds a measured propagation note naming the code, the words the proxy uses, and the verdict. Reveal does not carry it: its key already exists. SHARK-3620. The tool description said "The secret key material is never returned in the tool output" while the reply carries the endpoint token in full plus a URL with it embedded. The approval page was already accurate, so the description now IS that sentence rather than a second wording of it, shared as one constant and pinned by identity rather than by two regexes that can drift apart. Found while wiring that pairing: the approval page CLIPS every effect at 200 characters, and three lines were over it -- create's credential disclosure (cut at "which are live credentials", losing where they land), the platform-key mint's "without asking a human to approve anything and without a second factor", and the bulk logout's blast radius, whose truncation depended on how many sessions the account had. All three now sit under the bound, and no gated page may ship a truncated consequence again. SHARK-3622. freeze neither read the resulting state back nor mentioned the lag, while the allowlist write on the same key one minute earlier did both. It now reads the status back from the control plane -- authoritative immediately, unlike the data-plane propagation the allowlist writes rightly refuse to race -- and states the measured lag per direction: 10 to 21 seconds for freeze, unmeasured for unfreeze rather than borrowed from it. A read-back that disagrees with the request is reported without isError, because a separate route can trail a write and "retry me" here costs a second human approval. Gates: format, lint, typecheck, build clean; 1639/1639 tests; coverage on the changed files 98.3% lines / 90.5% branches / 100% functions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…not only the disagreeing one A client branching on the field would otherwise have to read `undefined` as agreement, which is the trap writeOutcome.ts closed for `observed`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`deploy/` held seven files describing an ingress-nginx deployment that was never applied to any cluster. They were marked DRAFT, and marking was not enough: a manifest in the repository reads as THE deployment to anyone who has not yet found the section explaining that it is not. The header of `deploy/ingress.yaml` still named `argocd-mrpc` as the managing repository, which does not exist, so the drafts were also carrying a wrong fact about the real path. Production routes through Istio out of `w3tech/infrastructure-k8s`, which is where the Gateway, both VirtualServices, the certificate and the ExternalSecret actually live. Nothing in `deploy/` was reachable from that path. Deleting files is only half of it; the references had to go too, and two of them were stating things that are false: - `src/bodyLimit.ts` explained the batch cap by saying the edge already bounds requests at `limit-rps 20`. It does not, and never did. The comment now says what is true and load-bearing for the cap's justification: there is no edge limit and no CDN, so this cap is the only thing between one request and ~25,000 outbound calls at shark-proxy. - `test/mgmt-dcr-registry.test.ts` cited the same non-existent nginx limit when explaining why an unauthenticated `/register` can be flooded. Same correction. - `DEPLOY-MGMT.md` carried a live `kubectl apply -f deploy/mgmt/*.yaml` runbook that ALSO minted the shim's RS256 signing key by hand. Both halves are wrong: those manifests are gone, and the running key is a stored Vault value read by an ExternalSecret. Generating a new one invalidates every live session at once and reads as an auth bug rather than a rotation. Replaced with the real path and an explicit "do not generate a signing key". - `DEPLOY.md`, `DEPLOY-RUNBOOK.md` and `REVIEW-READY.md` pointed at the deleted paths; each now points at what actually holds the answer. Note that `deploy/aapi-mcp-server-helm` and `deploy/*-helm-rc` are BRANCH names, not paths in this tree, and references to those are untouched and still correct. Gates after the change: typecheck, lint and format clean, 1701 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…them my own vacuous assertion Mutation run over freezeApiKey.ts (Stryker, repo config, 132 mutants) scored 67.42 and surfaced two survivors worth acting on. 1. The date assertion on the freeze propagation note was VACUOUS. The fixture key was named "acceptance-2026-08-07", so /2026-08-07/ matched the key's LABEL in the reply and would have passed with the date deleted from the note entirely -- which is exactly what the surviving mutant did. The fixture key is now "acceptance-run" and the assertion tests the note it was written for. 2. Flipping `||` to `&&` in the read-back guard survived: with an absent body the mutated guard throws reading `.frozen`, the throw is caught one frame later, and the result is still "accepted, not observed" -- passing every assertion. The unobserved test now pins the REASON, and a new case covers the other half of the guard: a body that exists but whose `frozen` is not a boolean, which must not render as `frozen: undefined`. Both re-verified by hand mutation: each mutant now fails, and the source was restored byte-for-byte (md5sum checked, not `git diff`). 1640/1640 tests; format, lint, typecheck, build clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The two per-plane ArgoCD applications and their two Helm charts were merged into one application and one chart on 2026-08-07 at about 13:45 UTC, after most of section 4b was written. Rather than let the file describe yesterday's shape, this re-reads it against the cluster and the manifests now in `main`. WHAT IMPROVED, and both were open items in this file: - The two VirtualServices are now ONE, carrying both routes in written order. That closes the ordering risk 4b flagged, and it is verified as SHIPPED rather than agreed: the manifest quoted in 4b is the one applied. - The two superseded chart branches take the pending `0.4.0` release decision with them, and PRs #30 and #31 against them are moot. The merged chart carries no ingress template at all and requests 256Mi for the data plane by default, so two of the three "artifacts describing a deployment nobody runs" are gone. WHAT BROKE, and it is the more instructive half. The merged chart labels its Services only under `spec.selector`, which selects PODS. A VMServiceScrape selects SERVICES by their own `metadata.labels`, which were dropped. Since about 13:45 UTC `up{namespace="agent-rpc-mcp"}` has returned no series and `mcp_ankr_build_info` has been absent, so "which build is running" stopped being answerable from a dashboard on the same day it started being answerable. The images did not move: data is still `e9a0b572…`, mgmt still `9176d12c…`, and `initialize` still answers `0.2.0+e9a0b572…` on the wire, so the code under review is the code serving. Fix in PR #35, needs `helmChartVersion` re-pinned to `0.1.1`. The point recorded for the reviewer is not about Helm. The pods stayed Healthy, ArgoCD stayed green and traffic kept flowing, so an observability outage is invisible to every signal a deployment normally offers. The only thing that catches it is an alert on the absence of the metrics themselves, and SHARK-3608 already contains exactly that alert, sitting unmerged in infrastructure-observability #310. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…fulness SHARK-3619/3620/3622: stop the key-lifecycle writes describing a world the data plane does not produce
Two things settled after the previous commit, both by the SRE who owns the deploy, and both recorded here as verified rather than as reported. THE SCRAPE OUTAGE IS OVER. PR #35 merged at 14:32Z, chart 0.1.1, re-pinned and rolled. Checked rather than accepted, which is the whole point given how the outage was found in the first place: `up{namespace="agent-rpc-mcp"}` is 1 for both jobs, `mcp_ankr_build_info` again carries `e9a0b572…` and `9176d12c…`, and the pod names have changed, so a rollout genuinely happened. The window was about 13:45 to 14:32 UTC. EDGE RATE LIMITING IS DECIDED, which empties section 6. The option space was narrower than this file assumed, and the wrong assumption was ours: an Istio local rate limit is a BLANKET limit in the sidecar with no per-client key, not the per-IP control we had written it up as. Per-IP at the edge needs a Global Rate Limit service with a Redis backend and a per-request gRPC hop. Decision taken jointly: neither, for now. The Global service is disproportionate to current traffic, the abuse shapes that worried us are already bounded in the application, and a blanket backstop was considered and also declined. The point to revisit is the one 4.2 already names: before GA, or when either plane moves past one replica. That is a decision a reviewer can disagree with, and the file now says so explicitly, because disagreeing with it would not change a line of this branch. Also refreshes the gate battery: 1714 tests pass after SHARK-3619/3620/3622 merged, and the live e2e is 26/26 against the deployment as it stands now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… MFA-gated synthetic one findKeyBySlot read index 0 as the account's own synthetic key whenever no team account was selected, so every key-addressed tool refused the slot a user can see in their own listing, explaining itself with a second factor that has nothing to do with it. Measured on prod: mgmt_list_api_keys shows `index 0: Default`, and mgmt_get_api_key_status(index 0) answers "Slot 0 is not a project key". Two different routes were being treated as one. GET /auth/jwt/all is the project listing and carries slot 0; GET /auth/jwt/getMySyntheticJwt is the account's own key, MFA-gated and deliberately never wrapped here. A personal account's slot 0 now resolves through the listing like slots 1 and up, and the synthetic route is still not reached on any path -- asserted, not assumed. mgmt_reveal_api_key keeps its refusal for this slot. It shares the resolver, so without an explicit guard this change would have silently reopened what SHARK-3567 closed on purpose: a tool whose job is handing over a credential makes a different trade from one that operates on a key without disclosing it. One existing pin flipped rather than being deleted: the fixture it used has no slot 0, so it now pins the refusal that stays true for it, an empty slot naming the slots that exist, with no claim about second factors. Gates: typecheck, lint, format, 1644 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… by default
The advertised OAuth endpoint served 75 account-administration tools and not one
chain read. Asked for an address balance, an agent connected to /mcp had no tool
for it: the data tools lived on a second server behind a raw key in a header, and
reaching them cost a second MCP entry, a hand-pasted credential and a client
restart. The restart is what actually stopped people.
/mcp now registers the same sixteen data tools /rpc does -- registerDataTools is
imported rather than copied, so the two endpoints cannot drift into different
surfaces -- and the key they spend is resolved server-side from the account the
session is already signed in as: slot 0, the Default project key, or whichever
slot mgmt_select_key names. Nothing is pasted and no credential enters the
conversation. The default selection moves from `core` to `core` plus `data`,
which only ever adds tools; the entry cost goes from ~2.4k to 8,823 o200k tokens
against an 8,900 ceiling the test asserts.
TWO CLASSIFICATION SUITES HAD TO NARROW THEIR SCOPE RATHER THAN GROW THEIR LISTS.
mgmt-annotations (SHARK-3540) and mgmt-role-capabilities (SHARK-3553) enumerate
everything registered on the management server and partition it, and both rules
are about acting on an ACCOUNT. A chain read is read-only and answers to a
different contract, held in test/annotations.test.ts -- so getAccountBalance was
being told its readOnlyHint should be false, which is the wrong complaint about
the right code. The boundary is now DATA_TOOL_NAMES in src/server.ts, exported
beside the registrar and held equal to the live /rpc surface by
test/data-tool-surface.test.ts. Verified rather than asserted: dropping one name
from that list fires all four gates at once, so a data tool cannot slip past
either classification by claiming to belong to the other.
mgmt_select_key is a management tool that rides with the group, so it stays in
both partitions. It registers through the account-scope wrapper -- naming slot 4
while the session is aimed somewhere you did not expect is exactly the mistake
worth refusing -- and takes JwtManagerRead, the capability the key LISTING takes:
a seat that may not see which projects exist has no business naming one by index.
FOUR DEFECTS FOUND IN REVIEW BEFORE THIS SHIPPED, all pinned by a test:
- The resolved token was cached by SLOT ALONE. mgmt_select_account moves a
session between the accounts a login holds a seat on, and slot 4 of one team
is a different key from slot 4 of another. The first resolution won, so every
later chain read went out on the previous account's key -- including the
slot-0 default nobody selects. The session would report one account through
mgmt_whoami and the account line every wrapped tool prints, while the reads
were billed to another. The cache key is now the account plus the slot.
- The chain tools' own contracts were not delivered on this endpoint. SHARK-3599
lifted that prose OUT of the 16 tool descriptions because instructions carry
it, which is only true where the instructions do; here they did not, and the
RAW BASE UNITS rule lives nowhere else, so an agent decoding a transfer would
have reported an amount wrong by 10^decimals with nothing contradicting it.
Contracts 2 to 5 are now a shared DATA_TOOL_CONTRACTS both planes compose
from; contract 1 stays per-endpoint, because the bound key is the one thing
the two genuinely disagree about.
- mgmt_select_key could name a key the account no longer has in that slot: a
session can delete and recreate a slot without leaving, and the cache sees
neither write. It now always re-reads, which is one gateway read and one
worker exchange, the same as the equivalent key tool pays.
- The deferred client Proxy answered every property with a function, which made
it THENABLE. One `await` or `Promise.resolve` near it would call
`then(resolve, reject)`, and the handler would resolve the real client, find
no `then`, return undefined and never call either callback -- a permanent
hang on a request path with nothing in a log. `then` is now absent.
MUTATION TESTING FOUND WHAT COVERAGE COULD NOT, on both files it ran over. The
new key session scored 68% on its first pass with fourteen survivors, and they
were not noise: nothing proved the cache was a cache, nothing proved a failed
resolution was retried rather than remembered as broken, nothing drove the
refusal path of a switch at all, and the account-switch test above turned out to
be passing through `select`, which always re-reads -- so it would have passed
against a cache keyed on the wrong thing. It also found that the `provider`
accessor, which hands the AAPI client to eight registered tools, was reachable
by no test: replacing it with one that yields undefined survived the whole
suite. Six tests later the file is at 95.45%, with two survivors that are
genuinely equivalent. Over toolsets.ts (97.17%) two more real gaps: listPhrase
was untested at three names, where slice(0, -1) and slice(0, 1) stop agreeing,
and the guarantee that a session carries `core` however it was built was never
exercised on a core-less input.
KNOWN, NOT FIXED, AND NOW STATED WHERE IT WAS PREVIOUSLY DENIED. Importing the
data plane here pulls gpt-tokenizer into the management binary at boot: measured
RSS 79 -> 189 MB and 1.04 s for src/mgmt/server.ts, the tokenizer being 65 MB and
386 ms of it, against a 512Mi pod. Two comments asserted the opposite ("the
management binary carries no tokenizer on purpose") and are corrected rather than
left to be believed. Deferring the import is not cheap -- `data` is in the
default and the group thunks run inside registerAsOneChange's synchronous window
-- so the two real options, a real token count in mgmt_list_toolsets and a lazy
tokenizer in torpc/tokens.ts, are recorded as open decisions.
Gates: typecheck, lint, format, 1662 tests, coverage (global 90/80/85 and
mgmt-scoped 80/75/80), build, mutation (toolsets.ts 97.17%, keySession.ts
95.45%).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n the module does SHARK-3629 made the management server import registerDataTools, which reaches torpc/tokens.ts, which imported gpt-tokenizer at the top level. So every management process paid 65 MB and 386 ms at boot for a tokenizer it might never reach: `import src/mgmt/server.ts` went RSS 79 -> 189 MB and took 1.04 s, against a 512Mi pod and a 5 s HEALTHCHECK, and a `?toolsets=core` session -- which has no chain tool in it at all -- paid the same as one serving the whole data plane. The load now happens on first use, and the warm-up moved to registerDataTools. That is the honest place for it: it is the function that puts chain tools on a server, so it is exactly the event after which a token count becomes reachable. Measured per process, one scenario each: import src/mgmt/server.ts 118 MB (was 177) not loaded ?toolsets=core session 104 MB (was 176) not loaded default core+data session 176 MB loaded /rpc createServer 170 MB loaded, as before A cold process is not at the 82 MB management-only baseline, and that is the change's boundary rather than a shortfall: the AAPI client and the sixteen tool modules are still statically imported. The tokenizer is the single largest piece and the only one a chain-free session provably never needs. createRequire RATHER THAN `await import()`. countTokensDetailed is called synchronously from every tool's response path, so making the deferral async would ripple through tokenMeta and all fourteen call sites for no behavioural gain. In an ESM package createRequire is how a synchronous deferral is spelled. There is deliberately NO fallback: if the module cannot be loaded this throws rather than quietly reverting to chars/4, which is the 40-60% understatement SHARK-3525 removed and would be worse arriving silently. AND THIS IS WHY mgmt_list_toolsets KEEPS ITS chars/4 ESTIMATE. The two halves are one decision. mgmt_list_toolsets is in `core`, i.e. on every session including the narrowest, so counting for real would pull those 65 MB straight back into exactly the connections this relieved -- undoing the change through the one tool that reports the numbers. Two comments claimed the binary "carries no tokenizer on purpose", which SHARK-3629 had falsified and d860d75 corrected to say so; they now state the posture that is actually true again. Pinned in test/tokenizer-lazy.test.ts, one CHILD PROCESS per scenario. A module registry is per process and write-once, so in-process the answer to "was it loaded?" would depend on test order -- the shape of a test that passes for the wrong reason. Both directions are asserted: the cold cases prove the deferral, and the warm ones prove it is a deferral and not a removal. Verified by hand mutation: deleting the warmTokenizer() call fails both warm tests. Also closes a vacuous assertion the mutation run exposed in the pre-existing >256 KB path. The extrapolation test asserted `meta.token_count === d.tokens`, which compares the computation with itself, so replacing `(tokens / counted) * text.length` with `/ text.length` or `tokens * counted` survived the whole suite. A uniform payload of twice the limit must extrapolate to about twice the count of one at the limit, which is a reference the function did not produce. Gates: typecheck, lint, format, 1668 tests, coverage (global 90/80/85 and mgmt-scoped 80/75/80), build, mutation (tokens.ts 82.35% before the added test; remaining survivors are the Math.min cost bound, which only a timing assertion could kill, and two equivalents). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings the observability work (#29), the live e2e suite (#32) and the docs commits onto this branch. One real conflict, in src/server.ts, and it was semantic rather than textual: SHARK-3607 added instrumentToolCalls, which must patch registerTool BEFORE any tool registers, while SHARK-3629 moved the registrations out of createServer into registerDataTools. Resolved by keeping both and ordering them: instrument the server, then register. The interaction leaves a gap neither side had on its own. instrumentToolCalls is applied only in createServer, so the chain tools the MANAGEMENT plane now serves are not counted by mcp_tool_calls_total. Recorded at the call site and in DEPLOY-MGMT.md rather than fixed here. Gates on the merged tree: typecheck, lint, format, 1742 tests, coverage (global and mgmt-scoped), build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…gement plane instrumentToolCalls is applied only in createServer, so mcp_ankr_tool_calls_total and mcp_ankr_tool_call_duration_seconds carry nothing from /mcp. That was invisible while the planes served disjoint surfaces; SHARK-3629 put the sixteen chain reads on both, so the same tool name is now counted from /rpc and not from /mcp, and a per-tool rate read off those families understates real usage silently. Not fixed here on purpose: widening a metric's coverage changes what every existing dashboard and alert on those names means, which belongs to whoever owns them rather than to a merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI failed the tokenizer test at 149 MB against a 140 MB ceiling, for a cold process that is 107-118 MB here, while the module registry correctly reported the tokenizer absent. The threshold was measuring the runner, not the change. The file's own comment already said RSS is noisy and the registry probe is exact, and the ceiling was added anyway on the argument that RSS is the number the ticket is about. It is, and that is an argument for reporting it, not for gating on it: baseline RSS moves with the Node build, the GC and the transform cache, and at 149 cold against 176 warm the bands overlap across environments, so no portable ceiling separates them. Locally the same scenario read 118 MB and then 107 MB on consecutive runs. The registry assertion, which is exact and environment independent, is unchanged and is what proved the change works. Each scenario now prints its RSS instead, so the figure stays visible without the gate depending on the machine. Gates: typecheck, lint, format, 1742 tests, both coverage scripts including the one CI failed on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comments across src/ recorded what the code USED TO be before they said what it is, so a reader met the archaeology first. This removes that layer without losing what it carried. The rule applied per comment: it stays if it explains something a competent reader cannot get from the code (a constraint, a non-obvious "why", a trap, an external contract, a security consequence); it goes if it only records that things were once different, which is what git history and REVIEW-READY.md are for. Where a history note carried a live lesson it is restated as a rule or a counterfactual, so the warning survives without the narrative. Notable, beyond the mechanical pass: - groupScope.ts explained the same trap four times (absence from the route set is NOT an opt-out; `resolveGroup` still inherits the selection). Stated once in the header, back-referenced from the four sites. - rpcCall.ts: the debug_ history becomes a better live warning, that the mutating word sits mid-camelCase so the verb rule cannot reach it. - buildInfo.ts asserted the deployment runs image tag `latest`. It does not: both planes pin a full git sha (REVIEW-READY 4b). Removed rather than reworded. - session-store.ts carried a comment about a REMOVED field, labelled as kept for history, ending in a truncated sentence fragment. Comment markers in src/ comments: 164 -> 19, of which 11 are in files owned by the open PR #36 and 8 are idioms ("can be used to", "the old one" meaning the previous ticket). The 26 marker hits inside string literals are product text under the SHARK-3599 token budget and were deliberately not touched. No behaviour change, proven rather than asserted: for each of the 46 changed files, comments were stripped from the committed and working versions via the TypeScript AST printer and compared byte-for-byte. All 46 are identical. Gate: typecheck, lint, format:check green; 1714 tests pass, 0 fail (unchanged). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DEPLOY-RUNBOOK.md section 2 was read on 2026-08-06 and describes the topology as it was that day. It was superseded the following afternoon, and REVIEW-READY.md 4b records the newer reading, so the runbook has been contradicting it since. A runbook that leads somewhere that no longer exists is worse than no runbook, because someone will follow it. Corrected against 4b: - ONE ArgoCD application (`aapi-do-fra1-03-agent-rpc-mcp-production`), not the two per-plane applications it named; - ONE source-of-truth path, `argocd/apps/aapi/resources/agent-rpc-mcp/`, not the two it listed, and the same correction in REVIEW-READY section 6; - routing is ONE VirtualService, and the runbook now says WHY it has to stay one: Istio merges VirtualServices on the same host and gateway, and the order of routes contributed by separate resources is not guaranteed, so a working split can silently start answering every /rpc request with the management plane's 401; - `/healthz` answers 200 and `/readyz` exists, both closed by SHARK-3607; the runbook still recorded the 404. `/metrics` 404 is stated as correct rather than left looking like a gap; - the two per-plane Helm chart branches are marked superseded by the merged chart and flagged for retirement, rather than described as if they were live options. Also drops the `argocd-mrpc` correction note from the runbook. That history is told once, in REVIEW-READY 4b, which is where it belongs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…fulness SHARK-3629/3635: serve the chain reads from the management endpoint, and stop paying for a tokenizer nobody reached
…nteg/mcp-prod-readiness
… files
Two groups.
FOUR STALE COMMENTS THE FIRST SWEEP MISSED, because none of them contains a
history marker word: they simply assert something the code stopped doing. Each
was confirmed by executing the code, not by reading it:
- oauth-provider.ts:704 said the nonce is "re-checked at /callback". Nothing
reads `ankrState` on the callback path, and the block seventy lines below
(added by the previous commit) says the check cannot fire and must not be
re-added. The file contradicted itself on a CSRF control.
- oauth-provider.ts:1097 likewise said "only the nonce is trusted at /callback".
What is trusted there is the one-time state key and the APPROVAL_COOKIE nonce.
- rpcCall.ts:286 said the txpool boundary is "those three NAMES". Executed:
txpool_flushPending and txpool_anythingElse are FORWARDED. With the read
allowlist gone there is no name boundary, only the write rules.
- rpcCall.ts:206 listed debug_chaindbCompact among methods the VERB rule
refuses. Executed: hasMutatingVerb("debug_chaindbcompact") is false, because
`_chaindbcompact` is not `_compact`. Only the debug_ namespace rule reaches it
which is what the header now says, so the file contradicted itself here too.
A RESTORED WARNING. The previous commit dropped the `argocd-mrpc` note from
DEPLOY-RUNBOOK.md as archaeology. That was wrong by this cleanup's own rule: the
name is still live in deploy/README.md and both ingress.yaml headers on the
`deploy/mcp-helm` branch, which is the branch the same table sends a reader to
for the chart, and the repository 404s. It is back as a table row.
The second group applies the same rule to the eleven files PR #36 owns, which
the first pass deliberately skipped to avoid conflicting with it. The
toolsets.ts deny-list-vs-allow-list reasoning is kept in full: the
`Set.prototype.forEach` third-argument hole is a live reason for the shape, not
a story about a previous one.
Archaeology markers in src/ comments: 164 -> 7, and all 7 are idioms ("can be
used to", "a ref that no longer resolves", "the old one" meaning the previous
ticket). The 26 hits inside string literals are product text and untouched.
Comments-only across all 52 changed files, proven by AST-printer comparison.
Gate: 1742 tests pass, 0 fail; typecheck, lint, format:check green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SHARK-3607 added per-tool metrics by patching registerTool once, before the tools
register, and applied that patch in createServer only. That was complete while
the data plane was the only thing with tools. SHARK-3629 ended it: the sixteen
chain reads run on the management endpoint as well, so the SAME tool name was
counted when served from /rpc and not counted when served from /mcp.
The failure mode is silence. Nothing errors and no series disappears; a per-tool
rate read off mcp_ankr_tool_calls_total simply understates real usage by whatever
share /mcp carries. None of the ~78 management tools had ever been counted
either, so the gate, the approvals and the key writes had no per-tool volume or
latency at all.
The helper moved to src/obs/toolMetrics.ts and both planes call it. It was a
data-plane helper while the data plane was its only caller; making the management
plane import it from the data plane's server module to get metrics would be
backwards.
THREE THINGS MAKE THE PATCH REACH THE WHOLE SURFACE, and each is a way it could
have reached only part, so each is a test:
- withAccountScope reads server.registerTool at CALL time rather than capturing
it, so the ~70 wrapped management tools go through the patch rather than
around it;
- the patch replaces registerTool on the server INSTANCE, so a group loaded
later through mgmt_load_toolset is wrapped too;
- the helper is now idempotent per server. Both planes call it and the
management server also receives the data plane's registrar, so without the
marker a second patch over the first would count every call on that server
twice.
WIDENS TWO METRIC FAMILIES, which changes what anything reading them means:
mcp_ankr_tool_calls_total and mcp_ankr_tool_call_duration_seconds will carry
management tool names that never appeared, and volume on the sixteen chain-read
names rises because the /mcp share stops being invisible. The plane label is
already a registry default, so the two stay separable; a panel that does not
split by it will merge them. Nothing narrows, so no existing series loses data.
Verified by hand mutation: removing the instrumentToolCalls call fails five of
the six new tests, and the sixth is the one that instruments explicitly. Restore
checked by md5sum.
Gates: typecheck, lint, format, 1748 tests, coverage (global 90/80/85 and
mgmt-scoped 80/75/80), build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…icting it Five root docs carried overlapping deployment content and disagreed with each other about what production runs. DEPLOY-RUNBOOK.md is now the single source of truth for deployment; DEPLOY.md is gone, folded into it. WHAT MOVED. DEPLOY.md's unique content is now runbook sections 8 (the served surface: endpoints, data-plane auth, the body and batch caps, the trust-proxy hop count) and 9 (observability: the metric table, the log-field allowlist, readiness and the drain). Its environment table duplicated the runbook's own, and three of its deployment instructions were stale and are not carried over: it routed `mcp.ankr.com/mcp` to the data plane "via ingress" when `/rpc` is what answers and Istio is what routes, it described a chart-0.4.0 deploy order against the pre-merge per-plane application file, and its "see Observability below" pointed upward. WHAT WAS CONTRADICTORY. DEPLOY-MGMT.md told whoever provisions the deploy to "generate a fresh key straight into the cluster Secret", and twenty lines later said "Do not generate a signing key" - about a key whose regeneration invalidates every live session at once. It also named the superseded ArgoCD path and two VirtualServices where there is one, and carried "Secret hygiene" twice. That whole block is now one instruction: the key is a stored Vault value read by an ExternalSecret, there is no `kubectl apply` step in this repository, and the runbook is the entry point. The single-VirtualService rule is stated with its reason, because splitting it again fails silently. Two stale traps in runbook section 7 went with them: "Nothing observes this service. No metrics, no dashboard, no alert" and "`/healthz` is both the liveness and the readiness target", both closed by SHARK-3607. The first is replaced by the gap that is actually still open, which is the alert on the ABSENCE of the scrape (SHARK-3608) - the failure that already happened and paged nobody. WHY NOT A `docs/` DIRECTORY, which the brief offered as an example. test/doc-gates.test.ts reads README.md, DEPLOY-MGMT.md and USER-STORIES.md by name from the repo root and asserts their claims against the code - it is the gate that keeps these docs honest. Those two docs are also referenced from 8 src files and 9 test files. Relocating them means rewiring a doc-honesty gate and 17 references on a branch already in production and under review, to gain a directory. The overlap was the actual complaint, and removing a file plus the contradictions addresses it without touching the gate. If the reviewer wants the move, it is a mechanical follow-up with the gate updated in the same change. Root docs: 5 files -> 4, with the deployment story told once. Gate: 1742 tests pass, 0 fail, doc-gates included; typecheck, lint, format green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nteg/mcp-prod-readiness
Cleanup pass before review: 4 commits, no behaviour changeThese four commits change comments and documentation only. They exist because the The rule applied per comment. It stays if it explains something you cannot get Archaeology markers in Proof that behaviour did not move, since "comments only" is easy to claim:
Four stale claims found and fixed, none of which contained a history marker
Docs. I did not create a Two things worth your judgement:
|
c79ad46 to
d48a6ff
Compare
Merges the data plane (#25) and the management plane (#6) into one branch, plus the rounds of fixes that only became possible once the trees were together.
REVIEW-READY.mdon this branch is the review document. It is written for someone who was in none of the sessions that produced this, and it states the decisions and their reasoning rather than assuming them. Section 4 is the one to read if you read one; section 4b is the one to read next.This PR replaces #25 and #6, which are closed in its favour.
Why the two had to merge before either could ship
Not tidiness. Both PRs had rewritten the same security bootstrap in
src/http.ts, independently, and each carried controls the other lacked:intEnv;guardHotPath,installLastResortHandlers, the fail-closedAllowlistConfigError, the origin allowlist passed to the transport, andtransport.close()on a failed initialize.Shipping either alone would have shipped a regression against the other in the file that holds the public endpoint's security bootstrap.
Gates
pnpm typecheck(src + tests)pnpm lint,pnpm format:checkpnpm testpnpm test:coverage(90/80/85)pnpm test:coverage:mgmt(80/75/80)pnpm buildBoth coverage gates now run in CI. They did not before, so the thresholds constrained nothing and the numbers in the review notes were hand-measured.
Mutation is scoped per file by hand (
coverageAnalysisis off, so every mutant costs a full suite run).src/bodyLimit.ts94.44,src/buildInfo.ts94.74, session-store 35.59 to 86.96, deleteApiKey 58.00 to 78.00.What the review round closed
Nine findings survived adversarial verification, four high, all fixed. Each names its evidence in
REVIEW-READY.mdsection 3. The four high ones:rpcCalladmitted 21 state-changing methods, two of which broadcast.bumpfeeandpsbtbumpfeecleared a substring allowlist on thefeetoken and were verified reaching upstream against a running data plane.tools/callentries in one 327 KB body, answered in 0.53 s. Now capped at 20 messages per request./authorizeanswered400 invalid_client.buildProvider's wiring toguardProviderwas untested: removing the guard left 1521 of 1521 tests passing while putting an incident string back into agent-visible text and removing the only AAPI deadline.The largest behaviour change
rpcCallstopped deciding which reads exist (section 4.5). It was a default-deny read allowlist in front of the write rules; it is now a write denylist only, and anything it does not refuse is forwarded. The read half failed in both directions at once, and which reads exist is decided by the chain's schema and the caller's tenant, both current by construction. 109 string literals removed.The risk, stated plainly: a method no write rule recognises now reaches the endpoint. Shark's
tx_broadcastingprofile forwards broadcasts, so for the WRITE class this guard remains the only chokepoint, and nothing here touches that half.What production actually runs (section 4b)
Read from ArgoCD on 2026-08-06, and it corrects two claims these notes had made. Production is Istio (Gateway plus VirtualService) via ArgoCD out of
argocd-mrpc, with an ExternalSecret and ECR images. Thedeploy/*.yamlin this repo are ingress-nginx and are not applied anywhere, so theirlimit-rpsannotations are not in force: there is no edge rate limiting on either plane and no CDN.DEPLOY-RUNBOOK.mdis new and is the single current deploy document.Also: nothing identified the running build.
serverInfo.versionwas a constant that differed between the two planes and frompackage.json, and both deployments runlatestwithIfNotPresent. It is now<version>+<commit>from aBUILD_COMMITbuild arg (SHARK-3606).Recorded decisions rather than open findings
/confirmself-approval path, shipped.Known gaps, tracked
latesttransactionHistory, breaks the invoice chainTwo checks need one real interactive login and are listed in SHARK-3461: whether UAuth echoes
ankrStateto/callback(thenMGMT_REQUIRE_ANKR_NONCEbecomes mandatory), and whether the login token and the exchanged session token carry the sameunique_id. If they differ, every HITL-gated write is permanently unapprovable, silently.Reproducing locally
Never raise Stryker's
concurrencyabove the pinned 2 and never run it unscoped: an unbounded run put a 20-core machine at load average 252.