SHARK-3629/3635: serve the chain reads from the management endpoint, and stop paying for a tokenizer nobody reached

DEPLOY-MGMT.md

@@ -78,21 +78,29 @@ client                         shim (mgmt-mcp)                 UAuth / gateway
 - `DELETE /mcp` — session teardown.
 
 **`?toolsets=` on the connection URL (SHARK-3600).** Which groups of tools the
-session registers: `core`, `keys`, `usage`, `billing`, `notifications`, `team`,
-`identity`, or `all`, comma-separated. `core` is always registered and cannot be
-dropped; **with no parameter a session gets `core` only** (10 tools, roughly
-2.3k o200k tokens, against ~27.7k for all 77). Callers who want everything must say
-`?toolsets=all`. Both figures are printed by `test/mgmt-toolsets.test.ts` on
-every run rather than being maintained here; read that output, not this sentence,
-when the number has to be exact.
+session registers: `core`, `data`, `keys`, `usage`, `billing`, `notifications`,
+`team`, `identity`, or `all`, comma-separated. `core` is always registered and
+cannot be dropped; **with no parameter a session gets `core` plus `data`**
+(27 tools, roughly 8.8k o200k tokens, against ~34.6k for all 94). Callers who
+want everything must say `?toolsets=all`; callers who want the account tools
+WITHOUT the chain reads say `?toolsets=core` (10 tools, ~2.4k). Every figure is
+printed by `test/mgmt-toolsets.test.ts` on every run rather than being maintained
+here; read that output, not this sentence, when the number has to be exact.
+
+**The default changed in SHARK-3629**, and it is the one behavioural change on
+this endpoint that an existing caller can notice: a connection that names no
+`?toolsets=` used to get `core` alone and now also gets the sixteen chain reads
+plus `mgmt_select_key`. It only ever ADDS tools, so nothing a caller already
+depended on moved, but the entry cost went from ~2.4k to ~8.8k tokens. A caller
+that wants the old listing asks for it by name.
 
 Those are REAL o200k counts. `mgmt_list_toolsets` prints slightly larger numbers
-for the same two listings (~2.5k and ~30.1k) because the served process carries
-no tokenizer and estimates at four characters per token. The estimate runs 5.6%
-to 10.7% high across the eight selections, measured on every test run and gated
-at 15%. Same quantity, two measurement methods, and the estimate is deliberately
-the one that overshoots: a caller is never surprised by a listing that costs more
-than it was told.
+for the same listings (~9.0k for the default and ~37.1k for all) because the
+served process carries no tokenizer and estimates at four characters per token.
+The estimate runs 2.6% to 10.9% high across the nine selections, measured on
+every test run and gated at 15%. Same quantity, two measurement methods, and the
+estimate is deliberately the one that overshoots: a caller is never surprised by
+a listing that costs more than it was told.
 
 Four properties this parameter has, and each one is a test:
 
@@ -132,7 +140,34 @@ cost was measured rather than assumed: building every group costs 1.006 ms and
 register-everything-disabled would have charged every default session ~0.87 ms
 and ~510 KB for tools it never lists. Against a 512Mi pod with a bounded session
 registry that is a real bill. The tool itself costs `core` one extra entry: 223
-o200k tokens, 2039 → 2262, still inside the 2400 budget the test asserts.
+o200k tokens, 2039 → 2262 when SHARK-3609 measured it. The budget the test
+asserts is on the DEFAULT selection rather than on `core`, and since SHARK-3629
+that is `core` plus `data`: 8,823 measured against an 8,900 ceiling.
+
+**Process memory, and what a chain-free session costs (SHARK-3635).** SHARK-3629
+made this binary import the data plane, which brought `gpt-tokenizer` with it:
+65 MB and 386 ms, paid at boot by every pod whether or not a session ever read a
+chain. The tokenizer now loads on FIRST USE and is warmed by `registerDataTools`,
+so it arrives with the chain tools instead of with the import statement.
+Measured per process, one scenario each:
+
+| Process                              | RSS              | tokenizer         |
+| ------------------------------------ | ---------------- | ----------------- |
+| `import src/mgmt/server.ts`          | 118 MB (was 177) | not loaded        |
+| `?toolsets=core` session             | 104 MB (was 176) | not loaded        |
+| default (`core` plus `data`) session | 176 MB           | loaded            |
+| `/rpc` `createServer`                | 170 MB           | loaded, as before |
+
+RSS figures are one run on one machine and move with the Node build, the GC and
+the transform cache: the same cold process read 118 MB locally and 149 MB on a CI
+runner. Read the column as an order of magnitude, not a budget.
+`test/tokenizer-lazy.test.ts` holds all four rows, one child process per row, and
+asserts the MODULE LOAD rather than the megabytes, which is exact and the same
+everywhere.
+
+This is why `mgmt_list_toolsets` still reports a chars/4 estimate rather than a
+real count: it is in `core`, so counting for real would pull those 65 MB back
+into exactly the sessions this relieved.
 
 Any session can call `mgmt_list_toolsets` (it is in `core`) for each group's tool
 count, approximate token cost and reconnect URL; the same catalogue is one line
@@ -201,15 +236,46 @@ this section records only what is specific to the management plane.
 Nothing here logs a credential: the log field set is a closed allowlist, and the
 UAuth bearer, the shim JWT, a TOTP and a `confirmToken` are all outside it.
 
+**KNOWN GAP: tool calls on this plane are not counted.** `instrumentToolCalls`
+patches `registerTool` once, before the tools register, and it is applied only in
+`createServer` (`src/server.ts`) — the raw-key data plane. The management server
+never instruments, so `mcp_ankr_tool_calls_total` and
+`mcp_ankr_tool_call_duration_seconds` carry nothing from `/mcp`.
+
+That was invisible while the two planes served disjoint surfaces. It stopped
+being invisible in SHARK-3629, which put the sixteen chain reads on this plane
+too: the same tool name is now counted when it is served from `/rpc` and not
+counted when it is served from `/mcp`, so a per-tool rate read off these metrics
+UNDERSTATES real usage by whatever share the management endpoint carries, and
+does so silently. Read those two families as "data plane only" until this is
+closed.
+
+Closing it is a one-line application of the same helper to the management
+server's raw McpServer, before `registerMgmtTools` runs; it is left out of
+SHARK-3629 because widening a metric's coverage changes what every existing
+dashboard and alert on those names means, and that is a decision for whoever owns
+them rather than a merge artefact.
+
 ## Tools (PoC)
 
-**77 tools are registered** on the management server (`?toolsets=all`; 75 before
-SHARK-3600 added `mgmt_list_toolsets` to `core` and SHARK-3609 added
-`mgmt_load_toolset` beside it), of which **32 are HITL-gated**.
-Both counts are held by `test/mgmt-annotations.test.ts`, which asserts the
-classified sets partition the registered surface exactly, so a new tool cannot
-land unclassified; `test/helpers/mgmtToolSurface.ts` is where the 77 are written
-out by name. The bullets below are the operationally interesting families, not
+**94 tools are registered** on the management server (`?toolsets=all`; 75 before
+SHARK-3600 added `mgmt_list_toolsets` to `core`, SHARK-3609 added
+`mgmt_load_toolset` beside it and SHARK-3629 added the `data` group's sixteen
+chain reads plus `mgmt_select_key`), of which **32 are HITL-gated**.
+The HITL count and the classification of the **78 management** tools are held by
+`test/mgmt-annotations.test.ts`, which asserts the classified sets partition the
+MANAGEMENT surface exactly, so a new tool cannot land unclassified. The **16**
+chain-read tools are governed by `test/annotations.test.ts` instead — the whole
+chain plane is read-only and open-world, which is the opposite of what the
+management rules assume — and the boundary between the two is `DATA_TOOL_NAMES`
+in `src/server.ts`, pinned against the live `/rpc` surface by
+`test/data-tool-surface.test.ts`.
+
+Note the two counts do not split the `data` GROUP down the middle by accident:
+the group has seventeen members, and the seventeenth is `mgmt_select_key`, which
+travels with the chain tools but acts on the session and stays in the management
+partition (`isDataToolName("mgmt_select_key") === false`, asserted). `test/helpers/mgmtToolSurface.ts` is where all
+94 are written out by name, split into their groups. The bullets below are the operationally interesting families, not
 the inventory; `tools/list` on a live pod is.
 
 - `mgmt_get_usage` (SHARK-3375) — read-only; `GET /auth/intervalUsage`.