diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4081265..006a296 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -14,7 +14,7 @@ jobs: strategy: fail-fast: false matrix: - paseo: ['0.11.0', '0.11.1', 'latest'] + paseo: ['0.11.0', '0.11.1', '0.12.0', 'latest'] name: Paseo SDK ${{ matrix.paseo }} steps: - uses: actions/checkout@v4 diff --git a/.gitignore b/.gitignore index 0ee1ac4..3b7d684 100644 --- a/.gitignore +++ b/.gitignore @@ -11,3 +11,5 @@ DEVELOPMENT_PLAN.md AGENTS.local.md # local Paseo agent settings (per-user) paseo.json +# private research notes (local paths, provider names, ledger data) +docs/private/ diff --git a/AGENTS.md b/AGENTS.md index aebf8c6..65e364a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,16 @@ # AGENTS.md — 开发调试速查 -TokenLedger 最低支持 Paseo v0.11.0，不设版本上限（id `token-ledger`，SDK 开发依赖固定 0.11.1；已验证宿主 0.11.1）。0.9.1–0.10.x 用户留在 v0.6.3。允许加载不代表所有未来版本均已实测；兼容策略与验证记录见 `docs/compatibility.md`。**若存在 `AGENTS.local.md`（gitignore，本机专用），先读它——本机路径、工具链怪癖、测试 provider、账号事项都在那里，其内容优先于本文件。** +TokenLedger 最低支持 Paseo v0.11.0，不设版本上限（id `token-ledger`，SDK 开发依赖固定 0.11.1；已验证宿主 0.11.1、0.12.0）。0.9.1–0.10.x 用户留在 v0.6.3。允许加载不代表所有未来版本均已实测；兼容策略与验证记录见 `docs/compatibility.md`。**若存在 `AGENTS.local.md`（gitignore，本机专用），先读它——本机路径、工具链怪癖、测试 provider、账号事项都在那里，其内容优先于本文件。** + +## UI 设计理念 + +Avoid unnecessary secondary labels and descriptions. Let the layout do the explaining. + +## 隐私红线 + +- **禁止发布 Stephen 私人网关的任何信息或数据**：网关名称、provider/model 名（含本机 provider 配置名）、价格表、汇率、额度、账号、端点，以及真实账本数据、agent/会话 ID、本机路径。范围包括源码、测试、文档、截图、commit message、PR/issue/Release 文本和 npm 包。 +- **新功能不能依赖私人网关**：只针对公开 provider/harness 的已验证语义设计；不内置私有价格种子或固定汇率。测试和文档用合成数据（如 `acme-web`、`openai/gpt-5`）。 +- 私人网关只用于本机实测，相关事项写在 `AGENTS.local.md`；含真实数据的调研笔记放 `docs/private/`（已 gitignore）。提交前 `git grep -i` 检查网关名和本机路径，发版前用 `npm pack --dry-run` 核对包内容。 ## 开发循环 @@ -29,8 +39,8 @@ paseo agent archive # 测完归档，别 - 服务端 contribute 拿到 `handle/on/before`，`paseo` 在 RPC 和生命周期回调中提供。`before(agent.session_open)` 启动订阅，`on(agent.turn_started)` 覆盖 reload 后已有会话；客户端 ensure 尽早连接已有会话。**新建 agent 在 session_open 时尚未进入目录，不能在此 refresh 新 agent。** - **wire 层 `agent_stream` 没有 `usage_updated`**：轮中 usage 走 `agent_update` upsert 快照（`lastUsage`/`activeTurn`），且必须先 `paseo.agents.list({ subscribe: {} })` 才会推送。turn 生命周期（started/completed/failed/canceled + turnId + 轮末 usage）走 `agents.ref(id).timeline.subscribe`。 - usage 语义（2026-09-04 实测）：Claude 轮末 usage 是**逐轮**汇总、`totalCostUsd` 是**会话累计**（所以记录里存 delta + raw 两份）；Codex 每次模型请求报一次 `last` 值，多请求轮靠去重求和。快照会把上一轮旧 usage 回放进新一轮——tracker 用 agent 级 `lastObservation` 基线挡掉。 -- **Paseo 丢弃 `cache_creation_input_tokens`**（2026-09-05 从 app.asar 的 `providers/claude/agent.js` `buildResultUsage` ~1424 行验证）：`cachedInputTokens` 只映射 `cache_read_input_tokens`，cache write 只用于上下文窗口内部估算、不进 `AgentUsage`。后果：用户输入和工具结果（几乎全是 cache write，计费 1.25× input 价）在 IN 和 CACHE 两列都不可见，IN 通常只有零头（个位数~几十）；`costUsd` 仍准确（上游总额含 write）。pricing.ts 的分项估算按牌价直算，总额与估算的差额单列为 `otherUsd`（COST 列下方显示 `+$x.xx`）——不再 rescale 摊进分项。token 数本身的缺口需上游 Paseo 加字段，插件侧无法拿到。 -- **input 语义因 provider 而异**（2026-09-05 从落库记录实测）：codex/OpenAI 系的 `inputTokens` **含** cached（如 input 213818 / cached 213252，真实新增仅 566）；Claude 的 input **不含** cache read。`semantics.ts` 维护按 provider（回退 model）匹配的语义表，归一化到"input = 未命中缓存的新输入"（Anthropic 式，信息不丢失）。磁盘 ledger.jsonl 永远存上游原始值，归一化只在读取/服务层做（store 的 summary、tracker 的 handleSync 和 inFlightFor），历史记录自动被正确重解读。新接 provider：跑一轮真实 turn → 看落库记录判断 input 是否含 cached → 往 SEMANTICS_TABLE 加一行 + 测试。未知 provider 原样透传，不猜。 +- **Paseo 0.11.2 仍丢弃 cache creation**：Claude `buildResultUsage` 只映射普通输入、cache read、输出与 SDK 报告费用。缺失不等于零；命中率必须有完整分母。`costUsd` 只是运行器报告费用，不保证等于网关实际扣款；分项估算残差不得反推 write token。 +- **输入口径只按已验证 harness，不按模型回退**：Codex input 包含 read/write；Claude 和 OMP input 已与缓存分开。`usageSemantics(provider)` 不接受 model。RPC 的 IN 是减掉 read 和已知 write 的未缓存输入，write 缺失时 Codex IN 仍可能含未分类写入；未知 provider 原样显示并排除轮次合计。磁盘原始值不重写。 - **turnId 不是全局唯一的**（2026-09-07 从 ledger.jsonl 实测）：Paseo 的 turnId 形如 `foreground-turn-N`，agent/daemon 会话重启后从 1 重新计数。任何以 `agentId:turnId` 当唯一键的地方都会跨会话冲突——曾导致面板 React key 重复、渲染出幽灵重复行。现在记录 id 拼入 `endedAt`（aggregate.ts），面板行 key 用 `seq`（兼容旧格式的历史记录）。新代码需要唯一键时同样别只用 turnId。 - **终止事件后快照可能回放 activeTurn**（同日实测，曾造成 3ms 内同一 turn 落库两条）：turn_completed 之后紧跟的 agent_update 快照仍带着刚结束的 turn，noteUsage 若据此重开 open turn，下个终止事件就会重复落库。tracker 用 `lastClosedTurnId` 挡掉（新轮保留旧 ID 以屏蔽迟到事件；provider session 确认改变时重置）。改 turn 生命周期逻辑时注意 timeline 事件流和快照流是两条异步通道，顺序无保证。 @@ -48,8 +58,8 @@ paseo agent archive # 测完归档，别 - `server/journal.ts` 保存 open turn、session/cost 基线及 pending records；先写 journal 再 append JSONL，open.key 是稳定 UUID。结束 hook 无 usage，短暂等待 timeline 终值后按已观测数据结算；清理时完成已知的结束 hook。 - `server/read-model.ts` 按 store/pricing revision 缓存归一化行与汇总；`client/data.ts` 共享 panel/pill 查询并合并未变化的 records。RPC 的空 records 配合相同 recordsRevision 表示未变化，不表示历史清空。 - `server/catalog.ts` 复用完整 agent 目录并订阅分页 workspace 目录；总览不再每次查询只取前 200 个 agent。 -- `providerSemantics` 只按已验证 harness 匹配：Claude whole-turn + session cost；Codex request observations；未知取最新观察且标 partial，原始费用保留但不推断累计差值。model fallback 仅用于 input/cache 口径。 -- 新 Codex 记录的 requests 保留原始逐请求 token，用于逐请求价格档位；旧记录没有请求明细，只能近似。pricing.json 最后档必须无上限，档位递增；运行时重新读取无效文件会保留上一份有效价格。 +- `providerSemantics` 区分 Claude turn、Codex request、OMP session/branch 和 unknown；`tokenScope` 在读取旧账本时按已验证 harness 补足解释。session/unknown 只作为快照显示，不做轮差、不计入轮次或日期范围合计、不估算单轮费用。`ledger.sync.usageSnapshot` 保留迟到的闭轮/idle 观察，不能据此重开一轮；无 epoch/终值归属时不猜测真实轮增量。 +- Codex `requests` 用于逐请求价格档位；缺字段时整个对应分项保持未知，原请求仍保留。价格匹配保留完整 ID，标准 Claude 纯数字版本仅兼容点号/连字符，其他差异使用显式 aliases。旧的 namespace/context/provider 折叠只用于提示迁移，不能选价；可能的旧覆盖价失配或多义匹配阻止参考价回退，并通过 pricingWarning/pricingWarningTurns 在 UI 提示。provider scope 仅匹配 provider，配置读回后自动清除已解决的提示。用户文件不重写，无内部价格种子或固定汇率。 - Timeline 使用 append + addTimelineRenderer：先落盘、每轮一次、稳定 ID、最多重试一次。不要高频更新或全历史回填；append 会改变 agent.updatedAt，且 plugin rows 不保证 daemon 重启后保留。JSONL 是权威账本。 ## 0.9.1 适配约束 @@ -64,17 +74,25 @@ paseo agent archive # 测完归档，别 ## 0.11 适配约束 - **发布代码禁止 import `@getpaseo/client` / `@getpaseo/protocol`**（含 type-only）。宿主编译插件时会解析 type-only import，但只提供 SDK specifier、zod、react、react-native、@tanstack/react-query、@types/node。git 安装不跑 `npm install`（只跑 manifest `build`），npm 安装不装 devDependencies，所以其他类型依赖会让安装失败（0.11.1 实测 `Could not resolve type dependency`）。客户端类型从 `client/paseo-types.ts` 取，服务端从 `server/paseo-types.ts` 取（都由宿主 `PaseoApi` 推导）。`shared/` 不能 import `/client` 或 `/server` SDK，用结构类型和泛型。测试文件不进 bundle，可以继续用 devDependencies。 -- 改动依赖或 import 后，用无 `node_modules` 的副本验证：复制源码到临时目录、改 manifest id、让 contribute 提前返回，`paseo plugin add <目录>`，看到 "Plugin ready" 后 `paseo plugin remove` 并删除目录。 +- 改动依赖或 import 后，用无 `node_modules` 的副本验证：复制源码到临时目录、改 manifest id、让两端 contribute 提前返回 `() => {}`（宿主要求 cleanup 函数，裸 return 会被拒绝），`paseo plugin add <目录>`，看到 "Plugin ready" 后 `paseo plugin remove` 并删除目录。 - 总览是 `addScreen`（id `ledger-overview` 不变），`range` 放在 screen params 里（`today`/`7d`/`30d`/`all`，缺省 all）作为**初始**范围；屏内切换是组件 state。screen props 没有 setParams，`client.openScreen` 是宿主 router.push——每调一次叠一条历史，所以屏内切换不能走导航（曾导致点 range 按钮刷屏历史记录）；标题用静态字符串，params 派生标题在本地切换后会过期。 - 侧边栏是 `addSidebarFooterItem` + `SidebarRow`，trailing 显示今日费用。"今天"按客户端本地午夜计算，以 ISO `since` 传给 `ledger.overview`；服务端按 `Date.parse(endedAt)` 过滤 read-model 行，不在服务端猜时区。 - `addSurface` / `addSidebarItem` / `openSurface` 已弃用，不要再用。不接 `registerUsageSource`（那是订阅额度窗口，不是逐轮 usage；用户已决定不放进 Usage 面板）。 -- 0.11.1 仍未公开 cache-write 和 provider 内部子代理 usage。 +- OMP 轮末 drain（0.11.2 源码验证）：`turn_completed` 不带 usage，先于 poller 的最终 `usage_updated` 发出；最终快照仍带 activeTurn，且 usage 未变化时不发。所以 session scope 的 completed 轮先记 `open.ending`（随 journal 恢复），在第一个新观察、下一轮开始或 2s 超时时结算，状态和 endedAt 取 ending。失败/取消立即结算。session/unknown 的 `modelCalls` 为 null（快照数不是调用数）。 +- 0.11.2 的 cache-write、Codex 内部子代理 usage、OMP branch epoch 缺口仍在。OMP 统计已包含部分子代理，不能无条件再叠加。徽标使用 Session token snapshot，不以 unknown 代替已知快照；未知 scope 不求 token 总数。历史 Timeline 缺少 scope 元数据时只声明元数据缺失，不声明账本排除。更新后重载客户端以使用新字段。 + +## 0.12 适配约束 + +- **background workspace 默认被过滤**：0.12 的 `agents.list` / `workspaces.list` 及其 update 流不带 `filter.includeBackground` 时排除 background workspace，被过滤的 agent 以 `remove` 推送。MCP 新建 workspace 会继承调用方的 background，子 agent 常在其中。所有目录 list 都经 `shared/background.ts` 的 `includingBackground`；同一订阅的翻页必须沿用 open 时的 filter。0.12+ 客户端 SDK 连旧 daemon 会抛 "Update the host to use background workspaces."，回退为无 filter，不抬高最低版本。 +- **`remove` 不等于删除**：tracker 收到 update 流的 `remove` 后先 `agents.ref(id).refresh()` 确认（按 ID 不受 background 过滤），agent 仍在则保留 open turn，确认消失才 retire。 +- CLI 的 `paseo agent archive ` 找不到 background agent；测试后用 `paseo workspace archive` 和 `paseo project delete` 清理。 +- 0.12 编译器允许 type-only import `@getpaseo/protocol/*` 子路径，但最低版本仍是 0.11，上面的 import 禁令不变。0.13 将弃用 `/client/react-native`（改用 `/client/ui`），届时再迁移。 ## 截图 发布用截图不能出现真实 agent 标题、workspace 或项目名。用独立 demo daemon 和合成数据： -- 新建 home 目录，config 只写 `daemon.listen`（换端口）、`relay.enabled: false`、`cors.allowedOrigins`（静态服务的 origin）、`pluginsEnabled: true`。**不要**复制 `~/.paseo/config.json`（含 provider 密钥）。`paseo daemon run --home ` 前台运行，用 `paseo daemon stop --home ` 只停它，不碰主 daemon。 +- 新建 home 目录，config 只写 `daemon.listen`（换端口）、`daemon.relay.enabled: false`、`daemon.cors.allowedOrigins`（静态服务 origin）、`pluginsEnabled: true`。0.11.2 的 relay/cors 在 daemon 下，不在根级。**不要**复制 `~/.paseo/config.json`（含 provider 密钥）。`paseo daemon run --home ` 前台运行，用 `paseo daemon stop --home ` 只停它，不碰主 daemon。演示 OMP 需在隔离配置中启用，并用 provider.env.PI_CODING_AGENT_DIR 指向临时目录，避免继承用户的 disabledProviders。 - `paseo project create --home `、`paseo workspace create --path --isolation local --title --home ` 建项目和 workspace。agent 记录直接写 `agents//.json`（daemon 启动时读），账本写 `plugins/token-ledger/ledger.jsonl`，然后 `paseo plugin add <仓库目录> --home `。 - Web 客户端在桌面安装目录的 `resources/app-dist`。静态服务时往 index.html 注入 `globalThis.__PASEO_INITIAL_DAEMON_CONNECTION__={listen:"127.0.0.1:",useTls:false}`，用 puppeteer-core 截图。总览可直接打开 `/h//plugin/token-ledger/surface/ledger-overview?param.range=7d`。 - 截完停 demo daemon，删除 demo 目录。 diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ab508e..5942a7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,22 @@ # Changelog -## Unreleased +## v0.8.0 (2026-10-23) — Paseo 0.12 background workspaces and accounting accuracy +Requires Paseo **0.11.0 or newer**; verified on 0.11.1 and 0.12.0. Existing ledger records are reinterpreted on read, not rewritten; existing custom price files are kept. Reload or reopen the app after updating. See the [release notes](docs/releases/v0.8.0.md). + +- Switch overview ranges inside the screen instead of pushing a navigation entry per click; the screen title is now fixed. Thanks to @ChtJT (#4, #5). +- Track agents in Paseo 0.12 background workspaces. Paseo 0.12 hides them from default agent and workspace lists, so their turns were closed as canceled with no usage and their workspace names were missing. Catalog lists now request background workspaces and fall back to unfiltered lists on hosts that predate them. A catalog removal is confirmed by agent ID before the agent's open turn is settled. - Add an overview screenshot (demo data, with the sidebar footer row) to the README and as the first manifest media image. +- Fix harness input semantics: OMP exclusive input is no longer reduced again by a GPT-model fallback; unverified providers are never inferred from model names. +- Separate per-turn usage from OMP current-session/branch totals and unknown-scope snapshots, including historical v1 records. Snapshots remain visible but are excluded from turn/date-range sums and estimates. Late idle usage updates refresh a separate latest-snapshot card, not an invented turn delta. +- Preserve unknown counts and cache rates rather than displaying zero. Only show a cache hit rate with a complete prompt denominator; keep partial request fields incomplete. +- Disclose incomplete usage, partial reference estimates and unknown-cost turns in the panel, pill, sidebar and overview. Runtime-reported amounts are not verified bills. Keep cost cells within narrow panel widths. +- Remove private gateway seed prices and fixed currency conversion. Start new price files empty without rewriting existing custom files. Match full model identities and explicit aliases; provider scopes no longer match model text, and ambiguous overrides are unpriced. +- Do not guess cache-read/write rates from ordinary-input rates, select tiers with an unknown prompt size, or interpret a falling session-cost counter as proof of a new session. +- Follow up upstream with public synthetic evidence for cache writes, request/branch/descendant attribution and OMP final-drain ordering; see [usage API requirements](docs/paseo-usage-api.md). Missing upstream data is not restored by this change. +- Preserve standard Claude dotted/hyphenated numeric-version aliases for custom prices. Block reference-price fallback and surface a pricing.json migration warning for ambiguous entries or matches that previously depended on unsafe namespace/context/provider folding; valid explicit aliases clear it without rewriting the file. +- Distinguish session token snapshots, unknown-scope snapshots, waiting for usage and no reported usage in the composer pill. Live per-turn token observations remain visible even without a price. +- Label old Timeline cards as missing scope metadata rather than falsely claiming they were excluded from ledger totals. Refresh the support matrix and screenshots with isolated synthetic data. ## v0.7.0 (2026-10-09) — Paseo 0.11 screens and sidebar diff --git a/OVERVIEW.md b/OVERVIEW.md index 5a13647..479cb3c 100644 --- a/OVERVIEW.md +++ b/OVERVIEW.md @@ -2,7 +2,9 @@ TokenLedger records the tokens and cost of each agent turn and keeps the history The agent panel shows session totals, the turn in progress with its context window, and one row per finished turn with status, time, duration, input, cache and output tokens, cost, and a data quality mark. The pill shows the session cost and context use, with an activity icon while a turn runs, and opens the panel. The sidebar row opens the overview, which groups agents by workspace and filters to today, the last 7 days, the last 30 days, or all time. Day ranges start at local midnight on the device that shows them. After each finished turn, the plugin can add one short usage summary to the timeline. -Token counts come from the provider and are never invented. Claude reports a total for each turn, which is marked exact. Codex reports each model request, so the plugin adds the requests and marks the total as approximate. Other providers show the latest report, marked partial. When the provider reports a cost, that cost is shown. Otherwise the plugin estimates the cost, marked with ≈, from your prices in `pricing.json`, then OpenRouter reference prices, then a small built-in price table. +Token counts come from the provider. Claude terminal aggregates and observed Codex requests contribute to turn subtotals. OMP current-session/branch totals and unverified-harness observations remain visible as snapshots but are excluded from turn sums and prices. A separate latest-snapshot card retains late updates without inventing turn attribution. Missing counts/rates display as unknown; cache hit rates require a complete prompt denominator. Partial estimates and unknown-cost turns are explicit in the panel, pill and overview. Runtime-reported costs take priority over your reference prices, OpenRouter references and built-in prices, but are not verified bills. + +The pill distinguishes known session snapshots (`Session ≈4.12M tok`), unknown-scope snapshots, waiting for usage and no reported usage. Old timeline cards without scope metadata are labelled as legacy; their ledger inclusion is not guessed from the card. The plugin settings page on each host has four options: @@ -11,11 +13,14 @@ The plugin settings page on each host has four options: - **Built-in fallback prices** controls estimates when no custom or OpenRouter price matches a model. - **Refresh frequency** selects normal or relaxed polling. Live updates still refresh the views immediately. -The ledger is a JSON Lines file in the plugin data folder on the daemon host, `~/.paseo/plugins/token-ledger/` (or the same path under `PASEO_HOME`). It keeps the last 2,000 turns. Each record holds timestamps, agent ID, provider, model, token counts, cost, status, duration, and quality. Prompts, responses, tool arguments, file paths, and credentials are not recorded. A `pricing.json` file in the same folder holds your own prices per million tokens. History does not sync between hosts. +The ledger is a JSON Lines file on the daemon host, `~/.paseo/plugins/token-ledger/` (or under `PASEO_HOME`), retaining 2,000 records. It stores observed usage, timestamps, identity, scope and quality, not prompts, responses, tool arguments or credentials. Older records remain readable and are reinterpreted without rewriting them. `pricing.json` starts empty and preserves your existing prices. Full model IDs and explicit aliases prevent unrelated routes from sharing a guessed price. History does not sync between hosts. + +Standard Claude numeric-version spellings such as `5.5` and `5-5` retain custom-price compatibility. Other old lossy matches or ambiguous entries show a pricing configuration warning and block reference-price fallback until an explicit alias/provider scope is reviewed. Existing files are not rewritten, and valid edits clear the warning automatically. Known limits: -- Paseo does not report cache write tokens, so Claude cache writes are missing from the token columns. The reported cost stays correct, and the difference shows as a residual under the cost. -- Paseo does not report the usage of subagents that run inside a provider, so totals can be lower than the provider bill. -- Usage during a time when the plugin or daemon is offline cannot be recovered. Affected turns are marked as a gap or as unknown. -- Estimated costs use reference prices and can differ from your actual contract. +- Missing cache writes are not zero; setting prices cannot recover omitted token counts. A reported-cost residual does not prove a cache-write charge. +- Child coverage differs by harness. Codex child usage can be omitted; OMP already includes some child usage. Blindly adding children can double-count. +- Session snapshots cannot safely become per-turn deltas without branch epochs and final usage attribution; they are not included in date-range totals. +- Offline observations and missing request identities cannot be reconstructed. Affected usage stays incomplete or unknown. +- Runtime and reference prices can differ from actual contracts. No private gateway rate table or fixed exchange rate is built in. diff --git a/README.md b/README.md index 59fed1c..00027a2 100644 --- a/README.md +++ b/README.md @@ -13,39 +13,41 @@ Per-turn LLM token usage and cost for [Paseo](https://paseo.sh) agents — an ag

- TokenLedger per-agent panel: session summary (turns, input/cache/output tokens, cost), context-window bar, and a per-turn history table + TokenLedger demo panel: session snapshot shown separately from unknown per-turn totals

## What it shows **Per-agent panel** (workspace/explorer, bound to one session — the header names the agent and model it is tracking): -- **Session summary** — total turns, input / cache / output tokens, and reported or estimated cost. Estimated totals use `≈`; missing-price turns are counted explicitly. -- **Turn in progress** — elapsed time, running token counts and estimated cost when available, and a context-window bar. When idle, the last known context-window usage is still shown. -- **Turn history** — one row per finished turn: status, time, duration, tokens, cost, and a data-quality mark. +- **Session summary** — known per-turn input / cache / output subtotals and runtime-reported or reference cost. Missing counts stay `–`; partial estimates and unknown-cost turns are counted explicitly. +- **Turn in progress** — elapsed time, observed token counts and cost when attributable to the turn, plus a context-window bar. Idle context usage remains visible. +- **Turn history** — one row per finished turn with accounting scope and coverage. OMP session/branch totals and unverified-harness observations are snapshots, not turn usage, and are excluded from totals. A separate latest-snapshot card also reflects late idle updates. -**Composer pill** — every live agent gets a compact pill next to its composer (`$0.43 · ctx 47%`, with an activity icon while a turn is running). The total includes the current turn's estimated cost when available and uses `≈` whenever estimates contribute. Pressing it opens that agent's panel. With split views, each session carries its own pill, so the session↔usage binding is always visible. +**Composer pill** — shows attributable current-turn totals when available. For OMP, `Session ≈4.12M tok` means the known tokens in the latest session/branch snapshot, not a per-turn total; `≈` marks missing components. Unknown-scope snapshots are named without summing potentially overlapping counts. No data shows **Waiting for usage** while running or **No usage reported** when idle. Reference estimates, partial costs and pricing configuration warnings remain explicit. Pressing the pill opens that agent's panel.

- TokenLedger composer pill showing cost and context usage next to the composer; pressing it opens the panel + TokenLedger demo pill showing a session token snapshot and context usage

**Sidebar** — a **TokenLedger** row in the sidebar footer shows today's cost (or tokens when no price is known), with an activity icon while a turn is running. "Today" starts at local midnight on the device that shows it. -**Overview** (sidebar footer → TokenLedger, or the command center) — all sessions in one place: grand totals, then per-agent rows grouped by workspace with live-turn indicator, last activity, cost/tokens, and turn count. Pick **Today**, **Last 7 days**, **Last 30 days**, or **All time**. The sidebar row opens it on Today; the command center opens All time. A range lists agents with turns that ended in it, plus agents with a turn in flight. Tapping a row jumps to that agent. +**Overview** (sidebar footer → TokenLedger, or the command center) — all sessions in one place: grand totals, then per-agent rows grouped by workspace with live-turn indicator, last activity, cost/tokens, and turn count. Pick **Today**, **Last 7 days**, **Last 30 days**, or **All time**. The sidebar row opens it on Today; the command center opens All time. Switching ranges stays on the same screen. A range lists agents with turns that ended in it, plus agents with a turn in flight. Tapping a row jumps to that agent.

- TokenLedger overview on Last 7 days with demo data: range buttons, totals for all sessions, and agent rows grouped by workspace; the TokenLedger row in the sidebar footer shows today's cost + TokenLedger Today overview with synthetic data, excluded snapshots and a custom-price migration warning

-**Timeline** — each newly finished turn gets one passive usage summary after its ledger record is saved. It uses the same normalized counts, cost source and quality marks as the panel. No polling or message replacement is involved. +**Timeline** — each newly finished turn gets one passive usage summary after its ledger record is saved. Older cards without scope metadata say **legacy scope metadata unavailable**; they do not claim exclusion from totals. The ledger remains authoritative. No polling or message replacement is involved. ## Compatibility -TokenLedger **v0.7.0** requires **Paseo 0.11.0 or newer**, on both the daemon and the app, without an upper version limit. It uses the screen and sidebar footer APIs that Paseo 0.11 added. The verified host is **0.11.1**; allowing newer versions is not a claim that every future release has been tested. SDK development dependencies are pinned to 0.11.1, with CI checks against the minimum, the pin, and the latest published SDK. See the [compatibility policy and verification notes](docs/compatibility.md). +TokenLedger **v0.8.0** requires **Paseo 0.11.0 or newer**, on both the daemon and the app, without an upper version limit. It uses the screen and sidebar footer APIs that Paseo 0.11 added. The verified hosts are **0.11.1** and **0.12.0**, including agents in Paseo 0.12 background workspaces; allowing newer versions is not a claim that every future release has been tested. SDK development dependencies are pinned to 0.11.1, with CI checks against the minimum, the pin, 0.12.0, and the latest published SDK. See the [compatibility policy and verification notes](docs/compatibility.md). Paseo 0.9.1–0.10.x users should stay on `v0.6.3`; Paseo 0.8 users on `v0.5.1`; Paseo 0.7 users on `v0.3.1`. Existing `ledger.jsonl` records remain readable; no migration or deletion is required. See the [v0.6.0 release notes](docs/releases/v0.6.0.md) for the 0.9.1 migration details. +The v0.8.0 accounting fixes were also exercised on Paseo 0.11.2. After updating, reload/reopen the client so it uses the new nullable summary fields, scope metadata and pricing warnings. Existing JSONL records are reinterpreted on read, not rewritten. The screenshots above use synthetic data on an isolated host. + ## Install Install from the [Paseo plugin registry](https://paseo.sh/plugins/stv1024/token-ledger) (Paseo 0.11.0 or newer): @@ -65,7 +67,7 @@ Or paste either source into **Settings → Plugins → Plugin source**. To install a tagged release from GitHub instead: ```bash -paseo plugin add github:stv1024/token-ledger --ref v0.7.0 +paseo plugin add github:stv1024/token-ledger --ref v0.8.0 ``` Since Paseo 0.11, a bare `owner/name` source means the official plugin registry, which rejects `--ref`. Keep the `github:` prefix for Git installs. On Paseo 0.9.1–0.10.x, use `paseo plugin add stv1024/token-ledger --ref v0.6.3`. @@ -102,20 +104,23 @@ All three switches default to on, and refresh frequency defaults to normal. A "turn" is one user message through to `turn_completed`, `turn_failed`, or `turn_canceled` — potentially spanning many model calls. Token counts always come from the provider; TokenLedger never invents missing usage: -| Situation | Value used | Quality | +| Situation | Value used | Accounting scope | | --- | --- | --- | -| Provider reports a whole-turn aggregate at turn end (Claude) | used as-is | `exact` | -| Provider reports per-request usage during the turn (Codex) | distinct observations summed | `partial` (shown as `≈`) | -| Only a single mid-turn observation | that observation | `partial` | -| Nothing reported | nothing shown | `unavailable` | +| Whole-turn terminal aggregate (Claude) | reported counts | turn; coverage can still be incomplete | +| Per-request observations (Codex) | observed requests summed | turn; approximate without stable request IDs | +| Current-session/branch aggregate (OMP) | latest reported snapshot | session; excluded from per-turn totals and prices | +| Unverified harness | latest reported snapshot | unknown; excluded from per-turn totals and prices | +| Nothing reported | unknown, not zero | unavailable | + +Counting conventions are selected by verified harness/provider identity, never by the model name. OMP input is already exclusive of cache; Codex input is inclusive. “IN” is uncached input after subtracting cache reads and known writes; if writes are omitted, Codex IN may still include unclassified writes. Unknown provider values stay as reported, without guessed normalization. -Cost uses the first available source: upstream reported cost, local override, OpenRouter reference pricing, then the small built-in fallback table. Claude reports cumulative session cost, so its per-turn delta is attributed to the turn. For unverified harnesses, raw cost is retained in `sessionCostUsd` but is not treated as a known per-turn charge. Model names do not establish aggregation or cost scope. +For attributable turn costs, the order is runtime-reported cost, local override, OpenRouter reference, then built-in fallback. Claude session-cost deltas use the known baseline; a lower counter alone does not prove a session reset. A runtime's reported cost can itself be an estimate: it is not a verified invoice, and custom reference pricing does not replace it. Residuals do not identify missing tokens or billing categories. -Estimated costs are marked `≈`. OpenRouter prices are references and may differ from the actual route or contract. New Codex records retain observed request counts, so tiered prices are chosen separately for each request. Historical records without request detail use the aggregate prompt as an approximate fallback. +Estimated costs are marked `≈`; missing components/rates or uncertain request coverage add `partial`. Summaries expose unknown-cost turns rather than presenting the known subtotal as a complete bill. Codex request observations are priced separately; legacy records without requests use the aggregate as an explicitly partial estimate. A tiered price cannot be selected when the complete prompt size is unknown. Per-request routed model identities are not exposed, so reference pricing uses the recorded selected model, not a claim of the actual billed model. -Paseo 0.9.1 still does not expose cache-write token counts through its public usage API. TokenLedger can retain and price them separately when supplied, but the current Claude and Codex adapters omit them. Missing counts stay unknown; historical writes are not inferred. Claude's reported total can include charges absent from the token counts, producing a positive residual under COST. Codex input includes cache reads (and cache writes when supplied); the UI separates known cache counts while the ledger retains raw counts. +Paseo 0.11.2 still omits cache-write counts from the public Claude, Codex and OMP usage paths. Missing counts remain unknown; setting a price cannot recover them. Cache hit rate is read tokens divided by the complete prompt size: Codex's inclusive input supplies that denominator, while Claude/OMP need input + read + write. Without a complete denominator the panel shows `?`, not a near-100% rate. If input/cache counts are inconsistent, the plugin does not clamp them into a fabricated zero. -Provider-internal subagents are not ordinary Paseo catalog agents. The public plugin API does not expose their usage; the Codex adapter discards child usage notifications. Session and overview totals therefore **may omit descendant usage**. TokenLedger uses only public Paseo APIs; it does not inspect provider logs or gateway billing endpoints. The [upstream API requirements](docs/paseo-usage-api.md) describe the missing fields, lineage, stable request IDs, and accounting scopes needed to close these gaps without double billing. +Provider-internal subagents are not ordinary Paseo catalog agents. The Codex adapter discards child usage notifications; OMP session stats already include some task-child usage. Neither just traversing the catalog nor blindly adding children fixes accounting. OMP's branch-view totals can decrease, and its final usage can arrive after completion or be superseded by the next turn. Without epochs and turn attribution the plugin does not invent per-turn deltas: cumulative history is shown as snapshots and late observations refresh the separate latest-snapshot card. These snapshots do not enter date-range totals. TokenLedger uses only public Paseo APIs, not provider logs or gateway billing endpoints. See the [upstream API requirements](docs/paseo-usage-api.md). Identical consecutive usage snapshots cannot be distinguished from separate equal-size model calls, so Codex totals remain `partial`. Open-turn observations, session identity and billing baselines are checkpointed locally, and graceful plugin reloads preserve them. A crash can lose observations since the last checkpoint; events missed while the plugin was offline cannot be reconstructed. Interrupted recovered turns are marked as partial/unavailable rather than claiming an exact offline outcome. `exact` describes the reported turn aggregate, not proof that every descendant or billing category was included. @@ -125,7 +130,7 @@ After a subscription error or reconnect, the tracker refreshes the affected agen ### Pricing overrides -On first use, TokenLedger creates `~/.paseo/plugins/token-ledger/pricing.json` (or under `PASEO_HOME`). It is a USD-per-million-token table and has highest priority among estimates. Entries can be scoped by provider substring and can contain prompt-size tiers. Changes to the file are picked up on the next usage request after a 30-second recheck interval; a reload also applies them immediately: +On first use, TokenLedger creates an **empty** `~/.paseo/plugins/token-ledger/pricing.json` (or under `PASEO_HOME`). No internal gateway prices or currency conversion are shipped. Existing files are preserved. Overrides are USD per million tokens and have highest priority among estimates. Provider substring scopes match only the provider, not the model string. Full model IDs are matched case-insensitively, with one safe spelling equivalence: standard Claude numeric versions such as `claude-opus-5.5` and `claude-opus-5-5` match each other. Namespaces, context/mode suffixes, arbitrary model punctuation and model arrays are not collapsed. Add explicit `aliases` for other verified identities. Scoped matches take priority over global ones; ambiguous matches are unpriced. Valid edits take effect after the next 30-second recheck, or a reload: ```json { @@ -135,6 +140,7 @@ On first use, TokenLedger creates `~/.paseo/plugins/token-ledger/pricing.json` ( { "providers": ["my-provider"], "model": "my-model", + "aliases": ["my-provider/my-model"], "tiers": [ { "input": 1.5, "cacheRead": 0.15, "cacheWrite": 1.875, "output": 6 } ] @@ -143,11 +149,13 @@ On first use, TokenLedger creates `~/.paseo/plugins/token-ledger/pricing.json` ( } ``` -`cacheWrite` is an optional, independent USD/MTok rate; the example is illustrative, not a universal write multiplier. Old files remain valid: when a write count is known but its price is omitted, it uses the ordinary input rate as an estimate. An explicit zero is a free write rate. Missing write counts cannot be recovered by setting a price. Prompt-size tiers include ordinary input, cache reads, and known writes; reported total cost stays authoritative and is never increased by adding the estimated components again. +**Upgrading old custom prices:** if a model would only have matched through the old namespace/context/provider folding, TokenLedger blocks OpenRouter/built-in fallback and shows **price config**. Review that model's full ID and provider scope, then add a verified alias or correct the entry; the warning clears after the valid configuration is re-read. The plugin does not edit your file or silently replace an intended override with a reference rate. Runtime-reported costs remain visible, but their reference breakdown can still require review. + +`cacheRead` and `cacheWrite` are optional independent USD/MTok rates. Missing rates are unknown; an explicit zero is free. A measured zero token count costs zero even when its rate is unknown. Missing rates no longer fall back to ordinary-input pricing. The example's write multiplier is illustrative, not universal. Tiers use the full prompt count when available. Runtime-reported totals are never increased by adding the estimated components again. OpenRouter's public model catalog, including `input_cache_write` when provided, is cached locally for 24 hours in `openrouter-pricing.json`. Local/cache prices are served before network refresh completes. If refresh fails, the last cache remains active, with retries no more often than every five minutes while the plugin is in use. -Built-in Claude rates were checked against [Anthropic's pricing table](https://platform.claude.com/docs/en/about-claude/pricing) on 2026-09-23. Opus 5.5 standard rates are input $4, cache read $0.20, output $20 per million tokens; the cache-write reference is $5 for the 5-minute lifetime. Other lifetimes, Fast Mode, discounts and gateway contracts can differ. The public usage API does not identify cache lifetimes or the billed service tier. Unknown model versions and explicit Fast/Batch suffixes do not receive a guessed built-in price. Configure a matching override when needed; no new TokenRouter contract prices are assumed. +Built-in Claude rates were checked against [Anthropic's pricing table](https://platform.claude.com/docs/en/about-claude/pricing) on 2026-09-23. Opus 5.5 standard rates are input $4, cache read $0.20, output $20 per million tokens; the cache-write reference is $5 for the 5-minute lifetime. Other lifetimes, Fast Mode, discounts and gateway contracts can differ. The public usage API does not identify cache lifetimes or the billed service tier. Unknown model versions and explicit Fast/Batch suffixes do not receive a guessed built-in price. Configure a matching override when needed; private contract prices are never assumed. ## Data & privacy @@ -160,19 +168,20 @@ Built-in Claude rates were checked against [Anthropic's pricing table](https://p ## Limitations -- Paseo 0.9.1 does not expose cache-write token counts; Claude cache writes are invisible in token columns (reported cost stays accurate, the gap shows as a `+$x.xx` residual). -- Usage of provider-internal subagents is not exposed by Paseo, so session and overview totals may omit it. -- Codex totals are summed per request and marked `partial` (`≈`); estimated costs use reference prices and may differ from your actual contract. -- Events missed while the plugin or daemon was offline cannot be reconstructed; affected turns are marked `gap` / `unknown`. -- Desktop and mobile UI were verified less thoroughly than the daemon side. +- Paseo 0.11.2 omits cache-write counts on current Claude/Codex/OMP usage paths. Missing writes are unknown, and may make the cache-rate denominator unavailable. A reported-cost residual is not proof of missing token charges. +- Child coverage differs by harness: Codex child usage can be omitted, while OMP already includes some child usage. Blindly adding children can double-count. +- OMP session/branch snapshots have no reliable per-turn boundary. They stay separate from per-turn and date-range sums; unknown providers are not priced by guessing semantics from model names. +- Codex request observations and reference estimates remain partial. Runtime-reported costs are not verified invoices. +- Events missed offline cannot be reconstructed; affected turns retain gap/unknown status. Mobile-device testing is more limited than desktop/web verification. ## Support matrix | Provider | Tokens | Cost | | --- | --- | --- | -| Claude / claude-* variants | per-turn aggregate (`exact`) | reported when available; estimate fallback | -| Codex / codex-* variants | summed per-request (`partial`) | estimate fallback when absent | -| Other providers (OpenCode, Pi, ACP harnesses) | latest observation, `partial`; `unavailable` when silent | raw cost retained; estimate when model pricing matches | +| Claude / claude-* variants | per-turn terminal aggregate; missing categories stay incomplete | runtime-reported when attributable; reference fallback | +| Codex / codex-* variants | summed request observations (`partial`) | partial reference estimate with known prices | +| OMP / omp-* variants | latest session/branch snapshot, excluded from turn totals | raw session cost displayed separately; no guessed turn cost | +| Other providers (OpenCode, Pi, ACP harnesses) | latest unknown-scope snapshot; no guessed aggregation | raw cost retained separately; no turn estimate | ## Development diff --git a/client/panel.tsx b/client/panel.tsx index 072e07b..b02b38d 100644 --- a/client/panel.tsx +++ b/client/panel.tsx @@ -58,7 +58,7 @@ function CtxBar({ ctx, theme }: { ctx: Ctx; theme: PluginTheme }) { function InFlightCard({ inFlight, seq, theme }: { inFlight: InFlight; seq: number; theme: PluginTheme }) { const elapsed = useElapsed(inFlight.startedAt); const tokens = tokensLine(inFlight.input, inFlight.cached, inFlight.output, inFlight.cacheWrite); - const pct = fmtPct(cacheRatio(inFlight.input, inFlight.cached, inFlight.cacheWrite)); + const pct = fmtPct(cacheRatio(inFlight.cached, inFlight.promptTokens)); const ctx = inFlight.ctxUsed !== null && inFlight.ctxMax !== null ? { used: inFlight.ctxUsed, max: inFlight.ctxMax } : null; return ( {tokens} - {pct ? ` (${pct} cached)` : ""} - {inFlight.modelCalls > 0 ? ` · ${inFlight.modelCalls} call${inFlight.modelCalls > 1 ? "s" : ""}` : ""} + {pct ? ` (${pct} cached)` : ' (cache rate unknown)'} + {inFlight.modelCalls != null && inFlight.modelCalls > 0 ? ` · ${inFlight.modelCalls} observation${inFlight.modelCalls > 1 ? 's' : ''}` : ''} ) : ( tokens & cost reported at turn end )} + {inFlight.tokenScope !== 'turn' ? + {inFlight.tokenScope} snapshot, not current-turn usage. Excluded from totals. + : null} {inFlight.effectiveCostUsd !== null ? ( - ≈{fmtCost(inFlight.effectiveCostUsd)} current turn + {inFlight.costSource === 'reported' ? '' : '≈'}{fmtCost(inFlight.effectiveCostUsd)} {inFlight.partialCost ? 'partial estimate' : 'current turn'} ) : null} + {inFlight.pricingWarning ? {inFlight.pricingWarning} : null} {ctx ? : null} {inFlight.usageGap ? Connection interrupted · usage may be incomplete. : null} @@ -115,9 +119,9 @@ function TurnCell({ theme: PluginTheme; }) { return ( - + {top} - + {bottom} @@ -128,16 +132,16 @@ function RecordRow({ record, theme, dense }: { record: TurnRow; theme: PluginThe const cols = turnColumns(dense); const hasUsage = record.input !== null || record.cached !== null || record.output !== null || record.cacheWrite != null; const split = record.costBreakdown; - const pct = fmtPct(cacheRatio(record.input, record.cached, record.cacheWrite)); + const pct = fmtPct(cacheRatio(record.cached, record.promptTokens)); const cacheCost = split ? fmtCostSmall(split.cacheUsd) : "–"; - const costDetail = + const costDetail = record.pricingWarning ? 'price config' : record.costSource === "override" ? "override" : record.costSource === "openrouter" ? "OR ref" : record.costSource === "builtin" ? "list" - : fmtResidual(split?.otherUsd ?? null); + : record.tokenScope !== 'turn' ? 'not turn' : `runtime ${fmtResidual(split?.otherUsd ?? null).trim()}`; const seqWidth = dense ? 20 : 24; return ( {fmtDuration(record.durationMs)} - {record.usageGap ? ' · gap' : record.quality === "partial" ? " ≈" : ""} + {record.tokenScope !== 'turn' ? ` · ${record.tokenScope} snapshot` : record.usageGap ? ' · gap' : record.incompleteUsage ? ' · incomplete' : ''} {hasUsage ? ( @@ -187,14 +191,12 @@ function RecordRow({ record, theme, dense }: { record: TurnRow; theme: PluginThe - {record.cacheWrite != null ? ( - - ) : null} + @@ -229,6 +231,7 @@ function RecordRow({ record, theme, dense }: { record: TurnRow; theme: PluginThe export function TokenLedgerPanel({ theme, layout, agentId }: PluginAgentPanelProps) { const { dense, setWidth } = useDenseLayout(layout.compact); const { data, error } = useLedger(agentId); + const pricingWarnings = [...new Set(data?.records.flatMap((record) => record.pricingWarning ? [record.pricingWarning] : []) ?? [])]; // Make the panel↔agent binding visible: which session is this ledger for? const agentLabel = useAgent(agentId, (agent) => { const name = agent.title ?? agentId.slice(0, 8); @@ -269,8 +272,24 @@ export function TokenLedgerPanel({ theme, layout, agentId }: PluginAgentPanelPro + {pricingWarnings.map((warning) => {warning})} {!data.inFlight && data.ctx ? : null} + {data.usageSnapshot ? + + Latest {data.usageSnapshot.scope === 'session' ? 'session / branch' : 'unknown-scope'} snapshot + + + {tokensLine(data.usageSnapshot.input, data.usageSnapshot.cached, data.usageSnapshot.output, data.usageSnapshot.cacheWrite)} + + {data.usageSnapshot.reportedCostUsd !== null ? + Runtime-reported cost: {fmtCost(data.usageSnapshot.reportedCostUsd)} · not a verified bill + : null} + + Not per-turn usage; excluded from totals. Late updates refresh this snapshot, not closed turns. + {data.usageSnapshot.observedAt ? ` Observed ${fmtTime(data.usageSnapshot.observedAt)}.` : ''} + + : null} {data.inFlight ? : null} @@ -286,10 +305,13 @@ export function TokenLedgerPanel({ theme, layout, agentId }: PluginAgentPanelPro // restarts (turnId reuse), which made React render ghost rows. ))} + + IN is uncached input; it may include unclassified writes. “?” means the cache-rate denominator is unknown. + Session/unknown snapshots are observations at settlement, not final turn totals. + {data.records.some((record) => fmtResidual(record.costBreakdown?.otherUsd ?? null).trim() !== "") ? ( - ± under COST: difference between reported cost and the token-based estimate, including charges - for usage Paseo does not expose. + ± under COST: runtime-reported cost minus known component estimates. A residual does not identify missing tokens or actual charges. ) : null} {data.records.some((record) => record.costSource && record.costSource !== "reported") ? ( diff --git a/client/pill.tsx b/client/pill.tsx index efc785d..d761ccb 100644 --- a/client/pill.tsx +++ b/client/pill.tsx @@ -5,23 +5,47 @@ import { useLedger } from "./data.ts"; import { useEffect } from "react"; import { subscribeAgents } from "../shared/agents.ts"; import { ledgerEnsure, type SyncResult } from "../shared/ledger.ts"; -import { fmtCost, fmtTokens } from "./ui.tsx"; +import { costOrTokens, fmtTokens } from "./ui.tsx"; import type { PanelPlacement } from "./layout.ts"; -function pillLabel({ summary, inFlight, ctx }: SyncResult): string { - const combinedCost = summary.effectiveCostUsd === null && inFlight?.effectiveCostUsd == null - ? null - : (summary.effectiveCostUsd ?? 0) + (inFlight?.effectiveCostUsd ?? 0); - const cost = fmtCost(combinedCost); - const estimated = summary.estimatedTurns > 0 || inFlight?.effectiveCostUsd != null; - const parts = [cost - ? `${estimated ? "≈" : ""}${cost}` - : `${fmtTokens(summary.input + summary.cached + (summary.cacheWrite ?? 0) + summary.output)} tok`]; +function pillLabel({ summary, inFlight, ctx, usageSnapshot }: SyncResult): string { + let label: string; + const snapshotHasTokens = usageSnapshot && (usageSnapshot.input !== null || usageSnapshot.cached !== null + || usageSnapshot.cacheWrite !== null || usageSnapshot.output !== null); + if (usageSnapshot && (snapshotHasTokens || usageSnapshot.reportedCostUsd !== null)) { + if (usageSnapshot.scope === 'session' && snapshotHasTokens) { + const partial = usageSnapshot.input === null || usageSnapshot.cached === null + || usageSnapshot.cacheWrite === null || usageSnapshot.output === null; + const total = (usageSnapshot.input ?? 0) + (usageSnapshot.cached ?? 0) + + (usageSnapshot.cacheWrite ?? 0) + (usageSnapshot.output ?? 0); + label = `Session ${partial ? '≈' : ''}${fmtTokens(total)} tok`; + } else label = usageSnapshot.scope === 'session' ? 'Session snapshot' : 'Usage snapshot · scope unknown'; + } else { + const live = inFlight?.tokenScope === 'turn' ? inFlight : null; + const liveCost = live?.effectiveCostUsd; + const combined = { ...summary, + effectiveCostUsd: summary.effectiveCostUsd === null && liveCost == null ? null + : (summary.effectiveCostUsd ?? 0) + (liveCost ?? 0), + estimatedTurns: summary.estimatedTurns + (liveCost != null && live?.costSource !== 'reported' ? 1 : 0), + partialCostTurns: summary.partialCostTurns + (liveCost != null && live?.partialCost ? 1 : 0), + unpricedTurns: summary.unpricedTurns + (live && liveCost == null ? 1 : 0), + pricingWarningTurns: summary.pricingWarningTurns + (live?.pricingWarning ? 1 : 0), + incompleteTurns: summary.incompleteTurns + (live ? 1 : 0), + }; + if (live) for (const field of ['input', 'cached', 'cacheWrite', 'output'] as const) { + if (live[field] != null) combined[field] = (combined[field] ?? 0) + live[field]; + } + const hasUsage = combined.effectiveCostUsd !== null || combined.input !== null || combined.cached !== null + || combined.cacheWrite != null || combined.output !== null; + label = hasUsage || combined.pricingWarningTurns > 0 ? costOrTokens(combined) + : inFlight ? 'Waiting for usage' : 'No usage reported'; + } + const parts = [label]; const liveCtx = inFlight?.ctxUsed != null && inFlight.ctxMax != null ? { used: inFlight.ctxUsed, max: inFlight.ctxMax } : ctx; if (liveCtx && liveCtx.max > 0) parts.push(`ctx ${Math.round(liveCtx.used / liveCtx.max * 100)}%`); - return parts.join(" · "); + return parts.join(' · '); } export function contributePills(client: PluginClientContext, placement: PanelPlacement): () => Promise { diff --git a/client/sidebar.tsx b/client/sidebar.tsx index 1f11f74..284feed 100644 --- a/client/sidebar.tsx +++ b/client/sidebar.tsx @@ -9,7 +9,7 @@ import { costOrTokens } from "./ui.tsx"; export function LedgerSidebarItem({ theme, currentScreen, openScreen }: PluginSidebarItemProps) { const { data } = useOverview(useRangeStart("today")); const working = !!data?.groups.some((group) => group.agents.some((agent) => agent.active)); - const label = !data ? null : data.totals.turns === 0 ? "$0.00" : costOrTokens(data.totals); + const label = data ? costOrTokens(data.totals) : null; return ( = 0.01 ? `$${value.toFixed(2)}` : `$${value.toFixed(4)}`; } -/** Cost when priced, else total tokens. "≈" marks a cost that includes estimates. */ +/** Compact totals always disclose omitted costs or incomplete token coverage. */ export function costOrTokens(summary: Summary): string { const cost = fmtCost(summary.effectiveCostUsd); - if (cost) return `${summary.estimatedTurns > 0 ? "≈" : ""}${cost}`; - return `${fmtTokens(summary.input + summary.cached + (summary.cacheWrite ?? 0) + summary.output)} tok`; + const unknown = summary.pricingWarningTurns > 0 ? ' · price config' + : summary.unpricedTurns > 0 ? ` · ${summary.unpricedTurns} unknown` : ''; + if (cost) return `${summary.estimatedTurns > 0 ? '≈' : ''}${cost}${summary.partialCostTurns > 0 ? ' partial' : ''}${unknown}`; + if (summary.input === null && summary.cached === null && summary.output === null && summary.cacheWrite == null) { + return summary.unmeasuredTurns > 0 ? `turn usage unavailable${unknown}` : `no usage reported${unknown}`; + } + const total = (summary.input ?? 0) + (summary.cached ?? 0) + (summary.cacheWrite ?? 0) + (summary.output ?? 0); + return `${fmtTokens(total)} tok${summary.incompleteTurns > 0 ? ' partial' : ''}${unknown}`; } /** Compact cost for per-component cells; "–" when unknown. */ @@ -29,12 +35,7 @@ export function fmtCostSmall(value: number | null): string { return `$${value.toPrecision(1)}`; } -/** - * Signed cost residual (reported total minus list-price component estimate) - * for the COST cell; blank when unknown or negligible. On Claude turns a - * positive residual is mostly cache writes, which upstream never reports as - * tokens. - */ +/** Reported runtime cost minus known component estimates, not missing-token attribution. */ export function fmtResidual(value: number | null): string { if (value === null || Math.abs(value) < 0.005) return " "; return `${value > 0 ? "+" : "−"}${fmtCostSmall(Math.abs(value))}`; @@ -72,7 +73,7 @@ export function statusColor(status: TurnRecord["status"], theme: PluginTheme): s export function tokensLine(input: number | null, cached: number | null, output: number | null, cacheWrite?: number | null): string | null { if (input === null && cached === null && output === null && cacheWrite == null) return null; - return `in ${fmtTokens(input)} · cache ${fmtTokens(cached)}${cacheWrite != null ? ` · write ${fmtTokens(cacheWrite)}` : ''} · out ${fmtTokens(output)}`; + return `in ${fmtTokens(input)} · cache ${fmtTokens(cached)} · write ${fmtTokens(cacheWrite ?? null)} · out ${fmtTokens(output)}`; } /** Column widths for the TURNS table; shared by the header and data rows. */ @@ -120,16 +121,25 @@ export function Stat({ label, value, theme, dense }: { label: string; value: str export function SummaryRow({ summary, theme, dense }: { summary: Summary; theme: PluginTheme; dense?: boolean }) { const cost = fmtCost(summary.effectiveCostUsd); - const costValue = cost ? `${summary.estimatedTurns > 0 ? "≈" : ""}${cost}` : null; - const costLabel = summary.unpricedTurns > 0 ? `cost · ${summary.unpricedTurns} unknown` : summary.estimatedTurns > 0 ? "cost · estimated" : "cost"; + const costValue = cost ? `${summary.estimatedTurns > 0 ? '≈' : ''}${cost}` : 'unknown'; + const costLabel = ['known cost', summary.partialCostTurns > 0 ? `${summary.partialCostTurns} partial` : null, + summary.unpricedTurns > 0 ? `${summary.unpricedTurns} unknown` : null].filter(Boolean).join(' · '); return ( - - - - - {summary.cacheWrite != null ? : null} - - {costValue ? : null} + + + + + + + + + + {summary.incompleteTurns > 0 ? + {summary.incompleteTurns} turn(s) with incomplete usage{summary.unmeasuredTurns > 0 ? ` · ${summary.unmeasuredTurns} snapshot(s) excluded from totals` : ''}. + : null} + {summary.pricingWarningTurns > 0 ? + {summary.pricingWarningTurns} turn(s) need pricing.json review. Reference-price fallback is blocked for ambiguous or unsafe old matches. + : null} ); } @@ -137,7 +147,7 @@ export function SummaryRow({ summary, theme, dense }: { summary: Summary; theme: export function UsageCoverageNote({ theme }: { theme: PluginTheme }) { return ( - Totals reflect usage available from Paseo; subagent usage and cache writes may be missing. + Known usage only. “–” means unknown, not zero. Runtime-reported and reference costs are not verified bills; descendant usage may be missing. ); } diff --git a/docs/compatibility.md b/docs/compatibility.md index f152bad..25bcd2d 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -28,7 +28,7 @@ the release for Paseo 0.9.1-0.10.x hosts. any other type import breaks installation. Updating the host does not require updating these pins in lockstep. - **Ongoing checks:** CI runs the typecheck and existing tests against 0.11.0, - 0.11.1, and npm `latest`, on pushes, PRs, manual runs, and weekly. These checks + 0.11.1, 0.12.0, and npm `latest`, on pushes, PRs, manual runs, and weekly. These checks detect type and SDK regressions; real provider turns remain necessary to verify token/cost semantics and host behavior. CI failures do not silently rewrite the compatibility range or disable installed plugins. @@ -46,6 +46,8 @@ exclude beta builds. | 0.10.2 | Installed desktop/daemon, published SDK, typecheck, 65 tests, plugin reload, real Claude/Codex turns on 2026-09-30 | Compatible with v0.6.3 | | 0.11.0 | Published SDK (plugin dist identical to 0.11.1), typecheck and 65 tests on 2026-10-09 | Minimum for v0.7.0; no installed host tested | | 0.11.1 | Installed desktop/daemon, published SDK, typecheck, 65 tests, dependency-free compile probe, plugin reload, real Claude/Codex turns, manual sidebar and overview check on 2026-10-09 | Supported by v0.7.0 | +| 0.12.0 | Installed desktop/daemon, published SDK, typecheck and 75 tests against both the 0.11.1 pin and 0.12.0, plugin reload, real Claude turns in a background workspace (before/after control) on 2026-10-23 | Requires the background-workspace fix below; otherwise compatible | +| 0.11.2 | Unreleased accounting fixes: pinned-SDK typecheck, 66 tests, installed-plugin reload, replay of existing Claude/OMP records, real host Web UI overview/pill/panel and live OMP snapshots on 2026-10-14 | Harness scopes and incomplete coverage verified; missing upstream data remains unavailable | The 0.11.1 investigation found: @@ -75,3 +77,61 @@ This review checked client types, bundle compilation, and the desktop sidebar row and overview ranges by hand; mobile UI was not checked. It does not certify other providers or future Paseo releases. Test agents were archived after verification. + +The 0.11.2 accounting review found two plugin errors: model-based input +normalization subtracted OMP cache twice, and session/branch snapshots were +added as turn counts. Historical snapshots are now excluded from turn totals, +while retaining their raw facts and showing late updates separately. Cache +hit rates require a complete denominator; costs disclose unknown/partial +coverage. Existing local prices and JSONL files were not rewritten. + +The UI check used the installed app-dist against the running daemon in an +isolated Chromium, including the narrow ledger panel. Local-network browser +policy was disabled only for that disposable verification process; document +assets were served locally at the daemon origin without changing its CORS or +auth configuration. No mobile device or new paid model probe was used. + +A copy without `node_modules` also compiled and reached "Plugin ready" on +0.11.2 under a disposable plugin ID. Its contributions returned cleanup +functions without registering tracking/UI; the probe was then removed. + +The local-trial follow-up retained 66 passing tests and added upgrade cases +for dotted/hyphenated Claude custom prices, unsafe legacy-match warnings, +reported-cost preservation and warning removal after a corrected alias. +The actual Timeline renderer was also exercised with legacy and scoped data: +legacy cards no longer assert exclusion. Native Text/View were represented +as leaf components for that render-branch smoke, not a full Timeline UI run. +An isolated 0.11.2 daemon with synthetic data verified the overview warning, +the `Session ≈4.12M tok` pill and its snapshot panel, and supplied the refreshed +screenshots. No model prompt was sent. The production plugin was reloaded +for local trial only; no package version was published. + +The 0.12.0 review compared the v0.11.2 and v0.12.0 sources: + +- Paseo 0.12 adds background workspaces. `agents.list`, `workspaces.list` + and their `agent_update` / `workspace_update` streams exclude them unless the + request sets `filter.includeBackground`, and an agent that is hidden by the + filter arrives as `remove`. TokenLedger previously retired such agents: one + real turn in a background workspace was recorded as a 3 ms `canceled` turn + with no usage. TokenLedger now requests background workspaces, confirms each + `remove` with `agents.ref(id).refresh()`, and keeps the open turn when the + agent still exists. The same turn sequence then recorded `completed` turns. +- A 0.12+ client SDK rejects `includeBackground` with "Update the host to use + background workspaces." when the host does not advertise the feature (a 0.12 + app on a 0.11 daemon). TokenLedger retries those lists without the filter, so + the minimum remains 0.11.0. Older daemons drop the unknown filter field. +- The plugin SDK client API, lifecycle hooks, timeline subscription and + `AgentUsage` are unchanged. Claude still omits cache creation and reports a + session-cumulative `totalCostUsd`; Codex still reports per-request `last` + usage with cached input included; OMP still emits `turn_completed` before + the final usage drain. The proxy context-meter fixes change only + `contextWindowUsedTokens`. +- The compiler now resolves type-only `@getpaseo/protocol/*` subpath imports. + TokenLedger does not rely on this while it supports 0.11.0. +- Agent-to-agent messages now steer a running turn instead of interrupting it, + so one record can cover several prompts. Plugin ACP providers without + `prompt.steer` still replace the turn. +- 0.13.0-beta.1 deprecates `/client/react-native` in favor of `/client/ui` + (alias kept until 2027-04-20), empties the deprecated `timeline` field of the + `turn_ended` hook (TokenLedger does not read it), and applies the background + filter to stored agents as well. diff --git a/docs/paseo-usage-api.md b/docs/paseo-usage-api.md index ef313a1..49e975f 100644 --- a/docs/paseo-usage-api.md +++ b/docs/paseo-usage-api.md @@ -5,6 +5,8 @@ Status: submitted as follow-up comments in existing upstream Discussions on - [Cache creation counts — discussion #4623](https://github.com/getpaseo/paseo/discussions/4623#discussioncomment-18550901). - [Descendant usage and attribution — discussion #4660](https://github.com/getpaseo/paseo/discussions/4660#discussioncomment-18550906). +- [2026-10-14 follow-up: OMP cache creation and input conventions](https://github.com/getpaseo/paseo/discussions/4623#discussioncomment-18924605). +- [2026-10-14 follow-up: final drain, branch epochs and attribution](https://github.com/getpaseo/paseo/discussions/4660#discussioncomment-18924607). Paseo's [contribution policy](https://github.com/getpaseo/paseo/blob/b2ce2bcb83dd40bd91c32098c59221f0b52d7ffd/CONTRIBUTING.md) directs missing extension points to Discussions. Existing threads covered both @@ -32,6 +34,27 @@ Rechecked on 2026-09-30 against the installed Paseo 0.10.2 provider source and published SDK: the same cache-creation and Codex child-usage gaps remain. See the [compatibility review](compatibility.md) for the scope of this verification. +Rechecked on 2026-10-14 against installed 0.11.2, separately from public `main` +at `cafa04d3fbdbc7e4a3fe8f4d4bea80f15cd4a1d0`. The follow-ups above were posted +as `stv1024` and read back through GitHub; submission does not imply acceptance. +The public OMP usage poller was exercised in memory with synthetic statistics +and controlled promises, not by running the full upstream daemon: + +- OMP maps `getSessionStats()` input/read/output/cost but drops `cacheWrite`. + The installed OMP statistics are already cache-exclusive and aggregate the + current session/branch, including some task-child usage. +- Completion can precede final usage; starting another turn can invalidate + the pending final sample. Public snapshots do not preserve its turn identity. + Require terminal drain or attributable late corrections, plus branch/reset + epochs; subtracting adjacent snapshots is not a reliable per-turn contract. +- Context-only OMP stats become zero token/cost values in the current mapper. + The plugin cannot distinguish those fabricated zeros from measured zero. +- Claude cache creation and Codex child usage remain omitted. Merged + [#6703](https://github.com/getpaseo/paseo/pull/6703) and + [#6106](https://github.com/getpaseo/paseo/pull/6106) improve context accounting, + not the missing billing/usage contract. They are not duplicate bug reports. + + ## 1. Preserve cache creation usage `@getpaseo/protocol/dist/agent-types.d.ts`, `AgentUsage`, currently exposes @@ -137,6 +160,8 @@ history/subscription. Persist identities before acknowledging ingestion, price each request with its own model, and select one accounting scope for each total. Keep the current snapshot path as an explicitly partial fallback. -Until then, TokenLedger's ordinary snapshot/terminal tracking remains active. -Historical missing counts and descendant charges cannot be reconstructed from -the current public API, and this change does not claim to restore them. +Until then, verified turn/request tracking remains active. OMP session/branch +and unknown-scope observations are displayed separately and excluded from +per-turn costs and sums; late observations refresh the latest-snapshot card. +Coverage and unknown prices are explicit. Historical missing counts and +descendant charges cannot be reconstructed from the current public API. diff --git a/docs/releases/v0.8.0.md b/docs/releases/v0.8.0.md new file mode 100644 index 0000000..5653a1c --- /dev/null +++ b/docs/releases/v0.8.0.md @@ -0,0 +1,42 @@ +# TokenLedger v0.8.0 + +Released 2026-10-23. Tracks agents in Paseo 0.12 background workspaces and +makes token and cost accounting more accurate and more explicit about what is +unknown. + +## Compatibility + +Requires **Paseo >=0.11.0** on both the daemon and app, without an upper version +limit. The verified hosts are **0.11.1** and **0.12.0**. SDK development +dependencies stay pinned to 0.11.1; CI checks 0.11.0, 0.11.1, 0.12.0, and the +latest SDK. See the [compatibility notes](../compatibility.md). + +Existing `ledger.jsonl` records are reinterpreted on read and are not +rewritten. Existing `pricing.json` files are kept. After updating, reload or +reopen the app so the client uses the new summary fields. + +## Changes + +- **Paseo 0.12 background workspaces.** Paseo 0.12 hides background workspaces + from default agent and workspace lists. Before this release, turns in those + workspaces were closed as canceled with no usage. TokenLedger now requests + background workspaces, falls back on older hosts, and confirms a removal by + agent ID before it settles an open turn. +- **Overview ranges.** Switching ranges stays on the same screen and no longer + adds a navigation entry per click. Thanks to @ChtJT (#4). +- **Usage scope.** Per-turn usage is kept apart from OMP session/branch totals + and unknown-scope snapshots. Snapshots stay visible but are not added to turn + or date-range sums. +- **Unknown is not zero.** Missing counts, cache rates, and costs show as + unknown. A cache hit rate shows only with a complete prompt denominator. The + panel, pill, sidebar, and overview mark incomplete usage and partial + estimates. +- **Pricing.** No private price seeds or fixed currency conversion are built + in; new price files start empty. Prices match full model IDs and explicit + aliases. Ambiguous or formerly unsafe matches block reference fallback and + show a pricing.json warning. + +## Known gaps + +Paseo 0.12.0 still omits cache-write counts and provider-internal subagent +usage. See [usage API requirements](../paseo-usage-api.md). diff --git a/images/screenshot-overview.png b/images/screenshot-overview.png index 947cc2a..034ec56 100644 Binary files a/images/screenshot-overview.png and b/images/screenshot-overview.png differ diff --git a/images/screenshot-panel.png b/images/screenshot-panel.png index d2f193c..21c833a 100644 Binary files a/images/screenshot-panel.png and b/images/screenshot-panel.png differ diff --git a/images/screenshot-pill.jpg b/images/screenshot-pill.jpg index 3b1c3fd..10aace4 100644 Binary files a/images/screenshot-pill.jpg and b/images/screenshot-pill.jpg differ diff --git a/package-lock.json b/package-lock.json index a517ae5..320c6ad 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "paseo-token-ledger", - "version": "0.7.0", + "version": "0.8.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "paseo-token-ledger", - "version": "0.7.0", + "version": "0.8.0", "license": "MIT", "devDependencies": { "@getpaseo/client": "0.11.1", diff --git a/package.json b/package.json index 49864a0..38c3483 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "paseo-token-ledger", - "version": "0.7.0", + "version": "0.8.0", "description": "Paseo plugin: per-turn token usage, cache counts, and cost tracking with a persistent local ledger.", "type": "module", "license": "MIT", diff --git a/server/background.test.ts b/server/background.test.ts new file mode 100644 index 0000000..d182a05 --- /dev/null +++ b/server/background.test.ts @@ -0,0 +1,79 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { PaseoApi, PaseoAgentTimelineEvent, PaseoAgentUpdate } from '@getpaseo/client'; +import { ownedSnapshot } from '../shared/subscription-test-helpers.ts'; + +test('a catalog remove for an agent that still exists keeps its turn open; a confirmed removal settles it', async () => { + const home = await mkdtemp(join(tmpdir(), 'token-ledger-background-')); + const previousHome = process.env.PASEO_HOME; + process.env.PASEO_HOME = home; + const dir = join(home, 'plugins', 'token-ledger'); + await mkdir(dir, { recursive: true }); + await writeFile(join(dir, 'pricing.json'), JSON.stringify({version: 1, currency: 'USD', prices: []})); + await writeFile(join(dir, 'openrouter-pricing.json'), JSON.stringify({fetchedAt: new Date().toISOString(), prices: []})); + const running = (id: string, turnId: string) => ({ id, provider: 'codex', model: 'gpt-5', workspaceId: 'w', status: 'running', + runtimeInfo: { sessionId: 's' }, activeTurn: { turnId, startedAt: '2026-10-23T00:00:00Z' }, + lastUsage: { inputTokens: 100, cachedInputTokens: 50, outputTokens: 10 } }); + // The background agent still resolves by ID; the deleted one does not. + const current = new Map([['bg', running('bg', 't1')], ['gone', running('gone', 't2')]]); + const timelines = new Map void>(); + const listFilters: unknown[] = []; + let update: (event: PaseoAgentUpdate) => void = () => {}; + const paseo = { agents: { + subscribe: () => () => {}, + list: async (options: { filter?: unknown; subscribe?: object }) => { + listFilters.push(options.filter); + const owned = ownedSnapshot({ entries: [...current.values()].map((agent) => ({ agent })), pageInfo: { nextCursor: null } }); + update = (event) => owned.update({ type: 'agent_update', payload: event }); + return owned.page; + }, + ref: (id: string) => ({ + refresh: async () => current.get(id) ?? null, + current: () => current.get(id) ?? null, + timeline: { + subscribe: (fn: (event: PaseoAgentTimelineEvent) => void) => { + timelines.set(id, fn); + return Object.assign(() => {}, { ready: Promise.resolve(), release: async () => {} }); + }, + append: async () => ({ seq: 1, epoch: 'test' }), + }, + }), + }, workspaces: { list: async (options: { filter?: unknown }) => { + listFilters.push(options.filter); + return ownedSnapshot({ entries: [], pageInfo: { nextCursor: null } }).page; + } } } as unknown as PaseoApi; + const tracker = await import('./tracker.ts'); + const store = await import('./store.ts'); + const records = () => store.allRecords().map((record) => [record.agentId, record.status]); + const waitForRecords = async (count: number) => { + for (let i = 0; i < 200 && records().length < count; i++) await new Promise((resolve) => setTimeout(resolve, 10)); + }; + try { + await tracker.ensureTracker(paseo); + for (const [id, turnId] of [['bg', 't1'], ['gone', 't2']]) { + timelines.get(id)!({ agentId: id, timestamp: '2026-10-23T00:00:00Z', event: { type: 'turn_started', provider: 'codex', turnId } }); + } + current.delete('gone'); + update({ kind: 'remove', agentId: 'bg' }); + update({ kind: 'remove', agentId: 'gone' }); + await waitForRecords(1); + // Let the background agent's confirmation finish too. + await new Promise((resolve) => setTimeout(resolve, 20)); + assert.deepEqual(records(), [['gone', 'canceled']]); + timelines.get('bg')!({ agentId: 'bg', timestamp: '2026-10-23T00:00:02Z', event: { type: 'turn_completed', provider: 'codex', turnId: 't1', + usage: { inputTokens: 120, cachedInputTokens: 60, outputTokens: 20 } } }); + // Terminal usage settles after the tracker's short stream window. + await waitForRecords(2); + assert.deepEqual(records(), [['gone', 'canceled'], ['bg', 'completed']]); + await tracker.handleOverview({}, { paseo }); + assert.ok(listFilters.length >= 2); + for (const filter of listFilters) assert.deepEqual(filter, { includeBackground: true }); + } finally { + await tracker.stopTracker(); + if (previousHome === undefined) delete process.env.PASEO_HOME; else process.env.PASEO_HOME = previousHome; + await rm(home, { recursive: true, force: true }); + } +}); diff --git a/server/catalog.ts b/server/catalog.ts index f40a96e..42627b3 100644 --- a/server/catalog.ts +++ b/server/catalog.ts @@ -1,4 +1,5 @@ import type { PaseoAgent, PaseoApi, PaseoWorkspace, PaseoWorkspaceListResult, PaseoWorkspaceUpdate } from './paseo-types.ts'; +import { includingBackground } from '../shared/background.ts'; import { subscribeCatalog, type CatalogSubscription } from '../shared/catalog-subscription.ts'; type Metadata = Pick; @@ -18,9 +19,14 @@ export class Catalog { ensureWorkspaces(paseo: PaseoApi): Promise { if (this.disposed) return Promise.resolve(); + let filter: {} = {}; this.subscription ??= subscribeCatalog({ - open: () => paseo.workspaces.list({ subscribe: {}, page: { limit: 200 } }), - next: (cursor) => paseo.workspaces.list({ page: { limit: 200, cursor } }), + open: () => includingBackground(async (option) => { + const page = await paseo.workspaces.list({ subscribe: {}, filter: option, page: { limit: 200 } }); + filter = option; + return page; + }), + next: (cursor) => paseo.workspaces.list({ filter, page: { limit: 200, cursor } }), entries: (page) => page.entries, id: (workspace) => workspace.id, update: (value) => { diff --git a/server/pricing.test.ts b/server/pricing.test.ts index 9100d41..fe42f89 100644 --- a/server/pricing.test.ts +++ b/server/pricing.test.ts @@ -1,36 +1,78 @@ -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { tokenRouterDefaultPricing } from "./pricing.ts"; +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { finalizeTurn } from '../shared/aggregate.ts'; -test("TokenRouter quota prices are converted to USD per million tokens", () => { - assert.deepEqual(tokenRouterDefaultPricing("gpt-5.6-sol", 100_000), { - input: 5.000222, - cacheRead: 0.499948, - output: 29.999852, - }); -}); - -test("TokenRouter aliases normalize dots, hyphens, vendor prefixes, and 1m suffixes", () => { - assert.deepEqual(tokenRouterDefaultPricing("tokenrouter/claude-opus-4-6[1m]", 100_000), { - input: 5.000222, - cacheRead: 0.499948, - output: 25.000369, - }); -}); - -test("TokenRouter tiered pricing selects the prompt-size tier", () => { - assert.deepEqual(tokenRouterDefaultPricing("gpt-6-astra", 272_000), { - input: 9.999705, - cacheRead: 0.999897, - output: 50, - }); - assert.deepEqual(tokenRouterDefaultPricing("gpt-6-astra", 272_001), { - input: 20.000148, - cacheRead: 1.999793, - output: 75.000369, - }); -}); - -test("unknown TokenRouter model has no invented price", () => { - assert.equal(tokenRouterDefaultPricing("future-model", 100), null); +test('pricing preserves model identity, provider scopes, explicit aliases and unknown tiers', async () => { + const home = await mkdtemp(join(tmpdir(), 'token-ledger-identities-')); + const oldHome = process.env.PASEO_HOME; + process.env.PASEO_HOME = home; + const dir = join(home, 'plugins', 'token-ledger'); + await mkdir(dir, { recursive: true }); + const prices = [ + { model: 'vendor-a/model-x', aliases: ['shared-alias'], tiers: [{ input: 1, cacheRead: 0, output: 0 }] }, + { model: 'vendor-b/model-x', aliases: ['shared-alias', 'explicit-alias'], tiers: [{ input: 10, cacheRead: 0, output: 0 }] }, + { model: 'acme/model-x', providers: ['acme'], tiers: [{ input: 99, cacheRead: 0, output: 0 }] }, + { model: 'tiered', tiers: [{ upToInputTokens: 100, input: 1, cacheRead: 0, output: 0 }, { input: 10, cacheRead: 0, output: 0 }] }, + { model: 'claude-opus-5.5', tiers: [{ input: 100, cacheRead: 0, cacheWrite: 0, output: 0 }] }, + { model: 'gateway/claude-sonnet-4-6', tiers: [{ input: 50, cacheRead: 0, cacheWrite: 0, output: 0 }] }, + { model: 'claude-haiku-4-5[1m]', tiers: [{ input: 20, cacheRead: 0, cacheWrite: 0, output: 0 }] }, + ]; + await writeFile(join(dir, 'pricing.json'), JSON.stringify({ version: 1, currency: 'USD', prices })); + await writeFile(join(dir, 'openrouter-pricing.json'), JSON.stringify({ fetchedAt: new Date().toISOString(), prices: [] })); + const originalNow = Date.now; + try { + // Module-load boundary: pricing captures PASEO_HOME; static import would target the real ledger. + const { ensurePricing, estimateUsageCost, enrichTurn } = await import('./pricing.ts'); + const { summaryFromRows } = await import('./read-model.ts'); + await ensurePricing(); + const usage = { provider: 'claude', model: 'vendor-b/model-x', input: 1_000_000, cached: 0, cacheWrite: 0, output: 0 }; + assert.equal(estimateUsageCost(usage).effectiveCostUsd, 10); + assert.equal(estimateUsageCost({ ...usage, model: 'vendor-a/model-x' }).effectiveCostUsd, 1); + assert.equal(estimateUsageCost({ ...usage, model: 'explicit-alias' }).effectiveCostUsd, 10); + assert.equal(estimateUsageCost({ ...usage, model: 'shared-alias' }).effectiveCostUsd, null); + assert.equal(estimateUsageCost({ ...usage, model: 'acme/model-x' }).effectiveCostUsd, null); + assert.equal(estimateUsageCost({ ...usage, provider: 'claude-acme', model: 'acme/model-x' }).effectiveCostUsd, 99); + assert.equal(estimateUsageCost({ ...usage, model: 'tiered', input: 80, cacheWrite: undefined }).effectiveCostUsd, null); + assert.equal(estimateUsageCost({ ...usage, model: 'tiered', input: 80 }).effectiveCostUsd, 0.00008); + assert.equal(estimateUsageCost({ ...usage, provider: 'omp' }).effectiveCostUsd, null); + assert.equal(estimateUsageCost({ ...usage, provider: 'unknown' }).effectiveCostUsd, null); + const dotted = estimateUsageCost({ ...usage, model: 'claude-opus-5.5' }); + const dashed = estimateUsageCost({ ...usage, model: 'claude-opus-5-5' }); + assert.equal(dotted.effectiveCostUsd, 100); + assert.equal(dashed.effectiveCostUsd, 100); + assert.equal(dashed.costSource, 'override'); + assert.equal(dashed.pricingWarning, null); + for (const model of ['claude-sonnet-4-6', 'claude-haiku-4-5', 'shared-alias', 'acme/model-x']) { + const result = estimateUsageCost({ ...usage, model }); + assert.equal(result.effectiveCostUsd, null); + assert.equal(result.costSource, null); + assert.ok(result.pricingWarning != null); + } + assert.equal(estimateUsageCost({ ...usage, model: 'claude-opus-4' }).effectiveCostUsd, 15); + const record = finalizeTurn({ agentId: 'a', turnId: 't', provider: 'claude', model: 'claude-sonnet-4-6', + startedAt: null, endedAt: '2026-10-14T00:00:00Z', status: 'completed', observations: [], prevSessionCostUsd: 0, + finalUsage: { inputTokens: 1_000_000, cachedInputTokens: 0, cacheWriteInputTokens: 0, outputTokens: 0 } }); + const blocked = enrichTurn(record, 1); + assert.equal(summaryFromRows([blocked]).pricingWarningTurns, 1); + // Reported costs remain visible even when reference breakdown pricing needs review. + const reported = enrichTurn({ ...record, costUsd: 7 }, 1); + assert.equal(reported.effectiveCostUsd, 7); + assert.ok(reported.pricingWarning != null); + const repaired = prices.map((entry) => entry.model === 'gateway/claude-sonnet-4-6' + ? { ...entry, aliases: ['claude-sonnet-4-6'] } : entry); + await writeFile(join(dir, 'pricing.json'), JSON.stringify({ version: 1, currency: 'USD', prices: repaired })); + Date.now = () => originalNow() + 31_000; + await ensurePricing(); + const fixed = enrichTurn(record, 1); + assert.equal(fixed.effectiveCostUsd, 50); + assert.equal(fixed.pricingWarning, null); + assert.equal(summaryFromRows([fixed]).pricingWarningTurns, 0); + } finally { + Date.now = originalNow; + if (oldHome === undefined) delete process.env.PASEO_HOME; else process.env.PASEO_HOME = oldHome; + await rm(home, { recursive: true, force: true }); + } }); diff --git a/server/pricing.ts b/server/pricing.ts index 01e41e2..c933f1f 100644 --- a/server/pricing.ts +++ b/server/pricing.ts @@ -3,12 +3,12 @@ import { homedir } from "node:os"; import { join } from "node:path"; import { z } from "zod"; import type { CostSource, TurnRecord, TurnRow } from "../shared/ledger.ts"; -import { canonicalModel, costBreakdownWithPricing, modelPricing, type ModelPricing } from "../shared/pricing.ts"; -import { freshInput, usageSemantics } from "../shared/semantics.ts"; +import { canonicalModel, costBreakdownWithPricing, knownCost, modelPricing, partialCost, type CostBreakdown, type ModelPricing } from "../shared/pricing.ts"; +import { freshInput, promptTokens, tokenScope, usageSemantics } from "../shared/semantics.ts"; import { getPreferences, preferencesRevision } from './preferences.ts'; type PriceTier = ModelPricing & { upToInputTokens?: number }; -type PriceEntry = { providers?: string[]; model: string; tiers: PriceTier[] }; +type PriceEntry = { providers?: string[]; model: string; aliases?: string[]; tiers: PriceTier[] }; export const PriceFileSchema = z.object({ version: z.literal(1), @@ -18,11 +18,12 @@ export const PriceFileSchema = z.object({ z.object({ providers: z.array(z.string()).optional(), model: z.string(), + aliases: z.array(z.string().min(1)).optional(), tiers: z.array( z.object({ upToInputTokens: z.number().int().positive().optional(), input: z.number().nonnegative(), - cacheRead: z.number().nonnegative(), + cacheRead: z.number().finite().nonnegative().optional(), cacheWrite: z.number().finite().nonnegative().optional(), output: z.number().nonnegative(), }), @@ -38,142 +39,24 @@ const DATA_DIR = join(process.env.PASEO_HOME ?? join(homedir(), ".paseo"), "plug const PRICE_FILE = join(DATA_DIR, "pricing.json"); const OPENROUTER_CACHE_FILE = join(DATA_DIR, "openrouter-pricing.json"); const OPENROUTER_MAX_AGE_MS = 24 * 60 * 60 * 1000; -const TOKENROUTER_QUOTA_PER_USD = 13.5414; // 1 quota = CNY 0.5 at CNY 6.7707 / USD -/* model|max prompt tokens|input quota|cache-read quota|output quota. Blank max = final tier. */ -const TOKENROUTER_ROWS = ` -claude-fable-5||135.41|13.54|677.07 -claude-haiku-4.5||13.54|1.35|67.71 -claude-opus-4.5||67.71|6.77|338.54 -claude-opus-4.6||67.71|6.77|338.54 -claude-opus-4.6-hq||67.71|6.77|338.54 -claude-opus-4.7||67.71|6.77|338.54 -claude-opus-4.8||67.71|6.77|338.54 -claude-opus-5||67.71|6.77|338.54 -claude-sonnet-4.5||40.62|4.06|203.12 -claude-sonnet-4.6||40.62|4.06|203.12 -claude-sonnet-5||27.08|2.71|135.41 -deepseek-v3.2||2|0.4|3 -deepseek-v4-flash||3|0.1|9 -deepseek-v4-flash-vision-exp||3|0.1|9 -deepseek-v4-pro||9|0.3|27 -gemini-3-flash-preview||6.77|0.68|40.62 -gemini-3.1-flash-lite||3.39|0.34|20.31 -gemini-3.1-pro-preview|200000|27.08|2.71|162.5 -gemini-3.1-pro-preview||54.17|5.42|243.75 -gemini-3.5-flash||20.31|2.03|121.87 -gemini-3.7-flash||10.16|1.02|50.78 -glm-5|32000|4|1|18 -glm-5||6|1.5|22 -glm-5-turbo|32000|5|1.2|22 -glm-5-turbo||7|1.8|26 -glm-5.1|32000|6|1.3|24 -glm-5.1||8|2|28 -glm-5.2||8|2|28 -glm-5.3||8|2|28 -glm-5.3-flash||0.8|0.23|2.8 -glm-5v-turbo|32000|5|1.2|22 -glm-5v-turbo||7|1.8|26 -google.gemini-2.5-pro|200000|16.93|1.69|135.41 -google.gemini-2.5-pro||33.85|3.39|203.12 -gpt-4o-mini||2.03|1.02|8.12 -gpt-5-mini||3.39|0.34|27.08 -gpt-5.4||33.85|3.39|203.12 -gpt-5.4-mini||10.16|1.02|60.94 -gpt-5.5||67.71|6.77|406.24 -gpt-5.6-luna||13.54|1.35|81.25 -gpt-5.6-sol||67.71|6.77|406.24 -gpt-5.6-terra||33.85|3.39|203.12 -gpt-6-astra|272000|135.41|13.54|677.07 -gpt-6-astra||270.83|27.08|1015.61 -hy3||1|0.25|4 -hy3-preview|15999|1.2|0.4|4 -hy3-preview|31999|1.6|0.6|6.4 -hy3-preview||2|0.8|8 -hy4-preview||6|0.3|18 -kimi-k2.5||4|0.7|21 -kimi-k2.6||6.5|1.1|27 -kimi-k2.7-code||6.43|1.29|27.08 -kimi-k2.7-code-highspeed||12.86|2.57|54.17 -kimi-k3||20|2|100 -mimo-v2.5-pro||3|0.03|6 -minimax-m2.5||2.1|0.21|8.4 -minimax-m2.7||2.1|0.42|8.4 -minimax-m3|512000|2.1|0.42|8.4 -minimax-m3||4.2|0.84|16.8 -qwen3.5-35b-a3b||0.95|0.95|6.77 -qwen3.5-flash|128000|0.2|0.02|2 -qwen3.5-flash|256000|0.8|0.08|8 -qwen3.5-flash||1.2|0.12|12 -qwen3.5-plus|128000|0.8|0.08|4.8 -qwen3.5-plus|256000|2|0.2|12 -qwen3.5-plus||4|0.4|24 -qwen3.6-27b||1.9|1.9|13.47 -qwen3.8-max-0902||12|1.5|36 -xai.grok-3|199999|16.93|2.71|33.85 -xai.grok-3||33.85|5.42|67.71 -xai.grok-4|199999|16.93|2.71|33.85 -xai.grok-4||33.85|5.42|67.71 -xai.grok-4.20-0309-non-reasoning|199999|16.93|2.71|33.85 -xai.grok-4.20-0309-non-reasoning||33.85|5.42|67.71 -xai.grok-4.20-0309-reasoning|199999|16.93|2.71|33.85 -xai.grok-4.20-0309-reasoning||33.85|5.42|67.71 -xai.grok-4.20-multi-agent|199999|16.93|2.71|33.85 -xai.grok-4.20-multi-agent||33.85|5.42|67.71 -xai.grok-4.20-multi-agent-0309|199999|16.93|2.71|33.85 -xai.grok-4.20-multi-agent-0309||33.85|5.42|67.71 -xai.grok-4.20-non-reasoning|199999|16.93|2.71|33.85 -xai.grok-4.20-non-reasoning||33.85|5.42|67.71 -xai.grok-4.20-reasoning|199999|16.93|2.71|33.85 -xai.grok-4.20-reasoning||33.85|5.42|67.71 -xai.grok-4.3|199999|16.93|2.71|33.85 -xai.grok-4.3||33.85|5.42|67.71 -xai.grok-4.6|199999|27.08|6.77|81.25 -xai.grok-4.6||54.17|13.54|162.5`; - -function seedPrices(): PriceEntry[] { - const usd = (quota: string) => Number((Number(quota) / TOKENROUTER_QUOTA_PER_USD).toFixed(6)); - const grouped = new Map(); - for (const line of TOKENROUTER_ROWS.trim().split("\n")) { - const [model, max, input, cacheRead, output] = line.split("|"); - let entry = grouped.get(model); - if (!entry) { - entry = { providers: ["tokenrouter", "deepseek-harness"], model, tiers: [] }; - grouped.set(model, entry); - } - entry.tiers.push({ - ...(max ? { upToInputTokens: Number(max) } : {}), - input: usd(input), - cacheRead: usd(cacheRead), - output: usd(output), - }); - } - return [...grouped.values()]; -} - -function providerMatches(entry: PriceEntry, provider: string | null, model: string | null): boolean { +function providerMatches(entry: PriceEntry, provider: string | null): boolean { if (!entry.providers?.length) return true; - const haystack = `${provider ?? ""} ${model ?? ""}`.toLowerCase(); - return entry.providers.some((part) => haystack.includes(part.toLowerCase())); + return entry.providers.some((part) => (provider ?? '').toLowerCase().includes(part.toLowerCase())); } -function chooseTier(entry: PriceEntry, promptTokens: number): PriceTier { - return entry.tiers.find((tier) => tier.upToInputTokens === undefined || promptTokens <= tier.upToInputTokens) ?? entry.tiers.at(-1)!; +function chooseTier(entry: PriceEntry, prompt: number | null): PriceTier | null { + if (entry.tiers.length === 1) return entry.tiers[0]; + if (prompt === null) return null; + return entry.tiers.find((tier) => tier.upToInputTokens === undefined || prompt <= tier.upToInputTokens) ?? null; } -/** Exposed for deterministic verification of the shipped TokenRouter seed table. */ -export function tokenRouterDefaultPricing(model: string, promptTokens: number): ModelPricing | null { - const entry = seedPrices().find((candidate) => canonicalModel(candidate.model) === canonicalModel(model)); - if (!entry) return null; - const { input, cacheRead, output } = chooseTier(entry, promptTokens); - return { input, cacheRead, output }; -} -type OpenRouterPrice = { id: string; prompt: number; completion: number; cacheRead: number; cacheWrite?: number }; +type OpenRouterPrice = { id: string; prompt: number; completion: number; cacheRead?: number; cacheWrite?: number }; const OpenRouterCacheSchema = z.object({ fetchedAt: z.string().datetime(), prices: z.array(z.object({ id: z.string(), prompt: z.number().finite().nonnegative(), - completion: z.number().finite().nonnegative(), cacheRead: z.number().finite().nonnegative(), + completion: z.number().finite().nonnegative(), cacheRead: z.number().finite().nonnegative().optional(), cacheWrite: z.number().finite().nonnegative().optional() })), }); @@ -188,14 +71,17 @@ async function loadOverrides(): Promise { try { overridePrices = PriceFileSchema.parse(JSON.parse(await readFile(PRICE_FILE, "utf8"))).prices; } catch (error) { - const seeded = { version: 1, currency: "USD", updatedAt: "2026-09-14", prices: seedPrices() } as const; - overridePrices = seeded.prices.map((entry) => ({ ...entry, tiers: [...entry.tiers] })); + if (!(error instanceof Error && 'code' in error && error.code === 'ENOENT')) { + console.error('token-ledger: invalid pricing.json; custom estimates unavailable', error); + return; + } + // Community installations start with no account-specific pricing policy. + const empty = { version: 1, currency: 'USD', prices: [] }; try { - await writeFile(PRICE_FILE, `${JSON.stringify(seeded, null, 2)}\n`, { encoding: "utf8", flag: "wx" }); - } catch { - if (!(error instanceof Error && "code" in error && error.code === "ENOENT")) { - console.error("token-ledger: invalid pricing.json; using TokenRouter defaults", error); - } + await writeFile(PRICE_FILE, `${JSON.stringify(empty, null, 2)}\n`, { encoding: 'utf8', flag: 'wx' }); + } catch (writeError) { + if (!(writeError instanceof Error && 'code' in writeError && writeError.code === 'EEXIST')) throw writeError; + overridePrices = PriceFileSchema.parse(JSON.parse(await readFile(PRICE_FILE, 'utf8'))).prices; } } } @@ -223,14 +109,15 @@ function refreshPrices(): void { const prompt = Number(item.pricing?.prompt); const completion = Number(item.pricing?.completion); if (!item.id || !Number.isFinite(prompt) || prompt < 0 || !Number.isFinite(completion) || completion < 0) return []; - const rawCache = Number(item.pricing?.input_cache_read); + const readPrice = item.pricing?.input_cache_read; + const rawCache = typeof readPrice === 'string' && readPrice.trim() !== '' ? Number(readPrice) : NaN; const writePrice = item.pricing?.input_cache_write; const rawWrite = typeof writePrice === 'string' && writePrice.trim() !== '' ? Number(writePrice) : NaN; return [{ id: item.id, prompt: prompt * 1_000_000, completion: completion * 1_000_000, - cacheRead: (Number.isFinite(rawCache) && rawCache >= 0 ? rawCache : prompt) * 1_000_000, + ...(Number.isFinite(rawCache) && rawCache >= 0 ? { cacheRead: rawCache * 1_000_000 } : {}), ...(Number.isFinite(rawWrite) && rawWrite >= 0 ? { cacheWrite: rawWrite * 1_000_000 } : {}), }]; }); @@ -270,11 +157,37 @@ function lookupOpenRouter(model: string): ModelPricing | null { return { input: price.prompt, cacheRead: price.cacheRead, cacheWrite: price.cacheWrite, output: price.completion }; } -function resolvePricing(provider: string | null, model: string | null, promptTokens: number): { pricing: ModelPricing; source: Exclude } | null { +/** Detect the old lossy matching rule only to warn; never select a price with it. */ +function legacyPriceKey(value: string): string { + let model = value.trim().toLowerCase(); + if (model.startsWith('[')) { + try { + const parsed: unknown = JSON.parse(model); + if (Array.isArray(parsed) && typeof parsed.at(-1) === 'string') model = parsed.at(-1); + } catch {} + } + return (model.split('/').at(-1) ?? model).replace(/\[1m\]$/, '').replace(/[._]/g, '-').replace(/-+/g, '-'); +} + + +function resolvePricing(provider: string | null, model: string | null, prompt: number | null): { pricing: ModelPricing; source: Exclude } | { warning: string } | null { if (!model) return null; const canonical = canonicalModel(model); - const override = overridePrices.find((entry) => canonicalModel(entry.model) === canonical && providerMatches(entry, provider, model)); - if (override) return { pricing: chooseTier(override, promptTokens), source: "override" }; + const matches = overridePrices.filter((entry) => + [entry.model, ...(entry.aliases ?? [])].some((id) => canonicalModel(id) === canonical) && providerMatches(entry, provider)); + const scoped = matches.filter((entry) => entry.providers?.length); + const candidates = scoped.length ? scoped : matches; + if (candidates.length > 1) return { warning: `Multiple custom prices match ${model}. Keep one matching entry in pricing.json; reference-price fallback is blocked.` }; + if (candidates.length === 1) { + const pricing = chooseTier(candidates[0], prompt); + return pricing ? { pricing, source: 'override' } : null; + } + const legacyKey = legacyPriceKey(model); + const formerScope = `${provider ?? ''} ${model}`.toLowerCase(); + const legacyMatch = overridePrices.some((entry) => + (!entry.providers?.length || entry.providers.some((scope) => formerScope.includes(scope.toLowerCase()))) + && [entry.model, ...(entry.aliases ?? [])].some((id) => legacyPriceKey(id) === legacyKey)); + if (legacyMatch) return { warning: `Review custom pricing for ${model}: add its full ID to model/aliases and check providers in pricing.json. The old name/provider folding is unsafe; reference-price fallback is blocked.` }; const openrouter = getPreferences().openRouterPricing ? lookupOpenRouter(model) : null; if (openrouter) return { pricing: openrouter, source: "openrouter" }; const builtin = getPreferences().builtinPricing ? modelPricing(model) : null; @@ -283,53 +196,65 @@ function resolvePricing(provider: string | null, model: string | null, promptTok type PriceableUsage = Pick & { requests?: TurnRecord['requests']; + tokenScope?: TurnRecord['tokenScope']; }; -/** Prices raw provider usage, including per-request tiers when observations are available. */ +/** Reference estimates only for usage with a verified per-turn accounting scope. */ export function estimateUsageCost(usage: PriceableUsage): { effectiveCostUsd: number | null; - costSource: Exclude | null; - costBreakdown: ReturnType; + costSource: Exclude | null; + costBreakdown: CostBreakdown | null; + partialCost: boolean; + pricingWarning: string | null; } { - const semantics = usageSemantics(usage.provider, usage.model); - const input = freshInput(semantics, usage.input, usage.cached, usage.cacheWrite); - const promptTokens = (input ?? 0) + (usage.cached ?? 0) + (usage.cacheWrite ?? 0); - const resolved = resolvePricing(usage.provider, usage.model, promptTokens); - let breakdown = resolved ? costBreakdownWithPricing({ ...usage, input, costUsd: null }, resolved.pricing) : null; - if (usage.requests?.length && resolved) { - const parts = usage.requests.map((request) => { - const fresh = freshInput(semantics, request.input, request.cached, request.cacheWrite); - const rate = resolvePricing(usage.provider, usage.model, (fresh ?? 0) + (request.cached ?? 0) + (request.cacheWrite ?? 0)); - return rate ? costBreakdownWithPricing({ ...request, input: fresh, costUsd: null }, rate.pricing) : null; - }); - if (parts.every((part) => part !== null)) { - const sum = parts.reduce((total, part) => ({ inUsd: total.inUsd + part!.inUsd, - cacheUsd: total.cacheUsd + part!.cacheUsd, cacheWriteUsd: total.cacheWriteUsd + (part!.cacheWriteUsd ?? 0), - outUsd: total.outUsd + part!.outUsd }), { inUsd: 0, cacheUsd: 0, cacheWriteUsd: 0, outUsd: 0 }); - const { cacheWriteUsd, ...known } = sum; - breakdown = { ...known, ...(parts.some((part) => part!.cacheWriteUsd !== undefined) ? { cacheWriteUsd } : {}), otherUsd: null }; + const unavailable = { effectiveCostUsd: null, costSource: null, costBreakdown: null, partialCost: true, pricingWarning: null }; + if (tokenScope(usage) !== 'turn' || !usageSemantics(usage.provider)) return unavailable; + const semantics = usageSemantics(usage.provider); + const requests = usage.requests?.length ? usage.requests : [usage]; + let breakdown: CostBreakdown | null = null; + let source: Exclude | null = null; + let incomplete = false; + let pricingWarning: string | null = null; + for (const request of requests) { + const resolved = resolvePricing(usage.provider, usage.model, promptTokens(usage.provider, request)); + if (!resolved) { incomplete = true; continue; } + if ('warning' in resolved) { pricingWarning ??= resolved.warning; incomplete = true; continue; } + const input = freshInput(semantics, request.input, request.cached, request.cacheWrite); + const part = costBreakdownWithPricing({ ...request, input, costUsd: null }, resolved.pricing); + if (!part) { incomplete = true; continue; } + source = resolved.source; + incomplete ||= partialCost(part); + if (!breakdown) breakdown = { ...part }; + else { + for (const field of ['inUsd', 'cacheUsd', 'cacheWriteUsd', 'outUsd'] as const) { + const value = part[field]; + if (value != null) breakdown[field] = (breakdown[field] ?? 0) + value; + } } } - return { - effectiveCostUsd: breakdown ? breakdown.inUsd + breakdown.cacheUsd + (breakdown.cacheWriteUsd ?? 0) + breakdown.outUsd : null, - costSource: resolved?.source ?? null, - costBreakdown: breakdown, - }; + return { effectiveCostUsd: knownCost(breakdown), costSource: source, costBreakdown: breakdown, partialCost: incomplete || !breakdown, pricingWarning }; } export function enrichTurn(record: TurnRecord, seq: number): TurnRow { - const input = freshInput(usageSemantics(record.provider, record.model), record.input, record.cached, record.cacheWrite); + const scope = tokenScope(record); + const input = freshInput(usageSemantics(record.provider), record.input, record.cached, record.cacheWrite); const estimated = estimateUsageCost(record); - const estimate = estimated.effectiveCostUsd; - const breakdown = estimated.costBreakdown && record.costUsd !== null - ? { ...estimated.costBreakdown, otherUsd: record.costUsd - estimate! } + const reported = scope === 'turn' ? record.costUsd : null; + const breakdown = estimated.costBreakdown && reported !== null + ? { ...estimated.costBreakdown, otherUsd: reported - estimated.effectiveCostUsd! } : estimated.costBreakdown; return { - ...record, - input, - seq, - effectiveCostUsd: record.costUsd ?? estimate, - costSource: record.costUsd !== null ? "reported" : estimated.costSource, + ...record, input, seq, tokenScope: scope, + // Historical session/unknown rows counted snapshots, not model calls. + modelCalls: scope === 'turn' ? record.modelCalls : null, + promptTokens: promptTokens(record.provider, record), + incompleteUsage: scope !== 'turn' || record.quality !== 'exact' || !!record.usageGap + || input === null || record.cached === null || record.cacheWrite == null || record.output === null, + partialCost: reported === null && (estimated.partialCost || record.quality !== 'exact' || !!record.usageGap), + pricingWarning: estimated.pricingWarning, + costUsd: reported, + effectiveCostUsd: reported ?? estimated.effectiveCostUsd, + costSource: reported !== null ? 'reported' : estimated.costSource, costBreakdown: breakdown, }; } diff --git a/server/read-model.test.ts b/server/read-model.test.ts new file mode 100644 index 0000000..00bba99 --- /dev/null +++ b/server/read-model.test.ts @@ -0,0 +1,36 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { finalizeTurn } from '../shared/aggregate.ts'; +import { enrichTurn } from './pricing.ts'; +import { summaryFromRows } from './read-model.ts'; + +const base = { agentId: 'a', turnId: 't', provider: 'omp', model: 'gpt-model', startedAt: null, + endedAt: '2026-10-14T00:00:00Z', status: 'completed' as const, observations: [], prevSessionCostUsd: null }; + +test('historical cumulative snapshots remain visible without duplicating turn totals or prices', () => { + const first = finalizeTurn({ ...base, finalUsage: { inputTokens: 1000, cachedInputTokens: 4000, outputTokens: 100, totalCostUsd: 1 } }); + const next = finalizeTurn({ ...base, finalUsage: { inputTokens: 2000, cachedInputTokens: 9000, outputTokens: 200, totalCostUsd: 2 } }); + const rows = [first, next].map((r, i) => enrichTurn({ ...r, tokenScope: undefined, costUsd: 9 }, i + 1)); + assert.equal(rows[1].input, 2000); + assert.equal(rows[1].tokenScope, 'session'); + assert.equal(rows[1].effectiveCostUsd, null); + const summary = summaryFromRows(rows); + assert.equal(summary.input, null); + assert.equal(summary.cached, null); + assert.equal(summary.effectiveCostUsd, null); + assert.equal(summary.unmeasuredTurns, 2); + assert.equal(summary.unpricedTurns, 2); +}); + +test('known turn values and measured zero survive beside unknown scope without hiding coverage', () => { + const known = finalizeTurn({ ...base, provider: 'claude', finalUsage: { inputTokens: 0, cachedInputTokens: 10, cacheWriteInputTokens: 0, outputTokens: 0, totalCostUsd: 1 } }); + const unknown = finalizeTurn({ ...base, provider: 'custom', finalUsage: { inputTokens: 9000, outputTokens: 9000, totalCostUsd: 20 } }); + const summary = summaryFromRows([enrichTurn(known, 1), enrichTurn(unknown, 2)]); + assert.equal(summary.input, 0); + assert.equal(summary.cached, 10); + assert.equal(summary.output, 0); + assert.equal(summary.effectiveCostUsd, 1); + assert.equal(summary.unmeasuredTurns, 1); + assert.equal(summary.incompleteTurns, 1); + assert.equal(summary.unpricedTurns, 1); +}); diff --git a/server/read-model.ts b/server/read-model.ts index ff9287b..8dcabea 100644 --- a/server/read-model.ts +++ b/server/read-model.ts @@ -3,13 +3,17 @@ import { allRecords, storeRevision } from './store.ts'; import { enrichTurn, pricingRevision } from './pricing.ts'; export const EMPTY_SUMMARY: Summary = { turns: 0, - input: 0, - cached: 0, - output: 0, + input: null, + cached: null, + output: null, costUsd: null, effectiveCostUsd: null, estimatedTurns: 0, unpricedTurns: 0, + unmeasuredTurns: 0, + incompleteTurns: 0, + partialCostTurns: 0, + pricingWarningTurns: 0, }; /** Sums enriched rows, so a caller can summarize any subset of the read model. */ @@ -17,14 +21,22 @@ export function summaryFromRows(rows: readonly TurnRow[]): Summary { const summary: Summary = { ...EMPTY_SUMMARY }; for (const row of rows) { summary.turns += 1; - summary.input += row.input ?? 0; - summary.cached += row.cached ?? 0; + if (row.incompleteUsage) summary.incompleteTurns += 1; + if (row.pricingWarning) summary.pricingWarningTurns += 1; + if (row.tokenScope !== 'turn') { + summary.unmeasuredTurns += 1; + summary.unpricedTurns += 1; + continue; + } + if (row.input !== null) summary.input = (summary.input ?? 0) + row.input; + if (row.cached !== null) summary.cached = (summary.cached ?? 0) + row.cached; if (row.cacheWrite != null) summary.cacheWrite = (summary.cacheWrite ?? 0) + row.cacheWrite; - summary.output += row.output ?? 0; + if (row.output !== null) summary.output = (summary.output ?? 0) + row.output; if (row.costUsd !== null) summary.costUsd = (summary.costUsd ?? 0) + row.costUsd; if (row.effectiveCostUsd !== null) { summary.effectiveCostUsd = (summary.effectiveCostUsd ?? 0) + row.effectiveCostUsd; if (row.costSource !== "reported") summary.estimatedTurns += 1; + if (row.partialCost) summary.partialCostTurns += 1; } else { summary.unpricedTurns += 1; } diff --git a/server/tracker.ts b/server/tracker.ts index c2565d9..a89d669 100644 --- a/server/tracker.ts +++ b/server/tracker.ts @@ -1,12 +1,13 @@ -import { closeTurn, createState, markUsageGap, reconcileUsage, noteUsage, startLifecycleTurn, streamTurn, type AgentState, type AgentSnapshotLike } from "./turns.ts"; +import { closeTurn, createState, markUsageGap, reconcileUsage, noteUsage, startLifecycleTurn, streamTurn, endLifecycleTurn, type AgentState, type AgentSnapshotLike } from "./turns.ts"; import { loadJournal, saveJournal } from "./journal.ts"; import type { PluginLifecycleEvents } from "@getpaseo/plugin/server"; import { subscribeAgents } from "../shared/agents.ts"; import type { CatalogSubscription } from '../shared/catalog-subscription.ts'; import type { PaseoAgent, PaseoAgentTimelineEvent, PaseoApi } from "./paseo-types.ts"; -import type { AgentUsageRow, InFlight, OverviewResult, Summary, SyncResult, TurnRecord } from "../shared/ledger.ts"; -import { freshInput, usageSemantics, providerSemantics } from "../shared/semantics.ts"; -import { ensurePricing, estimateUsageCost } from "./pricing.ts"; +import type { AgentUsageRow, InFlight, OverviewResult, Summary, SyncResult, TurnRecord, UsageSnapshot } from "../shared/ledger.ts"; +import { freshInput, tokenScope, usageSemantics } from "../shared/semantics.ts"; +import { finalizeTurn } from '../shared/aggregate.ts'; +import { ensurePricing, enrichTurn } from "./pricing.ts"; import { allRecords, flushStore, @@ -75,7 +76,28 @@ function stateFor(agentId: string): AgentState { return state; } export function observeStream(payload: PaseoAgentTimelineEvent): void { - if (!stopped) persist(streamTurn(stateFor(payload.agentId), payload)); + if (stopped) return; + const state = stateFor(payload.agentId); + persist(streamTurn(state, payload)); + if (state.open?.ending) scheduleSettle(payload.agentId, state); +} +// OMP publishes its final usage only when it changed, so a missing drain +// snapshot is normal. The bound covers its async stats read, not its 3s poll. +const SESSION_DRAIN_MS = 2000; +const STREAM_SETTLE_MS = 250; +function scheduleSettle(agentId: string, state: AgentState): void { + const open = state.open; + if (!open?.ending) return; + const key = open.key; + if (terminalTimers.has(key)) return; + const session = tokenScope({ provider: open.provider ?? state.provider }) === 'session'; + // The stream carries terminal usage; the hook independently carries outcome. + // Give the usage a brief chance to settle, then finalize observed facts only. + const settle = () => { + terminalTimers.delete(key); + if (state.open?.key === key) persist(closeTurn(agentId, state, state.open.ending?.status ?? 'unknown', null, new Date().toISOString())); + }; + terminalTimers.set(key, { timer: setTimeout(settle, session ? SESSION_DRAIN_MS : STREAM_SETTLE_MS), settle }); } export async function observeStart(event: PluginLifecycleEvents["agent.turn_started"], paseo: PaseoApi): Promise { await ensureTracker(paseo); @@ -89,17 +111,9 @@ export async function observeEnd(event: PluginLifecycleEvents["agent.turn_ended" await ensureTracker(paseo); if (stopped) return; const state = stateFor(event.agent.id); - if (!state.open || state.open.turnId !== event.turnId) return; - const key = state.open.key; - const endedAt = new Date().toISOString(); - clearTimeout(terminalTimers.get(key)?.timer); - // The stream carries terminal usage; the hook independently carries outcome. - // Give the stream a brief chance to settle, then finalize observed facts only. - const settle = () => { - terminalTimers.delete(key); - if (state.open?.key === key) persist(closeTurn(event.agent.id, state, event.outcome.kind, null, endedAt)); - }; - terminalTimers.set(key, { timer: setTimeout(settle, 250), settle }); + if (!endLifecycleTurn(state, event.turnId, event.outcome.kind, new Date().toISOString())) return; + checkpoint(); + scheduleSettle(event.agent.id, state); } function retireAgent(id: string, gap = false): void { needsReconcile.delete(id); @@ -168,17 +182,19 @@ export function ensureTracker(paseo: PaseoApi): Promise { if (open && !agents.get(event.agentId)?.open) needsReconcile.delete(event.agentId); }, gap: (id) => { needsReconcile.add(id); markUsageGap(stateFor(id)); checkpoint(); }, - restored: async (id) => { - const revision = agentRevisions.get(id); - const handle = paseo.agents.ref(id); - await handle.refresh(); - if (stopped || agentRevisions.get(id) !== revision) return; - const snapshot = handle.current(); - if (!snapshot) { retireAgent(id, true); return; } - applyAgent(snapshot, true); - }, + restored: (id) => reconcileAgent(id, true), error: (id, error) => console.error('token-ledger: timeline subscription failed', id, error), }); + // Re-read one agent by ID; a ref resolves agents that lists may hide. + const reconcileAgent = async (id: string, restored: boolean) => { + const revision = agentRevisions.get(id); + const handle = paseo.agents.ref(id); + await handle.refresh(); + if (stopped || agentRevisions.get(id) !== revision) return; + const snapshot = handle.current(); + if (!snapshot) { catalog.remove(id); retireAgent(id, restored); return; } + applyAgent(snapshot, restored); + }; const applyAgent = (agent: PaseoAgent, restored = false) => { restored = needsReconcile.delete(agent.id) || restored; bump(agent.id); @@ -199,7 +215,18 @@ export function ensureTracker(paseo: PaseoApi): Promise { onUpdate: (update) => { if (stopped) return; if (update.kind === 'upsert') applyAgent(update.agent); - else { bump(update.agentId); catalog.remove(update.agentId); retireAgent(update.agentId); } + else { + // A remove can mean the agent left this list's filter (such as a + // background workspace), not that it was archived or deleted. + const id = update.agentId; + bump(id); + const revision = agentRevisions.get(id); + void reconcileAgent(id, false).catch((error) => { + console.error('token-ledger: could not confirm agent removal', id, error); + if (stopped || agentRevisions.get(id) !== revision) return; + catalog.remove(id); retireAgent(id); + }); + } }, onError: (error) => { for (const [id, state] of agents) { needsReconcile.add(id); markUsageGap(state); } @@ -249,47 +276,42 @@ export function stopTracker(): Promise { function inFlightFor(agentId: string): InFlight | null { const state = agents.get(agentId); const open = state?.open; - if (!open) return null; - const semantics = usageSemantics(open.provider, open.model); - let input: number | null = null; - let rawInput: number | null = null; - let cached: number | null = null; - let cacheWrite: number | null = null; - let output: number | null = null; - const observations = providerSemantics(open.provider).tokens === "request" ? open.observations : open.observations.slice(-1); - for (const observation of observations) { - const fresh = freshInput(semantics, observation.input, observation.cached, observation.cacheWrite); - if (observation.input !== null) rawInput = (rawInput ?? 0) + observation.input; - if (fresh !== null) input = (input ?? 0) + fresh; - if (observation.cached !== null) cached = (cached ?? 0) + observation.cached; - if (observation.cacheWrite != null) cacheWrite = (cacheWrite ?? 0) + observation.cacheWrite; - if (observation.output !== null) output = (output ?? 0) + observation.output; - } - const usage = open.lastUsage; - const estimated = estimateUsageCost({ - provider: open.provider, - model: open.model, - input: rawInput, - cached, - cacheWrite, - output, - ...(providerSemantics(open.provider).tokens === "request" - ? { requests: open.observations.map(({ cost, ...tokens }) => tokens) } - : {}), - }); + if (!state || !open) return null; + const record = finalizeTurn({ agentId, turnId: open.turnId, provider: open.provider, model: open.model, + startedAt: open.startedAt, endedAt: new Date().toISOString(), status: 'unknown', + observations: open.observations, finalUsage: null, prevSessionCostUsd: state.prevSessionCostUsd }); + if (open.usageGap) record.usageGap = true; + if (state.costBaselineUncertain) record.costUsd = null; + const row = enrichTurn(record, 1); return { ...(open.usageGap ? { usageGap: true } : {}), - turnId: open.turnId, - startedAt: open.startedAt, - modelCalls: open.observations.length, - input, - cached, - cacheWrite, - output, - effectiveCostUsd: estimated.effectiveCostUsd, - costSource: estimated.costSource, - ctxUsed: typeof usage?.contextWindowUsedTokens === "number" ? usage.contextWindowUsedTokens : null, - ctxMax: typeof usage?.contextWindowMaxTokens === "number" ? usage.contextWindowMaxTokens : null, + turnId: open.turnId, startedAt: open.startedAt, modelCalls: row.modelCalls, + tokenScope: row.tokenScope, promptTokens: row.promptTokens, partialCost: row.partialCost, + pricingWarning: row.pricingWarning, + input: row.input, cached: row.cached, cacheWrite: row.cacheWrite, output: row.output, + effectiveCostUsd: row.effectiveCostUsd, costSource: row.costSource, + ctxUsed: open.lastUsage?.contextWindowUsedTokens ?? null, + ctxMax: open.lastUsage?.contextWindowMaxTokens ?? null, + }; +} + +function usageSnapshotFor(agentId: string): UsageSnapshot | null { + const state = agents.get(agentId); + const previous = lastRecordForAgent(agentId); + const provider = state?.provider ?? previous?.provider ?? null; + const scope = tokenScope({ provider }); + if (scope === 'turn') return null; + const compatible = !state?.sessionId || state.sessionId === previous?.sessionId; + const observation = state?.lastObservation ?? (compatible && previous ? { + input: previous.input, cached: previous.cached, cacheWrite: previous.cacheWrite, + output: previous.output, cost: previous.sessionCostUsd, + } : null); + if (!observation) return null; + return { + scope, input: freshInput(usageSemantics(provider), observation.input, observation.cached, observation.cacheWrite), + cached: observation.cached, cacheWrite: observation.cacheWrite ?? null, output: observation.output, + reportedCostUsd: observation.cost, + observedAt: state?.lastObservedAt ?? previous?.endedAt ?? null, }; } @@ -311,6 +333,7 @@ export async function handleSync( records: input.knownRecordsRevision === recordsRevision ? [] : (group?.rows.slice(0, input.limit ?? 50) ?? []), summary, ctx: agents.get(input.agentId)?.lastCtx ?? null, + usageSnapshot: usageSnapshotFor(input.agentId), }; } @@ -340,14 +363,18 @@ export async function handleOverview(input: { since?: string }, context: { paseo // A range lists only the agents that used tokens in it or that work now. if (since !== null && summary.turns === 0 && open === null) continue; totals.turns += summary.turns; - totals.input += summary.input; - totals.cached += summary.cached; + if (summary.input !== null) totals.input = (totals.input ?? 0) + summary.input; + if (summary.cached !== null) totals.cached = (totals.cached ?? 0) + summary.cached; if (summary.cacheWrite != null) totals.cacheWrite = (totals.cacheWrite ?? 0) + summary.cacheWrite; - totals.output += summary.output; + if (summary.output !== null) totals.output = (totals.output ?? 0) + summary.output; if (summary.costUsd !== null) totals.costUsd = (totals.costUsd ?? 0) + summary.costUsd; if (summary.effectiveCostUsd !== null) totals.effectiveCostUsd = (totals.effectiveCostUsd ?? 0) + summary.effectiveCostUsd; totals.estimatedTurns += summary.estimatedTurns; totals.unpricedTurns += summary.unpricedTurns; + totals.unmeasuredTurns += summary.unmeasuredTurns; + totals.incompleteTurns += summary.incompleteTurns; + totals.partialCostTurns += summary.partialCostTurns; + totals.pricingWarningTurns += summary.pricingWarningTurns; const row: AgentUsageRow = { agentId, title: snapshot?.title ?? null, diff --git a/server/turns.test.ts b/server/turns.test.ts index 5697e93..16a34ed 100644 --- a/server/turns.test.ts +++ b/server/turns.test.ts @@ -1,7 +1,7 @@ import assert from 'node:assert/strict'; import { test } from 'node:test'; import type { PaseoAgentStream } from '@getpaseo/client'; -import { AgentStateSchema, createState, markUsageGap, reconcileUsage, noteUsage, startLifecycleTurn, streamTurn } from './turns.ts'; +import { AgentStateSchema, closeTurn, createState, endLifecycleTurn, markUsageGap, reconcileUsage, noteUsage, startLifecycleTurn, streamTurn } from './turns.ts'; const start = '2026-09-14T00:00:00.000Z'; const end = '2026-09-14T00:00:01.000Z'; const usage = { inputTokens: 100, cachedInputTokens: 40, outputTokens: 10 }; @@ -167,3 +167,112 @@ test('a Codex delivery gap does not mark every subsequent uninterrupted turn as const next = streamTurn(state, finish('t2'))!; assert.equal(next.usageGap, undefined); }); + +test('late OMP usage settles a deferred turn once, then only refreshes the snapshot', () => { + const state = createState(); + const activeTurn = { turnId: 't1', startedAt: start }; + noteUsage(state, { id: 'a', provider: 'omp', model: 'gpt-model', activeTurn, + runtimeInfo: { sessionId: 's' }, lastUsage: { inputTokens: 1000, cachedInputTokens: 4000, outputTokens: 100 } }); + assert.equal(streamTurn(state, event({ type: 'turn_completed', provider: 'omp', turnId: 't1' })), null); + // OMP publishes the drain while its activeTurnId still names the ending turn. + const record = noteUsage(state, { id: 'a', provider: 'omp', activeTurn, + lastUsage: { inputTokens: 1300, cachedInputTokens: 4500, outputTokens: 200 } })!; + assert.equal(record.tokenScope, 'session'); + assert.equal(record.input, 1300); + assert.equal(record.status, 'completed'); + assert.equal(record.endedAt, end); + assert.equal(state.open, null); + noteUsage(state, { id: 'a', provider: 'omp', activeTurn, + lastUsage: { inputTokens: 1400, cachedInputTokens: 4600, outputTokens: 210 } }); + assert.equal(state.open, null, 'a replayed activeTurn does not reopen the settled turn'); + const restored = AgentStateSchema.parse(JSON.parse(JSON.stringify(state))); + assert.equal(restored.lastObservation?.input, 1400); + assert.equal(restored.lastObservation?.output, 210); + // A lower branch-view snapshot is not a fresh billing counter or a turn delta. + noteUsage(restored, { id: 'a', provider: 'omp', activeTurn: null, + lastUsage: { inputTokens: 100, cachedInputTokens: 0, outputTokens: 10 } }); + assert.equal(restored.lastObservation?.input, 100); + assert.equal(restored.open, null); + assert.equal(record.costUsd, null); +}); + +const omp = (type: 'turn_started' | 'turn_completed', turnId: string, timestamp = type === 'turn_started' ? start : end) => + event({ type, provider: 'omp', turnId }, timestamp); + +// Issue sample A (turn 83010c14): the second call's usage arrived 17ms after the terminal event. +test('OMP sample A: a final snapshot after the terminal event is not dropped', () => { + const state = createState(); + const activeTurn = { turnId: 't1', startedAt: start }; + streamTurn(state, omp('turn_started', 't1')); + noteUsage(state, { id: 'a', provider: 'omp', activeTurn, + lastUsage: { inputTokens: 2078, cachedInputTokens: 9984, outputTokens: 484, totalCostUsd: 0.00943752 } }); + assert.equal(streamTurn(state, omp('turn_completed', 't1')), null); + const record = noteUsage(state, { id: 'a', provider: 'omp', activeTurn, + lastUsage: { inputTokens: 2751, cachedInputTokens: 22016, outputTokens: 1103, totalCostUsd: 0.01553148 } })!; + assert.equal(record.input, 2751); assert.equal(record.cached, 22016); assert.equal(record.output, 1103); + assert.equal(record.sessionCostUsd, 0.01553148); + assert.equal(record.costUsd, null, 'OMP session cost is not differenced into turns'); + assert.equal(record.modelCalls, null, 'a snapshot count is not a model-call count'); +}); + +// Issue sample B (turn afd69f4c): the only call's usage arrived 9ms after the terminal event. +test('OMP sample B: a turn whose only snapshot drains late is not empty', () => { + const state = createState(); + noteUsage(state, { id: 'a', provider: 'omp', lastUsage: { inputTokens: 2751, cachedInputTokens: 22016, outputTokens: 1103 } }); + streamTurn(state, omp('turn_started', 't2')); + assert.equal(streamTurn(state, omp('turn_completed', 't2')), null); + const record = noteUsage(state, { id: 'a', provider: 'omp', activeTurn: { turnId: 't2', startedAt: start }, + lastUsage: { inputTokens: 3206, cachedInputTokens: 34688, outputTokens: 1410 } })!; + assert.equal(record.input, 3206); + assert.equal(record.source, 'last_observation'); + assert.equal(record.modelCalls, null); +}); + +test('OMP drain does not absorb a snapshot twice, and a new turn settles an unchanged drain', () => { + const state = createState(); + const s1 = { inputTokens: 100, cachedInputTokens: 0, outputTokens: 10 }; + streamTurn(state, omp('turn_started', 't1')); + streamTurn(state, omp('turn_completed', 't1')); + assert.equal(noteUsage(state, { id: 'a', provider: 'omp', activeTurn: { turnId: 't1', startedAt: start }, lastUsage: s1 })!.input, 100); + streamTurn(state, omp('turn_started', 't2', end)); + // A replay of the absorbed snapshot is not new usage for the next turn. + assert.equal(noteUsage(state, { id: 'a', provider: 'omp', activeTurn: { turnId: 't2', startedAt: end }, lastUsage: s1 }), null); + assert.equal(state.open?.observations.length, 0); + // Unchanged usage publishes no drain; the next turn start settles the ending one. + streamTurn(state, omp('turn_completed', 't2', '2026-09-14T00:00:02.000Z')); + const second = streamTurn(state, omp('turn_started', 't3', '2026-09-14T00:00:03.000Z'))!; + assert.equal(second.turnId, 't2'); + assert.equal(second.status, 'completed', 'a known outcome is not rewritten as canceled'); + assert.equal(second.endedAt, '2026-09-14T00:00:02.000Z'); + assert.equal(second.source, 'none'); + assert.equal(state.open?.turnId, 't3'); +}); + +test('OMP failures settle at once; Codex still counts its requests', () => { + const state = createState(); + streamTurn(state, omp('turn_started', 't1')); + assert.ok(streamTurn(state, event({ type: 'turn_failed', provider: 'omp', turnId: 't1', error: 'x' }))); + const codex = createState(); streamTurn(codex, begin()); + noteUsage(codex, { id: 'a', lastUsage: usage }); + assert.equal(streamTurn(codex, finish())!.modelCalls, 1); +}); + +test('a deferred outcome survives a checkpoint and keeps its reported status and time', () => { + const state = createState(); + streamTurn(state, omp('turn_started', 't1')); + streamTurn(state, omp('turn_completed', 't1')); + const restored = AgentStateSchema.parse(JSON.parse(JSON.stringify(state))); + assert.equal(restored.open?.ending?.status, 'completed'); + const record = closeTurn('a', restored, 'canceled', null, '2026-09-14T00:00:09.000Z')!; + assert.equal(record.status, 'completed'); assert.equal(record.endedAt, end); +}); + +test('the stream outcome overrides an earlier hook outcome', () => { + const state = createState(); + streamTurn(state, omp('turn_started', 't1')); + assert.equal(endLifecycleTurn(state, 'other', 'failed', end), false); + assert.equal(endLifecycleTurn(state, 't1', 'unknown', '2026-09-14T00:00:00.500Z'), true); + streamTurn(state, omp('turn_completed', 't1')); + const record = noteUsage(state, { id: 'a', provider: 'omp', activeTurn: { turnId: 't1', startedAt: start }, lastUsage: usage })!; + assert.equal(record.status, 'completed'); assert.equal(record.endedAt, end); +}); diff --git a/server/turns.ts b/server/turns.ts index 71e67df..4f74231 100644 --- a/server/turns.ts +++ b/server/turns.ts @@ -3,6 +3,7 @@ import { z } from "zod"; import type { PaseoAgentTimelineEvent } from "./paseo-types.ts"; import { finalizeTurn, mergeObservation, sameTokens, tokenObservation, type UsageLike } from "../shared/aggregate.ts"; import type { TurnRecord } from "../shared/ledger.ts"; +import { tokenScope } from '../shared/semantics.ts'; const nullableCount = z.number().finite().nonnegative().nullable(); const observationSchema = z.object({ input: nullableCount, cached: nullableCount, cacheWrite: nullableCount.optional(), output: nullableCount, cost: nullableCount }); @@ -11,11 +12,15 @@ export const AgentStateSchema = z.object({ provider: z.string().nullable(), model: z.string().nullable(), sessionId: z.string().nullable(), open: z.object({ key: z.string(), turnId: z.string().nullable(), startedAt: z.string(), usageGap: z.boolean().optional(), + /** Known outcome whose settlement waits for usage that drains after the terminal event. */ + ending: z.object({ status: z.enum(['completed', 'failed', 'canceled', 'unknown']), endedAt: z.string(), + from: z.enum(['stream', 'hook']) }).optional(), provider: z.string().nullable(), model: z.string().nullable(), sessionId: z.string().nullable(), observations: z.array(observationSchema), lastUsage: contextSchema.nullable(), }).nullable(), prevSessionCostUsd: nullableCount, lastObservation: observationSchema.nullable(), costBaselineUncertain: z.boolean().optional(), + lastObservedAt: z.string().nullable().optional(), usageGapPending: z.boolean().optional(), lastCtx: z.object({ used: z.number(), max: z.number() }).nullable(), lastClosedTurnId: z.string().nullable(), lastClosedAt: z.string().nullable(), @@ -42,6 +47,8 @@ function openTurn(state: AgentState, turnId: string | null, startedAt: string) { export function closeTurn(agentId: string, state: AgentState, status: TurnRecord["status"], finalUsage: UsageLike | null, endedAt: string): TurnRecord | null { const open = state.open; if (!open) return null; + // Interruption or a drain timeout cannot overrule an outcome already reported. + if (open.ending) { status = open.ending.status; endedAt = open.ending.endedAt; } state.open = null; state.usageGapPending = false; state.lastClosedTurnId = open.turnId; @@ -75,6 +82,7 @@ function sessionChanged(agentId: string, state: AgentState, sessionId: string | state.costBaselineUncertain = false; state.usageGapPending = false; state.lastObservation = null; + state.lastObservedAt = null; state.lastClosedTurnId = null; state.lastClosedAt = null; } @@ -83,7 +91,7 @@ function sessionChanged(agentId: string, state: AgentState, sessionId: string | return interrupted; } export function noteUsage(state: AgentState, snapshot: AgentSnapshotLike): TurnRecord | null { - const interrupted = sessionChanged(snapshot.id, state, snapshot.runtimeInfo?.sessionId); + let interrupted = sessionChanged(snapshot.id, state, snapshot.runtimeInfo?.sessionId); if (snapshot.provider) state.provider = snapshot.provider; if (snapshot.model !== undefined) state.model = snapshot.model; // Fill metadata that wasn't available at start; don't reattribute an old turn @@ -92,20 +100,29 @@ export function noteUsage(state: AgentState, snapshot: AgentSnapshotLike): TurnR state.open.provider ??= state.provider; state.open.model ??= state.model; } + const active = snapshot.activeTurn; + if (active && state.open && active.turnId !== state.open.turnId) { + // A newer turn proves that an ending turn has finished draining. + if (!state.open.ending) return interrupted; + interrupted = closeTurn(snapshot.id, state, 'unknown', null, new Date().toISOString()); + } + const closedReplay = active && active.turnId === state.lastClosedTurnId && active.turnId !== state.open?.turnId + && (!active.startedAt || !state.lastClosedAt || Date.parse(active.startedAt) <= Date.parse(state.lastClosedAt)); + if (closedReplay && tokenScope(state) === 'turn') return interrupted; + const observation = tokenObservation(snapshot.lastUsage); + const isNew = observation !== null && (state.lastObservation === null || !sameTokens(state.lastObservation, observation)); + if (observation) { + state.lastObservation = state.lastObservation && !isNew + ? mergeObservation(state.lastObservation, observation) : observation; + state.lastObservedAt = new Date().toISOString(); + } const usage = snapshot.lastUsage; if (usage && Number.isFinite(usage.contextWindowUsedTokens) && Number.isFinite(usage.contextWindowMaxTokens)) { state.lastCtx = { used: usage.contextWindowUsedTokens!, max: usage.contextWindowMaxTokens! }; } - const active = snapshot.activeTurn; - if (active && active.turnId === state.lastClosedTurnId && active.turnId !== state.open?.turnId) { - if (!active.startedAt || !state.lastClosedAt || Date.parse(active.startedAt) <= Date.parse(state.lastClosedAt)) return interrupted; - state.lastClosedTurnId = null; // verified newer occurrence of a reused ID - } - if (active && state.open && active.turnId !== state.open.turnId) return interrupted; - const observation = tokenObservation(usage); - const isNew = observation !== null && (state.lastObservation === null || !sameTokens(state.lastObservation, observation)); - if (observation) state.lastObservation = state.lastObservation && !isNew - ? mergeObservation(state.lastObservation, observation) : observation; + // Cumulative/unknown late observations remain visible as snapshots only. + if (closedReplay) return interrupted; + if (active?.turnId === state.lastClosedTurnId) state.lastClosedTurnId = null; if (!state.open && !active) { state.usageGapPending = false; if (typeof usage?.totalCostUsd === "number" && Number.isFinite(usage.totalCostUsd) && usage.totalCostUsd >= 0) { @@ -128,6 +145,11 @@ export function noteUsage(state: AgentState, snapshot: AgentSnapshotLike): TurnR open.observations[last] = mergeObservation(open.observations[last], observation); } } + // The first changed session snapshot after the outcome is the final drain: + // the harness publishes it only once, and only when usage changed. + if (open.ending && isNew && tokenScope({ provider: open.provider ?? state.provider }) === 'session') { + return closeTurn(snapshot.id, state, open.ending.status, null, open.ending.endedAt) ?? interrupted; + } return interrupted; } @@ -153,6 +175,12 @@ export function reconcileUsage(state: AgentState, snapshot: AgentSnapshotLike): if (interrupted) records.push(interrupted); return records; } +/** Records a hook outcome; the stream outcome, when present, remains authoritative. */ +export function endLifecycleTurn(state: AgentState, turnId: string | null, status: TurnRecord['status'], at: string): boolean { + if (!state.open || state.open.turnId !== turnId) return false; + state.open.ending ??= { status, endedAt: at, from: 'hook' }; + return true; +} export function startLifecycleTurn(agentId: string, state: AgentState, turnId: string | null, at: string): TurnRecord | null { // A delayed hook must not resurrect the turn already completed on the stream. if (state.open?.turnId === turnId || state.lastClosedTurnId === turnId) return null; @@ -183,11 +211,18 @@ export function streamTurn(state: AgentState, payload: PaseoAgentTimelineEvent): } case "turn_completed": case "turn_failed": - case "turn_canceled": + case "turn_canceled": { if (event.turnId && state.open?.turnId && event.turnId !== state.open.turnId) return null; - return closeTurn(payload.agentId, state, - event.type === "turn_completed" ? "completed" : event.type === "turn_failed" ? "failed" : "canceled", - event.type === "turn_completed" ? event.usage ?? null : null, at); + if (!state.open) return null; + const status = event.type === "turn_completed" ? "completed" : event.type === "turn_failed" ? "failed" : "canceled"; + const usage = event.type === "turn_completed" ? event.usage ?? null : null; + if (state.open.ending?.from !== 'stream') state.open.ending = { status, endedAt: at, from: 'stream' }; + // OMP reports no terminal usage and publishes its final session snapshot + // only after turn_completed, on the separate agent_update channel. + if (status === 'completed' && !tokenObservation(usage) + && tokenScope({ provider: state.open.provider ?? state.provider }) === 'session') return null; + return closeTurn(payload.agentId, state, status, usage, at); + } default: return null; } } diff --git a/shared/agents.test.ts b/shared/agents.test.ts index fce3bbb..9b29e0e 100644 --- a/shared/agents.test.ts +++ b/shared/agents.test.ts @@ -13,8 +13,8 @@ test('one-off catalog reads traverse all pages without creating subscriptions', } } } as unknown as PaseoApi; assert.deepEqual((await listAgents(paseo)).map(a => a.id), ['a', 'b']); assert.deepEqual(calls, [ - { page: { limit: 200 } }, - { page: { limit: 200, cursor: 'next' } }, + { filter: { includeBackground: true }, page: { limit: 200 } }, + { filter: { includeBackground: true }, page: { limit: 200, cursor: 'next' } }, ]); }); @@ -34,7 +34,7 @@ test('real 0.9 SDK accepts host-assigned catalog IDs, forwards restored pages an }); try { await subscription.ready; - assert.deepEqual(calls, [{ subscribe: {}, page: { limit: 200 } }]); + assert.deepEqual(calls, [{ subscribe: {}, filter: { includeBackground: true }, page: { limit: 200 } }]); assert.deepEqual(snapshots, [['a', 'b']]); owned.snapshot({ entries: [{ agent: { id: 'new' } }], pageInfo: { nextCursor: 'next' } }); await new Promise((resolve) => setImmediate(resolve)); @@ -46,3 +46,43 @@ test('repeated pagination cursor fails instead of looping forever', async () => const paseo = { agents: { list: async () => ({ entries: [], pageInfo: { nextCursor: 'same' } }) } } as unknown as PaseoApi; await assert.rejects(listAgents(paseo), /Repeated/); }); + +test('hosts without background workspaces get unfiltered lists on every page', async () => { + const calls: unknown[] = []; + const first = { entries: [{ agent: { id: 'a' } }], pageInfo: { nextCursor: 'next' } }; + const owned = ownedSnapshot(first); + // A 0.12+ client SDK rejects the filter before sending it to an older host. + const paseo = { agents: { list: async (options: { filter?: { includeBackground?: boolean }; subscribe?: {}; page: { cursor?: string } }) => { + calls.push(options); + if (options.filter?.includeBackground) throw new Error('Update the host to use background workspaces.'); + if (options.page.cursor) return { entries: [{ agent: { id: 'b' } }], pageInfo: { nextCursor: null } }; + return options.subscribe ? owned.page : first; + } } } as unknown as PaseoApi; + assert.deepEqual((await listAgents(paseo)).map((a) => a.id), ['a', 'b']); + assert.deepEqual(calls.splice(0), [ + { filter: { includeBackground: true }, page: { limit: 200 } }, + { filter: {}, page: { limit: 200 } }, + { filter: {}, page: { limit: 200, cursor: 'next' } }, + ]); + const snapshots: string[][] = []; + const subscription = subscribeAgents(paseo, { + onSnapshot: (agents) => snapshots.push(agents.map((agent) => agent.id)), + onUpdate: () => {}, onError: (error) => { throw error; }, + }); + try { + await subscription.ready; + assert.deepEqual(snapshots, [['a', 'b']]); + assert.deepEqual(calls, [ + { subscribe: {}, filter: { includeBackground: true }, page: { limit: 200 } }, + { subscribe: {}, filter: {}, page: { limit: 200 } }, + { filter: {}, page: { limit: 200, cursor: 'next' } }, + ]); + } finally { await subscription.release(); } +}); + +test('other list failures are not retried without the filter', async () => { + let calls = 0; + const paseo = { agents: { list: async () => { calls++; throw new Error('offline'); } } } as unknown as PaseoApi; + await assert.rejects(listAgents(paseo), /offline/); + assert.equal(calls, 1); +}); diff --git a/shared/agents.ts b/shared/agents.ts index 7ff259d..799682b 100644 --- a/shared/agents.ts +++ b/shared/agents.ts @@ -1,3 +1,4 @@ +import { includingBackground } from './background.ts'; import { subscribeCatalog, type CatalogSubscription, type OwnedSubscription } from './catalog-subscription.ts'; // Shared code cannot import host client or server SDK types. Callers pass the @@ -6,26 +7,28 @@ type AgentPage = { entries: readonly { agent: A }[]; pageInfo: { nextCursor?: export type AgentUpdate = { kind: 'upsert'; agent: A } | { kind: 'remove'; agentId: string }; export interface AgentCatalogApi { readonly agents: { - list(options: { subscribe: {}; page: { limit: number } }): Promise & { subscription: OwnedSubscription> }>; - list(options: { page: { limit: number; cursor?: string } }): Promise>; + list(options: { subscribe: {}; filter: {}; page: { limit: number } }): Promise & { subscription: OwnedSubscription> }>; + list(options: { filter: {}; page: { limit: number; cursor?: string } }): Promise>; }; } /** A list page is not the complete catalog. Subscribe only on the first page. */ -export async function listAgents(paseo: AgentCatalogApi): Promise { - const agents = new Map(); - let cursor: string | undefined; - const seen = new Set(); - do { - const page = await paseo.agents.list({ - page: { limit: 200, ...(cursor ? { cursor } : {}) }, - }); - for (const { agent } of page.entries) agents.set(agent.id, agent); - cursor = page.pageInfo.nextCursor ?? undefined; - if (cursor && seen.has(cursor)) throw new Error("Repeated agent pagination cursor"); - if (cursor) seen.add(cursor); - } while (cursor); - return [...agents.values()]; +export function listAgents(paseo: AgentCatalogApi): Promise { + return includingBackground(async (filter) => { + const agents = new Map(); + let cursor: string | undefined; + const seen = new Set(); + do { + const page = await paseo.agents.list({ + filter, page: { limit: 200, ...(cursor ? { cursor } : {}) }, + }); + for (const { agent } of page.entries) agents.set(agent.id, agent); + cursor = page.pageInfo.nextCursor ?? undefined; + if (cursor && seen.has(cursor)) throw new Error("Repeated agent pagination cursor"); + if (cursor) seen.add(cursor); + } while (cursor); + return [...agents.values()]; + }); } export function subscribeAgents(paseo: AgentCatalogApi, handlers: { @@ -33,9 +36,15 @@ export function subscribeAgents(paseo: AgentCatalogApi onUpdate(update: AgentUpdate): void; onError(error: unknown): void; }): CatalogSubscription { + // Pages must use the same filter as the subscription that opened them. + let filter: {} = {}; return subscribeCatalog({ - open: () => paseo.agents.list({ subscribe: {}, page: { limit: 200 } }), - next: (cursor) => paseo.agents.list({ page: { limit: 200, cursor } }), + open: () => includingBackground(async (option) => { + const page = await paseo.agents.list({ subscribe: {}, filter: option, page: { limit: 200 } }); + filter = option; + return page; + }), + next: (cursor) => paseo.agents.list({ filter, page: { limit: 200, cursor } }), entries: (page) => page.entries.map(({ agent }) => agent), id: (agent) => agent.id, update: (value) => { diff --git a/shared/aggregate.test.ts b/shared/aggregate.test.ts index 34eea4c..0313f50 100644 --- a/shared/aggregate.test.ts +++ b/shared/aggregate.test.ts @@ -104,13 +104,13 @@ test("cumulative session cost produces a per-turn delta", () => { assert.equal(record.sessionCostUsd, 0.08); }); -test("cost lower than previous raw is treated as a reset, raw kept", () => { +test("a lower cumulative cost is not a verified session reset", () => { const record = finalizeTurn({ ...base, finalUsage: { inputTokens: 10, outputTokens: 10, totalCostUsd: 0.02 }, prevSessionCostUsd: 0.05, }); - assert.equal(record.costUsd, 0.02); + assert.equal(record.costUsd, null); assert.equal(record.sessionCostUsd, 0.02); }); @@ -165,3 +165,13 @@ test('Codex includes an unseen terminal request and retains raw per-request coun assert.equal(record.input, 220); assert.equal(record.requests?.length, 2); assert.equal(record.quality, 'partial'); }); + +test('partial request fields do not produce falsely complete component totals', () => { + const record = finalizeTurn({ ...base, provider: 'codex', observations: [ + { input: 100, cached: 80, cacheWrite: 10, output: 20, cost: null }, + { input: 120, cached: 90, output: null, cost: null }, + ] }); + assert.equal(record.input, 220); + assert.ok(record.cacheWrite == null); + assert.equal(record.output, null); +}); diff --git a/shared/aggregate.ts b/shared/aggregate.ts index 4ccb7c2..c5ef840 100644 --- a/shared/aggregate.ts +++ b/shared/aggregate.ts @@ -1,4 +1,4 @@ -import { providerSemantics } from "./semantics.ts"; +import { providerSemantics, tokenScope } from "./semantics.ts"; import type { TurnRecord } from "./ledger.ts"; /** Reducer input. Paseo 0.8 exposes these fields except cacheWriteInputTokens. */ @@ -54,7 +54,8 @@ function sumField(observations: Observation[], field: "input" | "cached" | "cach let total: number | null = null; for (const observation of observations) { const value = observation[field]; - if (value != null) total = (total ?? 0) + value; + if (value == null) return null; + total = (total ?? 0) + value; } return total; } @@ -115,13 +116,16 @@ export function finalizeTurn(args: FinalizeInput): TurnRecord { const rawCost = num(args.finalUsage?.totalCostUsd) ?? observations.at(-1)?.cost ?? null; let costUsd: number | null = semantics.cost === "session" ? rawCost : null; - if (semantics.cost === "session" && rawCost !== null && args.prevSessionCostUsd !== null && rawCost >= args.prevSessionCostUsd) { - costUsd = Number((rawCost - args.prevSessionCostUsd).toFixed(6)); + if (semantics.cost === "session" && rawCost !== null && args.prevSessionCostUsd !== null) { + // Only a verified session change resets the baseline; a drop is not proof. + costUsd = rawCost >= args.prevSessionCostUsd + ? Number((rawCost - args.prevSessionCostUsd).toFixed(6)) : null; } const durationMs = args.startedAt !== null ? Math.max(0, Date.parse(args.endedAt) - Date.parse(args.startedAt)) : null; + const scope = tokenScope(args); return { v: 1, // endedAt is part of the id because Paseo turnIds ("foreground-turn-N") @@ -130,6 +134,7 @@ export function finalizeTurn(args: FinalizeInput): TurnRecord { agentId: args.agentId, turnId: args.turnId, provider: args.provider, + tokenScope: scope, model: args.model, startedAt: args.startedAt, endedAt: args.endedAt, @@ -143,7 +148,8 @@ export function finalizeTurn(args: FinalizeInput): TurnRecord { sessionCostUsd: rawCost, costScope: semantics.cost, ...(semantics.tokens === "request" ? { requests: observations.map(({ cost, ...tokens }) => tokens) } : {}), - modelCalls: observations.length, + // For session/unknown scope, snapshots may not correspond 1:1 with model calls. + modelCalls: scope === 'turn' ? observations.length : null, quality, source, }; diff --git a/shared/background.ts b/shared/background.ts new file mode 100644 index 0000000..a2d1556 --- /dev/null +++ b/shared/background.ts @@ -0,0 +1,21 @@ +// Paseo 0.12 hides agents in background workspaces from agents.list, +// workspaces.list and their agent_update streams unless the request opts in. +// Typed as {} so SDK pins that predate the field still accept it; older hosts +// have no background workspaces and drop the unknown field. +const INCLUDE_BACKGROUND: {} = { includeBackground: true }; + +// A 0.12+ client SDK rejects the filter before sending it to a host that does +// not advertise background workspaces. +function isUnsupported(error: unknown): boolean { + return error instanceof Error && /use background workspaces/i.test(error.message); +} + +/** Lists with background workspaces included, or unfiltered on hosts that lack them. */ +export async function includingBackground(call: (filter: {}) => Promise): Promise { + try { + return await call(INCLUDE_BACKGROUND); + } catch (error) { + if (!isUnsupported(error)) throw error; + return call({}); + } +} diff --git a/shared/ledger.ts b/shared/ledger.ts index 038e84f..16ec693 100644 --- a/shared/ledger.ts +++ b/shared/ledger.ts @@ -8,6 +8,8 @@ export const TurnRecordSchema = z.object({ turnId: z.string().nullable(), provider: z.string().nullable(), model: z.string().nullable(), + /** Omitted in v1 history; inferred only from verified harness semantics. */ + tokenScope: z.enum(['turn', 'session', 'unknown']).optional(), /** Provider session identity, when observed; optional for pre-0.8 records. */ sessionId: z.string().nullable().optional(), startedAt: z.string().nullable(), @@ -17,11 +19,10 @@ export const TurnRecordSchema = z.object({ /** A subscription gap may have hidden requests or the terminal outcome. */ usageGap: z.boolean().optional(), /** - * Turn token totals. On disk these are exactly as reported upstream (some - * providers count cache reads inside input — see semantics.ts); rows served - * over RPC normalize verified inclusive providers to ordinary input, with - * cache reads and known cache writes separate. Unknown semantics pass through. - * Null when the provider reported nothing. + * Raw observed counts on disk. Only turn-scoped observations enter totals. + * Session/unknown snapshots remain visible but cannot be added across turns. + * RPC input is uncached input for known harnesses, possibly including writes + * when their count was not exposed. Null means unavailable, not zero. */ input: z.number().nullable(), cached: z.number().nullable(), @@ -39,8 +40,11 @@ export const TurnRecordSchema = z.object({ input: z.number().nullable(), cached: z.number().nullable(), output: z.number().nullable(), cacheWrite: z.number().finite().nonnegative().nullable().optional(), })).optional(), - /** Number of distinct token-bearing usage observations during the turn. */ - modelCalls: z.number(), + /** + * Number of distinct token-bearing usage observations during the turn; null for + * session/unknown scope where snapshots do not map 1:1 to model calls. + */ + modelCalls: z.number().nullable(), quality: z.enum(["exact", "partial", "unavailable"]), /** How the token totals were derived: turn_usage | summed_observations | last_observation | none */ source: z.string(), @@ -51,16 +55,23 @@ export const CostSourceSchema = z.enum(["reported", "override", "openrouter", "b export type CostSource = z.infer; export const CostBreakdownSchema = z.object({ - inUsd: z.number(), - cacheUsd: z.number(), - cacheWriteUsd: z.number().optional(), - outUsd: z.number(), + inUsd: z.number().nullable(), + cacheUsd: z.number().nullable(), + cacheWriteUsd: z.number().nullable().optional(), + outUsd: z.number().nullable(), otherUsd: z.number().nullable(), }); /** A record enriched at read time. The JSONL remains untouched and contains only upstream facts. */ export const TurnRowSchema = TurnRecordSchema.extend({ seq: z.number().int().positive(), + tokenScope: z.enum(['turn', 'session', 'unknown']), + /** Null when the prompt denominator cannot be established. */ + promptTokens: z.number().nullable(), + incompleteUsage: z.boolean(), + /** Some components or rates are unavailable; cost is only a partial estimate. */ + partialCost: z.boolean(), + pricingWarning: z.string().nullable(), effectiveCostUsd: z.number().nullable(), costSource: CostSourceSchema.nullable(), costBreakdown: CostBreakdownSchema.nullable(), @@ -71,7 +82,11 @@ export const InFlightSchema = z.object({ usageGap: z.boolean().optional(), turnId: z.string().nullable(), startedAt: z.string(), - modelCalls: z.number(), + modelCalls: z.number().nullable(), + tokenScope: z.enum(['turn', 'session', 'unknown']), + promptTokens: z.number().nullable(), + partialCost: z.boolean(), + pricingWarning: z.string().nullable(), input: z.number().nullable(), cached: z.number().nullable(), cacheWrite: z.number().nullable().optional(), @@ -86,15 +101,20 @@ export type InFlight = z.infer; export const SummarySchema = z.object({ turns: z.number(), - input: z.number(), - cached: z.number(), - cacheWrite: z.number().optional(), - output: z.number(), + input: z.number().nullable(), + cached: z.number().nullable(), + cacheWrite: z.number().nullable().optional(), + output: z.number().nullable(), costUsd: z.number().nullable(), /** Reported + estimated costs for priced turns. */ effectiveCostUsd: z.number().nullable(), estimatedTurns: z.number().int().nonnegative(), unpricedTurns: z.number().int().nonnegative(), + /** Observations that are not attributable to an individual turn. */ + unmeasuredTurns: z.number().int().nonnegative(), + incompleteTurns: z.number().int().nonnegative(), + partialCostTurns: z.number().int().nonnegative(), + pricingWarningTurns: z.number().int().nonnegative(), }); export type Summary = z.infer; @@ -103,6 +123,15 @@ export const CtxSchema = z.object({ max: z.number(), }); export type Ctx = z.infer; +export const UsageSnapshotSchema = z.object({ + scope: z.enum(['session', 'unknown']), + input: z.number().nullable(), cached: z.number().nullable(), + cacheWrite: z.number().nullable(), output: z.number().nullable(), + reportedCostUsd: z.number().nullable(), + observedAt: z.string().nullable(), +}); +export type UsageSnapshot = z.infer; + const SyncOutputSchema = z.object({ recordsRevision: z.string(), @@ -111,6 +140,8 @@ const SyncOutputSchema = z.object({ summary: SummarySchema, /** Last known context-window usage for the agent, in or out of a turn. */ ctx: CtxSchema.nullable(), + /** Latest cumulative/unknown observation, including late idle updates. Not a turn total. */ + usageSnapshot: UsageSnapshotSchema.nullable(), }); export type SyncResult = z.infer; diff --git a/shared/pricing.test.ts b/shared/pricing.test.ts index 3556c85..496ec02 100644 --- a/shared/pricing.test.ts +++ b/shared/pricing.test.ts @@ -1,6 +1,6 @@ import assert from "node:assert/strict"; import { test } from "node:test"; -import { cacheRatio, costBreakdown, costBreakdownWithPricing, modelPricing } from "./pricing.ts"; +import { cacheRatio, costBreakdown, costBreakdownWithPricing, knownCost, modelPricing, partialCost } from "./pricing.ts"; test("model matching covers known families and rejects unknown", () => { assert.deepEqual(modelPricing("claude-opus-5"), { input: 5, cacheRead: 0.5, output: 25 }); @@ -13,10 +13,10 @@ test("model matching covers known families and rejects unknown", () => { test('Opus 5.5 uses verified standard prices across aliases without guessing Fast Mode or future models', () => { const standard = { input: 4, cacheRead: 0.2, cacheWrite: 5, output: 20 }; - for (const model of ['claude-opus-5.5', 'claude-opus-5-5', 'anthropic/claude-opus-5.5', 'claude-opus-5-5[1m]']) { + for (const model of ['claude-opus-5.5', 'claude-opus-5-5', 'anthropic/claude-opus-5.5']) { assert.deepEqual(modelPricing(model), standard); } - for (const model of ['claude-opus-5.5-fast', 'claude-opus-5.5:batch', 'claude-opus-6', 'claude-opus-5.6']) { + for (const model of ['claude-opus-5-5[1m]', 'vendor/claude-opus-5-5', '["claude-opus-4","claude-opus-5"]', 'claude-opus-5.5-fast', 'claude-opus-5.5:batch', 'claude-opus-6', 'claude-opus-5.6']) { assert.equal(modelPricing(model), null); } assert.deepEqual(modelPricing('claude-opus-4-1-20250805'), { input: 15, cacheRead: 1.5, output: 75 }); @@ -71,16 +71,17 @@ test("breakdown is null for unknown models or missing usage", () => { }); test("zero-token turn with a real total attributes everything to otherUsd", () => { - const split = costBreakdown({ model: "claude-opus-5", input: 0, cached: 0, output: 0, costUsd: 0.5 }); - assert.deepEqual(split, { inUsd: 0, cacheUsd: 0, outUsd: 0, otherUsd: 0.5 }); + const split = costBreakdown({ model: "claude-opus-5", input: 0, cached: 0, cacheWrite: 0, output: 0, costUsd: 0.5 }); + assert.deepEqual(split, { inUsd: 0, cacheUsd: 0, cacheWriteUsd: 0, outUsd: 0, otherUsd: 0.5 }); }); -test("cacheRatio", () => { - assert.equal(cacheRatio(2000, 8000), 0.8); - assert.equal(cacheRatio(null, 8000), 1); - assert.equal(cacheRatio(2000, null), null); +test('cache rate requires a complete, consistent denominator', () => { + assert.equal(cacheRatio(8000, 10000), 0.8); + assert.equal(cacheRatio(8000, null), null); + assert.equal(cacheRatio(null, 10000), null); assert.equal(cacheRatio(0, 0), null); - assert.equal(cacheRatio(1000, 8000, 1000), 0.8); + assert.equal(cacheRatio(10001, 10000), null); + assert.equal(cacheRatio(0, 10000), 0); }); test('cache writes have an independent price and do not increase reported totals', () => { @@ -88,7 +89,19 @@ test('cache writes have an independent price and do not increase reported totals const split = costBreakdownWithPricing(usage, { input: 1, cacheRead: 0.1, cacheWrite: 2, output: 5 })!; assert.equal(split.inUsd, 0.00001); assert.equal(split.cacheWriteUsd, 0.00006); - assert.equal(split.inUsd + split.cacheUsd + split.cacheWriteUsd! + split.outUsd + split.otherUsd!, 1); - assert.equal(costBreakdownWithPricing(usage, { input: 1, cacheRead: 0.1, output: 5 })!.cacheWriteUsd, 0.00003); + assert.equal(knownCost(split)! + split.otherUsd!, 1); + assert.equal(costBreakdownWithPricing(usage, { input: 1, cacheRead: 0.1, output: 5 })!.cacheWriteUsd, null); assert.equal(costBreakdownWithPricing(usage, { input: 1, cacheRead: 0.1, cacheWrite: 0, output: 5 })!.cacheWriteUsd, 0); }); + +test('unknown token counts and rates stay unknown, while explicit zero remains free', () => { + const missing = costBreakdownWithPricing({ input: null, cached: 8000, output: null, costUsd: null }, { input: 1, cacheRead: 0.1, output: 5 })!; + assert.equal(missing.inUsd, null); + assert.equal(missing.outUsd, null); + assert.equal(missing.cacheWriteUsd, null); + assert.equal(knownCost(missing), 0.0008); + assert.equal(partialCost(missing), true); + const zero = costBreakdownWithPricing({ input: 0, cached: 0, cacheWrite: 0, output: 0, costUsd: null }, { input: 1, output: 5 })!; + assert.equal(knownCost(zero), 0); + assert.equal(partialCost(zero), false); +}); diff --git a/shared/pricing.ts b/shared/pricing.ts index d0e7067..41d4ae5 100644 --- a/shared/pricing.ts +++ b/shared/pricing.ts @@ -1,18 +1,17 @@ /** * Per-model pricing used to estimate a cost breakdown per turn. Upstream only * reports a total costUsd, never per-component costs, so the split shown in - * the UI is an estimate: token counts × list price. When a real total exists, - * the difference between it and the estimate is surfaced as `otherUsd` rather - * than smeared across the components. Missing upstream usage (including - * cache writes in Paseo 0.8) can contribute to that residual. + * the UI is an estimate: token counts × reference price. Reported runtime cost + * is not proof of an invoice. Its difference from known estimated components + * is a residual, not an inferred cache-write charge. * * Prices are $/MTok. Cache write is independent of the input/read rates; * neither cache lifetime nor a provider's write premium is inferred. */ export type ModelPricing = { input: number; - cacheRead: number; - /** Omitted in older price files; writes then use the ordinary input rate. */ + cacheRead?: number; + /** Missing means the write rate is unknown; zero is an explicitly free rate. */ cacheWrite?: number; output: number; }; @@ -33,20 +32,16 @@ const PRICING_TABLE: Array<[RegExp, ModelPricing]> = [ ]; export function canonicalModel(value: string): string { - let model = value.trim().toLowerCase(); - if (model.startsWith('[')) { - try { - const parsed: unknown = JSON.parse(model); - if (Array.isArray(parsed) && typeof parsed.at(-1) === 'string') model = parsed.at(-1); - } catch {} - } - model = model.split('/').at(-1) ?? model; - return model.replace(/\[1m\]$/i, '').replace(/[._]/g, '-').replace(/-+/g, '-'); + const model = value.trim().toLowerCase(); + // Standard Claude numeric versions use both 4.5 and 4-5 spellings. Preserve + // namespaces, context/mode suffixes and arbitrary model identities. + return /^(?:anthropic\/)?claude-(?:opus|sonnet|haiku|fable|mythos)-\d+(?:[.-]\d+)*$/.test(model) + ? model.replace(/\./g, '-') : model; } export function modelPricing(model: string | null): ModelPricing | null { if (!model) return null; - const canonical = canonicalModel(model).replace(/-\d{8}$/, ''); + const canonical = canonicalModel(model).replace(/^anthropic\//, '').replace(/-\d{8}$/, '').replace(/(?<=\d)\.(?=\d)/g, '-'); for (const [pattern, pricing] of PRICING_TABLE) { if (pattern.test(canonical)) return pricing; } @@ -54,15 +49,14 @@ export function modelPricing(model: string | null): ModelPricing | null { } export type CostBreakdown = { - inUsd: number; - cacheUsd: number; - cacheWriteUsd?: number; - outUsd: number; + inUsd: number | null; + cacheUsd: number | null; + cacheWriteUsd?: number | null; + outUsd: number | null; /** - * costUsd minus the list-price estimate of the known components; null when - * no real total was reported. Positive residual is cost the reported token - * counts can't account for (chiefly cache writes on Claude); a small - * negative residual just means the hardcoded list prices overestimate. + * Reported runtime cost minus the sum of known estimated components. + * This includes price/source differences as well as unobserved categories; + * it does not establish actual billing or identify missing token counts. */ otherUsd: number | null; }; @@ -94,22 +88,33 @@ export function costBreakdownWithPricing( pricing: ModelPricing, ): CostBreakdown | null { if (turn.input === null && turn.cached === null && turn.output === null && turn.cacheWrite == null) return null; - const inUsd = ((turn.input ?? 0) * pricing.input) / PER_MTOK; - const cacheUsd = ((turn.cached ?? 0) * pricing.cacheRead) / PER_MTOK; - const cacheWriteUsd = ((turn.cacheWrite ?? 0) * (pricing.cacheWrite ?? pricing.input)) / PER_MTOK; - const outUsd = ((turn.output ?? 0) * pricing.output) / PER_MTOK; - const otherUsd = turn.costUsd !== null ? turn.costUsd - (inUsd + cacheUsd + cacheWriteUsd + outUsd) : null; - return { inUsd, cacheUsd, ...(turn.cacheWrite != null ? { cacheWriteUsd } : {}), outUsd, otherUsd }; + const inUsd = turn.input === null ? null : turn.input * pricing.input / PER_MTOK; + const cacheUsd = turn.cached === null ? null : turn.cached === 0 ? 0 + : pricing.cacheRead === undefined ? null : turn.cached * pricing.cacheRead / PER_MTOK; + const cacheWriteUsd = turn.cacheWrite == null ? null : turn.cacheWrite === 0 ? 0 + : pricing.cacheWrite === undefined ? null : turn.cacheWrite * pricing.cacheWrite / PER_MTOK; + const outUsd = turn.output === null ? null : turn.output * pricing.output / PER_MTOK; + const known = (inUsd ?? 0) + (cacheUsd ?? 0) + (cacheWriteUsd ?? 0) + (outUsd ?? 0); + const otherUsd = turn.costUsd !== null ? turn.costUsd - known : null; + return { inUsd, cacheUsd, cacheWriteUsd, outUsd, otherUsd }; } -/** - * Share of prompt tokens served from cache: cached / (input + cached + writes). - * Expects input in canonical fresh-token form (see semantics.ts) — the server - * normalizes inclusive-input providers before values reach clients. - */ -export function cacheRatio(input: number | null, cached: number | null, cacheWrite?: number | null): number | null { - if (cached === null) return null; - const prompt = (input ?? 0) + cached + (cacheWrite ?? 0); - if (prompt <= 0) return null; +/** Ratio only when the complete prompt denominator is available. */ +export function cacheRatio(cached: number | null, prompt: number | null): number | null { + if (cached === null || prompt === null || prompt <= 0 || cached > prompt) return null; return cached / prompt; } + +/** Known components only; callers must preserve the separate completeness flag. */ +export function knownCost(split: CostBreakdown | null): number | null { + if (!split) return null; + let sum: number | null = null; + for (const value of [split.inUsd, split.cacheUsd, split.cacheWriteUsd, split.outUsd]) { + if (value != null) sum = (sum ?? 0) + value; + } + return sum; +} + +export function partialCost(split: CostBreakdown | null): boolean { + return !split || split.inUsd === null || split.cacheUsd === null || split.cacheWriteUsd == null || split.outUsd === null; +} diff --git a/shared/semantics.test.ts b/shared/semantics.test.ts index 2d4b185..ff6680d 100644 --- a/shared/semantics.test.ts +++ b/shared/semantics.test.ts @@ -1,35 +1,30 @@ -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { freshInput, usageSemantics } from "./semantics.ts"; +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { freshInput, promptTokens, tokenScope, usageSemantics } from './semantics.ts'; -test("semantics lookup matches provider first, then model, else null", () => { - assert.deepEqual(usageSemantics("claude-tokenrouter", null), { inputIncludesCached: false }); - assert.deepEqual(usageSemantics("codex-tokenrouter", null), { inputIncludesCached: true }); - assert.deepEqual(usageSemantics(null, "gpt-5.6-sol"), { inputIncludesCached: true }); - assert.deepEqual(usageSemantics("some-router", "claude-fable-5"), { inputIncludesCached: false }); - assert.equal(usageSemantics("mystery-provider", "mystery-model"), null); - assert.equal(usageSemantics(null, null), null); +test('harness-normalized input is not normalized a second time', () => { + assert.equal(freshInput(usageSemantics('omp'), 160290, 2511639), 160290); + assert.equal(freshInput(usageSemantics('claude-router'), 6, 70751), 6); + assert.equal(freshInput(usageSemantics('codex-router'), 213818, 213252), 566); + assert.equal(tokenScope({ provider: 'omp' }), 'session'); + assert.equal(tokenScope({ provider: 'custom-gpt' }), 'unknown'); + assert.equal(usageSemantics('not-claude'), null); }); -test("freshInput subtracts cache reads for inclusive-input providers", () => { - const codex = usageSemantics("codex-tokenrouter", null); - // Real gpt-5.6-sol turn: inputTokens counts the whole prompt, cache hits included. - assert.equal(freshInput(codex, 213818, 213252), 566); - assert.equal(freshInput(codex, 100, null), 100); - assert.equal(freshInput(codex, null, 5000), null); - // Clamped rather than negative if a provider ever over-reports cache reads. - assert.equal(freshInput(codex, 100, 150), 0); +test('missing and inconsistent cache counts cannot fabricate fresh input', () => { + const codex = usageSemantics('codex'); + assert.equal(freshInput(codex, 100, null), null); + assert.equal(freshInput(codex, 100, 150), null); + assert.equal(freshInput(codex, 100, 60, 30), 10); + assert.equal(freshInput(codex, null, 60), null); }); -test("freshInput passes through for exclusive-input and unknown providers", () => { - const claude = usageSemantics("claude-tokenrouter", null); - assert.equal(freshInput(claude, 78, 2_974_078), 78); - assert.equal(freshInput(null, 78, 2_974_078), 78); - assert.equal(freshInput(null, null, null), null); -}); - -test('cache writes are separate from fresh input only for inclusive providers', () => { - assert.equal(freshInput(usageSemantics('codex', null), 100, 60, 30), 10); - assert.equal(freshInput(usageSemantics('claude', null), 10, 60, 30), 10); - assert.equal(freshInput(null, 100, 60, 30), 100); +test('prompt coverage follows the raw harness convention, not missing-value arithmetic', () => { + const usage = { input: 6, cached: 70751 }; + assert.equal(promptTokens('claude', usage), null); + assert.equal(promptTokens('claude', { ...usage, cacheWrite: 45829 }), 116586); + assert.equal(promptTokens('claude', { ...usage, cacheWrite: 0 }), 70757); + assert.equal(promptTokens('codex', { input: 100, cached: 80 }), 100); + assert.equal(promptTokens('codex', { input: 100, cached: 120 }), null); + assert.equal(promptTokens('unknown', { input: 100, cached: 80, cacheWrite: 0 }), null); }); diff --git a/shared/semantics.ts b/shared/semantics.ts index 1021422..567c207 100644 --- a/shared/semantics.ts +++ b/shared/semantics.ts @@ -1,58 +1,36 @@ -/** - * Providers disagree on what `inputTokens` means. Anthropic reports only the - * fresh (uncached) prompt tokens, with cache reads separate; OpenAI-style - * usage counts the whole prompt, cache hits included, so `inputTokens` already - * contains `cachedInputTokens`. Paseo's AgentUsage passes both through without - * defining which convention applies. - * - * The ledger stores values exactly as reported (so history can always be - * reinterpreted) and normalizes at read time to the canonical form that loses - * no information: input = ordinary input, cached = cache reads, cacheWrite = - * cache creation when reported. Inclusive-input providers are converted - * (input − cached − known writes); the reverse - * direction would be unrecoverable, which is why Anthropic semantics is the - * canonical one. - * - * Entries are added only after checking a real turn record against the - * provider's documented counting rules (see AGENTS.md for the workflow). - * Unknown providers get null and their numbers pass through unchanged — - * displayed as reported, never guessed. - */ -export type UsageSemantics = { - /** True when the provider's inputTokens already includes cachedInputTokens. */ - inputIncludesCached: boolean; -}; +/** Usage belongs to the harness contract, never to the selected model name. */ +export type UsageSemantics = { inputIncludesCached: boolean }; +export type TokenScope = 'turn' | 'session' | 'unknown'; + +const CLAUDE = /^(?:claude|anthropic)(?:$|[-/:])/i; +const CODEX = /^codex(?:$|[-/:])/i; +const OMP = /^omp(?:$|[-/:])/i; -/** Aggregation is a harness contract. A model name cannot establish it. */ export function providerSemantics(provider: string | null): { - tokens: 'turn' | 'request' | 'unknown'; cost: 'session' | 'unknown'; + tokens: 'turn' | 'request' | 'session' | 'unknown'; cost: 'session' | 'unknown'; } { - if (/claude|anthropic/i.test(provider ?? '')) return { tokens: 'turn', cost: 'session' }; - if (/codex/i.test(provider ?? '')) return { tokens: 'request', cost: 'unknown' }; + if (CLAUDE.test(provider ?? '')) return { tokens: 'turn', cost: 'session' }; + if (CODEX.test(provider ?? '')) return { tokens: 'request', cost: 'unknown' }; + // OMP exposes current-branch session totals, not monotonic billing counters. + // Without branch epochs and final-drain ordering, neither tokens nor cost + // can safely be differenced into turns. + if (OMP.test(provider ?? '')) return { tokens: 'session', cost: 'unknown' }; return { tokens: 'unknown', cost: 'unknown' }; } -const SEMANTICS_TABLE: Array<[RegExp, UsageSemantics]> = [ - [/claude|anthropic/i, { inputIncludesCached: false }], - [/codex|openai|gpt/i, { inputIncludesCached: true }], -]; +export function tokenScope(record: { provider: string | null; tokenScope?: TokenScope }): TokenScope { + if (record.tokenScope) return record.tokenScope; + const scope = providerSemantics(record.provider).tokens; + return scope === 'request' ? 'turn' : scope; +} -/** Looks up verified usage semantics by provider, falling back to model. */ -export function usageSemantics(provider: string | null, model: string | null): UsageSemantics | null { - for (const key of [provider, model]) { - if (!key) continue; - for (const [pattern, semantics] of SEMANTICS_TABLE) { - if (pattern.test(key)) return semantics; - } - } +export function usageSemantics(provider: string | null): UsageSemantics | null { + if (CLAUDE.test(provider ?? '') || OMP.test(provider ?? '')) return { inputIncludesCached: false }; + if (CODEX.test(provider ?? '')) return { inputIncludesCached: true }; return null; } -/** - * Canonical ordinary-input count for a single observation or record: subtracts - * reads and known writes when the provider counts them inside inputTokens, passes the - * value through otherwise (including unknown semantics). - */ +/** Uncached input; when writes are unavailable it can still contain writes. */ export function freshInput( semantics: UsageSemantics | null, input: number | null, @@ -60,5 +38,21 @@ export function freshInput( cacheWrite?: number | null, ): number | null { if (!semantics?.inputIncludesCached || input === null) return input; - return Math.max(0, input - (cached ?? 0) - (cacheWrite ?? 0)); + if (cached === null) return null; + const remaining = input - cached - (cacheWrite ?? 0); + return remaining >= 0 ? remaining : null; +} + +/** Complete prompt size from raw usage; missing counts are not measured zero. */ +export function promptTokens(provider: string | null, usage: { + input: number | null; cached: number | null; cacheWrite?: number | null; +}): number | null { + const semantics = usageSemantics(provider); + if (!semantics || usage.input === null) return null; + if (semantics.inputIncludesCached) { + if (usage.cached != null && usage.cached + (usage.cacheWrite ?? 0) > usage.input) return null; + return usage.input; + } + if (usage.cached === null || usage.cacheWrite == null) return null; + return usage.input + usage.cached + usage.cacheWrite; } diff --git a/shared/timeline.ts b/shared/timeline.ts index 90d797f..5e2c201 100644 --- a/shared/timeline.ts +++ b/shared/timeline.ts @@ -6,5 +6,6 @@ export const UsageTimelineSchema = TurnRowSchema.pick({ id: true, input: true, cached: true, cacheWrite: true, output: true, effectiveCostUsd: true, costSource: true, quality: true, status: true, durationMs: true, usageGap: true, -}); + tokenScope: true, incompleteUsage: true, partialCost: true, +}).partial({ tokenScope: true, incompleteUsage: true, partialCost: true }); export type UsageTimeline = z.infer;