Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## Unreleased

- Add a **Custom prices override reported costs** setting (off by default). A matching `pricing.json` price replaces the runtime-reported cost, which can be a list-price fallback for local or unbilled models. Reference prices never replace reported costs. The reported value stays in the ledger file (#1).
- Recognize custom providers that declare `extends: "claude"` or `extends: "codex"` in the daemon config. Their turns now use the base harness's usage and cost semantics instead of unknown-scope snapshots. New records store the resolved harness; earlier records of such providers stay unknown-scope snapshots (#1).

## v0.8.0 (2026-10-23) — Paseo 0.12 background workspaces and accounting accuracy

Requires Paseo **0.11.0 or newer**; verified on 0.11.1 and 0.12.0. Existing ledger records are reinterpreted on read, not rewritten; existing custom price files are kept. Reload or reopen the app after updating. See the [release notes](docs/releases/v0.8.0.md).
Expand Down
5 changes: 3 additions & 2 deletions OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@ TokenLedger records the tokens and cost of each agent turn and keeps the history

The agent panel shows session totals, the turn in progress with its context window, and one row per finished turn with status, time, duration, input, cache and output tokens, cost, and a data quality mark. The pill shows the session cost and context use, with an activity icon while a turn runs, and opens the panel. The sidebar row opens the overview, which groups agents by workspace and filters to today, the last 7 days, the last 30 days, or all time. Day ranges start at local midnight on the device that shows them. After each finished turn, the plugin can add one short usage summary to the timeline.

Token counts come from the provider. Claude terminal aggregates and observed Codex requests contribute to turn subtotals. OMP current-session/branch totals and unverified-harness observations remain visible as snapshots but are excluded from turn sums and prices. A separate latest-snapshot card retains late updates without inventing turn attribution. Missing counts/rates display as unknown; cache hit rates require a complete prompt denominator. Partial estimates and unknown-cost turns are explicit in the panel, pill and overview. Runtime-reported costs take priority over your reference prices, OpenRouter references and built-in prices, but are not verified bills.
Token counts come from the provider. Claude terminal aggregates and observed Codex requests contribute to turn subtotals. OMP current-session/branch totals and unverified-harness observations remain visible as snapshots but are excluded from turn sums and prices. A separate latest-snapshot card retains late updates without inventing turn attribution. Missing counts/rates display as unknown; cache hit rates require a complete prompt denominator. Partial estimates and unknown-cost turns are explicit in the panel, pill and overview. By default, runtime-reported costs take priority over your reference prices, OpenRouter references and built-in prices, but are not verified bills.

The pill distinguishes known session snapshots (`Session ≈4.12M tok`), unknown-scope snapshots, waiting for usage and no reported usage. Old timeline cards without scope metadata are labelled as legacy; their ledger inclusion is not guessed from the card.

The plugin settings page on each host has four options:
The plugin settings page on each host has five options:

- **Timeline summaries** turns the usage summary after each new turn on or off. Existing summaries stay.
- **OpenRouter reference prices** lets the plugin download the public model price list from `openrouter.ai` in the background and cache it for 24 hours. The request carries no tokens, prompts, or usage data. Turn it off to stop the download and stop using those prices.
- **Built-in fallback prices** controls estimates when no custom or OpenRouter price matches a model.
- **Custom prices override reported costs** uses your matching custom price instead of the cost the runtime reports. Use it for local or unbilled models. It is off by default.
- **Refresh frequency** selects normal or relaxed polling. Live updates still refresh the views immediately.

The ledger is a JSON Lines file on the daemon host, `~/.paseo/plugins/token-ledger/` (or under `PASEO_HOME`), retaining 2,000 records. It stores observed usage, timestamps, identity, scope and quality, not prompts, responses, tool arguments or credentials. Older records remain readable and are reinterpreted without rewriting them. `pricing.json` starts empty and preserves your existing prices. Full model IDs and explicit aliases prevent unrelated routes from sharing a guessed price. History does not sync between hosts.
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,12 @@ Open **Settings → Plugins → token-ledger → TokenLedger** on the selected h
- **Timeline summaries** controls summaries for newly finished turns; existing rows remain intact.
- **OpenRouter reference prices** controls both cached reference prices and background catalog refresh.
- **Built-in fallback prices** controls estimates when neither custom nor enabled OpenRouter pricing matches.
- **Custom prices override reported costs** (off by default) uses a matching `pricing.json` price instead of the runtime-reported cost, for example for local models that a runtime prices at a list-price fallback. Add `providers` to an entry to limit it to one provider. OpenRouter and built-in prices never replace a reported cost. The reported value stays in the ledger file, so turning the option off restores it.
- **Refresh frequency** selects normal (3 seconds active / 30 seconds idle) or relaxed (10 / 60 seconds) fallback polling. Live updates still trigger refreshes.

Settings apply without a plugin reload and are shared by clients of that host. Reported costs and `pricing.json` overrides always retain priority. These preferences do not alter ledger retention or erase history.
Settings apply without a plugin reload and are shared by clients of that host. By default, reported costs and `pricing.json` overrides retain priority. These preferences do not alter ledger retention or erase history.

All three switches default to on, and refresh frequency defaults to normal.
The other three switches default to on, and refresh frequency defaults to normal.

## How turns are measured

Expand All @@ -112,9 +113,9 @@ A "turn" is one user message through to `turn_completed`, `turn_failed`, or `tur
| Unverified harness | latest reported snapshot | unknown; excluded from per-turn totals and prices |
| Nothing reported | unknown, not zero | unavailable |

Counting conventions are selected by verified harness/provider identity, never by the model name. OMP input is already exclusive of cache; Codex input is inclusive. “IN” is uncached input after subtracting cache reads and known writes; if writes are omitted, Codex IN may still include unclassified writes. Unknown provider values stay as reported, without guessed normalization.
Counting conventions are selected by verified harness/provider identity, never by the model name. A custom provider that declares `extends: "claude"` or `extends: "codex"` in the daemon config follows its base harness; new records store the resolved harness. OMP input is already exclusive of cache; Codex input is inclusive. “IN” is uncached input after subtracting cache reads and known writes; if writes are omitted, Codex IN may still include unclassified writes. Unknown provider values stay as reported, without guessed normalization.

For attributable turn costs, the order is runtime-reported cost, local override, OpenRouter reference, then built-in fallback. Claude session-cost deltas use the known baseline; a lower counter alone does not prove a session reset. A runtime's reported cost can itself be an estimate: it is not a verified invoice, and custom reference pricing does not replace it. Residuals do not identify missing tokens or billing categories.
For attributable turn costs, the order is runtime-reported cost, local override, OpenRouter reference, then built-in fallback; **Custom prices override reported costs** puts a matching local override first. Claude session-cost deltas use the known baseline; a lower counter alone does not prove a session reset. A runtime's reported cost can itself be an estimate: it is not a verified invoice, and custom pricing replaces it only when **Custom prices override reported costs** is on. Residuals do not identify missing tokens or billing categories.

Estimated costs are marked `≈`; missing components/rates or uncertain request coverage add `partial`. Summaries expose unknown-cost turns rather than presenting the known subtotal as a complete bill. Codex request observations are priced separately; legacy records without requests use the aggregate as an explicitly partial estimate. A tiered price cannot be selected when the complete prompt size is unknown. Per-request routed model identities are not exposed, so reference pricing uses the recorded selected model, not a claim of the actual billed model.

Expand Down
1 change: 1 addition & 0 deletions client/settings.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ export function TokenLedgerSettings({ theme }: PluginSurfaceProps) {
<SettingsCard>
<SettingsSwitch label="OpenRouter reference prices" hint="Use cached reference prices and refresh the public catalog in the background." value={settings.values.openRouterPricing} disabled={settings.saving} onValueChange={(openRouterPricing) => save({ openRouterPricing })} />
<SettingsSwitch label="Built-in fallback prices" hint="Use known model prices when no custom or OpenRouter price matches." value={settings.values.builtinPricing} disabled={settings.saving} onValueChange={(builtinPricing) => save({ builtinPricing })} />
<SettingsSwitch label="Custom prices override reported costs" hint="For local or unbilled models. Scope entries with providers in pricing.json." value={settings.values.customPricesOverride} disabled={settings.saving} onValueChange={(customPricesOverride) => save({ customPricesOverride })} />
<SettingsRow label="Custom prices" hint="Edit pricing.json in the TokenLedger data folder on this host. Changes are picked up within 30 seconds of the next usage request." />
</SettingsCard>
</SettingsSection>
Expand Down
15 changes: 9 additions & 6 deletions index.server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,23 @@ import type { PluginServerContext } from "@getpaseo/plugin/server";
import { ledgerEnsure, ledgerOverview, ledgerSync } from "./shared/ledger.ts";
import { ensureTracker, handleEnsure, handleOverview, handleSync, observeStart, observeEnd, prepareAgent, stopTracker } from "./server/tracker.ts";
import { initializePreferences } from './server/preferences.ts';
import { refreshProviderBases } from './server/providers.ts';
import type { PaseoApi } from './server/paseo-types.ts';

export default function contribute(server: PluginServerContext) {
const preferences = initializePreferences(server);
server.handle(ledgerSync, async (input, context) => { await preferences.ready; return handleSync(input, context); });
server.handle(ledgerEnsure, async (input, context) => { await preferences.ready; return handleEnsure(input, context); });
server.handle(ledgerOverview, async (input, context) => { await preferences.ready; return handleOverview(input, context); });
const ready = async (paseo: PaseoApi) => { await preferences.ready; await refreshProviderBases(paseo); };
server.handle(ledgerSync, async (input, context) => { await ready(context.paseo); return handleSync(input, context); });
server.handle(ledgerEnsure, async (input, context) => { await ready(context.paseo); return handleEnsure(input, context); });
server.handle(ledgerOverview, async (input, context) => { await ready(context.paseo); return handleOverview(input, context); });
// Establish timeline demand before the provider can emit its first turn.
// Lifecycle hooks run even with no desktop/mobile client connected.
const beforeOpen = server.before("agent.session_open", async ({ request }, { paseo }) => {
if (request.purpose !== "interactive") return;
// A newly-created agent is not in the daemon catalog until session_open
// returns. Establish catalog demand now; its first upsert attaches usage.
try {
await preferences.ready;
await ready(paseo);
if (request.reason === "create") await ensureTracker(paseo);
else await prepareAgent(paseo, request.agentId);
} catch (error) {
Expand All @@ -24,10 +27,10 @@ export default function contribute(server: PluginServerContext) {
});
// Covers a plugin reload while an existing provider session stays open.
const onStart = server.on("agent.turn_started", async (event, { paseo }) => {
try { await preferences.ready; await observeStart(event, paseo); }
try { await ready(paseo); await observeStart(event, paseo); }
catch (error) { console.error("token-ledger: could not attach turn tracking", error); }
});
const onEnd = server.on("agent.turn_ended", async (event, { paseo }) => { await preferences.ready; await observeEnd(event, paseo); });
const onEnd = server.on("agent.turn_ended", async (event, { paseo }) => { await ready(paseo); await observeEnd(event, paseo); });
return async () => {
beforeOpen();
onStart();
Expand Down
41 changes: 40 additions & 1 deletion server/preferences.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ test('settings changes win over a stale read, update pricing immediately, and pr
const pricing = await import('./pricing.ts');
const { TimelinePublisher } = await import('./timeline.ts');
try {
notify({ status: 'ready', revision: 'new', values: { ...DEFAULT_PREFERENCES, timelineSummaries: false, openRouterPricing: false, builtinPricing: false } });
notify({ status: 'ready', revision: 'new', values: { ...DEFAULT_PREFERENCES, timelineSummaries: false, openRouterPricing: false, builtinPricing: false, customPricesOverride: true } });
resolveRead({ status: 'ready', revision: 'old', values: DEFAULT_PREFERENCES });
await settings.ready;
assert.equal(getPreferences().openRouterPricing, false);
Expand All @@ -36,7 +36,13 @@ test('settings changes win over a stale read, update pricing immediately, and pr
assert.equal(pricing.estimateUsageCost({ ...usage, model: 'custom' }).effectiveCostUsd, 9);
const { finalizeTurn } = await import('../shared/aggregate.ts');
const record = finalizeTurn({ agentId: 'a', turnId: 't', provider: 'claude', model: 'claude-opus-5.5', startedAt: null, endedAt: new Date().toISOString(), status: 'completed', observations: [], finalUsage: { inputTokens: 1_000_000, totalCostUsd: 2 }, prevSessionCostUsd: 0 });
// Without a custom price the reported cost stays.
assert.equal(pricing.enrichTurn(record, 1).effectiveCostUsd, 2);
const local = pricing.enrichTurn({ ...record, model: 'custom' }, 1);
assert.equal(local.effectiveCostUsd, 9);
assert.equal(local.costSource, 'override');
assert.equal(local.costUsd, null);
assert.equal(local.costBreakdown?.otherUsd, null);
const publisher = new TimelinePublisher();
publisher.publish({} as never, record); // Disabled summaries must never call the timeline API.
await publisher.flush();
Expand All @@ -45,9 +51,42 @@ test('settings changes win over a stale read, update pricing immediately, and pr
assert.notEqual(pricing.pricingRevision(), previous);
assert.equal(pricing.estimateUsageCost(usage).effectiveCostUsd, 4);
assert.equal(pricing.estimateUsageCost({ ...usage, model: 'remote' }).effectiveCostUsd, 1);
assert.equal(pricing.enrichTurn({ ...record, model: 'custom' }, 1).effectiveCostUsd, 2, 'reported costs win by default');
notify({ status: 'ready', revision: 'override', values: { ...DEFAULT_PREFERENCES, customPricesOverride: true } });
const builtin = pricing.enrichTurn(record, 1);
assert.equal(builtin.effectiveCostUsd, 2, 'reference prices never replace reported costs');
assert.equal(builtin.costSource, 'reported');
assert.equal(pricing.enrichTurn({ ...record, model: 'custom' }, 1).effectiveCostUsd, 9);
} finally {
await settings.dispose();
if (oldHome === undefined) delete process.env.PASEO_HOME; else process.env.PASEO_HOME = oldHome;
await rm(home, { recursive: true, force: true });
}
});

test('custom providers that extend a known harness follow its usage semantics', async () => {
const { setProviderBases, providerBasesRevision } = await import('../shared/semantics.ts');
const { finalizeTurn } = await import('../shared/aggregate.ts');
const pricing = await import('./pricing.ts');
const turn = () => finalizeTurn({ agentId: 'a', turnId: 't', provider: 'local-claude', model: 'acme/local-1', startedAt: null, endedAt: new Date().toISOString(), status: 'completed', observations: [], finalUsage: { inputTokens: 10, cachedInputTokens: 0, outputTokens: 5, totalCostUsd: 2 }, prevSessionCostUsd: 0 });
try {
const unresolved = turn();
assert.equal(unresolved.tokenScope, 'unknown');
assert.equal(unresolved.harness, undefined);
const previous = pricing.pricingRevision();
setProviderBases({ 'local-claude': 'claude' });
assert.notEqual(pricing.pricingRevision(), previous);
const resolved = turn();
assert.equal(resolved.tokenScope, 'turn');
assert.equal(resolved.harness, 'claude');
assert.equal(resolved.costUsd, 2);
const revision = providerBasesRevision();
setProviderBases({ 'local-claude': 'claude' });
assert.equal(providerBasesRevision(), revision, 'an unchanged config keeps cached rows');
// The recorded harness keeps semantics when the config later changes.
setProviderBases({});
assert.equal(pricing.enrichTurn(resolved, 1).effectiveCostUsd, 2);
} finally {
setProviderBases({});
}
});
Loading
Loading