Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/sdk-cost-per-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"uptide": patch
---

`fix` computes the agent's cost itself, per model and token class, from Uptide's price table: `report.json` and `--json` list each model's uncached input, cache read, 5-minute and 1-hour cache write and output tokens, the rate applied to each and the cost, so the total can be redone by hand. The Agent SDK's own figure is kept apart as `sdkReportedCostUsd`; `--max-cost` counts the higher of the two, a gap above 5% is printed as a warning, and a model missing from the table uses the SDK's figure, marked unverified. `llm.inputTokens` now counts uncached input only.
5 changes: 5 additions & 0 deletions .changeset/sonnet-cache-read-rate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"uptide": patch
---

Sonnet 5.5 cache reads are priced at the published $0.10 per million tokens (0.05x input), not $0.20. Every Anthropic rate in the price table was checked against Anthropic's pricing page on 2026-10-20, and a test now pins each one.
46 changes: 40 additions & 6 deletions docs/model-pricing.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# Model pricing and budget accounting

Reviewed **2026-10-04**. USD per million tokens, standard text inference. Source of
Reviewed **2026-10-04**; every Anthropic cell re-checked against the
[Anthropic pricing page](https://platform.claude.com/docs/en/about-claude/pricing) on
**2026-10-20**. USD per million tokens, standard text inference. Source of
truth in code: `packages/core/src/llm/pricing.ts`. No paid built-in tools, batch,
priority processing, explicit prompt caching or service-tier upgrades are requested.

| Provider/model | Input | Output | Cached input | Cache write (5m / 1h where applicable) |
| --- | ---: | ---: | ---: | ---: |
| Anthropic Sonnet 4.6 | 3 | 15 | 0.30 | 3.75 / 6 |
| Anthropic Sonnet 5.5 | 2 | 10 | 0.20 | 2.50 / 4 |
| Anthropic Sonnet 5.5 | 2 | 10 | 0.10 | 2.50 / 4 |
| Anthropic Opus 5.5 | 4 | 20 | 0.20 | 5 / 8 |
| Anthropic Fable 5.1 | 10 | 50 | 0.25 | 12.50 / 20 |
| Anthropic Haiku 4.5 (`20251001`) | 1 | 5 | 0.10 | 1.25 / 2 |
Expand All @@ -33,10 +35,42 @@ rates end on 2026-12-31; from 2027-01-01 the table automatically uses 1.50 input

## Agent SDK budget

`fix` runs only with the Agent SDK executor. Its reported session cost comes from the SDK;
the package's session receives its budget, and enforcement after each turn can overshoot
by one turn. The reservation model below applies to the patch executor and the OpenAI and
Gemini rates, which remain for Uptide's own evaluations (`UPTIDE_INTERNAL=1`).
`fix` runs only with the Agent SDK executor. The package's session receives its budget, and
the SDK's enforcement after each turn can overshoot by one turn. The reservation model below
applies to the patch executor and the OpenAI and Gemini rates, which remain for Uptide's own
evaluations (`UPTIDE_INTERNAL=1`).

`llm.costUsd` is computed by Uptide from the table above, so anyone can redo it by hand.
`report.json` and `fix --json` list each model under `llm.models`:

| Field | Meaning |
| --- | --- |
| `inputTokens` | uncached input |
| `cacheReadTokens` | cache hits and refreshes |
| `cacheWriteTokens` / `cacheWriteHourTokens` | 5-minute / 1-hour cache writes |
| `outputTokens` | output, thinking included |
| `rates` | the USD per million tokens applied to each class |
| `costUsd` | the sum of tokens x rate / 1,000,000, rounded up to the nanodollar |
| `sdkReportedCostUsd` | the SDK's own figure for that model |

`llm.costUsd` is the total of the rows, and `llm.sdkReportedCostUsd` is the SDK's
`total_cost_usd`, from the SDK's built-in price table, kept apart and never mixed in.
`llm.inputTokens` counts uncached input only, summed over models; before UPT-127 it also
counted cache reads and writes. The per-session records (`llm.tasks`) carry the same fields.

The SDK gives tokens per model but splits cache writes by TTL only for the main loop:
those 1-hour writes are priced on the session's model. **Assumption:** every cache write by
any other model (a subagent, compaction) is priced as a 5-minute write. Had it been a
1-hour write, the computed cost is low by the difference (on Sonnet 5.5, $1.50 per million
written tokens); the 5% check against the SDK's figure and the `--max-cost` rule below
still apply.

`--max-cost` counts the higher of the computed and the SDK-reported cost, so a wrong
table never lets a run spend past the limit by Uptide's own accounting. When the two
differ by more than 5% of the higher, the run prints both on the terminal and lists the
line in `llm.warnings` (never in the PR body). A model with no row in the table gets no
guessed rate: its cost is the SDK's figure, the row has no `rates` and is marked
`unverified`, `llm.costUnverified` is set, and the run warns.

## Reservation and actual spend

Expand Down
5 changes: 4 additions & 1 deletion docs/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,10 @@ it can overshoot by one turn.
The summary's `Phases` row and the PR's run details give wall time per phase (analyze,
baseline, install, rules, agent, verify, feedback) and the agent's spend in each.

Unknown models use Anthropic's highest listed rates with a warning. See
`llm.costUsd` is computed from Uptide's price table per model and token class; the SDK's
own figure is `llm.sdkReportedCostUsd`, and `--max-cost` counts the higher of the two.
With the SDK, an unknown model's cost is the SDK's figure, marked unverified; with the
patch executor, unknown models use Anthropic's highest listed rates with a warning. See
[model pricing](model-pricing.md) for rates, token estimation and the scope of accounting.

## Privacy
Expand Down
2 changes: 2 additions & 0 deletions packages/cli/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1262,6 +1262,8 @@ ${runnable(PRIVACY)}`,
`\n${formatFixSummary(report, { color: ui.color, invocation: INVOCATION, suggest, cwd, ms: io.now() - started })}`,
);
io.err(whereItRan(report));
// Cost accounting warnings: the terminal's (and --json's), never the PR body's.
for (const warning of report.llm.warnings ?? []) io.err(`${warning}\n`);
// The plan and its warnings, now that no spinner is drawing.
if (report.publicationLog && !flags.json) io.err(`\n${report.publicationLog}\n`);
if (report.publication?.refused.length)
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/action/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { isMajorUpgrade, NO_PACK_YET, noPackBlockers } from '../fix/no-pack.js';
import { sdkRequired } from '../fix/sdk-agent.js';
import type { FixReport } from '../fix/types.js';
import type { ListedDependency, ListReport } from '../list/list.js';
import { chargedUsd } from '../llm/pricing.js';
import { loadUptideConfig } from '../onboarding/node.js';
import { DEFAULT_CONFIG, UPTIDE_CONFIG_FILE, type UptideConfig } from '../onboarding/schema.js';
import { packStatus, registeredPacks } from '../packs/index.js';
Expand Down Expand Up @@ -421,7 +422,7 @@ export async function runAction(
);
continue;
}
budget -= (fixed.llm?.costUsd ?? 0) + (fixed.llm?.unreportedCostUsd ?? 0);
budget -= (fixed.llm ? chargedUsd(fixed.llm) : 0) + (fixed.llm?.unreportedCostUsd ?? 0);
let pr: { number: number; url: string } | null = null;
let error: string | undefined;
// `uptide pr` refuses these too; asking it would only turn the reason into a failure. A
Expand Down
6 changes: 2 additions & 4 deletions packages/core/src/fix/assisted.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { Node, Project, SyntaxKind } from 'ts-morph';
import type { ProgressListener } from '../domain/progress.js';
import { ACCEPTED_KEYS } from '../llm/config.js';
import { CostLimitError, DEFAULT_MAX_COST_USD } from '../llm/fixer.js';
import { nanos } from '../llm/pricing.js';
import { addSpend, nanos } from '../llm/pricing.js';
import type { MigrationPack, PackContext } from '../packs/types.js';
import { resetSharedState } from '../shared-state.js';
import { applySitePatch } from './patch.js';
Expand Down Expand Up @@ -375,9 +375,7 @@ export async function assist(
log.costUsd = response.costUsd ?? 0;
log.explanation = response.explanation ?? '';
log.diff = response.diff;
llm.inputTokens += response.inputTokens;
llm.outputTokens += response.outputTokens;
llm.costUsd = (nanos(llm.costUsd) + nanos(response.costUsd ?? 0)) / 1e9;
addSpend(llm, response);
if (response.unreportedCostUsd)
llm.unreportedCostUsd =
(nanos(llm.unreportedCostUsd ?? 0) + nanos(response.unreportedCostUsd)) / 1e9;
Expand Down
16 changes: 16 additions & 0 deletions packages/core/src/fix/feedback.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,22 @@ it('starts no session the remaining budget is below a typical session for (UPT-1
);
});

it('counts the higher of computed and SDK-reported cost against --max-cost (UPT-127)', async () => {
for (const [costUsd, sdkReportedCostUsd] of [
[0.05, 0.1],
[0.1, 0.05],
] as const) {
const { root, services } = await fixture();
const runner = vi.fn<TaskRunner>(async () => ({ ...runResult, costUsd, sdkReportedCostUsd }));
const result = await fix(options(root), { ...services, agentRunner: runner });
// Either way $0.10 a session counts: the second gets $0.15, and no third starts.
expect(runner.mock.calls.map(([input]) => input.maxBudgetUsd)).toEqual([0.25, 0.15]);
expect(result.llm.feedback?.stopped).toEqual({ remainingUsd: 0.05, typicalUsd: 0.1 });
expect(result.llm.costUsd).toBeCloseTo(2 * costUsd, 9);
expect(result.llm.sdkReportedCostUsd).toBeCloseTo(2 * sdkReportedCostUsd, 9);
}
});

it('does not send failures already present at the parent to the agent', async () => {
const { root, services } = await fixture();
services.tests = async () => [{ workspace: '.', status: 'failed', output: 'old failure' }];
Expand Down
2 changes: 2 additions & 0 deletions packages/core/src/fix/feedback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,8 @@ export function feedbackPatchRunner(fixer: Fixer, finding: Finding): TaskRunner
unreportedCostUsd,
inputTokens: answer.inputTokens,
outputTokens: answer.outputTokens,
...(answer.costUnverified ? { costUnverified: true as const } : {}),
...(answer.models ? { models: answer.models } : {}),
summary: {
changed: [...edits.keys()].map((file) => ({
file,
Expand Down
12 changes: 5 additions & 7 deletions packages/core/src/fix/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
providerFixer,
sdkCostEstimate,
} from '../llm/fixer.js';
import { nanos } from '../llm/pricing.js';
import { addSpend, chargedUsd, nanos } from '../llm/pricing.js';
import { packageChangelog, usedNames } from '../packs/changelog.js';
import { decidedRules, decisionInventory, packDecisions } from '../packs/decisions.js';
import { genericPack } from '../packs/generic.js';
Expand Down Expand Up @@ -1269,14 +1269,14 @@ async function fixPackage(options: FixOptions, services: FixServices): Promise<F
);
if (!feedback && !(round === 1 && fallback.length)) break;
const remaining =
(nanos(maxCostUsd) - nanos(llm.costUsd) - nanos(llm.unreportedCostUsd ?? 0)) / 1e9;
(nanos(maxCostUsd) - nanos(chargedUsd(llm)) - nanos(llm.unreportedCostUsd ?? 0)) / 1e9;
if (remaining < 0.02 || llm.unreportedCostUsd) break;
// A session that would likely cross the limit is not started: one that does stops at the
// limit mid-edit, and its spend buys nothing (UPT-111).
// ponytail: the mean of this run's sessions; a per-pack history if sessions vary widely.
const sessions = (llm.tasks ?? []).filter((t) => t.costUsd > 0);
const sessions = (llm.tasks ?? []).filter((t) => chargedUsd(t) > 0);
const typicalUsd = sessions.length
? sessions.reduce((n, t) => n + t.costUsd, 0) / sessions.length
? sessions.reduce((n, t) => n + chargedUsd(t), 0) / sessions.length
: 0;
if (remaining < typicalUsd) {
feedbackLimits.stopped = { remainingUsd: remaining, typicalUsd };
Expand Down Expand Up @@ -1325,9 +1325,7 @@ async function fixPackage(options: FixOptions, services: FixServices): Promise<F
feedbackLimits.roundsUsed++;
const lastAttempt = next.tasks?.at(-1);
previousAttempt = lastAttempt?.rejection ?? lastAttempt?.stop ?? '';
llm.inputTokens += next.inputTokens;
llm.outputTokens += next.outputTokens;
llm.costUsd = (nanos(llm.costUsd) + nanos(next.costUsd)) / 1e9;
addSpend(llm, next);
if (next.unreportedCostUsd)
llm.unreportedCostUsd =
(nanos(llm.unreportedCostUsd ?? 0) + nanos(next.unreportedCostUsd)) / 1e9;
Expand Down
75 changes: 60 additions & 15 deletions packages/core/src/fix/sdk-agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { ts } from 'ts-morph';
import { defaultCacheDir } from '../cache/paths.js';
import type { ProgressListener } from '../domain/progress.js';
import { modelCapabilities } from '../llm/capabilities.js';
import { addSpend, type ModelCost, modelCost, type Spend } from '../llm/pricing.js';
import { packDecisions } from '../packs/decisions.js';
import { recordPrompt } from '../packs/records.js';
import type { MigrationPack, PackContext } from '../packs/types.js';
Expand Down Expand Up @@ -66,11 +67,9 @@ export interface AgentTask {
files: string[];
}
/** One session's result, from the SDK or from a test double. */
export interface TaskRun {
export interface TaskRun extends Spend {
summary?: AgentSummary;
turns: number;
inputTokens: number;
outputTokens: number;
costUsd: number;
unreportedCostUsd?: number;
durationMs: number;
Expand Down Expand Up @@ -792,21 +791,73 @@ export async function assistWithSdk(input: SdkAssistInput): Promise<FixReport['l
} else accept(pending);
}

const sum = (key: 'inputTokens' | 'outputTokens' | 'costUsd') =>
records.reduce((n, r) => n + r[key], 0);
const spent: Spend & { costUsd: number } = { inputTokens: 0, outputTokens: 0, costUsd: 0 };
for (const record of records) addSpend(spent, record);
return {
provider: 'anthropic',
model: input.model,
available: true,
inputTokens: sum('inputTokens'),
outputTokens: sum('outputTokens'),
costUsd: sum('costUsd'),
...spent,
unreportedCostUsd: records.reduce((sum, r) => sum + (r.unreportedCostUsd ?? 0), 0),
agent: 'sdk',
tasks: records,
};
}

type SdkResult = Extract<SDKMessage, { type: 'result' }>;
/** Cost and SDK cost disagree by more than this share before the run says so. */
export const SDK_COST_TOLERANCE = 0.05;
/**
* The session's spend from the SDK result: tokens per model and class (`modelUsage`), priced
* from pricing.ts. `total_cost_usd` comes from the SDK's own table and is kept apart.
* `modelUsage` has no cache TTL split; `usage.cache_creation` (main loop only) has one, so its
* 1-hour writes go to the session model and every other write is priced as 5-minute.
*/
// ponytail: other models' 1-hour writes price as 5-minute; the 5% check and max() cap cover it.
export function sdkSpend(result: SdkResult, model: string): Spend & { costUsd: number } {
const hour =
(result.usage as { cache_creation?: { ephemeral_1h_input_tokens?: number } }).cache_creation
?.ephemeral_1h_input_tokens ?? 0;
const warnings: string[] = [];
const models: ModelCost[] = Object.entries(result.modelUsage ?? {}).map(([name, m]) => {
const hourWrites = name === model ? Math.min(hour, m.cacheCreationInputTokens) : 0;
const row = modelCost('anthropic', name, {
inputTokens: m.inputTokens,
outputTokens: m.outputTokens,
cacheReadTokens: m.cacheReadInputTokens,
cacheWriteTokens: m.cacheCreationInputTokens - hourWrites,
cacheWriteHourTokens: hourWrites,
});
if (!row.unverified) return { ...row, sdkReportedCostUsd: m.costUSD };
// Never guess a rate: the SDK's figure stands, marked unverified.
warnings.push(
`Cost of ${name}: no rate in Uptide's pricing table; using the SDK's $${m.costUSD.toFixed(4)}, unverified.`,
);
const { rates: _, ...unpriced } = row;
return { ...unpriced, costUsd: m.costUSD, sdkReportedCostUsd: m.costUSD };
});
const sdk = result.total_cost_usd;
const unverified = models.some((m) => m.unverified) || (!models.length && sdk > 0);
const costUsd = models.length ? models.reduce((n, m) => n + m.costUsd, 0) : sdk;
if (!models.length && sdk > 0)
warnings.push(
`Cost: the SDK reported no per-model usage; using its $${sdk.toFixed(4)}, unverified.`,
);
else if (Math.abs(costUsd - sdk) > SDK_COST_TOLERANCE * Math.max(costUsd, sdk))
warnings.push(
`Cost check: computed $${costUsd.toFixed(4)} from Uptide's pricing table, the SDK reported $${sdk.toFixed(4)}; --max-cost counts the higher.`,
);
return {
inputTokens: models.reduce((n, m) => n + m.inputTokens, 0),
outputTokens: models.reduce((n, m) => n + m.outputTokens, 0),
costUsd: Math.round(costUsd * 1e9) / 1e9,
sdkReportedCostUsd: sdk,
...(unverified ? { costUnverified: true as const } : {}),
models,
...(warnings.length ? { warnings } : {}),
};
}

const SDK = '@anthropic-ai/claude-agent-sdk';
/** The SDK version Uptide is tested with (core's devDependency), fetched when none is installed. */
export const SDK_VERSION = '0.3.296';
Expand Down Expand Up @@ -1053,15 +1104,9 @@ const sdkRunner: TaskRunner = async ({
if (!result) throw err;
}
if (!result) throw new Error('the SDK session ended without a result');
const usage = result.usage as unknown as Record<string, number | undefined>;
return {
turns: result.num_turns,
inputTokens:
(usage.input_tokens ?? 0) +
(usage.cache_read_input_tokens ?? 0) +
(usage.cache_creation_input_tokens ?? 0),
outputTokens: usage.output_tokens ?? 0,
costUsd: result.total_cost_usd,
...sdkSpend(result, model),
durationMs: Date.now() - started,
toolCalls,
...(result.subtype !== 'success'
Expand Down
Loading
Loading