Skip to content

fix: price Sonnet 5.5 cache reads at $0.10 and make SDK cost auditable per model (UPT-128, UPT-127) - #268

Merged
lucasbraum merged 3 commits into
mainfrom
lucasbraum/upt-127-uptide-fix-json-cost-cannot-be-audited
Oct 20, 2026
Merged

lucasbraum merged 3 commits into
mainfrom
lucasbraum/upt-127-uptide-fix-json-cost-cannot-be-audited

Conversation

@lucasbraum

@lucasbraum lucasbraum commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Two commits, in order: UPT-128 (the price table), then UPT-127 (auditable SDK cost).

UPT-128: price table

Source: https://platform.claude.com/docs/en/about-claude/pricing, read 2026-10-20 (cited at the top of the Anthropic table in pricing.ts).

Model Input Output Cache read 5m write 1h write Change
claude-sonnet-4-6 3 15 0.30 3.75 6 none
claude-sonnet-5-5 2 10 0.20 → 0.10 2.50 4 cache read is 0.05x input, per the page's footnote 2
claude-opus-5-5 4 20 0.20 5 8 none (also 0.05x)
claude-fable-5-1 10 50 0.25 12.50 20 none (0.025x)
claude-haiku-4-5-20251001 1 5 0.10 1.25 2 none

There is one change: Sonnet 5.5 cache reads. docs/model-pricing.md is updated to match.

  • pricing.test.ts pins every Anthropic cell to a single literal expected table (toEqual). Setting Sonnet 5.5 back to 0.20 fails it. OpenAI and Gemini are not re-checked and not pinned (out of scope).
  • sdkCostEstimate (UPT-94) does not move. It prices uncached token equivalents only (cachedInputTokens: 0), so the cache-read rate never enters it. cost-estimate.test.ts passes unmodified before and after (recorded runs: $0.23 for alanagoyal; $0.14 and $0.27 for the unit cases; all four within 1.3–2.5× of actual). SDK_ESTIMATE_MARGIN and cost-runs.json are unchanged. The actualUsd values there are the SDK's own totals with no token breakdown, so they cannot be repriced.

UPT-127: auditable SDK cost

Where tokens were merged: packages/core/src/fix/sdk-agent.ts:1058-1062 (on main) set inputTokens = input_tokens + cache_read_input_tokens + cache_creation_input_tokens and costUsd = result.total_cost_usd. The patch path did the same at packages/core/src/llm/fixer.ts:217 (billedInput), on top of adapters.ts:89, which already adds cache reads to input.

  • sdkSpend(result, model) reads result.modelUsage, which the SDK documents as the field for token and cost accounting (it covers subagents and compaction, which usage does not). For each model it takes uncached input, output, cache reads and cache writes.
    • TTL split: modelUsage has none. result.usage.cache_creation has one, but for the main loop only. Its 1-hour writes are assigned to the session model; any other model's writes are priced as 5-minute writes.
  • costUsd = tokens × pricing.ts rate, per class and model, rounded up to the nanodollar. sdkReportedCostUsd keeps the SDK's total_cost_usd separate, and each row also carries its own SDK figure.
  • report.json and --json gain llm.models[]: inputTokens, cacheReadTokens, cacheWriteTokens, cacheWriteHourTokens, outputTokens, rates, costUsd, sdkReportedCostUsd. The same fields appear on each llm.tasks[] record.
    • costUsd and inputTokens keep their names. inputTokens now means uncached input only, documented in docs/model-pricing.md and the types.
  • --max-cost counts max(computed, SDK-reported) (chargedUsd) for the feedback-round budget, the typical-session check and the Action's per-run budget.
    • Test: feedback.test.ts "counts the higher…" runs computed $0.05 / SDK $0.10 and the reverse. Both stop exactly where $0.10 sessions do. Making chargedUsd ignore the SDK figure fails the test.
  • A gap above 5% (of the higher figure) adds a line to llm.warnings with both numbers, printed to stderr. It is never in the PR body: notes feed the PR, warnings do not.
  • Unknown model: no guessed rate. The row has no rates and is marked unverified: true, costUsd is the SDK's figure, llm.costUnverified is set, and the run warns. If there is no modelUsage at all, the SDK total is used, also marked unverified. Tested in sdk-cost.test.ts.
  • Patch executor: it did not have a breakdown before. It now returns the same single-model row per call, and assisted.ts and run.ts add rows up by model (addSpend).
  • schema/uptide.run.v1.json is untouched: the run report copies named fields only, and its costUsd is now the computed figure.

Real run: hand calculation

uptide fix --only zod --max-cost 0.25 --json on a local clone of public consetto/calmcp at bdaef39^ (zod ^3.24.1 → 4.6.7). Origin was removed from the clone, and nothing was pushed or commented. Fix ran in its temporary clone and verification passed.

  • UTC window: 2026-10-20T16:58:27Z → 2026-10-20T16:59:53Z
  • Model: claude-sonnet-5-5, one SDK session, 31 turns, no feedback rounds. Preflight estimate $0.15 for 10 sites.
"inputTokens": 24,
"outputTokens": 6581,
"costUsd": 0.1622007,
"sdkReportedCostUsd": 0.1622007,
"models": [{
  "model": "claude-sonnet-5-5",
  "inputTokens": 24, "outputTokens": 6581,
  "cacheReadTokens": 256402, "cacheWriteTokens": 28281, "cacheWriteHourTokens": 0,
  "rates": { "input": 2, "output": 10, "cacheRead": 0.1, "cacheWrite": 2.5, "cacheWriteHour": 4 },
  "costUsd": 0.1622007, "sdkReportedCostUsd": 0.1622007
}]
Class Tokens $/MTok Token × rate
uncached input 24 2.00 48.0
cache read 256,402 0.10 25,640.2
cache write 5m 28,281 2.50 70,702.5
cache write 1h 0 4.00 0
output 6,581 10.00 65,810.0
total 162,200.7 / 1e6 = $0.1622007

Computed $0.1622007 = SDK-reported $0.1622007, a 0% gap. At the old $0.20 cache-read rate the computed figure would have been $0.1878409, 15.8% above the SDK. So the SDK does use Sonnet 5.5's $0.10 rate, and the UPT-128 fix is right. These numbers are pinned in sdk-cost.test.ts, with the arithmetic in the comment.

Verification

  • pnpm test: core 2460 passed / 37 skipped (205 files), cli 538 passed (48 files)
  • pnpm typecheck: 3/3 tasks successful
  • pnpm exec biome check .: no errors; the 2 warnings and 1 info are the same as on main

Follow-up commit (66f17f1)

CI's "No private material" step failed both test jobs. .changeset/auditable-sdk-cost.md matched PRIVATE_DOC (audit[^/]*\.md) in scripts/public-tree.mjs:41, so it is renamed sdk-cost-per-model.md. Locally that step exits early without UPTIDE_PRIVATE_DENYLIST, so it never ran. Run with a dummy denylist, it now reports clean. The same commit drops the unverified OpenAI/Gemini snapshot and states the 5-minute assumption for non-session-model cache writes in docs/model-pricing.md.

Open questions (answered)

  1. The SDK enforces maxBudgetUsd inside a session with its own table. If that table ever undercharged, one session could pass the cap by Uptide's figures. max() only protects what comes after the session. Should the session's budget be scaled by the observed computed/SDK ratio?
  2. 1-hour cache writes from a model other than the session model are priced at the 5-minute rate, because the SDK gives no per-model TTL split. Today's run had one model and no 1-hour writes.
  3. The 5% gap is measured against the higher of the two figures. Say if you would rather measure it against the SDK figure.
  4. The OpenAI and Gemini rows were not re-checked (out of scope; they are snapshot-pinned only).

🤖 Generated with Claude Code

lucasbraum and others added 2 commits October 20, 2026 10:05
…ok (UPT-128)

Anthropic's pricing page (read 2026-10-20) prices Sonnet 5.5 and Opus 5.5
cache hits at 0.05x base input: $0.10/MTok on Sonnet 5.5, not $0.20. Every
other Anthropic cell (input, output, cache read, 5m and 1h cache write) for
the five listed models matches the page.

pricing.test.ts pins each Anthropic cell to one expected table, and the
OpenAI and Gemini rows as an inline snapshot, so a changed cell fails CI.

sdkCostEstimate prices uncached token equivalents only, so its numbers and
the recorded cost-runs.json expectations do not move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
…, keep the SDK figure apart (UPT-127)

The SDK executor reported costUsd straight from the Agent SDK's total_cost_usd,
whose price table is inside its binary, and merged uncached input, cache reads
and cache writes into one inputTokens (sdk-agent.ts:1058-1062). Nothing could
be recomputed by hand.

- sdkSpend reads the result's modelUsage: per model, uncached input, output,
  cache reads and cache writes. modelUsage has no TTL split; the main loop's
  usage.cache_creation does, so its 1-hour writes go to the session model.
- costUsd is tokens x pricing.ts rates per class; the SDK total is
  sdkReportedCostUsd, never mixed in. llm.models lists tokens, rates and cost
  per model in report.json and --json; inputTokens is uncached input only.
- --max-cost (feedback rounds, typical session, the Action's budget) counts
  the higher of computed and SDK-reported cost.
- A gap above 5% goes to llm.warnings and the terminal, not the PR body.
- A model missing from pricing.ts gets no guessed rate: the SDK figure,
  marked unverified, with a warning.
- The patch executor reports the same per-model row (one model per call).

Real run, zod 3 -> 4.6.7 on consetto/calmcp: computed $0.1622007, SDK
$0.1622007.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
@tenki-reviewer

tenki-reviewer Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review complete. No blocking issues — approved ✅; 3 nitpicks below.

🧹 Nitpicks (3) — 🟢 3 low
  • 🟢 modelCost rates ignore the long-context price tier (pricing.ts:195) — modelCost builds rates from priceFor(provider, model) with no token count (packages/core/src/llm/pricing.ts:195, 199-205), while costUsd is computed via usageCost, which prices with priceFor(provider, model, billedInput) and switches to the long tier above the threshold (packages/core/src/llm/pricing.ts:127-128).
  • 🟢 Failure paths drop models and costUnverified from the task record (feedback.ts:121) — feedbackPatchRunner forwards the provider's per-model cost breakdown and its unverified-cost flag only on the success path (packages/core/src/fix/feedback.ts:131-132); the answer.failure return at line 121 and the catch-block return at lines 147-153 forward costUsd/unreportedCostUsd but omit models and costUnverified, even when the fixer reported usage before failing.
  • 🟢 addSpend duplicates identical cost warnings every merge (pricing.ts:226) — addSpend appends b.warnings wholesale without deduplication (packages/core/src/llm/pricing.ts:226), so the same per-session warning text is copied again on every merge.

Reworks LLM cost accounting in the fix pipeline around the SDK's own reported spend, adds cache-aware pricing and a per-model cost breakdown, and corrects the Sonnet cache-read rate.

Files Change
packages/core/src/llm/pricing.ts, pricing.test.ts New pricing engine: modelCost, addSpend, cache rates, long-context tier, warning collection.
packages/core/src/fix/sdk-agent.ts, sdk-cost.test.ts SDK agent reads provider-reported spend and per-model usage, merging it into the spend accumulator.
packages/core/src/fix/feedback.ts, types.ts FixReport.llm extends Spend; feedback forwarding carries per-model rows and costUnverified on success.
packages/core/src/fix/run.ts, assisted.ts, action/run.ts Budget gating switches to chargedUsd (max of computed and SDK totals).
packages/core/src/llm/fixer.ts Fixer emits token usage consistent with new uncached-input semantics.
packages/cli/src/cli.ts CLI prints SDK-reported cost and accumulated warnings to stderr.
docs/model-pricing.md, docs/models.md, .changeset/* Document the pricing table, report schema changes, and release notes.

Three low-severity issues remain: modelCost rows ignore the long-context tier, failure paths drop the per-model breakdown, and repeated warnings are not deduplicated.

Reviewed commit: b95d7fa

…n; drop the unverified provider snapshot

CI's "No private material" step (scripts/private-material.mjs) failed both test
jobs: `.changeset/auditable-sdk-cost.md` matches PRIVATE_DOC in
scripts/public-tree.mjs (`audit[^/]*\.md`). It is now sdk-cost-per-model.md.
Locally the step exits early without UPTIDE_PRIVATE_DENYLIST, so it never ran.

pricing.test.ts no longer pins the OpenAI and Gemini rows as an inline
snapshot: they were not re-checked (out of scope), and a pin would read as
verified. The Anthropic rows stay pinned against a literal table.

docs/model-pricing.md states the 5-minute assumption for cache writes by any
model other than the session's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
@lucasbraum
lucasbraum merged commit 4cee03a into main Oct 20, 2026
12 checks passed
@lucasbraum
lucasbraum deleted the lucasbraum/upt-127-uptide-fix-json-cost-cannot-be-audited branch October 20, 2026 17:31
@uptide-release uptide-release Bot mentioned this pull request Oct 20, 2026
lucasbraum pushed a commit that referenced this pull request Oct 20, 2026
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## uptide@0.18.2

### Patch Changes

- [#267](#267)
[`907dffb`](907dffb)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - The agent can no
longer change dependencies outside the upgrade. An agent edit that
changes or removes an existing dependency in any `package.json`, or
touches `overrides`, `resolutions` or the `pnpm` field, is refused and
reverted. Uptide sends the refusal back to the agent in the next
feedback round, naming the package, and records it in `report.json`
(`outOfScopeDependencyEdits`). That includes the upgraded package and
its planned companions, whose versions the engine wrote: only a rewrite
to the same value is allowed. The agent may still add a new dependency:
it is kept, recorded (`agentAddedDependencies`) and listed under "Added
by the agent" in the PR body. Lockfiles, `.npmrc`, `.yarnrc(.yml)`,
`.pnpmfile.cjs`, `pnpm-workspace.yaml` and new `package.json` files are
read-only for the agent.

- [#267](#267)
[`907dffb`](907dffb)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - A companion that
would move a new major is planned only when the upgraded package asks
for it: its own manifest at the target pins the companion, the
companion's manifest declares the upgraded package, or the upgraded
package's pack names it. When a third package asks for it
(`@guardian/libs` 33 pinning `@guardian/ophan-tracker-js` 3.1.0 during a
typescript upgrade), Uptide no longer moves it. The PR lists it under
"Blocked" with the declaring package, the manifest field and the range.
Release pairs still move with their lead: `react-dom` with `react`,
`@ai-sdk/*` with `ai`, `@sentry/profiling-node` with `@sentry/node`.

- [#267](#267)
[`907dffb`](907dffb)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - A package that
does not move no longer drags another one through its pins: `fix
typescript` on guardian/support-dotcom-components moved
`@guardian/ophan-tracker-js` 2.8.3 → 3.1.0 because `@guardian/libs`
33.0.0, which stayed, pins it. A pin the same manifest also declares as
a peer the install satisfies moves nothing either, and a pin reason now
names the manifest field and the installed version.

- [#266](#266)
[`a5e3f57`](a5e3f57)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix` no
longer upgrades a peer blocker across a major that no migration pack
covers. Before, `redis` 5 → 6 next to `typeorm` 0.3 (which peers on
redis ^5) moved typeorm to 1.1.0 and ended in an agent-led draft. Now
the blocker keeps its version, with a peer override scoped to it in
package.json: npm `overrides.<blocker>.<peer>`, pnpm
`pnpm.peerDependencyRules.allowedVersions`, Yarn
`resolutions["<blocker>/<peer>"]`. The lockfile scope check still covers
the install. The PR body names the override, which release was not taken
and why. In `report.json`, each `peerConflicts` entry now has `strategy`
(`override`, `blocked` or `none`), `declined` and `override`. If
verification fails with the override, the blocker is reported as a
blocker and is still not upgraded. A blocker you name on the command
line, or one whose major a pack covers, is upgraded as before.

- [#264](#264)
[`2089d0a`](2089d0a)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix` no
longer stops when the package manager resolves a different version than
the target. Yarn classic ignored the pin for a direct dependency and
resolved a range such as `^7.0.105` to its newest match (7.0.146). Now
the lockfile resolves exactly the target with npm, pnpm, Yarn classic
and Yarn Berry, and the manifest keeps its range. If the package manager
still resolves another version, the run continues only when that version
is in the same major, inside the manifest range and covered by the pack.
The PR body, the terminal summary and `report.json` (`resolvedVersion`,
`lockfile.resolved`) then name both versions. Otherwise the run stops
with an error that names the target, the resolved version and the
reason.

- [#268](#268)
[`4cee03a`](4cee03a)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` computes
the agent's cost itself, per model and token class, from Uptide's price
table: `report.json` and `--json` list each model's uncached input,
cache read, 5-minute and 1-hour cache write and output tokens, the rate
applied to each and the cost, so the total can be redone by hand. The
Agent SDK's own figure is kept apart as `sdkReportedCostUsd`;
`--max-cost` counts the higher of the two, a gap above 5% is printed as
a warning, and a model missing from the table uses the SDK's figure,
marked unverified. `llm.inputTokens` now counts uncached input only.

- [#268](#268)
[`4cee03a`](4cee03a)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - Sonnet 5.5 cache
reads are priced at the published $0.10 per million tokens (0.05x
input), not $0.20. Every Anthropic rate in the price table was checked
against Anthropic's pricing page on 2026-10-20, and a test now pins each
one.

Co-authored-by: uptide-release[bot] <338861287+uptide-release[bot]@users.noreply.github.com>
lucasbraum added a commit that referenced this pull request Oct 20, 2026
…and, PR-template rule, dev set (UPT-24)

ecommerce's RESP3 is missed at the location; the committed mention rule is a
footnote. Unrelated changes are judged by hand per run: Uptide's
ophan-tracker 2->3 makes support-dotcom-components not mergeable without
edits; Claude Code's strut test edits follow the required aieo bump. yea's
PR-description check is out of the bar for both arms. Arm A cost cites #268.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
lucasbraum added a commit that referenced this pull request Oct 22, 2026
…o the upgrade's copy (UPT-134, UPT-135) (#271)

Hotfix for 0.18.3. Three signed-off commits, in order: UPT-134, UPT-135,
and the small group-title fix.

## UPT-134: the agent silently does nothing

**Root cause** (`packages/core/src/fix/sdk-agent.ts`, `sdkRunner`, lines
1102–1117 at c64fb88). When the Anthropic API refuses the request
(invalid key, empty credit balance, a model the key cannot use), Agent
SDK 0.3.296 emits one `result` with `subtype: "success"`, `is_error:
true`, `num_turns: 1`, `total_cost_usd: 0` and the API's message in
`result`, then throws `Claude Code returned an error result: …`. The
runner's `catch` kept the result and dropped the throw (`if (!result)
throw err`), and the return only looked at `subtype !== 'success'`. So
the task came back as a clean session with no summary: gate `no-change`,
every site "SDK agent made no change", and up to 3 feedback rounds that
hit the same error. That is the reported signature: `agent` a few
seconds, `feedback` a few seconds, `$0.00`.

Measured directly against SDK 0.3.296 with a bad key and with a model
the key cannot use: both return
`{"subtype":"success","is_error":true,"num_turns":1,"total_cost_usd":0}`
in about 1.2 s, then throw.

**Not a 0.18.2 regression.** Both lines date from 405f429 (2026-10-12).
0.18.1 from npm reproduces the same thing (below). PRs #267 and #268 are
not involved.

**UpgradeBench riseup-oss/mcp (jest 29 → 30) is a different case.**
Uptide found 0 sites, and types, tests and build pass after the bump, so
no agent task was planned (`tasks: []`). The ground truth agrees: the
bump alone equals the human commit. That $0 was legitimate; the run just
never said why. It does now.

**Fix**
- `sdkRunner` keeps the API's message (`sessionError`). A `stop_reason:
refusal` is recorded as `stop: 'refusal'`.
- That task's gate is `error`, and whatever the session wrote before the
error is reset. Feedback starts no further session after an error.
- Safety net, whatever the cause: every task that keeps nothing records
why in `llm.tasks[].noChange`. The reasons are the API's error, no turn,
no output tokens, a refusal, a stop, every edit rejected (with the
gate's reason), or the agent's own `couldNotDo`.
- When nothing the agent did stayed, `llm.agentIdle` carries the reason.
The terminal and the PR body print "The agent did not change anything:
<reason>". The PR body no longer says "The code was migrated" when every
site is still manual.

## UPT-135: pnpm lock guard false positive

I could not get pnpm 10.34 to produce the reported resolutions on a
fresh fixture, so the shape is reproduced at the guard with synthetic
lockfiles (`fixtures/lockfiles/onto-upgrade`, pnpm, npm and Yarn Berry).
The "same version, different resolution" entries were not peer suffixes.
pnpm moved unchanged packages' dependencies (`qs`, `iconv-lite`) onto
the newer copies express 5 brings. The copies they left were outside the
upgrade's subtree, so each one read as "→ none", and every package
depending on them read as "X → X (same version, different resolution)".
On c64fb88 the fixture gives exactly that list. A run like this falls
back to an agent-led draft.

**Fix** (`packages/core/src/fix/managers/lock-guard.ts`, shared by all
three managers)
- An edge of an unmoved package may move onto the upgrade's copy when it
stays in the same major (the same minor below 1.0) and is strictly
newer. Importers' own declarations still move only through the plan.
- A copy that left is in scope when everything that reached it is: the
subtree, an accepted dependent, or another such copy.
- Each move is listed as housekeeping: `housekeeping.onto` in
report.json, a line under **Lockfile housekeeping** in the PR body, and
a count in the terminal.
- **npm** had the same false positive (a nested copy dropped for the
hoisted one) and is fixed by the same code.
- **Yarn Berry** had it too, in a different form: the descriptor
`qs@npm:^6.10.0` changes 6.12.0 → 6.16.1 in place. Yarn keys descriptors
apart from the upgrade's own `qs@npm:^6.14.0`, so the upgrade's copy is
matched by content there, and an in-place descriptor move is accepted
under the same rule. npm placements are still matched by key only: a
nested npm copy that changes in place is another copy on disk, and stays
refused (existing `npm-lock.test.ts` case).
- Still refused: a move to a release the upgrade did not bring (the
control), a downgrade, a move across a major, a 0.x minor, and anything
the repository declares itself.

## Small: group titles

The PR body headed each group of sites with its key. That was the rule
id (`params-string-array`) or, for sites no pack rule covers, the
changed declaration's path. `e.query` is how @types/express names
`express.query`. The run now records each site rule's summary
(`ruleSummaries`), and groups are titled by it. A declaration path gets
the compiler's title at its sites, or "`e.query` was removed in express
5.3.0". Stored reports without summaries keep the id.

## Evidence

Case A: riseup-oss/mcp at `0d0e55a` (bench secondary base), read-only
clone, remotes removed. Case B: a local express 4.21.2 → 5.3.0 fixture
on pnpm, with 6 `req.params`-as-string sites and 1 `express.query()`
site (all manual: the express pack has no rewrites), plus
`@bull-board/express` 5.23.0, which keeps express 4 as its own
dependency. Every run used `--max-cost 0.25`, with the key loaded per
command.

| Run | Uptide | Key | Result | `llm` |
|---|---|---|---|---|
| A before | 0.18.2 npm | real | 0 sites, nothing said | tasks `[]`, $0,
0/0 tokens |
| A before | 0.18.1 npm | real | same | tasks `[]`, $0 |
| A after | this branch | real | PR body: "The agent did not change
anything: nothing was left for it: no site needed it, and the
type-check, tests and build had no new failure to repair." | tasks `[]`,
$0 |
| B before | 0.18.2 npm | rejected by the API | 7 manual, "SDK agent
made no change", 4 sessions (site task + 3 feedback rounds) | each:
turns 1, 0/0 tokens, $0, gate `no-change`, no stop, no error |
| B before | 0.18.1 npm | rejected by the API | identical | identical |
| B after | this branch | rejected by the API | 7 manual; terminal and
PR body: "The agent did not change anything: the session failed: Invalid
API key · Fix external API key"; 1 session, no feedback round | turns 1,
0/0 tokens, $0, gate `error`, `error` and `noChange` set |
| B | 0.18.2 npm | real | 7/7 fixed by the agent | turns 14, 2,354
output tokens, $0.0605, gate `accepted` |
| B after | this branch | real | 7/7 fixed by the agent, no `agentIdle`
| turns 14, 2,508 output tokens, $0.0636, 22.0 s, gate `accepted` |

With a working key, case B runs fine on 0.18.2. Its before-state ($0, 0
fixed) needs the API to refuse the request, which is the failure this
fixes. Case B on pnpm 10.34 wrote no out-of-scope entries on either
version; the reported out-of-scope list is reproduced at the guard
(above).

Spend: $0.12 in total ($0.0605 + $0.0636); every other run and probe was
$0.

## Checks (run locally the way CI runs them, API keys unset)

```text
pnpm install --frozen-lockfile --ignore-scripts   ok
pnpm lint                                         Checked 952 files. No fixes applied. 2 warnings, 1 info (all in files this PR does not touch)
node scripts/private-material.mjs                 1513 tracked files, 1 private identifiers: clean   (EVENT=pull_request, dummy UPTIDE_PRIVATE_DENYLIST)
pnpm build                                        Tasks: 2 successful, 2 total
pnpm typecheck                                    Tasks: 3 successful, 3 total
pnpm test  @uptide/core                           Test Files 203 passed | 4 skipped (207)   Tests 2487 passed | 37 skipped (2524)
pnpm test  uptide (cli)                           Test Files 48 passed (48)   Tests 539 passed (539)
pnpm docs:packs --check / pnpm docs:links         ok; 84 Markdown files, every relative link resolves
```

Under `turbo run test` on my machine, 6 core tests (npmrc, list) and 5
CLI tests (failures, smoke) fail on this branch and on a clean c64fb88
alike. Turbo gives the tests `TMPDIR=/private/tmp`, and a stray
`/private/tmp/package.json` left by an earlier bench run makes `/tmp`
look like a project root. The totals above are each package's suite run
directly. CI's runner has neither.

Mutation checks: 12/12 mutations killed for UPT-134, 14/14 for UPT-135
(after two tests were added for the two that first survived), and 5/5
for the titles.

## Open questions

- The tester's actual API error is unknown. With this build their run
will print it. Worth asking them to re-run, or to send `llm.tasks` from
the 0.18.2 report.json if they still have it.
- UPT-135 accepts a dependency moving forward onto the upgrade's copy
within a major. That is a real resolution change for an unchanged
package (another package's `qs` moves forward), only listed, not
refused. Is that the line you want, or should it stay refused outside
agent-led runs?
- `uptide fix` on a checkout without `node_modules` fails with "express
has no upgrade to 5.3.0" (`fix/run.ts:496`): the check finds no
installed version. That is separate from this PR and is not fixed here.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant