fix(pricing): gpt-5.6 terra/luna carried the launch tiers; luna was 5x over - #29
Merged
Conversation
…x over terra 2.5 / 15 / 0.25 -> 2 / 12 / 0.2 luna 1 / 6 / 0.1 -> 0.2 / 1.2 / 0.02 Not a new discovery — promptster-backend settled this on 2026-07-31 in f4500a27, Paarth confirming LiteLLM against the vendor: both models were repriced after GA and the rows still carried launch day. This copy never got the correction. WHICH IS THE ACTUAL DEFECT. src/vendor/pricing.ts is a hand-copied mirror of promptster-backend packages/config-cost, and it has no subscriber to that repo — cc-audit deliberately does not depend on it, so the mirror can only go stale silently, and did, for two weeks. Diffed the two tables in full while here: 29 OpenAI rows and 16 Anthropic rows on both sides, identical key sets, and these two rows were the ONLY divergence. One missed sync, not a pattern. pricingDrift.test.ts is the one thing that notices, and it worked — it went red the first time CI ran after the repricing (main has been red on it since 2026-08-13) and stayed red rather than being relaxed. The suite is now 405/405 for the first time. A red run there is an instruction to find out who is right, not to update the expectation; the header comment now says so. SHIPPED IMPACT: 0.7.0 is the published version and went out 2026-07-29, two days before the correction. Any user auditing Codex sessions on gpt-5.6-luna has been seeing a 5x overstatement of those turns, terra 1.25x. A wrong-but-present rate is quieter than the failures this table is famous for — a missing model bills $0 and vanishes, a bad prefix match bills 12x under — because every downstream number still looks like a number. Needs a release, not just a merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Greptile SummaryThis PR corrects the mirrored OpenAI pricing for GPT-5.6 Terra and Luna after their post-GA repricing.
Confidence Score: 4/5The PR appears safe to merge, with the non-blocking concern that the corrected rates are not protected by an offline regression test. The changed rates flow correctly into existing OpenAI cost calculations and match the stated correction, but network-degraded test runs cannot detect a future regression of these exact values. Files Needing Attention: src/vendor/pricing.ts
|
| Filename | Overview |
|---|---|
| src/vendor/pricing.ts | Corrects the two stale pricing rows and improves mirror documentation, but leaves the corrected values without deterministic offline regression coverage. |
Comments Outside Diff (1)
-
src/vendor/pricing.ts, line 391-392 (link)Corrected rates lack offline coverage
The corrected
gpt-5.6-terraandgpt-5.6-lunavalues are only covered by the network-dependent drift check, so an offline or network-degraded test run can pass after these rates regress and allow the same silent spend overstatement to return. Please add deterministic assertions for both corrected rows.
Reviews (1): Last reviewed commit: "fix(pricing): gpt-5.6 terra/luna carried..." | Re-trigger Greptile
pa-arth
added a commit
that referenced
this pull request
Aug 15, 2026
…later turn (#30) Review findings from #28 and #29, both real. ### 1. Resumed prefix used the wrong turn (#28, Greptile) `injectedOpen` closes at the first main-chain assistant row — 34 lines BEFORE the cross-file `seen` dedup — while `computeAlwaysOn` measures the prefix from `firstMain`, the first RETAINED turn. On a resumed transcript those are different rows: the closing row is a replay owned by an earlier file and gets dropped, so `firstMain` lands on a later turn whose cacheRead has the whole replayed conversation folded in. Turn-1-sized `attributed` against a much larger `measured` INFLATES `fixedPrefixTokens`, and `reconcile()` cannot catch it because it only fires on a NEGATIVE remainder. Inflation is the silent direction. Closing before the dedup was deliberate and stays — it is what stops a whole resumed session's attachments folding into "turn 1". The bug was the other half. Fixed by recording which row closed the window and which row `firstMain` will pick, and declining to measure the session when they differ — counted, not silently dropped. `standingContextTokens` is untouched: a resumed turn really did carry that much. Only the BREAKDOWN is unanswerable, and `unmeasured` now says so with its own reason rather than borrowing the no-attachments one. **ZERO INSTANCES IN THE AUTHORING CORPUS: 0 of 483 sessions with attachments, of 926.** Verified by same-instant runs of origin/main and this branch — 0 of 30 fields moved. So this fixes no number today; it closes a path that would have been invisible when it did fire. That also means the case cannot be recorded, and a suite running only real local data would stay green with the guard deleted. It is built from RECORDED bytes anyway: a resumed replay is "rows this parse has seen, then rows it has not", so seeding `seen` from a prefix of a recorded transcript and re-parsing the whole of it reproduces the shape exactly, with no invented rows. Two recorder bugs surfaced doing that, both the fixture-encodes-an-impossible-shape failure the recorder exists to prevent: - `message.id` was filled with CONSTANT filler, so every assistant row shared an id and the parser merged them — a "two-turn" recording came back as one turn with one prefix. Identity keys are now a memoized bijection: distinctness and intentional sharing both survive, no real value is carried. The old comment claimed ids "key nothing across fixtures"; they key the dedup and the streamed-row merge. - the recorder counted assistant ROWS, not turns. Now counts distinct message ids. ### 2. Corrected rates had no offline coverage (#29, Greptile) `pricingDrift` degrades to pass when it cannot fetch — correct, and it means an offline run proves nothing about the rows it just corrected. `pricingPinned.test.ts` is the deterministic companion: drift asks whether the VENDOR moved, this asks whether OUR table moved underneath us. Only rows with a reason to be pinned are in it; pinning all 29 would train people to update the expectation to match the code. ### 3. A stale comment that would have caused the next mis-pricing Writing that test failed it, on `gpt-5.5-pro`. The header claimed "the `pro` tiers publish no cached-input rate" and are set to cachedInput == input. **False since gpt-5.4-pro/5.5-pro**: both publish 3/M, the ordinary 10%, confirmed against LiteLLM, and the table correctly carries it. A reader trusting that comment would have "restored" them to equal-input and over-billed every cached token they read. The real exception set is three rows and is now named with reasons rather than derived from the word "pro" — gpt-5-pro and gpt-5.2-pro (genuinely no published rate) and codex-mini-latest (genuinely 25%). 410/410 tests, typecheck and lint clean. Mutation-checked: reverting luna to the launch tier, "fixing" gpt-5.5-pro to equal-input, and dropping the prefix-turn gate each fail only their own test. The two-turn fixture asserts its own precondition (turn 2's prefix 61,014 -> 61,863) so a future re-recording cannot quietly leave the guard proving nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Greptile P2 on #30, valid — and the failure was one step worse than reported. `tool_result.tool_use_id` names a `tool_use.id`, which `claudeCode.ts:270` reads to fold is_error and the result timestamp back onto the issuing turn. `id` went through the identity bijection; `tool_use_id` fell through to length-preserving filler. Because it is LENGTH-preserving, the two real ids were the same length and both results came out as the same string — so the fixture did not merely lose the link, it pointed two distinct calls at one key. A tool_result naming a tool_use that does not exist is a transcript Claude Code cannot emit, which is the exact class of defect recording fixtures exists to prevent (and the second instance of it in this PR). Added to IDENTITY and the two-turn fixture re-recorded from the same transcript: 4 lines change, the recorded prefixes are byte-identical (61,014 -> 61,863), so the resumed-transcript assertions rest on the same numbers as before. Also asserted, across EVERY injected-prefix fixture rather than the one that regressed: every tool_use_id resolves to a tool_use in the same file, and distinct results name distinct calls. The second half is not redundant — containment alone passes the moment filler collides with a real id, which is how this hid. 414/414 (the fixture join check runs once per fixture file), typecheck and lint clean. Checked the guard bites: the pre-fix fixture fails it by name ("dangling tool_use_id redacted-fixture-content redac"). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merged
pa-arth
added a commit
that referenced
this pull request
Aug 15, 2026
Four merged PRs since 0.7.0 (#27 #28 #29 #30), and two of them change numbers a user has already read, which is what makes this a release rather than a merge. MINOR, not patch: #28 changes what the always-on breakdown MEANS — measured off the transcript instead of censused off disk — and takes the aggregate schemaVersion 9 -> 10, removing `skillDescriptionTokens` rather than keeping a field whose name outlived its meaning. THE ONE TO SHIP FOR: 0.7.0 prices gpt-5.6-luna 5x over and terra 1.25x over on every Codex session it has audited since the vendor repriced (backend f4500a27, 2026-07-31, two days after 0.7.0 went out). That was fixed on main on 2026-08-14 and reaches nobody until this publishes — a merged pricing fix in an unreleased CLI is a fix that does not exist for users. Verified before tagging, not just built: - 414/414 tests, typecheck and lint clean - `npm run build:npm` -> 411.9kb single file, `--version` reports 0.8.0 (so the esbuild --define baked the bump, which is the thing a version bump can silently miss) - ran the BUNDLE against the real local corpus, not just `--help`: schemaVersion 10, standingContextTokens 60,963 — within 0.4% of the 60,738 measured independently by differencing `claude -p` runs — `skillDescriptionTokens` absent from the payload as #28 intended, `unmeasured` empty. Tag v0.8.0 after merge; .github/workflows/publish.yml guards tag == package.json and publishes + attaches the standalone binaries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns main green.
pricingDrifthas been the only failure since 2026-08-13 and the suite is now 405/405 for the first time.Who was right
config-costgpt-5.6-terragpt-5.6-lunaNot a new discovery. promptster-backend settled it on 2026-07-31 in
f4500a27, with Paarth confirming LiteLLM against the vendor: "luna was repriced after GA and our rows still carried the launch tier." This copy never got the correction.I checked the agreement wasn't circular before trusting it.
39e47f6e(backend #319) introduced2.5/0.25/15and1/0.1/6; cc-audit #19 ported those verbatim and was correct at the time;f4500a27corrected the backend against the vendor; cc-audit didn't move. Both tables also already agree with LiteLLM ongpt-5.6andgpt-5.6-sol(5/0.5/30), so the drift is isolated to the two repriced rows. The 10%-of-input cached-rate invariant in the header still holds at the new numbers.The actual defect is the mirror, not the two numbers
src/vendor/pricing.tsis hand-copied frompackages/config-costand has no subscriber to that repo — cc-audit deliberately does not depend on it. A mirror like that can only go stale silently, and did, for two weeks.So I diffed the two tables in full rather than patching the two rows I already knew about: 29 OpenAI rows and 16 Anthropic rows on both sides, identical key sets, and these two rows are the only divergence. One missed sync, not a pattern. Recorded in the header so the next reader knows the mirror is a mirror.
pricingDrift.test.tsis the one thing that notices, and it did its job — it went red the first time CI ran after the repricing and stayed red rather than being relaxed to fit. Its instruction, inherited from the commit that fixed the backend side: find out who is right, not update the expectation. That is now written at the top of the table.Shipped impact — needs a release, not just a merge
0.7.0is the published version and went out 2026-07-29, two days before the correction. Any user auditing Codex sessions ongpt-5.6-lunahas been seeing a 5x overstatement of those turns; terra 1.25x.A wrong-but-present rate is quieter than this table's famous failures — a missing model bills $0 and vanishes from every spend view, a too-eager prefix match bills 12x under — because every downstream number still looks like a number. Nothing is zero, nothing is absent, the totals are just wrong.
Uploaded aggregates already in
cost_audit_aggregates/solo_captureskeep the overstated figures. This fixes the rate, not the stored rows.Verification
npx tsc --noEmitclean.npx vitest run— 405 passed (405), 40 files, 0 failures.🤖 Generated with Claude Code