Repository navigation
Conversation
The new library adds a per-breakpoint TTL phase, media pricing and validation bounds. The suite for dash, proxy, apply, keepalive and components passes unchanged, so no figure changes. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: DAVID BOAZ <DAVIDBO@il.ibm.com>
…#909) The live path booked no split credit. The cause was in the library, not in the proxy path. On a short conversation the stable half the provider reads is within the library's read tolerance (128 tokens plus 2%) of the whole previous entry. The library took that read as proof that nothing broke, moved B* past the changed system block and dropped the split effect. The proxy sent the right facts and the right split operation. The fix is accountant fix/split-short-turn (23e2ed1). This commit pins that version and adds a proxy test with a short conversation, which fails on the old pin and passes on the new one. Before (fake upstream, 4 requests, tail moves each turn): cachesplit_saved_usd 0, split_credited 0. After: 0.04433, split_credited 3. Truth: 3 x 4325 x (3.75 - 0.30) / 1M = 0.04477. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: DAVID BOAZ <DAVIDBO@il.ibm.com>
#910) Summarize hit ops_dropped on most requests, so it booked its model cost and no saving. The splice itself was reported. The rebuild after a count change broke the sizes. It matched a kept message to the first unused slot with the same bytes. A user message that holds a tool_result and a text block (for example "continue") has the same bytes on every turn. A kept message took the slot of a removed turn, so the forwarded body held that turn's orphan tool_result and not the kept message. The byte sizes then failed wireSizes and the library dropped all operations. A byte match now prefers a slot of a wire message that a kept tool message already names by its tool_call_id. Before (fake upstream, 10 requests): net_saved_usd -0.00274, 6 requests ops_dropped. After: +0.00404, 0 fallbacks. Truth from the fake's own log (control 0.0721437 minus treatment 0.0681045): +0.00404. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: DAVID BOAZ <DAVIDBO@il.ibm.com>
…the manual figure an estimate (#913) The issue feared that a manual ping, which stores no facts, would leave the next request cold. The scenario was run with the pinned library: request at t0 writes 100,000 tokens, manual ping at t0+4m reads them, request at t0+8m reads them again. The gap is 8 minutes, past the 5 minute TTL. The phase is hot, not cold. The library takes a billed read that covers the previous extent as proof that the entry was alive, so it needs no LastPing. The same request after an automatic ping is hot too and books a rescue. The manual run books none, so the write is not counted twice. No code change was needed for the phase. The test pins it. Storing the facts with the rescue disabled, as the issue suggested, would add state and a row flag for no change in any figure, so it is not done. ManualSavedUSD is the user's estimate of the avoided write. The Overview now labels the line 'estimate' and the Cache info text and docs/dashboard.md say it is not a measurement. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: DAVID BOAZ <DAVIDBO@il.ibm.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
integ/batch1(PR Fix: Combine the accountant fixes for #906-#919 on the re-land #927). Draft.fix/split-short-turn(23e2ed1), pinned in go.mod. Merge that library PR first and re-pin to its merge commit.What changed
cachesplit_saved_usd0,split_credited0 on the live pathfix/split-short-turn: the rule skips a delivered split on a changed system block. Pin bumped. Proxy test with a short conversation added./api/statsnow agree.ops_droppedon 6 of 10 requests, net -0.00274tool_resultand the text "CALL_TOOL again" has the same bytes every turn. A kept message took the slot of a removed turn. The forwarded body held that turn's orphantool_result, so sizes failedwireSizesand all ops were dropped.rebuildCountChangedprefers a slot of a wire message that a kept tool message already names bytool_call_id.LastPingis needed.Side effect of the #910 fix: before it, the forwarded body for such transcripts held an orphan
tool_result. It is now correct. This is a behavior change for summarize on transcripts with repeated sibling text.Skipped: storing facts for a manual ping with the rescue disabled, and a
KeepAliveManualrow flag. #913 shows no wrong phase, so it would add state and a column for no change. A human can decide if they still want it.Data dependence: no stored field is added.
docs/dashboard.mdsays the manual figure is an estimate.How it was tested
/tmp/review-893/features, before and after, against the fake upstream's own cost log.TestSavings_ASplitBooksItsEffectOnAShortConversationWhenTheTailMoves(fails on the old pin),TestSpliceKeepsSurvivorsWithIdenticalSiblingText(fails before the fix),TestTheRequestAfterAManualPingIsHotNotCold. In accountant: casesplit/short-conversation/tail-changed.go test -count=1 ./proxy ./dash: ok../apply/... ./keepalive/... ./components/...: ok.gofmt -l,go vet ./...,go build ./...,go mod tidy -diff: clean.Assisted-By: Claude (Anthropic AI) noreply@anthropic.com