Repository navigation
Conversation
`cacheRatio` (cache reads over the whole prompt) has existed in shared/pricing.ts since v0.6.0, but the only place it reached the UI was the CACHE cell's bottom line, and only when the turn also had a known price — so turns from unpriced models hid the ratio entirely and the session/overview summary never showed it. Render it in three places instead: - the session and overview summary rows, aggregated over turns, as a "cache hit" stat next to the raw cache token count; - each finished turn's CACHE cell, independent of whether a price is known (the cost keeps precedence when there is one); - the in-progress turn line, which already carried the token counts. Input is the provider-normalized fresh-token count, so the ratio stays comparable between exclusive-input (Claude) and inclusive-input (Codex) adapters. No data-layer change: the ratio is derived at render time, so existing ledger records are reinterpreted correctly.
… and hit rate Paseo exposes no per-token or first-token timing, so the only speed signal available to a plugin is the span between a turn's start and end. Summing that span per summary (and per overview range) gives an average output throughput: output tokens over wall clock. Wall clock includes thinking, tool calls, and waiting on the provider, so this is a floor on generation speed rather than the model's token rate — the README and the code comments say so, and the label is "avg out" rather than "speed". Summary.durationMs is added as a non-negative field that defaults to 0, so a client reading summaries from a server that predates it still parses. It is summed alongside the token counts in both summaryFromRows and the overview accumulator; rows without a timing contribute nothing instead of dropping the turn. The composer pill then gains the throughput, and gains the cache hit rate that was previously panel-only. The host caps the pill's button at 160px, and the icon, gaps and padding take 50px, so the label has about 110px before the host ellipsizes it. Measured widths per glyph (12px system-ui, tabular numerals) show `%` at 11.1px against `1` at 5.6px, so the label is assembled against a measured advance-width table instead of a character count, and drops the throughput (then the hit rate) rather than letting the host clip it. Context-window fill is dropped from the pill: the chat surface already shows it next to the composer, so spending pill width on it duplicated the host at the expense of the number that cannot be seen there. Formatting helpers move to shared/format.ts so the pill, panel and overview render from one implementation, and the pill label becomes a pure function with its own tests.
Both cells now show three lines: the token count, the reported cost, then the derived number that describes the count — the hit rate under CACHE, the throughput under OUT. The previous arrangement put the throughput on OUT's second line, which only worked because the second line was empty for unpriced models. As soon as a price is configured the cost takes that line back and the throughput vanishes, which is exactly what a reader wants to see least: configuring pricing should not cost them the speed figure. A third line keeps the reported cost in place and gives each derived value a stable home. The hit rate moves off CACHE's cost line for the same reason: it was sharing `$0.01 · 98%` with the cost, so a long cost could crowd it.
Author
|
Withdrawn for now — no upstream activity, and the design is still being iterated locally (the pill and the per-turn rows changed again since this was opened). I'll reopen or repost once it settles. The branch is kept. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
cacheRatio(cache reads over the whole prompt) has been inshared/pricing.tssince v0.6.0, but it only reached the UI inside the CACHE cell's bottom line when the turn also had a known price. For unpriced models that line fell back to–, so the ratio disappeared even though the token counts were right there. The session and overview summaries never showed it at all, which is where a cache-hit number is most useful.Separately, nothing in the UI showed how fast a session was running, and the composer pill was spending its width on the context-window percentage — a figure the chat surface already displays next to the pill.
What
Cache hit rate — rendered in three places, derived at render time so existing ledger records are reinterpreted correctly (no migration, no new persisted field):
cache hitstat aggregated over turns;$0.01 · 69%when priced, otherwise just69%);Because
inputis already provider-normalized to fresh-token form bysemantics.ts, the ratio is comparable across exclusive-input (Claude) and inclusive-input (Codex) adapters.Average output throughput — Paseo reports no per-token or first-token timing, so the only signal is the span between a turn's start and end.
Summary.durationMs(non-negative, defaulting to0so an older server still parses) is summed next to the token counts insummaryFromRowsand in the overview accumulator, andoutput ÷ durationis shown in the session/overview summary, per turn, and in the pill. Wall clock includes thinking, tool calls, and provider wait, so this is a floor on generation speed, not the model's token rate; the README says so explicitly rather than implying a measurement the protocol cannot make.Per-turn values share the OUT column's second line, which unpriced turns left as
–. A priced turn keeps its output cost there, since that is a reported figure and the rate is one press away in the summary.Composer pill — now
33.99M(96%) 70t/s, or$0.43(90%) 70t/swhen the model has a price.Why context fill left the pill
It was already on screen. The chat surface renders the context-window bar and its
context 267.6k / 950.0klabel immediately beside the composer, so the pill was repeating a number the user could read one element to the left — while omitting the throughput, which appears nowhere else in the host. With the pill capped at 160px there is room for tokens, hit rate, or context, not all three; the panel still shows the full context bar (and keeps it when idle).The cap is 160px, of which the icon, gaps and padding take 50px, leaving ~110px of label before the host ellipsizes. Measured advance widths per glyph (12px system-ui, tabular numerals) put
%at 11.1px against1and(at ~5.6px, so the label is assembled against a measured width table rather than a character count. If the exact count is too wide it is re-rendered more coarsely (33.99M→34M), then the throughput is dropped, and last the hit rate — the count itself is never dropped.Two details found by measuring rather than estimating:
33.99M(96%) 70t/sneeds a thin space, not a plain one: 109px versus 111px, and 111px is the width that clips.Formatting helpers move to
shared/format.tsso the pill, panel and overview render from one implementation, andpillLabelbecomes a pure function with its own tests.Verification
tsc --noEmitclean;node --test shared/*.test.ts server/*.test.ts→ 77/77 pass (new tests cover the hit-rate ratio, the throughput derivation, the duration aggregation, and the pill's fit-and-degrade order).33.99M(96%) 70t/sat 109px inside a 159px button, unclipped; session summary showscache hit 97%andavg out 68 tok/s; per-turnCACHEcolumn shows69%/83%/96%/97%; per-turnOUTcolumn shows56t/s/48t/s/86t/s/108t/s. Console clean.Notes for review
shared/format.ts,shared/pill-label.ts, and the two view files.