bench(chat-history): hybrid arm's top-spacer height drifts — deep scroll-back excursion stalls at 114/200 messages

[jdmanring]

Summary

tests/bench/vendor/hybrid_bench.js (the benchmark's hybrid arm — warm-band detach over cold-tail eviction) under-reports the height of unrendered history in its top spacer. Scrolling up, scrollTop reaches 0 while ~1470 messages are still unrendered, so a scroll excursion terminates far short of its target.

Measured at n=2000, excursion target 200 messages:

arm	messages actually traversed
naive	201
evict	200
hybrid	114 (stalls, scrollTop hits 0 with lo=1470)

Impact — contained, but real

The benchmark's completeness guard catches this: upBy() reports complete: moved >= k, and run_cell nulls the cell rather than publishing a number from a short excursion. So no false number is published — hybrid's deep-scroll-back timing cells are blank in the artifact, by design.

Unaffected, and explicitly still trusted:

• hybrid's memory numbers (USS, retained nodes): sampled at the top after the full bottom-to-top sweep, which does reach lo == 0.
• tests/test_chat_history_hybrid_bench_js.py asserts lo == 0 after the sweep and a bounded window throughout — both still pass.

The benchmark's conclusion does not rest on the affected numbers. Eviction dominates the hybrid on memory structurally (detach-preserve retains a superset of what eviction retains over the same window), not by measurement.

Suspected cause

Hybrid.prototype._spacerHeights estimates unmeasured message heights as heights[i] || avgH, where avgH is seeded from the initial WINDOW_SIZE render. Suspects, in order:

1. avgH under-estimates the true corpus mean (the initial window is the newest 50 messages, not a representative sample), so the top spacer is systematically short by a factor that grows with lo.
2. The scroll-anchor box.scrollTop += box.scrollHeight - before in _pageOlder may compound a per-page error rather than correcting it.

A ~10% avgH error accounts for only ~25k px of the ~220k px shortfall observed, so (1) alone does not explain it. Read the geometry before changing code.

Scope

This is a proof-of-concept benchmark arm, not shipping code, and the hypothesis it was built to test (that a warm detach band adds value over eviction alone) was refuted on other grounds. Fixing it is worthwhile only to make the withheld timing cells publishable, which would strengthen the artifact but change no conclusion.

Deliberately not fixed under the run that found it: the harness must not be edited while a measurement run is in flight.

Repro

venv/bin/python -c "
import sys; sys.path.insert(0,'tests/bench')
from playwright.sync_api import sync_playwright
import chat_history_bench as B
with sync_playwright() as pw:
    ctx = pw.chromium.launch_persistent_context('/tmp/hb', headless=True, args=B.LAUNCH_ARGS, viewport=B.VIEWPORT)
    page = ctx.new_page(); page.set_content(B._HARNESS_HTML)
    B.load_arm(page, 'hybrid', 2000)
    page.evaluate(B._deepback_probe_js())
    print(page.evaluate('()=>window.hybridBench.stats()'))
    print('scrollTop', page.evaluate(\"()=>document.getElementById('chat-history').scrollTop\"))
    ctx.close()
"

Expect lo well above 0 with scrollTop at ~0.

[github-actions]

⚠️ Issue description is incomplete. Please update the following sections:

• Install Method — select how you installed Odysseus
• Operating System — select your OS
• Steps to Reproduce — must include at least one numbered or bulleted step
• Expected Behaviour — section is empty or too short
• Actual Behaviour — section is empty or too short

This comment is deleted automatically once all sections are complete.

[jdmanring]

Second, distinct cause of a withheld cell — evict, not hybrid

The clean artifact (commit 8425c28d) blanks the deep-scroll-back cell for both bounded arms, but for different reasons. Only one of them is this issue.

arm	excursion moved (target 200)	reached newest message?	why the cell is blank
hybrid	116 (66 at n=5000)	—	this issue: top spacer drifts, excursion falls short
evict	200 ✅	no	the return walk never arrives

So evict walks its full excursion and is still withheld: the driver's scrollTop += clientHeight * 0.9 walk never reaches the newest message. chatHistory.js re-anchors scrollTop while paging (to hold the viewport steady), which fights a programmatic stepped scroll. An early version of the walk stalled at scrollTop 12756 of 16718 with the newest message never rendered — and reported ~0ms, because almost nothing had happened. That is why walkDown() now returns complete and the runner discards the cell.

Consequence for this repo's claims: the benchmark supports no statement about eviction's deep scroll-back cost. Anything asserting one is unsupported. The hybrid refutation is unaffected — it rests on retained memory, which is structural.

Is this a real bug? Unknown — do not assume

It may indicate a genuine "scrolling down never reaches the newest message" defect under programmatic scroll, or it may be an artifact of stepping scrollTop against the anchoring logic in a way a real user's wheel/trackpad input never does.

Not filed as a bug, because it has not been reproduced in the real app. Reproduce it against the running application (wheel input and scrollTo, not stepped scrollTop assignment) before reporting it as one. If it reproduces there, it deserves its own issue and is considerably more serious than this one.

Related: #127 (the same file's spacer never represents unrendered history, so its scrollbar reports 2.5% of a 2000-message conversation). The anchoring behaviour and the missing spacer are plausibly the same geometry problem seen from two sides — worth reading together.