Skip to content

Control channel: scope buffer page reads to referenced pool entries and attachments#4523

Open
ransomr wants to merge 6 commits into
UKGovernmentBEIS:mainfrom
meridianlabs-ai:claude/issue-80-20260714-2155
Open

Control channel: scope buffer page reads to referenced pool entries and attachments#4523
ransomr wants to merge 6 commits into
UKGovernmentBEIS:mainfrom
meridianlabs-ai:claude/issue-80-20260714-2155

Conversation

@ransomr

@ransomr ransomr commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

This PR contains:

  • New features
  • Changes to dev-tools e.g. CI config / github tooling
  • Docs
  • Bug fixes
  • Code refactor

What is the current behavior? (You can also link to an open issue here)

Fixes meridianlabs-ai#80

When a control-channel events page is served from the realtime sample buffer (open_sample_history_from_sample_history), the page limit only bounds the event rows. Each page request also unconditionally loads and json.loads the sample's entire message pool and call pool, and fetches every attachment body for the sample (tens of MB on image-heavy transcripts). Since the read runs synchronously on the eval's own event loop, a read-only monitor polling inspect ctl events imposes repeated blocking pauses on in-flight samples.

What is the new behavior?

Fix option 1 from the issue — page-scoped pool/attachment loading:

  • SampleHistory pools are now position-keyed mappings (dict[int, ...]) rather than dense lists. Full-history reads carry every entry (contiguous keys from 0); page reads carry only referenced positions.
  • After fetching a page's event rows, _sample_history(page_scoped=True) collects the input_refs/call_refs positional ranges and the attachment:// hashes referenced by the page's event JSON (plus refs inside the loaded pool entries, since large message content is itself stored as an attachment), and loads only those rows (WHERE pos IN (...) via ROW_NUMBER rank, WHERE hash IN (...) scoped to the sample; IN lists chunked under SQLite's bound-parameter limit).
  • Both paged entry points (open_sample_history_from and open_sample_history_tail) are page-scoped; open_sample_history (full reads: streaming flush, recorder, retry errors) still loads complete pools.
  • _expand_refs / resolve_model_event_inputs / resolve_model_event_calls / materialize_pooled_events accept position-keyed mappings alongside dense lists; the buffer transcript history provider resolves against the mappings.
  • SampleHistory.events_data (dense pools for .eval serialization) became a property that raises on a page-scoped (sparse) history rather than silently misaligning refs.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

No — all changed interfaces are internal (inspect_ai.log._recorders.buffer.*, inspect_ai.event._pool). Public resolve_* helpers only widened their accepted parameter types.

Other information:

Verification: pytest tests/log tests/checkpoint tests/_control tests/_eval/test_retry_error_events.py (all passing), ruff check/ruff format, and mypy clean on changed files. New tests in tests/log/test_sample_history.py cover page-scoped message/call pool loading, attachment scoping (event-level and pool-entry-level), end-to-end materialization through BufferTranscriptHistoryProvider.events_from, and the events_data guard.

The keyset-cursor option (#2 in the issue) is not included; the ROW_NUMBER CTE still scans the sample's event rows per page, but no longer parses or ships unreferenced pool/attachment content.

🤖 Generated with Claude Code

github-actions Bot and others added 6 commits July 14, 2026 22:15
Paged sample-history reads from the realtime buffer (open_sample_history_from
/ open_sample_history_tail) previously loaded and parsed the sample's entire
message pool, call pool, and every attachment body on every page request,
making control-channel event polls O(sample) instead of O(page).

SampleHistory pools are now position-keyed mappings; page-scoped reads
collect the input_refs/call_refs positions and attachment hashes actually
referenced by the page's events (including refs inside the loaded pool
entries) and fetch only those rows. Full-history reads still load complete
pools, and events_data (dense pools for .eval serialization) now guards
against misuse on page-scoped histories. Pool-ref resolution accepts
position-keyed mappings alongside dense lists.

Fixes #80

Co-authored-by: Ransom Richardson <1209015+ransomr@users.noreply.github.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the contiguity heuristic in SampleHistory.events_data with an
explicit page_scoped field set by the construction site, so pages that
reference a contiguous pool prefix also raise instead of returning
truncated pools. Bind events_data to a local in
eval_retry_error_from_history to avoid rebuilding the dense pools twice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add POOL_REF_FIELDS in event._pool as the single registry of event fields
carrying range-encoded pool refs, with collect_pool_ref_positions()
replacing the hardcoded collector in the buffer database. A new test
introspects the event models for *_refs fields and fails when one isn't
registered, so page-scoped reads can't silently drop pool entries for a
future ref field. Also add strict=True to the SampleHistory pool
re-keying zips so a length mismatch raises instead of truncating.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-80-20260714-2155

# Conflicts:
#	CHANGELOG.md
… histories on the SampleBuffer interface

Review round 2 feedback: move the export path's pool-ref remapping into
event._pool as remap_pool_refs(), walking the POOL_REF_FIELDS registry so
a newly registered ref field is remapped without a code change, and note
on the abstract open_sample_history_tail/_from docstrings that they yield
page-scoped histories (sparse pools, events_data raises).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Control channel: buffer page reads load full pools/attachments and scan all event rows

2 participants