Skip to content

feat(arenabench): transcript as its own searchable page + dedupe streamed text - #1884

Open
macanderson wants to merge 2 commits into
mainfrom
arenabench-transcript-page
Open

feat(arenabench): transcript as its own searchable page + dedupe streamed text#1884
macanderson wants to merge 2 commits into
mainfrom
arenabench-transcript-page

Conversation

@macanderson

@macanderson macanderson commented Aug 6, 2026

Copy link
Copy Markdown
Owner

What

Two halves of one user-facing outcome — the transcript surface:

1. Every response rendered twice (bug). Stella's event stream carries both text_delta fragments (payload field: text) and one consolidated text event per message (payload field: delta) — the names are crossed on the wire. TranscriptReader coalesced both into the same run, so each message appeared once assembled from fragments and once whole, usually split around the step_usage line that closed the run between them. The consolidated event now replaces its fragment run under the same seq (clients keyed by seq replace in place), located via a pending_text pointer that deliberately outlives the run-closing step_usage. Witnesses in tests/test_telemetry.py::TestTranscript — all three fail on main (doubled body / two seqs per message / concatenated lone messages), pass here.

2. The transcript is now its own page (feature). /transcript?match=…&task=…&seat=… replaces the fly-out drawer: a URL you can bookmark, reload, and send, with the reading tools a drawer had no room for —

  • kind filters: responses / thinking / tools / results / usage / stages
  • full-text search with <mark> highlighting and an entry count; an active search bypasses replay pacing (a search answers over the whole transcript, never behind a timer)
  • task picker + seat picker write back to the URL, so the view you're on is always shareable
  • playback semantics carried over intact: paced replay with 1×–6× for finished trials, pause/resume for live ones

Rendering follows the Command Deck's transcript grammar (crates/stella-tui/src/render/entry.rs is the exemplar): stages are section rules (the label is the stage), reasoning is dim italic collapsed to a preview with its line count, the label is coloured and the value is read (tool names take the accent, bodies stay plain), long results fold at twelve lines.

server._static learns the exporter's file naming (<route>.html, index.html under a directory) so the clean route resolves; the path-containment check still runs first.

Verification

  • uv run pytest — whole suite green (4 Docker-dependent skips).
  • tsc --noEmit clean; next build exports /, /transcript, /_not-found.
  • Witness direction checked by swapping in origin/main's telemetry.py: the three new tests fail there with the exact doubled-text shapes reported from match ba6464fca798.

Residue

Summary by Sourcery

Deduplicate streamed transcript text events and promote transcripts from a drawer to a dedicated, bookmarkable page with search and filtering.

Bug Fixes:

  • Ensure consolidated transcript text events replace their fragment runs so each response renders only once.

Enhancements:

  • Add a dedicated transcript page with task/seat selection, kind filters, full-text search with highlighting, and paced playback semantics aligned with the Command Deck.
  • Wire arena task table rows to the new transcript page via shareable links and simplify arena view state accordingly.
  • Teach the static file server to resolve clean route paths to the corresponding exported HTML files.

Tests:

  • Add regression tests around consolidated vs. fragment transcript text events to prevent duplicated or missing messages.

Stella Test added 2 commits August 6, 2026 04:09
Stella's event stream carries both text_delta fragments (field: text) and
one consolidated text event per message (field: delta) — the names are
crossed on the wire. TranscriptReader appended both into the same run, so
every response rendered twice: once assembled from fragments, once whole,
usually split around the step_usage line that closed the run in between.

The consolidated event now REPLACES its fragment run under the same seq
(clients keyed by seq replace in place), found via a pending_text pointer
that outlives the run-closing step_usage. A stream with no fragments still
renders one entry per consolidated event.
The transcript drawer becomes /transcript?match=&task=&seat= — a URL you
can bookmark, reload, and send. New reading tools the drawer had no room
for: kind filters (responses / thinking / tools / results / usage /
stages) and case-insensitive full-text search with highlighting across
every entry; an active search bypasses replay pacing, because a search
answers over the whole transcript.

Rendering follows the Command Deck's transcript grammar: stages are
section rules (the label is the stage), reasoning is dim, italic and
collapsed to a preview with its line count, tool names take the accent
while bodies stay plain, and long results fold at twelve lines.

The task table's transcript button is now a link; the drawer is deleted.
server._static learns the exporter's naming (<route>.html, index.html in
a directory) so the new route resolves without touching the URL shape.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
stella-cli-docs Ignored Ignored Aug 6, 2026 11:21am

@sourcery-ai

sourcery-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Fixes duplicated transcript text by making consolidated text events replace their preceding fragment runs in the telemetry reader, and replaces the in-page transcript drawer with a dedicated, URL-addressable /transcript page that supports task/seat selection, filters, search, and paced playback, while updating static asset resolution to serve clean route URLs.

Sequence diagram for deduping streamed transcript text in TranscriptReader

sequenceDiagram
    participant Stream as StellaEventStream
    participant Reader as TranscriptReader
    participant State as TranscriptState
    participant Bodies as TranscriptReader_bodies

    Stream->>Reader: entry(kind="text_delta", event)
    Reader->>Reader: bucket = "text"
    Reader->>Reader: fragment = event.get("delta") or event.get("text")
    alt no open text seq
        Reader->>Reader: seq = _next_seq(State)
        Reader->>State: open_entries["text"] = seq
        Reader->>State: pending_text = seq
    else existing text seq
        Reader->>State: seq = open_entries["text"]
    end
    Reader->>Bodies: bodies[(path_key, seq)] += fragment

    Stream->>Reader: entry(kind="text", event)
    Reader->>Reader: full = event.get("delta") or event.get("text")
    alt existing open text or pending_text
        Reader->>State: seq = open_entries["text"] or pending_text
    else no existing seq
        Reader->>Reader: seq = _next_seq(State)
    end
    Reader->>State: open_entries.clear()
    Reader->>State: pending_text = None
    Reader->>Bodies: bodies[(path_key, seq)] = full
    Reader-->>Stream: entry(seq, "text", "response", full)
Loading

File-Level Changes

Change Details Files
Make consolidated transcript text events replace prior fragment-built runs to avoid duplicated messages and handle late arrivals.
  • Extend TranscriptState with a pending_text pointer that tracks the last fragment-built text entry beyond run closure.
  • Adjust streaming text handling so text_delta and reasoning fragments coalesce entries, recording pending_text for text runs.
  • Introduce special handling for text events that treats them as consolidated messages: resolve the correct seq via open_entries or pending_text, replace the stored body, and clear trackers.
  • Ensure non-delta events still close open text/reasoning runs and add regression tests covering replacement, late consolidated events, and lone text events.
arenabench/arenabench/telemetry.py
arenabench/tests/test_telemetry.py
Replace the transcript drawer with a dedicated transcript page that is navigated via link from the task table and supports filters, search, and playback controls.
  • Simplify TaskTable to drop transcript drawer state/props and render a styled anchor linking to /transcript with match and task query parameters.
  • Remove TranscriptDrawer usage and related activeTask state from ArenaView.
  • Introduce a client-side TranscriptPage component that reads URL params, pulls match snapshots and streaming transcript entries via SSE, and renders them using a grammar aligned with the Command Deck (stages as section rules, dim/collapsible reasoning, tool/result styling, usage lines).
  • Add reading tools to TranscriptPage including kind filters, case-insensitive text search with highlighting, per-entry folding, and paced playback with speed control for finished trials while keeping live-stream pause/resume semantics.
  • Wire TranscriptPage into the Next.js app by adding the /transcript route component.
arenabench/ui/components/arena/task-table.tsx
arenabench/ui/components/arena/arena-view.tsx
arenabench/ui/components/arena/transcript-page.tsx
arenabench/ui/app/transcript/page.tsx
arenabench/ui/components/arena/transcript-drawer.tsx
Improve static asset resolution so exported HTML routes can be served via clean URLs.
  • Update server._static to map directory paths to index.html and fall back to a .html sibling when the requested path has no suffix, after enforcing path-containment under the web root.
  • Leave existing security/path checks intact while ensuring clean route URLs resolve correctly for the static export.
arenabench/arenabench/server.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant