Description
The Trace panel in the VS Code extension freezes when a pipeline returns audio,
video, or image media. The final-result renderer cannot recognise the media
result shapes, so it falls through to the raw JsonTree fallback — which prints
the entire base64 payload (a multi-megabyte string for any real video) into a
single DOM text node with no length limit.
Two independent defects combine to cause this:
-
The type guards never match the result shape. render_final.tsx wraps the
audio/video lanes as {audio: v} / {video: v}, then calls isAudio / isVideo
on that envelope. But those guards look for .action / .format at the top
level — they are the AVI stream-trace guards, reused for the final result,
which is a different shape. So the guard is undefined and the fallback is
unconditional for audio and video. (isImage can fire, but only on the
{format, width} metadata shape, never on {mime_type, image: <base64>}.)
-
The fallback has no length cap. packages/shared-ui/.../trace/JsonTree.tsx
renders string leaves as JSON.stringify(value) with wordBreak: 'break-all'
and defaultExpanded={1}, so the full base64 string lands in the DOM on first
paint. The sibling component at apps/events-ui/src/components/JsonTree.tsx:43
already caps strings at 200 chars — the shared-ui copy is simply missing it.
Scope note: the video and image dump is pre-existing on develop (its
writeVideo/writeImage already emit base64). PR #1525 does not cause it and
touches no UI files. PR #1525 only makes the audio lane worse — it changes
that lane from a tiny {url, aviAction, mimeType, size} dict (harmless in the
fallback) to {mime_type, audio: <base64>, metadata}, which then hits the same
uncapped fallback.
Steps to Reproduce
Reproduces on plain develop — no PR branches, no merge required:
git switch develop && ./builder build vscode- Open the VS Code extension → Project view. Load a video pipeline:
dropper → parse → frame_grabber → detect → video_composer → response_video.
- Drop a video file (e.g.
testdata/misc/BBC - Tear down this wall.mp4).
- Open the Trace panel → final result → the
video section.
- The section shows a
JsonTree with mime_type and a multi-megabyte video
string; the webview stalls.
For the audio regression specifically (requires the #1525 branch): use a
dropper → parse → audio_tts → response_audio pipeline and observe the base64
dump in the audio section; the same pipe on develop renders a tiny 4-field
dict instead.
Expected vs Actual Behavior
Expected: each media lane renders an inline player — <video controls>,
<audio controls>, or <img> — sourced from the payload, with the descriptor
metadata shown alongside. At an absolute minimum, no unbounded string is ever
written to the DOM.
Actual: the raw base64 payload is dumped into a single DOM text node,
freezing the webview for any real media file.
Severity
P1 - High (major feature broken, no workaround)
Platform
(The code path is platform-independent; all platforms are affected.)
Version / Environment
Branch develop; VS Code extension webview (shared-ui trace renderers).
Also affects the dropper UI: apps/dropper-ui/src/utils/dropperUtils.ts:202
has no audio/video case, so those lanes hit the default branch and only
console.warn.
Proposed fix (for reference)
Description
The Trace panel in the VS Code extension freezes when a pipeline returns audio,
video, or image media. The final-result renderer cannot recognise the media
result shapes, so it falls through to the raw
JsonTreefallback — which printsthe entire base64 payload (a multi-megabyte string for any real video) into a
single DOM text node with no length limit.
Two independent defects combine to cause this:
The type guards never match the result shape.
render_final.tsxwraps theaudio/video lanes as
{audio: v}/{video: v}, then callsisAudio/isVideoon that envelope. But those guards look for
.action/.formatat the toplevel — they are the AVI stream-trace guards, reused for the final result,
which is a different shape. So the guard is
undefinedand the fallback isunconditional for audio and video. (
isImagecan fire, but only on the{format, width}metadata shape, never on{mime_type, image: <base64>}.)The fallback has no length cap.
packages/shared-ui/.../trace/JsonTree.tsxrenders string leaves as
JSON.stringify(value)withwordBreak: 'break-all'and
defaultExpanded={1}, so the full base64 string lands in the DOM on firstpaint. The sibling component at
apps/events-ui/src/components/JsonTree.tsx:43already caps strings at 200 chars — the shared-ui copy is simply missing it.
Scope note: the video and image dump is pre-existing on
develop(itswriteVideo/writeImagealready emit base64). PR #1525 does not cause it andtouches no UI files. PR #1525 only makes the audio lane worse — it changes
that lane from a tiny
{url, aviAction, mimeType, size}dict (harmless in thefallback) to
{mime_type, audio: <base64>, metadata}, which then hits the sameuncapped fallback.
Steps to Reproduce
Reproduces on plain
develop— no PR branches, no merge required:git switch develop && ./builder build vscodedropper → parse → frame_grabber → detect → video_composer → response_video.testdata/misc/BBC - Tear down this wall.mp4).videosection.JsonTreewithmime_typeand a multi-megabytevideostring; the webview stalls.
For the audio regression specifically (requires the #1525 branch): use a
dropper → parse → audio_tts → response_audiopipeline and observe the base64dump in the
audiosection; the same pipe ondeveloprenders a tiny 4-fielddict instead.
Expected vs Actual Behavior
Expected: each media lane renders an inline player —
<video controls>,<audio controls>, or<img>— sourced from the payload, with the descriptormetadatashown alongside. At an absolute minimum, no unbounded string is everwritten to the DOM.
Actual: the raw base64 payload is dumped into a single DOM text node,
freezing the webview for any real media file.
Severity
P1 - High (major feature broken, no workaround)
Platform
(The code path is platform-independent; all platforms are affected.)
Version / Environment
Branch
develop; VS Code extension webview (shared-ui trace renderers).Also affects the dropper UI:
apps/dropper-ui/src/utils/dropperUtils.ts:202has no
audio/videocase, so those lanes hit thedefaultbranch and onlyconsole.warn.Proposed fix (for reference)
isImage/isAudio/isVideothe result shapes{mime_type, <lane>, metadata?}and
{mime_type, path}(the latter for forward-compat with PR feat(media): stream produced audio and video as they are produced #1374), and fix theaudio/video
wrapinrender_final.tsxto the identity wrap (asimagealready uses).<img>/<audio>/<video>from adata:URL (or thepath).audio/videocases todropperUtils.ts.JsonTree, matching the events-ui copy, as ageneric safety net.
dropperUtils.tschange overlaps with PR feat(media): stream produced audio and video as they are produced #1374 (closed, may return); whoeverrevives feat(media): stream produced audio and video as they are produced #1374 resolves the conflict.