Skip to content

observe(): no event carries partial tool-call arguments (pi-ai toolcall_delta is dropped) #450

Description

@dknecht

Summary

The runtime's conversation stream store forwards pi-ai streaming events to
Flue events for assistant text and thinking, but silently drops
toolcall_delta. There is no Flue event type that carries partial tool-call
arguments, and session.task() has no per-call streaming callbacks, so an
application cannot observe a tool call's argument text while the model is
still emitting it — only the completed arguments at toolcall_end.

Where the drop happens

In conversation-stream-store (dist/conversation-stream-store-*.mjs,
case "message_update"):

  • text_start / text_delta / text_end → canonical assistant_text_*
    records + emitted text_delta events
  • thinking_start / thinking_delta / thinking_end → canonical
    assistant_reasoning_* + emitted thinking_* events
  • toolcall_end → canonical assistant_tool_call (complete arguments)
  • toolcall_start / toolcall_delta → no branch; dropped

The deltas demonstrably exist upstream of the drop: the runtime's own
OpenAI-compat streaming adapter produces them
(stream.push({ type: "toolcall_delta", contentIndex, delta, partial }) in
dist/cloudflare/internal.mjs).

Use case

An application can submit its primary artifact as tool-call arguments to a
schema-enforced submit tool. That boundary is deliberate: moving the artifact
body into assistant text would double output-token cost and weaken the
schema-enforced submit contract. To stream a live preview while preserving
the submit-tool boundary, the application needs the partial argument text of
that tool call as it is generated: it can tap the argument stream in-process,
incrementally extract one JSON string field, and push coalesced deltas to its
own UI channel.

Requested behavior

Forward toolcall_delta as a Flue observation event delivered to
observe() subscribers, carrying at least:

{
  type: "toolcall_delta",
  toolCallId: string,
  toolName: string,
  argumentTextDelta: string, // raw partial-JSON argument text chunk
}

Constraints that keep the change cheap:

  • In-process only. Deliver to observe() subscribers like the existing
    text_delta/thinking_* events; explicitly exclude the event from
    buffered run streams and persisted stores. No storage or replay
    obligations — per-token persistence is exactly what we need to avoid.
  • No change to toolcall_end / canonical assistant_tool_call semantics;
    complete arguments remain the source of truth.
  • toolcall_start forwarding would be nice for symmetry (it carries the
    tool name before the first delta) but is not required if
    toolcall_delta includes toolName.

Alternatives considered

  • Streaming the artifact as assistant text before the submit tool call: pays
    for the body twice in output tokens and weakens the schema-enforced
    submit boundary.
  • Polling materialized views faster: never exposes partial arguments at any
    poll rate; multiplies read load.
  • Patching the runtime locally: feasible as a pinned patch, but this seems
    useful as a runtime capability.

Metadata

Metadata

Assignees

No one assigned

    Labels

    fix pendingFixed on a development branch; lands with the next release

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions