Skip to content

chat: voice mode speaks the raw partial envelope when a reply is cut short #709

Description

@Aristocles

What

In voice mode the model is instructed to answer with a JSON envelope,
{"speak": "...", "display": "..."}, and extractJsonReply in server.js
unwraps it. When a generation is cut short (finish_reason: "length", or any
other early finish; see #695) the envelope arrives unterminated:

{"speak":"You have three cards and the sleep one is

extractJsonReply cannot parse that (the direct parse fails and the brace walker
finds no balanced candidate), so it returns null and the fallback branch runs:

const speak = finalText.replace(/\p{Extended_Pictographic}/gu, '').trim();
return withFollowup({ reply: finalText || EMPTY_REPLY_MESSAGE, speak, ... });

So the raw JSON fragment becomes both the displayed reply and the spoken text.
The user hears the punctuation and the key names read aloud, and #695's
explanatory suffix is appended inside the broken envelope where it reads as part
of the fragment.

Why

Voice is the mode with the least recovery affordance: there is no visible text to
squint at, so an unparseable envelope is pure noise. The fallback exists for a
model that ignored the format instruction and answered in prose, which is a
reasonable thing to speak. A truncated envelope is not prose and must not go
through the same door.

Acceptance criteria

  • A voice-mode turn whose generation was cut short and whose text does not
    parse as an envelope speaks a short human sentence explaining the cut,
    not the raw fragment
  • Any prose the fragment did carry is still shown in the transcript (progress
    is not thrown away), and the reply is still flagged capped so "keep going"
    is offered
  • A voice-mode turn that completed cleanly, and one that answered in prose
    instead of the envelope, both behave exactly as they do today
  • Coverage at the API layer: an unterminated envelope with a length finish
    asserts the spoken text contains no brace or key name

Notes

Fall-out from #695, which made an early finish a first-class outcome. The bug
predates it (any truncation hit this), but #695 is what makes the path routine.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:chatChat widget + gateway integration + voicebugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions