[Feature Request] WebSocket event to replace system prompt mid-conversation

[thisames]

Problem

The current contextual_update WebSocket event appends context to the conversation state.
There is no way to replace the system prompt mid-conversation via WebSocket.

Use case

In voice agent applications with multi-step flows (e.g., debt negotiation), the agent transitions
through distinct phases — each requiring a completely different system prompt with different
instructions, constraints, and persona behavior. The previous prompt must be fully replaced,
not augmented, because:

1. Conflicting instructions — the old prompt's rules may contradict the new phase's behavior
2. Token waste — accumulated contextual_update injections bloat the context window with
   outdated instructions
3. Non-deterministic behavior — when old and new prompts coexist, the LLM must resolve
   conflicts on its own, leading to inconsistent responses

Current workaround

We route the agent through a custom LLM proxy (customLlm) and intercept /v1/chat/completions
to swap the system message before it reaches the LLM. This works but:

• Adds latency (extra network hop)
• Requires maintaining proxy infrastructure
• Doesn't work with native ElevenLabs LLMs (only custom-llm mode)
• contextual_update is still sent as a fallback but doesn't truly replace the prompt

Proposed solution

A new client-to-server WebSocket message type (e.g., prompt_override) that replaces the
active system prompt:

{
  "type": "prompt_override",                                                                     
  "text": "You are now in phase 2. Your new instructions are..."                               
}

Expected behavior

- The previous system prompt is discarded
- The new text becomes the sole system prompt for all subsequent LLM calls
- Works with both native ElevenLabs LLMs and custom-llm mode                                     
- Does not interrupt the ongoing conversation or TTS output
                                                                                                 
Alternative: replace mode on contextual_update                                                   
 
If adding a new message type is not desirable, a mode flag on the existing contextual_update     
would also solve this:                                                                         

{
  "type": "contextual_update",
  "text": "New full system prompt...",                                                           
  "mode": "replace"
}                                                                                                
                                                                                               
Where mode defaults to "append" (current behavior) for backwards compatibility.                  

[kraenhansen]

Thanks for the detailed write-up! The use case for fully replacing (rather than appending to) the system prompt mid-conversation makes a lot of sense, especially for multi-phase agent flows.

This would require a change to the Conversational AI WebSocket API itself, so we're flagging it as upstream. We'll keep this open to track interest: if others have similar needs, please upvote with a 👍 reaction to help us prioritize.

[kraenhansen]

Following up on this: the long-term solution we're considering is dynamic variables. You would template your system prompt with a variable like {{phase_instructions}}, and update its value mid-conversation. Since the variable is re-evaluated on each LLM call, updating it would effectively replace that part of the prompt rather than appending to it.

However, updating dynamic variables mid-conversation is not supported yet. Introducing a new client event for that is the desired direction. We'll keep this issue open to track demand: please upvote with a 👍 if you need this.

[thisames]

Thanks for the follow-up! The dynamic variables direction makes sense to us. 👍 Will keep an eye on the issue for updates