Handoff Orchestration sample - doesn't work with OpenAI?

[Brian Murphy (brainmurphy)]

I've tried to run Step04_Handoff against OpenAI with gpt-4o-2024-08-06. I'm getting this back:

An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_Yd1hIw4CVtwFdpzuxgPrZAIu))))))

All I've done is git clone, set the secrets (OpenAI:ChatModelId and OpenAI:ApiKey), then dotnet test --filter Step04_Handoff.

Should this just work?

The conversation log:

# INPUT:
I am a customer that needs help with my orders

REGISTER HandoffOrchestration Start: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68/default
REGISTER ACTOR HandoffOrchestration MEMBER #4: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_4
REGISTER ACTOR HandoffOrchestration MEMBER #3: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_3
REGISTER ACTOR HandoffOrchestration MEMBER #2: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_2
REGISTER ACTOR HandoffOrchestration MEMBER #1: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1
REGISTER HandoffOrchestration Complete: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68/default
INVOKE HandoffOrchestration: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68/default
YIELD HandoffOrchestration: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68/default
AWAIT HandoffOrchestration: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68/default
INIT HandoffOrchestration: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Boot/default
START HandoffOrchestration: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Boot/default
REQUEST Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]

# RESPONSE assistant - TriageAgent: Could you please provide more details about the specific issue you're facing with your order? This will help me direct you to the appropriate support agent. Are you looking to check the status of an order, process a return, or request a refund?

# STREAMED Assistant - TriageAgent: ("Could")(" you")(" please")(" provide")(" more")(" details")(" about")(" the")(" specific")(" issue")(" you\u0027re")(" facing")(" with")(" your")(" order")("?")(" This")(" will")(" help")(" me")(" direct")(" you")(" to")(" the")(" appropriate")(" support")(" agent")(".")(" Are")(" you")(" looking")(" to")(" check")(" the")(" status")(" of")(" an")(" order")(",")(" process")(" a")(" return")(",")(" or")(" request")(" a")(" refund")("?")

RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]: Could you please provide more details about the specific issue you're facing with your order? This will help me direct you to the appropriate support agent. Are you looking to check the status of an order, process a return, or request a refund?

# INPUT: I'd like to track the status of my order

TOOL Handoff [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]: OrderStatusAgent
RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]: 
REQUEST Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_2/default]

# RESPONSE assistant - OrderStatusAgent: Please provide the order ID for the order you would like to track, and I will check the status for you.

# STREAMED Assistant - OrderStatusAgent: ("Please")(" provide")(" the")(" order")(" ID")(" for")(" the")(" order")(" you")(" would")(" like")(" to")(" track")(",")(" and")(" I")(" will")(" check")(" the")(" status")(" for")(" you")(".")

RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_2/default]: Please provide the order ID for the order you would like to track, and I will check the status for you.

# INPUT: My order ID is 123


# RESPONSE assistant - OrderStatusAgent: Your order (ID: 123) has been shipped and is expected to arrive in 2-3 days. If you have any more questions or need further assistance, feel free to ask!

# STREAMED Assistant - OrderStatusAgent: ("Your")(" order")(" (")("ID")(":")(" ")("123")(")")(" has")(" been")(" shipped")(" and")(" is")(" expected")(" to")(" arrive")(" in")(" ")("2")("-")("3")(" days")(".")(" If")(" you")(" have")(" any")(" more")(" questions")(" or")(" need")(" further")(" assistance")(",")(" feel")(" free")(" to")(" ask")("!")

RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_2/default]: 

# INPUT: I want to return another order of mine

TOOL Handoff [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_2/default]: TriageAgent
RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_2/default]: 
REQUEST Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]
TOOL Handoff [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]: OrderReturnAgent
RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]: 
REQUEST Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_3/default]
TOOL Handoff [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_3/default]: TriageAgent
RESULT Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_3/default]: 
REQUEST Handoff agent [HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Agent_1/default]
FAILURE HandoffOrchestration: HandoffOrchestration_500b0d48b5bd40569fa7a1a0a4f2de68_Boot/default

[Brian Murphy (brainmurphy)]

This might be #12663 🤔

[Brian Murphy (brainmurphy)]

Running the same sample with Bedrock/Claude, I get:

One or more errors occurred. (One or more exceptions occurred while processing the message. (One or more exceptions occurred while processing the message. (The text field in the ContentBlock object at messages.3.content.0 is blank. Add text to the text field, and try again.)))

I modified BaseTest.AddChatClientToKernel to look like this:

    protected IChatClient AddChatClientToKernel(IKernelBuilder builder)
    {
#pragma warning disable CA2000 // Dispose objects before losing scope
        IChatClient chatClient;

        var bedrock = new AmazonBedrockRuntimeClient();
        chatClient = bedrock.AsIChatClient("eu.anthropic.claude-3-7-sonnet-20250219-v1:0");

        IChatClient functionCallingChatClient = chatClient.AsBuilder().UseKernelFunctionInvocation().Build();
        builder.Services.AddSingleton(functionCallingChatClient);
        return functionCallingChatClient;
#pragma warning restore CA2000 // Dispose objects before losing scope
    }

It looks like the tool back-and-forth results in the last FunctionResultContent becoming transformed into an empty string which gets passed as part of the conversation? Not sure I follow the logic in ChatClientChatCompletionService.GetChatMessageContentsAsync

[Evan Mattson (moonbox3)]

Tao Chen (@TaoChenOSU) any thoughts here?

[Lucas Kim (kimnamu)]

Thanks for posting the full conversation log and for trying the Bedrock variant too; having both errors side by side made this quick to trace.

On the latest packages (1.80.1, whose HandoffActor.cs is identical to main ca40aa7), the blank-text error is gone, but the non-streaming Handoff run still fails at the first handoff: the triage agent's transfer_to_OrderStatusAgent tool call is forwarded to the next agent without its tool result. The streaming path (setting StreamingResponseCallback, as the sample's streamedResponse: true case does) finished in my runs. I only ran the Bedrock variant, so I can't confirm the OpenAI side, although its error describes the same shape.

Packages	streamedResponse: false	streamedResponse: true
1.60.0 (July 2025)	❌ blank text block at messages.3	❌ blank text block at messages.3
1.80.1 (latest)	❌ tool_use without tool_result at messages.3 (3/3 runs)	✅ finished with the summary (2/2 runs)

Setup: Step04_Handoff with your AsIChatClient(...).AsBuilder().UseKernelFunctionInvocation() client, Claude Haiku 4.5 (us.anthropic.claude-haiku-4-5-20251001-v1:0, us-east-1), plus a pass-through .Use(...) that only prints what each agent sends:

SemanticKernel 1.80.1 | Agents.Orchestration 1.80.1-preview | model us.anthropic.claude-haiku-4-5-20251001-v1:0

=== streamedResponse=False
# INPUT: I am a customer that needs help with my orders
# RESPONSE TriageAgent: I'd be happy to help you with your orders! To better assist you, could you please tell me ...
# INPUT: I'd like to track the status of my order
  [OrderStatusAgent request] messages.3 role=assistant has tool call HandoffPlugin_transfer_to_OrderStatusAgent with no tool result after it
FAILED: ValidationException: The model returned the following errors: messages.3: `tool_use` ids were found without `tool_result` blocks immediately after: tooluse_UJ50PseLDazUFAvHCcCkZU. Each `tool_use` block must have a corresponding `tool_result` block in the next message.

Where that message comes from

HandoffActor copies each agent's final response into the other agents' caches (HandoffMessages.Response, HandoffActor.cs:101-110). Before publishing it only checks the role (HandoffActor.cs:124-130 on main):

// The response can potentially be a TOOL message from the Handoff plugin due to the filter
// which will terminate the conversation when a function from the handoff plugin is called.
// Since we don't want to publish that message, so we only publish if the response is an ASSISTANT message.
if (response.Role == AuthorRole.Assistant)

In the non-streaming run the handoff turn comes back as an assistant message holding the transfer_to_OrderStatusAgent call, so it passes this check and becomes messages.3 of the next agent's first request, with no tool result after it.

On 1.60.0 the same slot held an assistant message with blank text, which matches the messages.3.content.0 is blank error in your comment:

=== streamedResponse=False
# INPUT: I am a customer that needs help with my orders
# RESPONSE TriageAgent: Hello! I'm here to help you with your orders. To better assist you, could you please tell ...
# INPUT: I'd like to track the status of my order
  [OrderStatusAgent request] messages.3 role=assistant has a blank text block
FAILED: ValidationException: The model returned the following errors: messages: text content blocks must be non-empty

[Ícaro Galvão do Nascimento (Icaro0310)]

The garbage at the end of that call id ())))))) is the tell. SK's handoff orchestration wrote an assistant message with tool_calls into the history but the matching tool result never made it in, and the id looks like it was rebuilt from the call text instead of the id OpenAI returned. OpenAI validates this hard: every tool_call must be followed by a role=tool message.

This was a real bug in the Agents.Orchestration preview packages (the internal handoff transfer ran, but the FunctionResultContent wasn't appended before the next model call). It's fixed on recent versions, so first make sure Microsoft.SemanticKernel and the Agents.* packages are all on the same current version. Mixed versions reproduce exactly this failure.

If you're already current, enable Trace logging for Microsoft.SemanticKernel and inspect the outgoing request. The missing tool message will be right there, and that's worth filing.

—
Check out my Repositories: github.com/Icaro0310 · Website: icaro0310.github.io