Summary
Issue #6683 was previously closed by inactivity, but the behavior is still present and impacts async pipelines.
In AgentExecutor.ainvoke()/run_async flow, LLM calls are routed through the shared synchronous path (get_llm_response()), even when the provider implements an async path (LLM.acall() / aget_llm_response()).
As a result, async workloads can be constrained by thread-pool capacity and lose the expected async execution benefits.
Current behavior
AgentExecutor methods call_llm_and_parse and call_llm_native_tools call get_llm_response().
get_llm_response() invokes LLM.call().
LLM.call() uses sync execution path, requiring threadpool offload for providers that are actually async-capable.
Expected behavior
- Async execution paths should use
aget_llm_response() and LLM.acall() when available.
- Preserve compatibility for custom LLMs that only implement sync behavior by falling back to
asyncio.to_thread(...) only on NotImplementedError.
Suggested fix
- Convert the two router methods to
async and await aget_llm_response().
- In
aget_llm_response(), implement narrow fallback:
- try async path first
- on
NotImplementedError, delegate to asyncio.to_thread(llm.call, ...) (or equivalent)
- Keep sync behavior unchanged for non-async contexts.
Repro (minimal)
- Create a custom LLM double that records calls for both
call() and acall().
- Attach it to an
Agent and invoke await agent.kickoff_async("probe").
- Validate that
acall() is invoked when available.
If needed, I can provide a tiny regression test draft.
Context
This is a re-open-style report derived from #6683 and current source inspection.
Summary
Issue
#6683was previously closed by inactivity, but the behavior is still present and impacts async pipelines.In
AgentExecutor.ainvoke()/run_asyncflow, LLM calls are routed through the shared synchronous path (get_llm_response()), even when the provider implements an async path (LLM.acall()/aget_llm_response()).As a result, async workloads can be constrained by thread-pool capacity and lose the expected async execution benefits.
Current behavior
AgentExecutormethodscall_llm_and_parseandcall_llm_native_toolscallget_llm_response().get_llm_response()invokesLLM.call().LLM.call()uses sync execution path, requiring threadpool offload for providers that are actually async-capable.Expected behavior
aget_llm_response()andLLM.acall()when available.asyncio.to_thread(...)only onNotImplementedError.Suggested fix
asyncandawait aget_llm_response().aget_llm_response(), implement narrow fallback:NotImplementedError, delegate toasyncio.to_thread(llm.call, ...)(or equivalent)Repro (minimal)
call()andacall().Agentand invokeawait agent.kickoff_async("probe").acall()is invoked when available.If needed, I can provide a tiny regression test draft.
Context
This is a re-open-style report derived from
#6683and current source inspection.