Skip to content

[Reopen] AgentExecutor async path still routes through LLM.call #7339

Description

@suqinghen

Summary

Issue #6683 was previously closed by inactivity, but the behavior is still present and impacts async pipelines.

In AgentExecutor.ainvoke()/run_async flow, LLM calls are routed through the shared synchronous path (get_llm_response()), even when the provider implements an async path (LLM.acall() / aget_llm_response()).

As a result, async workloads can be constrained by thread-pool capacity and lose the expected async execution benefits.

Current behavior

  • AgentExecutor methods call_llm_and_parse and call_llm_native_tools call get_llm_response().
  • get_llm_response() invokes LLM.call().
  • LLM.call() uses sync execution path, requiring threadpool offload for providers that are actually async-capable.

Expected behavior

  • Async execution paths should use aget_llm_response() and LLM.acall() when available.
  • Preserve compatibility for custom LLMs that only implement sync behavior by falling back to asyncio.to_thread(...) only on NotImplementedError.

Suggested fix

  1. Convert the two router methods to async and await aget_llm_response().
  2. In aget_llm_response(), implement narrow fallback:
    • try async path first
    • on NotImplementedError, delegate to asyncio.to_thread(llm.call, ...) (or equivalent)
  3. Keep sync behavior unchanged for non-async contexts.

Repro (minimal)

  1. Create a custom LLM double that records calls for both call() and acall().
  2. Attach it to an Agent and invoke await agent.kickoff_async("probe").
  3. Validate that acall() is invoked when available.

If needed, I can provide a tiny regression test draft.

Context

This is a re-open-style report derived from #6683 and current source inspection.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions