Skip to content

fix(graph): prefer node tools before synthetic handoff routing - #90

Merged
andrewklatzke merged 1 commit into
mainfrom
aklatzke/AIC-3365/tighten-graph-handoffs
Sep 16, 2026
Merged

fix(graph): prefer node tools before synthetic handoff routing#90
andrewklatzke merged 1 commit into
mainfrom
aklatzke/AIC-3365/tighten-graph-handoffs

Conversation

@andrewklatzke

@andrewklatzke andrewklatzke commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

When a graph node has multiple outgoing edges, route() augments the node's config with synthetic __handoff_* tools. In practice this crowded out the node's own tools — a node that reliably called its real tool standalone would call only a handoff tool inside a graph, so the response carried routing information instead of the tool's data.

Three things in the routing augmentation pushed the model that way, all fixed here:

  • The appended routing directive led with "Select exactly one transfer tool to route to the next agent", making the transfer read as the task and the node's real work optional. It now asks the model to complete its task with its available tools first, and only then call exactly one transfer tool.
  • Handoff tool descriptions fell back to the target node's instructions verbatim, so __handoff_<target> advertised itself as the tool that does the target's work. Descriptions now always lead with Transfer control to <key>., with any handoff.description or target-instructions text appended as trailing detail.
  • The handoff handler returned Transferring to <key>, which reads as though control had already left, so the model wrapped up instead of continuing. Selecting an edge only records the choice — execution continues until the model produces its final text — so the result now says the handoff is recorded and asks the model to finish its own work.

Behavior is unchanged for nodes with zero or one outgoing edge, which never enter this path. Framework-native runners build their own transfer_to_* tools and are untouched.

Test plan

  • Existing graph suite passes (27 tests, packages/client/tests/test_graph.py)
  • Verified against a real multi-node graph whose root node was skipping its tool: the node's own tool now fires and the handoff still routes correctly
  • Reviewer sanity check on the routing copy — this is model-steering, so wording matters

Note: the matching change for the JS SDK is launchdarkly/js-ai-sdk#59

@andrewklatzke @cursoragent
fix(graph): prefer node tools before synthetic handoff routing
119b021 
Handoff tools were crowding out real node tools because routing copy led with "select a transfer tool", descriptions reused the target agent's instructions, and the tool result implied the turn was already over.

Co-authored-by: Cursor <cursoragent@cursor.com>
@andrewklatzke andrewklatzke mentioned this pull request Sep 16, 2026
Merged
2 of 3 tasks
@andrewklatzke
andrewklatzke merged commit f830b2c into main Sep 16, 2026
8 checks passed
@andrewklatzke
andrewklatzke deleted the aklatzke/AIC-3365/tighten-graph-handoffs branch September 16, 2026 17:44
@github-actions github-actions Bot mentioned this pull request Sep 16, 2026
Merged
andrewklatzke added a commit to launchdarkly/js-ai-sdk that referenced this pull request Sep 16, 2026
@andrewklatzke @cursoragent
fix(graph): prefer node tools before synthetic handoff routing (#59)
0578819 
## Summary

When a graph node has multiple outgoing edges, `route()` augments the
node's config with synthetic `__handoff_*` tools. In practice this
crowded out the node's own tools — a node that reliably called its real
tool standalone would call only a handoff tool inside a graph, so the
response carried routing information instead of the tool's data.

Three things in the routing augmentation pushed the model that way, all
fixed here:

- The appended routing directive led with "Select exactly one transfer
tool to route to the next agent", making the transfer read as the task
and the node's real work optional. It now asks the model to complete its
task with its available tools first, and only then call exactly one
transfer tool.
- Handoff tool descriptions fell back to the *target* node's
instructions verbatim, so `__handoff_<target>` advertised itself as the
tool that does the target's work. Descriptions now always lead with
`Transfer control to <key>.`, with any `handoff.description` or
target-instructions text appended as trailing detail.
- The handoff handler returned `Transferring to <key>`, which reads as
though control had already left, so the model wrapped up instead of
continuing. Selecting an edge only records the choice — execution
continues until the model produces its final text — so the result now
says the handoff is recorded and asks the model to finish its own work.

Behavior is unchanged for nodes with zero or one outgoing edge, which
never enter this path. Framework-native runners build their own
`transfer_to_*` tools and are untouched.

## Test plan

- [x] Existing client suite passes (30 tests, includes
`src/__tests__/graph.test.ts`)
- [x] Verified against a real multi-node graph whose root node was
skipping its tool: the node's own tool now fires and the handoff still
routes correctly
- [ ] Reviewer sanity check on the routing copy — this is
model-steering, so wording matters

Note: the matching change for the Python SDK is
launchdarkly/python-ai-sdk#90

Made with [Cursor](https://cursor.com)

Co-authored-by: Cursor <cursoragent@cursor.com>
andrewklatzke pushed a commit that referenced this pull request Sep 22, 2026
@github-actions
chore: release main (#91)
520b7e3 
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-ai-server: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-server-0.2.2...launchdarkly-ai-server-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))
* **client:** stamp modelKey and modelVersion from _ldMeta on tracking
events
([ef103d3](ef103d3))
* **client:** stamp modelKey and modelVersion from _ldMeta on tracking
events ([#97](#97))
([4e6998a](4e6998a))
* **evaluations:** add LD judge event support
([#63](#63))
([ce61644](ce61644))


### Bug Fixes

* **client:** harden model stamps and keep judge results from inheriting
parent model identity
([5d1e678](5d1e678))
* **client:** only stamp a non-empty string modelKey
([8424452](8424452))
* **evaluations:** dedup criteria case-insensitively, matching the API
([4126d7b](4126d7b))
* **evaluations:** offline judge message_history must carry
FORMATTING_INSTRUCTIONS
([0b81f4e](0b81f4e))
* **evaluations:** prefer an exact-provider judge handler over a
wildcard
([c61b3a5](c61b3a5))
* **evaluations:** route judge configs to a compatible handler
([1a5e099](1a5e099))
* extract LangChain content-block text and apply model parameters after
eval ([#80](#80))
([a4e1aad](a4e1aad))
* **graph:** prefer node tools before synthetic handoff routing
([#90](#90))
([f830b2c](f830b2c))


### Documentation

* **evaluations:** document criteria, judge handlers, and scoring policy
([5f5a626](5f5a626))
* **evaluations:** use a customer-style judge key in examples
([ceca049](ceca049))
* remove the internal staging host from a public repo
([00b8dba](00b8dba))
* remove the internal staging host from a public repo
([#96](#96))
([1ad638d](1ad638d))
* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-python: 0.1.7</summary>

##
[0.1.7](launchdarkly-ai-python-0.1.6...launchdarkly-ai-python-0.1.7)
(2026-09-21)


### Features

* **evaluations:** add LD judge event support
([#63](#63))
([ce61644](ce61644))


### Documentation

* **evaluations:** document criteria, judge handlers, and scoring policy
([5f5a626](5f5a626))
* **evaluations:** use a customer-style judge key in examples
([ceca049](ceca049))
* remove the internal staging host from a public repo
([00b8dba](00b8dba))
* remove the internal staging host from a public repo
([#96](#96))
([1ad638d](1ad638d))
* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-claude-agents: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-claude-agents-0.2.2...launchdarkly-ai-claude-agents-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))


### Documentation

* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-claude-messages: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-claude-messages-0.2.2...launchdarkly-ai-claude-messages-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))


### Documentation

* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-openai-agents: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-openai-agents-0.2.2...launchdarkly-ai-openai-agents-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))


### Documentation

* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-openai-messages: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-openai-messages-0.2.2...launchdarkly-ai-openai-messages-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))


### Documentation

* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-langchain-agents: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-langchain-agents-0.2.2...launchdarkly-ai-langchain-agents-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))


### Bug Fixes

* **AIC-3382:** support Bedrock configs in LangChain handlers
([#101](#101))
([04a67c7](04a67c7))
* apply Claude thinking parameters from the evaluated LangChain config
([#99](#99))
([8be2318](8be2318))
* extract LangChain content-block text and apply model parameters after
eval ([#80](#80))
([a4e1aad](a4e1aad))


### Documentation

* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

<details><summary>launchdarkly-ai-langchain-messages: 0.2.3</summary>

##
[0.2.3](launchdarkly-ai-langchain-messages-0.2.2...launchdarkly-ai-langchain-messages-0.2.3)
(2026-09-21)


### Features

* **AIC-3106:** add multimodal history support to graph().invoke()
([#26](#26))
([c91132d](c91132d))


### Bug Fixes

* **AIC-3382:** support Bedrock configs in LangChain handlers
([#101](#101))
([04a67c7](04a67c7))
* apply Claude thinking parameters from the evaluated LangChain config
([#99](#99))
([8be2318](8be2318))
* extract LangChain content-block text and apply model parameters after
eval ([#80](#80))
([a4e1aad](a4e1aad))


### Documentation

* replace stale launchdarkly-ai package name with launchdarkly-ai-python
([#40](#40))
([3256db7](3256db7))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> **Release Please** cut that bumps all Python AI SDK packages and
records their release notes—no runtime code changes in this diff.
> 
> **`launchdarkly-ai-server` 0.2.3** and **`launchdarkly-ai-python`
0.1.7** ship the accumulated work since the last release: **multimodal
conversation history** in `graph().invoke()`, **LaunchDarkly judge
evaluation events**, and **model identity** (`modelKey` / `modelVersion`
from `_ldMeta`) on tracking events, plus fixes for judge handler
routing, criteria dedup, offline judge formatting, graph tool vs handoff
ordering, and LangChain content-block / post-eval parameters.
> 
> Integration packages (**Claude/OpenAI/LangChain** agents and messages
at **0.2.3**) pick up the multimodal `graph().invoke()` behavior via the
core client; **LangChain** packages additionally note **Bedrock config**
support and **Claude thinking** parameters from evaluated configs.
Versions are updated in `.release-please-manifest.json`, each package’s
`pyproject.toml`, `__version__`, and `CHANGELOG.md`.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
3a40793. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@XieX XieX mentioned this pull request Oct 27, 2026
Merged
XieX added a commit that referenced this pull request Oct 28, 2026
@XieX @claude
fix(AIC-3601): stop graph routing text inviting a router to answer fo…
27b8d09 
…r its target (#230)

## Summary

The text for graph nodes with more than one outgoing edge told every
such node to "Finish your own work and provide your final response". It
also said to call a transfer tool "Only once you have your final
answer". A pure router has no answer of its own, so this wording invites
it to write the target agent's answer. In `stream()` that reply reaches
the caller as `chunk` events, and it reaches the target as the
`[Previous agent response]` block. In the Whodunit test app the router
answered in another agent's voice. The wording is a likely cause, not a
proven one.

Both strings in `build_handoff_routing` now match
launchdarkly/ai-sdks-monorepo#104 (`TESTING.md` §3.15a) byte for byte.
TypeScript and Go get the same change in separate PRs.

- **Routing suffix** (appended to the node's instructions):
`\n\nComplete your task using your available tools first. Only once your
own part is done, call exactly one transfer tool to route to the next
agent. Do not answer the parts of the request that belong to the agent
you route to.`
- **Handoff tool result**: `Handoff to <target> recorded. <target> takes
over when your turn ends. Finish your own part, then end your turn. Do
not answer for <target>.` (`<target>` is the raw `target_key`, not the
sanitized tool name.)

The new text keeps `recorded` and `using your available tools first`.
Those words are why a node with a real tool still calls it before
routing (#90). The comment above the handoff function now gives both
constraints. The text does not branch on whether a node is a router,
because the graph schema has no router role. No other routing behaviour
changes.

## Tests

`TestGraphStreamMultiEdge::test_routing_text_matches_the_spec` checks
both strings exactly through `graph().invoke()` and `graph().stream()`.
It covers a multi-edge root that has its own tool and one that has none.
The assertions read the config and tool handlers the handler actually
received: the instructions, the result each `__handoff_*` handler
returns, and the set of tools. With the old strings, all four cases
fail.

Client suite: 1907 passed, 1 skipped. `ruff check`, `ruff format
--check`, `mypy packages/*/src` and the API report check are all clean.


`packages/langchain-agents/tests/test_handler.py::test_handoff_handler_records_destination`
keeps its own copy of the old result text. It is a local fixture that
only checks passthrough, so I left it alone here. Changing it would also
bump that package.

## Live checks not run

The spec asks for two live checks on one model before a release ships
this. **Neither has been run:**

1. In a two-edge graph whose root has no tools, the root does not answer
for the agent it picks.
2. In a two-edge graph whose root has a real tool, the root still calls
that tool before routing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Updates **multi-edge graph routing prompts** in
`build_handoff_routing` so routers are not told to deliver a “final
answer/response,” which was encouraging pure routing nodes to speak for
the next agent (visible in stream chunks and in the `[Previous agent
response]` handoff block).
> 
> The **instruction suffix** now tells the model to finish its own part,
call one transfer tool, and **not answer** portions of the request owned
by the routed agent. The **`__handoff_*` tool return string** is
rewritten to say the target takes over when the turn ends and explicitly
**not to answer for** that target. Both strings match **TESTING.md
§3.15a** byte-for-byte across SDKs; the inline comment documents why
`recorded` and the no–final-response wording stay.
> 
> Adds **`test_routing_text_matches_the_spec`**, asserting exact copy
through **`graph().invoke()`** and **`graph().stream()`** for roots with
and without their own tools.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
deebf59. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@apucacao apucacao apucacao approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@andrewklatzke @apucacao