Repository navigation
fix(AIC-3570)!: re-resolve graph flags on every invoke and stream - #11
Merged
Merged
Conversation
Graph() cached the resolved definition per context for the life of the AgentGraph, so flag changes and the graph kill switch never reached a reused instance. Removes the cache and the public MaxGraphCacheSize constant. BREAKING CHANGE: MaxGraphCacheSize is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 20, 2026
andrewklatzke
approved these changes
Oct 20, 2026
jeffdupont
added a commit
to launchdarkly/python-ai-sdk
that referenced
this pull request
Oct 21, 2026
Fixes [AIC-3570](https://launchdarkly.atlassian.net/browse/AIC-3570). ## Why `GraphInstance` kept the built graph in `self._cache`, keyed by the JSON of the context, for the life of the instance. An app that reuses one `graph()` never saw flag changes again: new node instructions, a new model, or turning the graph off (the kill switch) did not reach it. A transient variation failure was cached the same way, so the graph stayed disabled for that context. Flag evaluation is local and cheap, so there is nothing worth caching. The spec rule is updated to match in the monorepo PR (TESTING.md §3.11). The other half of the ticket, one `runId` per run, was already fixed on main (`invoke` and `stream` each mint a fresh `runId`). Those tests still pass. ## Fix - `packages/client/src/launchdarkly_ai_server/graph.py`: `invoke` and `stream` call `_build_graph(...)` on every call. Removed `self._cache`, `MAX_GRAPH_CACHE_SIZE` and the unused `json` import. Updated the `_RUN_TRACK_DATA` comment, which described the cache. - `MAX_GRAPH_CACHE_SIZE` was not exported from `launchdarkly_ai_server` (not in `__init__.py` or `test_public_surface.py`). It could still be imported from the `launchdarkly_ai_server.graph` module, so code that did that will break. Pre-1.0 removal is approved. ## Testing - `packages/client/tests/test_graph.py`: `test_node_variations_resolved_only_once` encoded the bug, so it is now `test_node_variations_resolved_on_every_call` (the variation call count doubles across two calls). Deleted `test_cache_is_bounded` and `test_equal_content_contexts_share_cache_entry`, since there is no cache any more. Added `test_flag_change_reaches_reused_graph`: instructions changed from v1 to v2 reach the second call, then disabling the graph makes `invoke` and `stream` raise. Added `test_failed_resolution_is_not_reused`. `test_per_call_variables_isolated` is unchanged. - With the fix reverted, the 3 new or changed tests fail. With the fix, all 33 tests in test_graph.py pass. - `make test`: 2544 passed, 14 skipped. `make typecheck`: no issues in 75 files. `make lint`: all checks passed. `make format-check`: clean. Related: launchdarkly/js-ai-sdk#145, launchdarkly/go-ai-sdk#11, launchdarkly/ai-sdks-monorepo#71 🤖 Generated with [Claude Code](https://claude.com/claude-code) [AIC-3570]: https://launchdarkly.atlassian.net/browse/AIC-3570?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
added a commit
to launchdarkly/js-ai-sdk
that referenced
this pull request
Oct 21, 2026
Fixes [AIC-3570](https://launchdarkly.atlassian.net/browse/AIC-3570). ## Why `graph()` kept the built graph (topology plus every node's config) in a per-context `WeakMap` for the life of the returned caller. An app that creates one `graph()` and reuses it for a user never saw flag changes again: new instructions, a new model, or turning the graph off (the kill switch) did not reach that caller. A transient variation failure was cached the same way, so the graph stayed disabled for that context. Flag evaluation is local and cheap, so there is nothing worth caching. The spec rule is updated to match in the monorepo PR (TESTING.md §3.11). The other half of the ticket, one `runId` per run, was already fixed on main (`graph.ts` mints a fresh `runId` per stream call). Those tests still pass. ## Fix - `packages/client/src/graph.ts`: removed `nodeCache` and `resolveBuilt` from `graphInternal`. `streamEvents` calls `buildGraph` directly on every call. - `invoke()` no longer resolves the graph itself before draining `stream()`, so each call builds the graph once, not twice. `stream()` still throws `Agent graph "<key>" is disabled`, and `invoke()` passes that error up. - Callers of `graphInternal` (`graph()`, claude-agents, langchain-agents, vercel-agents, openai-agents) did not depend on the cache. ## Testing - `packages/client/src/__tests__/graph.test.ts`: the test "resolves node variations only once per graph() instance" encoded the bug, so it now asserts the opposite: the variation call count doubles across two calls. Added: a node flag changed from v1 to v2 between calls reaches the second call; disabling the graph flag makes the next `invoke()` and `stream()` throw; a failed variation on call 1 does not carry over, and call 2 succeeds. The per-call variables isolation test is unchanged. - With the fix reverted, the 4 new or changed tests fail. With the fix, all 66 tests in graph.test.ts pass. - `yarn build`, `yarn typecheck`, `yarn code:check`, `yarn workspace @launchdarkly/ai-server api:check` all pass. `yarn test` passes in every workspace (client: 1404 passed, 11 skipped). Related: launchdarkly/python-ai-sdk#187, launchdarkly/go-ai-sdk#11, launchdarkly/ai-sdks-monorepo#71 🤖 Generated with [Claude Code](https://claude.com/claude-code) [AIC-3570]: https://launchdarkly.atlassian.net/browse/AIC-3570?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
pushed a commit
that referenced
this pull request
Nov 2, 2026
🤖 I have created a release *beep* *boop* --- ## 0.1.0 (2026-10-30) ### ⚠ BREAKING CHANGES * **skills:** rename FDv2SkillStore to LaunchDarklySkillStore ([#15](#15)) * **AIC-3570:** re-resolve graph flags on every invoke and stream ([#11](#11)) * **AIC-3614:** hide SDK-only entry points in internal/sdkapi ([#8](#8)) * **AIC-3612:** remove the SDK-owned global registry ([#6](#6)) ### Features * **AIC-3569:** add the Go AI SDK ([#1](#1)) ([b92042b](b92042b)) * **skills:** rename FDv2SkillStore to LaunchDarklySkillStore ([#15](#15)) ([0b87819](0b87819)) ### Bug Fixes * **AIC-3570:** re-resolve graph flags on every invoke and stream ([#11](#11)) ([ada5a25](ada5a25)) * **AIC-3571:** visit every child before its parent in ReverseTraverse ([#12](#12)) ([43f86d0](43f86d0)) * **AIC-3576:** record deferred judge scores under the evaluated config ([#13](#13)) ([cd71325](cd71325)) * **AIC-3601:** stop multi-edge routing text from inviting a final answer ([#17](#17)) ([e85260c](e85260c)) * **AIC-3612:** remove the SDK-owned global registry ([#6](#6)) ([6152dc9](6152dc9)) * **AIC-3614:** hide SDK-only entry points in internal/sdkapi ([#8](#8)) ([acca582](acca582)) * **AIC-3726:** realign three usage IDs with the 1.0 factory names ([#23](#23)) ([805ece7](805ece7)) * Register every package that reports an AI SDK identity ([#10](#10)) ([cba37e5](cba37e5)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > This PR cuts the **0.1.0** release via Release Please by bumping `.release-please-manifest.json` from `0.0.0` to `0.1.0` and adding a new `CHANGELOG.md` for **2026-10-30**. > > The changelog records the first public release of the Go AI SDK and calls out **breaking changes** integrators should plan for: removing the SDK-owned global registry, hiding `internal/sdkapi` entry points, re-resolving graph flags on every invoke/stream, and renaming `FDv2SkillStore` to `LaunchDarklySkillStore`. It also lists follow-on fixes (graph traversal, judge scoring, routing text, usage ID alignment, and AI SDK identity registration) that ship in this tag. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 0751fcf. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes AIC-3570.
Why
AgentGraphkept each enabled resolved definition, keyed by the JSON of the context, for its whole life, with up toMaxGraphCacheSize(512) entries. A service that reuses oneGraph()never saw flag changes again: new node instructions, a new model, or turning the graph off (the kill switch) did not reach it. Flag evaluation is local and cheap, so there is nothing worth caching. The spec rule is updated to match in the monorepo PR (TESTING.md §3.11).The other half of the ticket, one
runIdper run, was already fixed on main (each run sets its ownRunID). Those tests still pass.Fix
aiserver/graph.go:definition()is nowreturn resolveDefinition(ctx, g.key, userCtx, g.opts). Removed themu/cache/orderfields and the test-onlycacheLen. Updated theRunIDcomment, which described the cache.MaxGraphCacheSize. Also removed it fromaiserver/surface_test.goandaiserver/agents.md. Pre-1.0 removal is approved. The monorepo has no Go api-surface list, so nothing else changes.Testing
aiserver/graph_test.go:TestGraphCachingAndIsolationencoded the bug, so it is nowTestGraphResolvesEachCallAndIsolation. It checks that the graph and node flags are evaluated on both calls, and keeps the variables isolation check. The cache-size loop is gone. AddedTestGraphSeesFlagChanges: instructions changed from v1 to v2 reach the second call, then disabling the graph makesInvokeandStreamreturnErrGraphDisabled. The transient-failure case is already covered byTestGraphFailedResolutionIsNotCached, which still passes. Go already skipped caching disabled results.make test(go test -race ./...): all 16 packages ok, 844 tests passed.go vet ./...andmake lintare clean.Related: launchdarkly/js-ai-sdk#145, launchdarkly/python-ai-sdk#187, https://github.com/launchdarkly/ai-sdks-monorepo/pull/71
🤖 Generated with Claude Code