Repository navigation
feat(client)!: capture conversation content by default and add configure() - #190
Conversation
…o text A handler that passes an undefined text (a structured-output result with no output, say) made partsToText call .length on undefined and fail the run. That only happened with capture on; capture is about to become the default, so the helper now treats a missing text as empty.
…core
Python already has configure(options=SDKOptions(...)) and captures content
by default; TESTING.md 3.9a names the TypeScript pair configure and
SDKOptions and marks it pending.
configure({ options }) stores SDKOptions on the lifecycle singleton slot.
captureContent defaults to true. It can be replaced until the first AI
invocation, after which only identical options are accepted, and
shutdown() restores the built-in defaults before any await.
createHandler now makes handler.captureContent a getter: an explicit true
or false is an override, an omitted value is unset and reads the configured
default each time, and assigning sets the override. A handler's stream
latches configure when it is called, and the SDK latches before every
handler call it makes (config, graph and evaluations).
The judge explanation and the evaluation spans the runner writes read the
same resolver, so a handler or plain function with no explicit value now
exports the transcript unless configure turns the default off.
Each factory defaulted captureContent to false and froze it at construction, so the configure() default could never reach a factory-built handler. The factories (claude-agents, claude-messages, langchain-agents, langchain-messages, litellm-agents, litellm-messages, openai-agents, openai-messages, typesafe, vercel-agents, vercel-messages) now leave the option unset when it is omitted, pass it to createHandler as an override only when it is given, and read handler.captureContent at the start of each run and stream. A handler built before configure() follows it. The convenience wrappers forward an omitted option as unset. vercelEvaluate, which is not a provider handler, resolves the same gate for its call. Docs that said content was off by default now describe the resolution order. Tests in each package cover: on by default, explicit false, configure turning the default off, a handler built before configure, and explicit true over a configured false.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 62f88f0. Configure here.
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Approving. This matches the spec's §3.9a on monorepo main (built-in default true, precedence, lock, shutdown() reset, initClient ignoring the key), and every handler, wrapper, runJudges, the evaluation spans and vercelEvaluate go through the one resolver.
Nit, fine as a follow-up: the configure lock only latches on SDK paths (executeAndTrack, executeAndStream, stream start, the evaluation runners). A caller that invokes a handler directly (await handler(config, input)) or calls vercelEvaluate makes an AI call without latching, so a later configure() with a different value is accepted instead of throwing. Wrapping the blocking call in createHandlerInternal the same way stream is wrapped (noteAiInvocation() then the real call) would close it. Bugbot flags the same thing; its graph-node case is already covered, since graph.ts goes through executeAndStream.
|
Tracked the |
…aths createHandler now returns a wrapper that latches configure() and then calls the original function with the same this and arguments. A direct `await handler(config, input)` call therefore locks the options the same way executeAndTrack, a stream, or an evaluation run does, which matches the Python SDK, where ProviderHandler.__call__ latches. The wrapper carries the original function's own properties, so wrapping an existing handler again keeps its stream and anything else attached to it. BREAKING CHANGE: createHandler no longer returns the function it was given. Code that compared the result with the original function by identity has to compare with the returned handler instead.
… model vercelEvaluate makes its AI call without a provider handler, so a later configure() with a different captureContent was accepted. It now calls its per-call handler as the evaluation starts, which latches configure() like any other handler call.
…r handler packages vercelEvaluate is not a provider handler, so it built a throwaway handler to read the effective captureContent and called it to latch configure(). The client now exports effectiveCaptureContent and noteAiInvocation as internal plumbing, beside createHandlerInternal and reportUsage, and vercelEvaluate calls them directly.
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Requesting changes on 56811b4. My earlier approval was for 62f88f0, and two things changed after it.
The §3.9a work itself still looks right against the spec on monorepo main (cb50d0c). Every factory and wrapper, both judge paths, the evaluation spans and vercelEvaluate go through the one resolver. The lock latches on direct calls, stream start and the SDK paths, shutdown() resets it before any await, and initClient ignores the key with the generic warning. It also matches python-ai-sdk main on the contract: default true, unset vs explicit, the property setter, the lock, and no env var.
Must fix:
-
createHandler's return contract (utils.ts:76-80, from 42ffeb1). TESTING.md §3.14 ("TypeScript mutates in place and returns the same function reference") and Appendix A.8 still require the old behaviour, and this commit deletes the spec's "returns the same function reference" test. Code that calls
createHandler(p, fn)and keeps usingfnnow routes without providesFor: I ran it at 56811b4, andfn.providesForis undefined. I think the wrapper is the right call, since it closes AIC-3703 and matches Python's ProviderHandler. But the contract freezes at 1.0, so please land a monorepo PR that updates §3.14 and A.8 before this merges. -
Migration note (§3162). The README section "Conversation content on spans" describes the new behaviour, but it doesn't say the default used to be off, and it doesn't mention the createHandler change. Please add one line for each with the way back:
configure({ options: { captureContent: false } }), and "use the handler createHandler returns". The repo squash-merges, so the BREAKING CHANGE footer in 42ffeb1 won't survive. Please put both breaking changes in the squash message.
Should fix:
- Re-wrapping drops an explicit override.
createHandler(q, createHandler(p, fn, undefined, false)).captureContentis true (ran it). The inner spans still honour false, but runJudges reads the outer handler, so a re-tagged judge would write gen_ai.evaluation.explanation. Inheriting the source's explicit value when the argument is undefined would fix it. - AGENTS.md:521 still says createHandler returns the same function reference.
- 42ffeb1 fixes AIC-3703. Please reference it in the PR body so the ticket closes.
- Tests the spec asks for that I couldn't find: explicit false on a convenience wrapper beating a configured true, with omitted forwarded as unset (§3.9a). The deferred runJudge explanation gate. inspectConfig not latching.
- Spec bookkeeping to go with the §3.14 PR: drop "[configure: TypeScript pending]" in §3.9a, make configure / SDKOptions plain names in api-surface/typescript/ai-server.txt, and add the two new "-" internals (effectiveCaptureContent, noteAiInvocation).
Not for this PR: the native graph adapters (toOpenAIAgents and the others) never latch configure in either SDK, and the spec doesn't say whether they should. I'll file that in Jira.
…is wrapped again createHandler without a captureContent argument now inherits the source handler's explicit override, so re-tagging a handler built with false no longer turns content capture back on for runJudges. An unset source stays unset and keeps following configure().
…itted one follows it
…andler return value
Brings in the createHandler options object, the positional-form TypeError and wildcard `providers` from #111. createHandler still returns a new wrapper; `providers` is validated, deduped and frozen, then set on the wrapper, and a re-wrap without `providers` keeps the list copied from the wrapped handler. Tests and docs from this branch now use the options form.
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Thanks, this covers everything from my last review. Checked 56811b4..37aaae6:
- createHandler identity: AGENTS.md, both READMEs and the utils.ts doc comment now say it returns a new function, and the body links the §3.14 / A.8 change in ai-sdks-monorepo#113.
- Migration notes: client README "Migrating from an earlier version" covers both breaking changes, and the body has BREAKING CHANGE footers for both, so a body-based squash keeps them. AIC-3703 is linked.
- Re-wrap override (46c65d9): createHandler(q, createHandler(p, fn, undefined, false)).captureContent is now false, an unset inner handler still follows a later configure(), and an explicit true on the outer one wins. I re-ran my probes on 37aaae6 and they all pass; the client suite passes too (2018).
- New tests: wrapper precedence (claudeAgents, openaiMessages), the deferred runJudge gate, and inspectConfig not latching.
I'm holding approval for one reason: #111 is approved, and it conflicts with this branch in six files, all around createHandler. Since both PRs freeze that signature at 1.0, I'd like to approve the rebased version. Please rebase onto #111 once it merges, and keep:
- #111's options object and its TypeError for the old positional form. The fourth-argument mentions here (client README line 109 and the positional calls in sdk-options.test.ts) move to { captureContent }.
- This PR's wrapper: latching on call and stream start, the copied properties, the run-time accessor and the inherited explicit override.
- #111's promise that a re-wrap keeps earlier providers as well as captureContent. A test next to your captureContent re-wrap tests would pin it.
- Drop #111's "returns the same function reference" test and its "in place" wording, which #113's §3.14 / A.8 now contradict.
Two small spec notes for #113, not this PR. Its new §3.14 sentence about keeping the inner override on re-wrap isn't true of Python main yet: create_handler(("B","messages"), create_handler(("A","messages"), fn, None, False)).capture_content is True. So Python needs a matching change, or the sentence needs a TypeScript scope. Also, the inherited override is taken at wrap time; assigning inner.captureContent afterwards doesn't reach the outer handler. That seems fine, but the spec could say "when it is wrapped".
|
Approved at d000dc1. For the record, I checked the merge of main (which brings in #111) against the four things my last comment asked the rebase to keep:
Separately, for #113: its new §3.14 re-wrap sentence isn't true of Python main yet, so the Python SDK needs the same change before 1.0. |
… ag/capture-content-default # Conflicts: # packages/client/src/tracking.ts
Main added a second captureContentFor in content.ts with a hard-coded false default. Delete it and point tracking.ts and graph.ts at the sdk-options.ts resolver, so the graph root and plain function handlers follow configure() and its built-in default of true, like every other content gate. The graph suite now drives capture with configure() and resetSdkOptions() instead of mocking content.js, and covers the graph root under the built-in default. The executeAndTrack suite covers a plain function handler under the built-in default and under configure().
…er the default A handler not built with createHandler carries no captureContent flag, so it follows the process default. Under the built-in default it writes the raw variation as feature_flag.result.value through config().invoke, and the variation key once configure() turns capture off.
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Re-checked d000dc1..7d7a7fe after the #186 merge.
The duplicate captureContentFor is gone. content.ts's hard-coded-false copy and DEFAULT_CAPTURE_CONTENT were removed, and tracking.ts, graph.ts, judges.ts and evaluations-spans.ts all import the sdk-options.ts one. I probed it through executeAndTrack and executeAndStream on 7d7a7fe:
- explicit false gives the key
- unset with configure(false) gives the key
- explicit true with configure(false) gives the full value
- unset createHandler and a plain function under the default give the full value
- a plain function with configure(false) gives the key
- streaming with explicit false gives the key
Breaking the gate at tracking.ts:253 makes those cases fail, so the probe really tests it. handlerVariables now deletes a caller-supplied __ldValue, which closes the spread I mentioned on #186. The graph-root test now drives capture through configure() instead of mocking content.js. The wrapper, the re-wrap override, the options object and the migration notes are all unchanged, and the client suite passes (2089).
One small docs ask, fine as a follow-up: the "Conversation content on spans" list in packages/client/README.md (line 96) should also say that the AI Config variation itself goes out by default as feature_flag.result.value on the root and graph spans, since that's new since #186. The existing configure({ options: { captureContent: false } }) opt-out already covers it.
…sult.value by default
jeffdupont
left a comment
jeffdupont
left a comment
There was a problem hiding this comment.
Re-checked 7d7a7fe..e2bab1d (README only). The sentence is right that the raw variation now goes out by default and that capture off falls back to the key (setLdSpanAttributes, packages/client/src/utils.ts:670-678).
One wording point: feature_flag.result.value is an attribute of the feature_flag span event, not of the span (utils.ts:689, and TESTING.md :396 on #113). It is also written on judge spans, not only root and graph spans (types.ts:662), and it is omitted when the variation key is empty, which graph spans can have (utils.ts:636). Suggestion inline.
Not blocking, so approving at e2bab1d. Take the suggestion if you push again before the cut.
Fixes AIC-3703. The spec change for TESTING.md §3.14 and Appendix A.8 (
createHandlerreturning a new function) is in launchdarkly/ai-sdks-monorepo#113.BEGIN_COMMIT_OVERRIDE
feat(client)!: capture conversation content on spans by default and add configure()
BREAKING CHANGE: conversation content is now captured on spans by default. To keep it off, call configure({ options: { captureContent: false } }) once at startup.
fix(client): return a new handler from createHandler instead of changing the function passed in
BREAKING CHANGE: createHandler returns a new wrapper function rather than adding providesFor to the function you pass and returning that same function. Use the handler createHandler returns; the function you passed in no longer carries providesFor.
END_COMMIT_OVERRIDE
Why
The cross-SDK spec (TESTING.md §3.9a) says conversation content is captured on spans by default and that a process-wide
configure()sets that default, with TypeScript marked pending. The Python SDK already works this way. This SDK did the opposite: every handler factory defaultedcaptureContenttofalse, evaluation spans had their ownfalsedefault, and there was noconfigure(). This PR brings the TypeScript SDK in line with the spec and with Python.What changed
configure({ options })andSDKOptions({ captureContent?: boolean }, built-in defaulttrue) exported from@launchdarkly/ai-server. The options live on the lifecycle singleton, not oninitClient.configurecan replace them until the first AI invocation. Every call to a handler built withcreateHandleror a factory counts, whether the SDK makes it or your code callsawait handler(config, input)directly, as does the start of a handler's stream, an SDK call to a plain function handler, and avercelEvaluatecall. After that, identical options are accepted and different ones throw. This matches Python, where every handler call latches.shutdown()restores the built-in defaults, even when no client was ever initialized.createHandlernow returns a new function that latchesconfigureand then calls the one you passed, with the samethisand arguments, returning its result. It keeps the original function's own properties, so wrapping a handler again keeps itsstreamandproviders. The options object,createHandler(providesFor, fn, { stream, captureContent, providers }), is unchanged. It no longer returns the function it was given, and that function no longer carriesprovidesFor, so code that callscreateHandler(p, fn)and keeps usingfnmust use the returned handler instead. When thecaptureContentoption is omitted and the function is already a handler with an explicit override, the new handler keeps that override; an unset handler stays unset and keeps followingconfigure(). An omittedprovidersoption likewise keeps the wrapped handler's list, and a passed one replaces it on the new handler only.handler.captureContenton anything built withcreateHandleris now resolved when it is read: the handler's explicittrueorfalseif it has one, otherwise the configured default, otherwisetrue. Assigning to it sets the handler's override and leaves the process default alone.configure()follows it. Convenience wrappers forward an omitted option as unset.vercelEvaluatefollows the same rule.feature_flag.result.valuealso goes through this resolver. feat(client)!: add result variant and value to the feature_flag span event #186 added a secondcaptureContentForincontent.tswith a fixedfalsedefault; it is deleted, andtracking.tsandgraph.tsnow use the one insdk-options.ts. So the graph root and plain function handlers (not built withcreateHandler) followconfigure()and its built-intrue, and write the raw variation by default.createHandlerreturns the same function. The API report gainsconfigureandSDKOptions, both additions already listed as joining inapi-surface/ai-server.txt.Behaviour change
If you never set
captureContent, your spans will now include conversation content: prompts and system instructions, model output, tool definitions, tool arguments and results, judge explanations, and the same fields on offline evaluation spans. The full variation is also exported asfeature_flag.result.value, on handler spans and on the graph root, because #186 gates it on content capture, which now resolves totrueby default. Model names, token counts, timings and tool names were already exported and are unchanged.createHandleralso changed: it returns a new function, and the function you pass in is left withoutprovidesFor. Use the returned handler.How to opt out
For the whole process, once at startup, before the first AI call:
For one handler, pass
captureContent: falseto its factory, for examplecreateOpenAIMessagesHandler({ captureContent: false }). An explicittrueon a factory still captures when the process default isfalse.Testing
configure: default on, read at run time for a handler built earlier, replacement before the first invocation, explicitfalseandtrueoverrides in both directions, the property setter, rejection after an invocation and acceptance of identical options, latching on a plain function call through the SDK, on a directawait handler(...)call (as soon as it starts, and when wrapped twice), on a handler stream that is never iterated, and on a streamed SDK call,shutdown()restoring the default,captureContentoninitClientbeing ignored with a warning, andrunJudgeswritinggen_ai.evaluation.explanationonly when the judge's effective flag is true.createHandler(q, createHandler(p, fn, { captureContent: false })).captureContentstaysfalseunder a configuredtrue, a re-wrapped unset handler still follows a laterconfigure(), andrunJudgesomits the explanation for a re-wrapped judge that kept an explicitfalse.runJudge(task, handlers)path writesgen_ai.evaluation.explanationonly when the judge's effective flag is true: built-in default, explicitfalse,configureturning it off, and explicittrueover a configuredfalse.inspectConfigreads a variation without latchingconfigure.openaiMessagesandclaudeAgents: an explicitcaptureContent: falsebeats a configuredtrue, and an omitted option is forwarded as unset, so the handler followsconfigure().createHandlertests check that the returned handler forwardsthisand arguments, returns the original result, latchesconfigurewhen called directly, keeps properties attached to the wrapped function, and keeps or replacesproviderson a re-wrap.vercelEvaluatetests check that a call latchesconfigure(also when the model call fails), that identical options are still accepted afterwards, and that a call rejected for a disabled config does not latch.configureor an explicitfalse, kept by an explicittrueover a configuredfalse, and that a handler built beforeconfigurefollows it.false,configureturning it off, a handler built beforeconfigure, and explicittrueover a configuredfalse.feature_flag.result.value: a plain function handler under the built-in default gets the raw variation throughexecuteAndTrackand throughconfig().invoke, and gets the variation key onceconfigureturns capture off. The graph root writes the raw graph variation under the built-in default and under a configuredtrue, and the variation key under a configuredfalse. The graph suite drives this withconfigure()andresetSdkOptions()instead of mocking the resolver. These tests fail against the old fixedfalse. feat(client)!: add result variant and value to the feature_flag span event #186's tests, including the caller-supplied__ldValueone, still pass.yarn test,yarn typecheck,yarn code:check,yarn lint:pkg,yarn api:update(no report change),yarn api:check,yarn api:surface,yarn test:api-surface,yarn test:api-breaking,yarn check:packagingandscripts/api-breaking.mjs prall pass locally.Note
Overview
Aligns the TypeScript SDK with the cross-SDK spec and Python: conversation content on OpenTelemetry spans is on by default, and integrators can set process-wide defaults with new
configure({ options: { captureContent } }).Content capture no longer defaults to
falseon every handler factory. UnsetcaptureContentfollowsconfigure()(built-intrue), resolved when each run starts so handlers created beforeconfigure()still pick up the default. Per-handlercaptureContent: false(or convenience-wrapper overrides) still wins. The same gate applies to judgegen_ai.evaluation.explanation, offline evaluation runner spans, andfeature_flag.result.value(full variation when capture is on, variation key when off).shutdown()restores built-in defaults and clears the post-invocation lock onconfigure.createHandlernow returns a new wrapper that carriesprovidesFor, latches the first AI call, and forwards to the function you pass in—the original function is not mutated and must not be used for routing. HandlercaptureContentis a runtime getter with an assignable override.Docs, examples, and tests across the client and all provider packages were updated for the new defaults and resolution order.
Reviewed by Cursor Bugbot for commit ede7625. Bugbot is set up for automated code reviews on this repo. Configure here.