Skip to content

feat(server-utils): Instrument the OpenAI decisions API as gen_ai.decide spans - #25285

Merged
andreiborza merged 27 commits into
developfrom
ab/openai-decisions
Oct 14, 2026
Merged

andreiborza merged 27 commits into
developfrom
ab/openai-decisions

Conversation

@andreiborza

@andreiborza andreiborza commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

What

OpenAI decisions.create calls now create decide <model> spans with op gen_ai.decide, recorded in the same shape as Jev calls so Sentry renders them the same way:

  • input is recorded as state, images included as they are.
  • Questions and answers are keyed by question name, as the conventions define.
  • Choice and score question options go in criteria, like Jev: choices as { [value]: description }, score levels as their labels.
  • Choice and score probabilities are keyed by option or score value, and score labels go in a legend.
  • predicate answers keep OpenAI's { type, probability }, which Sentry renders as yes/no.
OpenAI sends / returns Recorded on the span
Evaluated input input: "I want a refund" state: "I want a refund"
Questions [{ name: "wants_refund", type: "predicate", ... }] { wants_refund: { type: "predicate", ... } }
Choice question choices: [{ value: "billing", description: "Charges and refunds" }, { value: "other" }] criteria: { billing: "Charges and refunds", other: "" }
Score question levels: [{ label: "low" }, { label: "high", description: "Blocks the customer" }] criteria: ["low", "high"], descriptions: { "1": "Blocks the customer" }
Answers [{ name: "wants_refund", type: "predicate", ... }] { wants_refund: { type: "predicate", ... } }
Predicate answer { type: "predicate", probability: 0.97 } unchanged
Choice answer choice: true, probabilities: [{ value: true, probability: 0.91 }, ...] choice: "true", probabilities: { "true": 0.91, ... }
Score answer probabilities: [{ value: 1, label: "high", probability: 0.95 }, ...] probabilities: { "1": 0.95, ... }, legend: { "1": "high", ... }

OpenAI decisions have a few things Jev does not:

  • predicate questions and answers instead of noul, with the yes probability in probability.
  • Choice values can be booleans as well as strings. The answer's choice is recorded as a string so Sentry renders it.
  • Score levels can have descriptions, recorded in descriptions keyed by score value. Sentry does not render them yet.
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["low", "medium", "high"],
      "descriptions": { "1": "Needs a reply today", "2": "Blocks the customer" }  <-- Jev does not have these
    }
  • A question can come back as a refusal, { type: "refusal" }, with no probability or reason. Sentry does not render it yet.
    "answers": {
      "wants_refund": { "type": "predicate", "probability": 0.97 },
      "urgency": { "type": "refusal" }  <-- Jev does not have these 
    }
  • The input can hold images as base64 data URLs.
    "state": [
      {
        "role": "user",
        "content": [
          { "type": "input_text", "text": "Is this image mostly red?" },
          { "type": "input_image", "image_url": "data:image/png;base64,iVBORw0KGgo..." }  <-- Jev does not have these
        ]
      }
    ]

Why

OpenAI added a Decisions API that works like Jev, and these calls had no span.

Related: #25253

…ide spans

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread dev-packages/node-integration-tests/suites/tracing/openai/v7/test.ts Outdated
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

⚠️ Warning: Base artifact is not the latest one, because the latest workflow run is not done yet. This may lead to incorrect results. Try to re-run all tests to get up to date results.

Path Size % Change Change
@sentry/browser 29.83 kB - -
@sentry/browser - with treeshaking flags 27.96 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 27.87 kB - -
@sentry/browser (incl. Tracing) 52.02 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.02 kB - -
@sentry/browser (incl. Tracing, Profiling) 54.96 kB - -
@sentry/browser (incl. Tracing, Replay) 91.75 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 80.62 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 96.47 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 109.43 kB - -
@sentry/browser (incl. Feedback) 47.35 kB - -
@sentry/browser (incl. sendFeedback) 34.89 kB - -
@sentry/browser (incl. FeedbackAsync) 40 kB - -
@sentry/browser (incl. Metrics) 30.85 kB - -
@sentry/browser (incl. Logs) 31.13 kB - -
@sentry/browser (incl. Metrics & Logs) 31.78 kB - -
@sentry/react 31.64 kB - -
@sentry/react (incl. Tracing) 54.34 kB - -
@sentry/vue 37.98 kB - -
@sentry/vue (incl. Tracing) 54.95 kB - -
@sentry/svelte 29.86 kB - -
@sentry/remix (Remix 3 client bundle) 57.02 kB - -
CDN Bundle 31.56 kB - -
CDN Bundle (incl. Tracing) 52.52 kB - -
CDN Bundle (incl. Logs, Metrics) 33.77 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 54.46 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 74.7 kB - -
CDN Bundle (incl. Tracing, Replay) 90.19 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 92.11 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 96.34 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 98.28 kB - -
CDN Bundle - uncompressed 92.98 kB - -
CDN Bundle (incl. Tracing) - uncompressed 155.83 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 99.52 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 161.79 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 229.56 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 276 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 281.94 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 289.7 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 295.63 kB - -
@sentry/nextjs (client) 56.75 kB - -
@sentry/sveltekit (client) 52.41 kB - -
@sentry/core/server 41.12 kB - -
@sentry/core/browser 13.81 kB - -
@sentry/node 152.43 kB +0.31% +460 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 83.81 kB +0.04% +30 B 🔺
@sentry/node - without tracing 94.57 kB +0.04% +34 B 🔺
@sentry/node - without channel injection 130.66 kB +0.38% +484 B 🔺
@sentry/aws-serverless 102.73 kB +0.04% +39 B 🔺
@sentry/cloudflare (withSentry) - minified 210.38 kB - -
@sentry/cloudflare (withSentry) 521.61 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 228.06 kB - -

View base workflow run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0cf6661. Configure here.

Comment thread packages/server-utils/src/ai/openai/utils.ts Outdated
Comment thread packages/server-utils/src/ai/openai/index.ts
@andreiborza
andreiborza added this pull request to stack #25332 October 13, 2026 13:41
@andreiborza
andreiborza marked this pull request as ready for review October 13, 2026 13:57
@andreiborza
andreiborza requested a review from a team as a code owner October 13, 2026 13:57
@andreiborza
andreiborza requested review from mydea and removed request for a team October 13, 2026 13:57
Comment thread dev-packages/node-integration-tests/suites/tracing/openai/v7/test.ts Outdated
Comment thread dev-packages/node-integration-tests/suites/tracing/openai/v7/test.ts Outdated
Comment thread packages/server-utils/src/ai/openai/index.ts
Comment thread packages/server-utils/src/ai/openai/index.ts Outdated
Comment thread packages/server-utils/src/ai/openai/mapping.ts Outdated
Comment thread packages/server-utils/src/ai/openai/mapping.ts
Comment thread packages/server-utils/src/ai/openai/mapping.ts Outdated
Comment thread packages/server-utils/src/ai/openai/mapping.ts Outdated
Comment thread packages/server-utils/src/ai/openai/utils.ts
Comment thread packages/server-utils/src/ai/openai/mapping.ts

@nicohrubec nicohrubec left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for updating, lgtm

@JPeer264 JPeer264 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{ type: 'predicate', name: 'lgtm', probability: 0.99 }

@andreiborza
andreiborza merged commit 8ec5fe4 into develop Oct 14, 2026
732 of 734 checks passed
@andreiborza
andreiborza deleted the ab/openai-decisions branch October 14, 2026 14:24
renovate Bot added a commit to andrei-picus-tink/auto-renovate that referenced this pull request Oct 17, 2026
| datasource | package             | from  | to     |
| ---------- | ------------------- | ----- | ------ |
| npm        | @sentry/vite-plugin | 5.4.1 | 11.8.0 |


## [v11.8.0](https://github.com/getsentry/sentry-javascript/blob/HEAD/CHANGELOG.md#1180)

##### Important Changes

- **feat(bundler-plugins): Release `@sentry/webpack-plugin`, `@sentry/vite-plugin`, `@sentry/rollup-plugin` and `@sentry/esbuild-plugin` from the SDK repository ([#25095](getsentry/sentry-javascript#25095

  The four standalone bundler plugins now release together with the SDK, so their version jumps from 5.4.1 to the SDK version. Each package re-exports the matching `@sentry/bundler-plugins` entry point, which we recommend for new projects. The plugin options do not change, but the update brings these breaking changes:

  - Node.js 20.19.0 or higher is required.
  - `@sentry/webpack-plugin` requires webpack 5.1 or higher. The `@sentry/webpack-plugin/webpack5` entry point still works and exports the same plugin as `@sentry/webpack-plugin`.
  - The plugins use the `sentry` CLI SDK instead of `@sentry/cli`. Sentry self-hosted 26.4.2 or higher is supported. Lower versions may continue to work.
  - Deploys that the plugins create on Vercel use the value of `VERCEL_TARGET_ENV` (`production`, `preview` or a custom name) as their environment, instead of `vercel-production` or `vercel-preview`. To keep the old value, set `release.deploy.env`.

- **feat(server-utils): Instrument the OpenAI decisions API as `gen_ai.decide` spans ([#25285](getsentry/sentry-javascript#25285

  OpenAI `decisions.create` calls now create `gen_ai.decide` spans.

##### Other Changes

- feat(core): Support SuppressedError in LinkedErrors integration ([#25246](getsentry/sentry-javascript#25246))
- fix(astro): Delete generated source maps for static output ([#25310](getsentry/sentry-javascript#25310))
- fix(astro): Upload source maps of Astro 7's prerender build ([#25309](getsentry/sentry-javascript#25309))
- fix(browser): Apply `urlQueryParams` to the Referer header ([#25324](getsentry/sentry-javascript#25324))
- fix(cloudflare): auto-instrument when wrangler `main` is a package ([#25343](getsentry/sentry-javascript#25343))
- fix(hono): Expose runtime-specific SDK options ([#25290](getsentry/sentry-javascript#25290))
- fix(node): Instrument Prisma 8 CollectionBase from rc.15 ([#25301](getsentry/sentry-javascript#25301))
- fix(nuxt): Skip 3xx/4xx Nuxt errors in the vue:error hook ([#25341](getsentry/sentry-javascript#25341))
- fix(server-utils): Read pi-durable 1.2.0 tool results from `output` ([#25312](getsentry/sentry-javascript#25312))
- fix(server-utils): Transform modules once when two Sentry Vite plugins add orchestrion ([#25043](getsentry/sentry-javascript#25043))
- fix(sveltekit): add op and origin to Kit's `sveltekit.endpoint` span ([#25376](getsentry/sentry-javascript#25376))
- fix(tanstackstart-react): Keep parametrized route names in Nitro 3 builds ([#25307](getsentry/sentry-javascript#25307))
- fix(tanstackstart-react): Rewrite source paths of split route modules ([#25361](getsentry/sentry-javascript#25361))

<details>
  <summary> <strong>Internal Changes</strong> </summary>

- chore: Add external contributor to CHANGELOG.md ([#25375](getsentry/sentry-javascript#25375))
- ci: Install the pnpm version that Volta pins ([#25287](getsentry/sentry-javascript#25287))
- license: Add cli FSL notice to package licenses ([#25314](getsentry/sentry-javascript#25314))
- ref(core): use shared MCP attribute conventions ([#25059](getsentry/sentry-javascript#25059))
- ref(tanstackstart-react): Reuse shared component annotation options ([#25313](getsentry/sentry-javascript#25313))
- test(browser): Keep whitespace text out of the handlers-lcp page ([#25280](getsentry/sentry-javascript#25280))
- test(bun): Restore [@sentry/core](https://github.com/sentry/core) spies after the Bun.serve tests ([#25276](getsentry/sentry-javascript#25276))
- test(bundler-plugins): Disable CLI telemetry in the CLI contract tests ([#25308](getsentry/sentry-javascript#25308))
- test(e2e): Add node-openai-decisions-send-to-sentry test app ([#25331](getsentry/sentry-javascript#25331))
- test(e2e): Add node-openai-send-to-sentry test app ([#24824](getsentry/sentry-javascript#24824))
- test(e2e): Add nuxt-5-sourcemaps test ([#25268](getsentry/sentry-javascript#25268))
- test(e2e): Inherit root Volta config in all e2e test apps ([#25369](getsentry/sentry-javascript#25369))
- test(e2e): Use an external host address for the Lambda DSN on loopback hostnames ([#25282](getsentry/sentry-javascript#25282))
- test(nextjs): Run nextjs-16-tunnel on Cloudflare ([#25161](getsentry/sentry-javascript#25161))
- test(nextjs): Switch DEBUG_BUILD with a getter instead of re-importing the SDK ([#25279](getsentry/sentry-javascript#25279))
- test(node): Await the child exit in the "should exit" integration tests ([#25277](getsentry/sentry-javascript#25277))
- test(node): Cover ioredis v6 in the diagnostics_channel suite ([#25300](getsentry/sentry-javascript#25300))
- test(node): Cover LangChain model calls made from a tool without config ([#25326](getsentry/sentry-javascript#25326))
- test(node): Log slow and failed Docker Compose starts in integration tests ([#25278](getsentry/sentry-javascript#25278))
- test(node): Unflake captureMessage with_level test ([#25358](getsentry/sentry-javascript#25358))
- test(sveltekit): Pin rollup to 4.64.5 ([#25328](getsentry/sentry-javascript#25328))

</details>

Work in this release was contributed by [@kunal649](https://github.com/kunal649) and [@mketiku](https://github.com/mketiku). Thank you for your contributions!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants