Support for LLM Observability

[Temitaio]

Is your feature request related to a problem? Please describe

1. Problem Statement
   Monoscope customers are shipping LLM-powered features (chatbots, document extraction, fraud-triage assistants) on top of infrastructure we already monitor. Today, if a customer instruments an LLM call, that telemetry either goes nowhere or goes to a second vendor (usually because they're already paying for Datadog LLM Observability, or self-hosting Langfuse).

At the same time, we should not build this from scratch. OpenTelemetry's GenAI Semantic Conventions already define the shape of this data; span names, attribute keys, metric names and three major vendors (Datadog, Sentry, Grafana) have converged on supporting it natively rather than inventing their own schema. Our task is teach our existing OTel ingestion pipeline to recognize GenAI semantic convention spans and metrics, and build the minimum UI needed to make that data useful.

2. Goals
   G1: A customer using any GenAI semconv-compliant SDK (OpenAI, Anthropic, Google GenAI, LiteLLM, community OpenTelemetry instrumentation) sees their LLM calls in Monoscope with zero custom code, the moment their OTel exporter points at us.
   G2: Per-model, per-provider breakdown of latency, token usage (input/output), and cost; cost computed from a maintained provider pricing table, not left to the customer.
   G3: A single LLM call renders as a readable trace (not raw JSON) model name, prompt/response (if capture is enabled), token counts, latency, and any error/finish reason.
   G4: This ships as part of the existing Monoscope trace/APM view, not a separate product surface the customer has to learn or switch into.
3. Non-Goals (explicitly out of scope)
   Multi-step agent tracing (planner/tool-call/sub-agent chains) - Phase 2.
   Any quality evaluation (hallucination detection, LLM-as-judge, regex rules) - Phase 2.
   Natural-language querying/copilot.
   PII scanning/redaction beyond respecting the customer's own capture settings.
   Self-hosted model / GPU infra monitoring

Describe the solution you'd like

User Personas & Stories

Primary: Backend/platform engineer , already a Monoscope customer for infra/APM.

"As an engineer who just shipped a support chatbot on OpenAI's API, I want to see it in the same dashboard as my payment service latency, without adding a new SDK or a new vendor."
"As an on-call engineer, when a customer reports 'the AI answered a payment status question wrong,' I want to find the actual trace; prompt, response, model, timestamp in under a minute."
"As an engineering lead, I want to know what our OpenAI/Anthropic bill is going to look like before the invoice arrives, broken down by which feature is driving it."

Secondary: Monoscope sales, using this in a demo or POC.

"As a salesperson in a POC with a fintech CTO, I want to show LLM cost and latency sitting next to their existing infra dashboards within the same 30-minute call, with no separate setup."
5. Success Metrics
≥80% of GenAI semconv-compliant spans sent to Monoscope render correctly (model, tokens, cost, latency all populated) with zero customer-side mapping config.
Time-to-first-LLM-trace (from SDK install to first visible trace) under 10 minutes, matching our existing infra onboarding benchmark.
Cost dashboard accuracy within 5% of actual provider invoice for the top 4 supported providers (OpenAI, Anthropic, Google Gemini, AWS Bedrock), validated against at least 2 design-partner customers before GA.

6. Scope
   P0 (must ship for GA)

i. Ingestion: GenAI semconv span recognition: Parse gen_ai.operation.name, gen_ai.request.model, gen_ai.provider.name, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.response.finish_reasons from incoming OTLP spans

ii. Ingestion: GenAI semconv metrics: Ingest gen_ai.client.operation.duration (histogram) and gen_ai.client.token.usage (histogram, split by token.type: input/output)

iii. Provider pricing table: Maintained lookup table (model → $/1K input tokens, $/1K output tokens) for OpenAI, Anthropic, Google Gemini, AWS Bedrock model families. Versioned, updatable without a deploy (config-driven)

iv. Cost computation: Derive $ per span and rolled-up $ per service / per day from token usage × pricing table

v. Trace UI (LLM call view): Render a gen_ai span with a dedicated card (not raw attribute dump): model, provider, latency, tokens (in/out), computed cost, finish reason. Prompt/response shown only if content capture is enabled (see Data Handling below)

vi. Dashboard (LLM overview):Pre-built dashboard - cost over time (by model/provider/service), P50/P95 latency by model, token volume trend, error rate

vii. Alerting integration: LLM latency, error rate, and daily cost usable as alert conditions in existing Monoscope alerting - no new alerting subsystem

viii. Docs: "Instrument your LLM calls" guide: Copy-paste setup for OpenAI Python/Node SDKs (community OTel instrumentation) plus a manual-span example for unsupported SDKs

7. UI/UX
   LLM call spans appear inline in the existing trace waterfall (no new navigation) but render with a specialized card instead of generic key-value attributes.
   New "AI" tab in the existing service/dashboard navigation, not a separate top-level product area. Reinforces "one pane of glass."
   Dashboard should ship as a pre-built, editable dashboard (consistent with how other Monoscope pre-built dashboards work), not a bespoke one-off screen.

8. API
   Extend existing trace/span query API to accept signal_type=llm_call filter.
   New read endpoint for the cost rollup (GET /api/v1/llm/cost?groupBy=model|service|feature&range=...) needed for both the dashboard and for the alerting integration.

Describe alternatives you've considered

No response

Are you able to assist to bring the feature to reality?

yes, I can

Additional context

No response