Skip to content

OpenTelemetry span for each prompt and chain, with purpose - #8

Draft
asg017 wants to merge 2 commits into
mainfrom
asg017/otel-v2
Draft

asg017 wants to merge 2 commits into
mainfrom
asg017/otel-v2

Conversation

@asg017

@asg017 asg017 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor
🤖 Claude-generated PR description

Each prompt() and chain() call on a wrapped model or conversation now emits one OpenTelemetry span, datasette_llm.prompt (scope datasette_llm). It records the one thing llm itself can't know: the purpose a Datasette plugin declared for the call.

Attributes

  • datasette_llm.kind: "prompt" or "chain"
  • datasette_llm.purpose: the declared purpose, or "none"
  • gen_ai.request.model: the model ID
  • datasette_llm.actor.present: a boolean, never the actor itself
  • error.type: the exception class name on failure. llm.PauseChain is not an error.

No prompt or response text, actor details or error messages are recorded. The span has no exception events and no status description.

Shape

  • The span is entered first on each existing AsyncExitStack (WrappedAsyncChainResponse.responses(), WrappedConversation.prompt(), WrappedAsyncModel.prompt()). It is current for the llm_prompt_context hooks and, for chains, for the whole iteration, so llm's own chat / execute_tool spans (from an llm release with OpenTelemetry support) nest under it. __init__.py gains 20 added lines and no re-indents.
  • prompt() awaits the llm response, which forces the whole model call. The span therefore covers hooks plus the full call.
  • The only new dependency is opentelemetry-api>=1.37. When no SDK is configured the span is non-recording, and the code then never attaches a context. So behaviour without an SDK is unchanged, including when a caller abandons a chain mid-iteration.
  • With an SDK, a chain abandoned without aclose() (for example, a client disconnect mid-stream) still ends cleanly:
    • The span ends with status UNSET and datasette_llm.outcome="abandoned".
    • Nothing logs "Failed to detach context" when asyncio later finalizes the generator in another Context. The context is restored through a private sentinel ContextVar rather than context.detach().
    • The span does stay current in the abandoning task until then, which is inherent to keeping it current while the caller iterates. The README recommends contextlib.aclosing(...) to avoid it.
  • PauseChain is looked up with getattr, so llm>=0.32a1 (before PauseChain existed) still works.

Tests
tests/test_telemetry.py uses a session-scoped TracerProvider + InMemorySpanExporter. It covers:

  • prompt and chain on a model and on a conversation, with exact attribute dicts
  • hook spans nesting under the prompt span
  • the span being current during chain iteration
  • no purpose, and actor present
  • a hook raising ValueError, and a hook raising PauseChain
  • closing a chain early restores the context
  • no context attach at all with a no-op tracer

uv run pytest: 65 passed. Tests also cover an abandoned chain (no detach error, outcome abandoned) and a hook that leaks a context (not propagated to the caller). opentelemetry-sdk>=1.37 is added to the dev group only.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant