Skip to content

[OpenTelemetry] Fix OpenTelemetry span kinds and add write spans in PubsubIO and KafkaIO - #40483

Merged
stankiewicz merged 2 commits into
apache:masterfrom
stankiewicz:pubsub_kafka_otel_fixes
Oct 13, 2026
Merged

stankiewicz merged 2 commits into
apache:masterfrom
stankiewicz:pubsub_kafka_otel_fixes

Conversation

@stankiewicz

Copy link
Copy Markdown
Contributor

Follow-up to #39150 (PubsubIO) and #39151 (KafkaIO) to properly set OpenTelemetry messaging span kinds and create producer spans in the write-side header propagation DoFns:

  1. Read-Side Span Kind (SpanKind.CONSUMER):
    • In PubsubIO.Read.OpenTelemetryHeaderConsumer, set .setSpanKind(SpanKind.CONSUMER) on the "PubSubIO.Read" span.
    • In KafkaIO.OpenTelemetryHeaderConsumer, set .setSpanKind(SpanKind.CONSUMER) on the "KafkaIO.Read" span.
  2. Write-Side Producer Span Creation (SpanKind.PRODUCER):
    • Previously, OpenTelemetryHeaderPropagator in both PubsubIO and KafkaIO injected Context.current() directly without creating a producer span. If no upstream span was active on the worker thread, no traceparent was injected and no write span was recorded.
    • In PubsubIO.Write.OpenTelemetryHeaderPropagator, initialize the "PubSubIO" Tracer via PipelineOptions (SdkHarnessOptions) in @Setup, start a "PubSubIO.Write" span with SpanKind.PRODUCER and parent Context.current(), and make it current before injecting W3C trace context (googclient_traceparent / googclient_tracestate) and outputting the PubsubMessage.
    • In KafkaIO.OpenTelemetryHeaderPropagator, initialize the "KafkaIO" Tracer via PipelineOptions (SdkHarnessOptions) in @Setup, start a "KafkaIO.Write" span with SpanKind.PRODUCER and parent Context.current(), and make it current before injecting W3C trace context into the ProducerRecord headers and outputting the record.

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @Abacn for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stankiewicz
stankiewicz force-pushed the pubsub_kafka_otel_fixes branch from f2737d5 to 91362da Compare October 13, 2026 11:27
@stankiewicz
stankiewicz requested a review from Abacn October 13, 2026 11:27
@stankiewicz
stankiewicz force-pushed the pubsub_kafka_otel_fixes branch from 91362da to a90d3ce Compare October 13, 2026 11:42
@stankiewicz
stankiewicz merged commit bfa17fd into apache:master Oct 13, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants