What happened?
Native Elasticsearch trace summaries report an incorrect error count when a span
contains only the http.response.status_code attribute with a value in the 5xx
range and has no explicit OpenTelemetry error status.
The full-trace fallback correctly classifies the span as an error, but the native
summary aggregation does not. This causes the native and fallback summary paths
to return different error counts for the same kind of trace.
Steps to reproduce
-
Start Elasticsearch using the repository's Elasticsearch 9 compose file.
-
Start Jaeger with native trace summaries enabled.
-
Send one OTLP span with:
service.name: repro-http-5xx
http.response.status_code: 500
- no explicit
status.code
- no
error=true attribute
-
Query /api/v3/trace-summaries for the repro-http-5xx service.
-
Stop Jaeger and restart it with native summaries disabled:
go run ./cmd/jaeger \
--config cmd/jaeger/config-elasticsearch.yaml \
--feature-gates=-jaeger.es.nativeTraceSummaries
-
Run the same trace-summary query again.
The complete commands are documented above in this file.
Actual Behavior
With native summaries enabled, the response reports zero errors:
{
"spanCount": 1,
"errorSpanCount": 0,
"services": [
{
"name": "repro-http-5xx",
"spanCount": 1,
"errorSpanCount": 0
}
]
}
With native summaries disabled, the full-trace fallback reports one error:
{
"spanCount": 1,
"errorSpanCount": 1,
"services": [
{
"name": "repro-http-5xx",
"spanCount": 1,
"errorSpanCount": 1
}
]
}
The API omits zero-valued protobuf fields in the raw response; the displayed
values above were normalized with jq using // 0.
Expected behavior
Native Elasticsearch summaries should classify spans consistently with the
full-trace conversion path. A span with http.response.status_code=500 should
contribute one to both:
- the trace-level
errorSpanCount
- the per-service
errorSpanCount
Explicit status and error tags should retain their existing precedence and
behavior.
Relevant log output
These are the outputs collected during the local reproduction. The first query
was run against the fallback path and returned one error:
{
"traceId": "6b73d1dc57a65e99076ff9039499e2c3",
"spanCount": 1,
"errorSpanCount": 1,
"services": [
{
"name": "repro-http-5xx",
"spanCount": 1,
"errorSpanCount": 1
}
]
}
The second query was run against the native summary path and returned zero
errors:
{
"traceId": "6e3eb473c58607d9da14e6fdba200cb1",
"spanCount": 1,
"errorSpanCount": 0,
"services": [
{
"name": "repro-http-5xx",
"spanCount": 1,
"errorSpanCount": 0
}
]
}
Screenshot
Additional context
- Jaeger: local source build from
main
- Storage: Elasticsearch 9.x
- Jaeger configuration:
cmd/jaeger/config-elasticsearch.yaml
- Endpoint:
GET /api/v3/trace-summaries
- Feature gate:
jaeger.es.nativeTraceSummaries
Suspected Cause
The native Elasticsearch summary aggregation in
internal/storage/v2/elasticsearch/tracestore/core/summary.go currently counts
errors using only the explicit error=true tag.
The write path does not add that tag when a span has an HTTP 5xx attribute but no
explicit OpenTelemetry error status. In contrast,
internal/storage/v2/elasticsearch/tracestore/from_dbmodel.go infers
StatusCodeError from the HTTP status during normal span conversion.
The query service accepts native summary results directly and falls back only
when the backend returns errors.ErrUnsupported, so this mismatch is exposed to
API clients.
Relevant paths:
internal/storage/v2/elasticsearch/tracestore/core/summary.go
internal/storage/v2/elasticsearch/tracestore/from_dbmodel.go
cmd/jaeger/internal/extension/jaegerquery/querysvc/summary.go
cmd/jaeger/internal/extension/jaegerquery/querysvc/service.go
Jaeger backend version
No response
SDK
No response
Pipeline
No response
Stogage backend
No response
Operating system
Linux POP_OS
Deployment model
No response
Deployment configs
What happened?
Native Elasticsearch trace summaries report an incorrect error count when a span
contains only the
http.response.status_codeattribute with a value in the 5xxrange and has no explicit OpenTelemetry error status.
The full-trace fallback correctly classifies the span as an error, but the native
summary aggregation does not. This causes the native and fallback summary paths
to return different error counts for the same kind of trace.
Steps to reproduce
Start Elasticsearch using the repository's Elasticsearch 9 compose file.
Start Jaeger with native trace summaries enabled.
Send one OTLP span with:
service.name: repro-http-5xxhttp.response.status_code: 500status.codeerror=trueattributeQuery
/api/v3/trace-summariesfor therepro-http-5xxservice.Stop Jaeger and restart it with native summaries disabled:
Run the same trace-summary query again.
The complete commands are documented above in this file.
Actual Behavior
With native summaries enabled, the response reports zero errors:
{ "spanCount": 1, "errorSpanCount": 0, "services": [ { "name": "repro-http-5xx", "spanCount": 1, "errorSpanCount": 0 } ] }With native summaries disabled, the full-trace fallback reports one error:
{ "spanCount": 1, "errorSpanCount": 1, "services": [ { "name": "repro-http-5xx", "spanCount": 1, "errorSpanCount": 1 } ] }The API omits zero-valued protobuf fields in the raw response; the displayed
values above were normalized with
jqusing// 0.Expected behavior
Native Elasticsearch summaries should classify spans consistently with the
full-trace conversion path. A span with
http.response.status_code=500shouldcontribute one to both:
errorSpanCounterrorSpanCountExplicit status and error tags should retain their existing precedence and
behavior.
Relevant log output
Screenshot
Additional context
maincmd/jaeger/config-elasticsearch.yamlGET /api/v3/trace-summariesjaeger.es.nativeTraceSummariesSuspected Cause
The native Elasticsearch summary aggregation in
internal/storage/v2/elasticsearch/tracestore/core/summary.gocurrently countserrors using only the explicit
error=truetag.The write path does not add that tag when a span has an HTTP 5xx attribute but no
explicit OpenTelemetry error status. In contrast,
internal/storage/v2/elasticsearch/tracestore/from_dbmodel.goinfersStatusCodeErrorfrom the HTTP status during normal span conversion.The query service accepts native summary results directly and falls back only
when the backend returns
errors.ErrUnsupported, so this mismatch is exposed toAPI clients.
Relevant paths:
internal/storage/v2/elasticsearch/tracestore/core/summary.gointernal/storage/v2/elasticsearch/tracestore/from_dbmodel.gocmd/jaeger/internal/extension/jaegerquery/querysvc/summary.gocmd/jaeger/internal/extension/jaegerquery/querysvc/service.goJaeger backend version
No response
SDK
No response
Pipeline
No response
Stogage backend
No response
Operating system
Linux POP_OS
Deployment model
No response
Deployment configs