Skip to content

Fix duplicate index throttling metric registration - #1171

Open
wdtte wants to merge 1 commit into
prometheus-community:masterfrom
wdtte:fix/index-stats-is-throttled-conflict
Open

Fix duplicate index throttling metric registration#1171
wdtte wants to merge 1 commit into
prometheus-community:masterfrom
wdtte:fix/index-stats-is-throttled-conflict

Conversation

@wdtte

@wdtte wdtte commented May 26, 2026

Copy link
Copy Markdown

Summary

  • rename the index-level throttling metric to elasticsearch_index_stats_indexing_is_throttled
  • avoid the registry collision with the node-level elasticsearch_indices_indexing_is_throttled metric
  • add a regression test to ensure NewNodes and NewIndices can be registered together

Reproduction

The issue was reproduced in multi-target /probe mode with these flags:

  • --es.uri=
  • --config.file=/etc/elasticsearch-exporter/exporter-config.yml
  • --web.listen-address=:9114
  • --es.all
  • --es.shards
  • --es.ssl-skip-verify

Representative runtime logs:

time=2026-05-26T12:48:36.027Z level=INFO source=tls_config.go:354 msg="Listening on" address=[::]:9114
time=2026-05-26T12:48:36.027Z level=INFO source=tls_config.go:357 msg="TLS is disabled." http2=false address=[::]:9114
2026/05/26 12:49:02 http: panic serving 10.34.29.6:36890: a previously registered descriptor with the same fully-qualified name as Desc{fqName: "elasticsearch_indices_indexing_is_throttled", help: "Whether indexing is currently throttled for an index (1=throttled, 0=not throttled)", constLabels: {}, variableLabels: {index,cluster}} has different label names or a different help string
github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(...)
	/go/pkg/mod/github.com/prometheus/client_golang@v1.23.2/prometheus/registry.go:406
main.main.func3({0xdc6940, 0x298a9f4b0690}, 0x298a9f3fb900)
	/app/main.go:403 +0xe3c

The same panic repeats on every scrape while NewNodes() and NewIndices() are both registered.

Testing

  • env GOCACHE=/Users/wdte/GolandProjects/elasticsearch_exporter/.gocache go test ./collector -run TestNodesAndIndicesRegisterTogether -count=1

Notes

  • full go test ./collector/... is blocked in the current sandbox because existing tests use httptest.NewServer and local port binding is not permitted in this environment

Signed-off-by: yanxin <yanx10594@fxiaoke.com>
@wdtte
wdtte force-pushed the fix/index-stats-is-throttled-conflict branch from 5090a10 to 027560b Compare May 26, 2026 13:22
@wdtte

wdtte commented May 27, 2026

Copy link
Copy Markdown
Author

This PR is ready for review.

It fixes a reproducible /probe panic caused by duplicate registration of elasticsearch_indices_indexing_is_throttled when NewNodes() and NewIndices() are both registered.

The change is intentionally small:

  • rename the index-level metric to elasticsearch_index_stats_indexing_is_throttled
  • keep the node-level metric unchanged
  • add a regression test for co-registration

DCO is green now. A maintainer review would be appreciated.

@sysadmind could you please take a look when you have time?

This seems to be a regression caused by the interaction between multi-target /probe registration and the index-level indexing metrics added in #1125. I added a minimal fix plus a regression test.

If this is merged, an updated official image would be very helpful, since I am currently hitting this issue in production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant