Skip to content

feat: ccr stats - #1145

Open
kizuna-lek wants to merge 4 commits into
prometheus-community:masterfrom
kizuna-lek:feature/ccr-stats
Open

feat: ccr stats#1145
kizuna-lek wants to merge 4 commits into
prometheus-community:masterfrom
kizuna-lek:feature/ccr-stats

Conversation

@kizuna-lek

@kizuna-lek kizuna-lek commented Mar 9, 2026

Copy link
Copy Markdown

Comment thread collector/ccr.go Outdated
[]string{"follower_index"},
nil,
)
ccrFollowShardInfo = []ccrShardMetric{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these should be package vars, not nested in a slice. It's our standard for all new collectors. It helps us prevent unused metric descriptions.

Comment thread collector/ccr.go Outdated
nil,
)
ccrAutoFollowedClusterTimeSinceLastCheckMilliseconds = prometheus.NewDesc(
prometheus.BuildFQName(namespace, "ccr_auto_followed_cluster", "time_since_last_check_milliseconds"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread collector/ccr.go Outdated
},
{
desc: prometheus.NewDesc(
prometheus.BuildFQName(namespace, "ccr_follow_shard", "time_since_last_read_milliseconds"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should also be normalized to seconds

Comment thread collector/ccr.go Outdated
registerCollector("ccr", defaultDisabled, NewCCR)
}

type ccrShardMetric struct {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What value does this struct provide? I think we can drop it.

Comment thread collector/ccr_test.go Outdated
if err != nil {
t.Fatal(err)
}
if !hasMetric(descs, "elasticsearch_ccr_follow_index_global_checkpoint_lag") {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests in this project for new collectors test the full output. This helps us catch breaking changes more easily. See a good example here: https://github.com/prometheus-community/elasticsearch_exporter/blob/master/collector/slm_test.go#L30

Comment thread metrics.md Outdated
| elasticsearch_ccr_auto_followed_cluster_last_seen_metadata_version | gauge | 1 | Last seen metadata version for an auto-followed cluster |
| elasticsearch_ccr_auto_followed_cluster_time_since_last_check_milliseconds | gauge | 1 | Time since last auto-follow check in milliseconds for an auto-followed cluster |
| elasticsearch_ccr_follow_index_global_checkpoint_lag | gauge | 1 | Total global checkpoint lag for a follower index |
| elasticsearch_ccr_follow_shard_successful_read_requests_total | counter | 4 | Successful read requests for a follower shard (`collector.ccr.detailed=true`) |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is missing a large number of metrics that are added by this PR.

Comment thread metrics.md Outdated
| elasticsearch_ccr_auto_follow_successful_follow_indices_total | counter | 0 | Number of indices auto-follow successfully followed |
| elasticsearch_ccr_auto_follow_recent_errors_total | counter | 0 | Number of recent auto-follow errors currently reported |
| elasticsearch_ccr_auto_followed_cluster_last_seen_metadata_version | gauge | 1 | Last seen metadata version for an auto-followed cluster |
| elasticsearch_ccr_auto_followed_cluster_time_since_last_check_milliseconds | gauge | 1 | Time since last auto-follow check in milliseconds for an auto-followed cluster |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These will need to be updated when the metrics are changed in ccr.go

@kizuna-lek

kizuna-lek commented Mar 18, 2026

Copy link
Copy Markdown
Author

Thanks for the thorough review, @sysadmind I've addressed all of your feedback:

  • Package-level vars: Extracted all shard metric descriptors from the []ccrShardMetric slice into individual package-level var declarations, and removed the ccrShardMetric struct.

  • Time normalization: Converted time_since_last_check_milliseconds and time_since_last_read_milliseconds to seconds (time_since_last_check_seconds / time_since_last_read_seconds), with /1000 conversion applied to the values.

  • Full output tests: Rewrote the tests using testutil.CollectAndCompare (following the slm_test.go pattern) to validate the complete metric output for both minimal and detailed modes.

  • metrics.md: Updated the CCR section to include all metrics (including the previously missing shard-level and follower parameter metrics), and aligned the time unit naming.

Please take another look when you get a chance. Happy to make further adjustments if needed

Signed-off-by: kizuna-lek <asd98041@qq.com>
Signed-off-by: kizuna-lek <asd98041@qq.com>
Signed-off-by: kizuna-lek <asd98041@qq.com>
Signed-off-by: kizuna-lek <asd98041@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prometheus Elasticsearch exporter for receiving CCR stats

2 participants