Skip to content

Add optional Prometheus metrics endpoint - #62

Open
danbryan wants to merge 1 commit into
cosmos:mainfrom
danbryan:metrics
Open

Add optional Prometheus metrics endpoint#62
danbryan wants to merge 1 commit into
cosmos:mainfrom
danbryan:metrics

Conversation

@danbryan

@danbryan danbryan commented Aug 23, 2026

Copy link
Copy Markdown
image

Summary

Adds an opt-in Prometheus endpoint to the signer. A metrics config block
enables an HTTP listener serving GET /metrics with signer metrics plus the
standard Go and process collectors. Nothing changes when the block is absent:
collection is always on, serving is opt-in, and no existing behavior or config
is touched.

metrics:
  listen: 0.0.0.0:8545

Also defaults the file backend algorithm to ed25519, which the README
and the kms init template already document as the default; before this
change a backend: file key without an explicit algorithm failed at
startup with file: unknown key type.

Motivation

A remote signer is the one component a validator cannot see from the chain
side until blocks are already being missed. Operators need to alert on the
signer itself: is it connected, is it signing, did the double-sign guard
refuse something, is the key backend healthy, and how fast is signing.

Metrics

A reasonable default set, based on surveying what the remote signers in the
ecosystem expose and what operators alert on. All series are prefixed kms_,
and signer series carry chain_id, since one process can sign for several
chains. Full reference with starter alert rules is in docs/metrics.md.

Group Metrics
Connection kms_validator_connected, kms_validator_connected_since_timestamp_seconds, kms_validator_dials_total{result}
Signing kms_requests_total{type,result} where result="refused" marks a double-sign-guard decline, kms_sign_duration_seconds{type}, kms_last_signed_height/_round/_timestamp_seconds{type}
Double-sign state kms_sign_state_height/_round/_step mirroring the persisted FilePV floor
Key backend kms_backend_sign_duration_seconds{backend,algorithm}, kms_backend_errors_total{backend,algorithm}; the pkcs11 and awskms backends put an HSM or a network API in the signing hot path, so this is where their health shows
Metadata kms_build_info{version,go_version}, kms_key_info{chain_id,backend,algorithm,address} with the public consensus address, so operators can assert the signer holds the key they expect

Plus the stock go_* and process_* collectors from promhttp.

@danbryan
danbryan requested a review from a team as a code owner August 23, 2026 23:05
@danbryan
danbryan marked this pull request as draft August 23, 2026 23:09
@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds an opt-in Prometheus endpoint and instruments validator connectivity, signing requests, backend operations, signing state, and key/build metadata. It also makes Ed25519 the effective default for file-backed consensus keys.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename Overview
internal/manager/manager.go Adds validator connection, dial, request, signing-latency, last-signature, and sign-state metrics.
internal/metrics/metrics.go Defines the private Prometheus registry, collectors, HTTP server, and backend signer instrumentation.
cmd/kms/main.go Starts and closes the optional metrics listener from the configured metrics block.
internal/app/build.go Wraps consensus signers with backend metrics, publishes key metadata, and defaults file keys to Ed25519.
config/config.go Adds the optional metrics listener configuration schema.

Reviews (2): Last reviewed commit: "Add optional Prometheus metrics endpoint" | Re-trigger Greptile

Comment thread internal/manager/manager.go
@danbryan
danbryan force-pushed the metrics branch 4 times, most recently from f169334 to c60b617 Compare August 23, 2026 23:42
@danbryan

Copy link
Copy Markdown
Author

@greptileai review please

@danbryan
danbryan marked this pull request as ready for review August 24, 2026 00:12
Serves /metrics from a metrics.listen config block: connection state and
dial results, requests by type and result including double-sign refusals,
sign latency, the persisted sign state, raw backend latency and errors,
and build/key metadata, plus the standard Go and process collectors.
Also defaults the file backend algorithm to ed25519 as documented.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant