Skip to content

obs: expose TLS/H2 cutover transport telemetry - #88

Open
seonghobae wants to merge 11 commits into
fix/downstream-forwarded-scheme-v1from
obs/h2-cutover-transport-metrics-v1
Open

obs: expose TLS/H2 cutover transport telemetry#88
seonghobae wants to merge 11 commits into
fix/downstream-forwarded-scheme-v1from
obs/h2-cutover-transport-metrics-v1

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Refs #51.

Writer-safe child of #87 exact 5868fb61e8f66dc5d79bb15a6e7887156dc7684a. This increment targets rollback/canary observability without adding product labels, deployment identity, client-IP authority, Keyverse identity, or Wardnet/EgressWeave verdicts to the gateway.

The aggregate request/error counters could not distinguish a TLS/H2 canary regression from healthy cleartext/H1 traffic. RED-only commit 701faf4390225ae650870dedf64dc921e838e278 added a CA-verified real-wire requirement that an actual H2-over-TLS request expose cwl_pingora_gateway_requests_by_transport_total{outcome="ok",protocol="h2",transport="tls"} 1. Its hosted CI was superseded while compile/test was still running, so it is not claimed as a terminal hosted RED receipt.

Production commit f0231178a0eb21c66b3f81b6ed1239cb608ff6c1 adds one finite-cardinality CounterVec with only outcome={ok,error}, protocol={h1,h2}, and transport={cleartext,tls}. Protocol comes from the accepted Pingora session (Session::is_http2()); TLS comes from the accepted session digest. The eight possible series are pre-created. Routes, headers, cookies, hostnames, product/customer IDs, client IPs, certificate paths and deployment revisions remain excluded; existing aggregate counters remain backward-compatible.

CI 34601751211 on f023117... failed only at Rust 1.98.0 cargo fmt --check; 7834aad1c6b991e57b663590e240489ef06a4446 applies exactly the emitted formatter diff. Exact cc9067006296b6dd1fb5f89d2b26fe5ea4e9e77a then passed formatting, compile/test including the real-wire H2/TLS metric assertion, Clippy, rustdoc, load-contract, OCI runtime, Supply Chain and bounded-origin capacity, but CI 34602099784 failed final complete-owned-production coverage. Artifact coverage-cc9067006296b6dd1fb5f89d2b26fe5ea4e9e77a (sha256:39bb9923363809341feca489bdecfd95ffec1f0f355e84305b88be13021c13d2) isolated the only sub-100% file to src/observability.rs; the two zero-count regions were duplicate-registration failure instrumentation in the transport CounterVec initializer.

Ordinary-forward 2e2674033667007872be7bc95261f1eb60f5e0e0 factors CounterVec registration through a fail-closed helper and adds the missing duplicate-vector registration regression without changing metric names, labels, pre-created series, classification or failure semantics. 87f354e70b4e33847110840a30c0aa1d721ff028 records that RCA in docs/doctoring/CUTOVER_OBSERVABILITY_TRACEABILITY.md. A code-current sweep then repaired stale canonical ARCHITECTURE.md/TRD.md descriptions through 86e53139277f1a8055ba4041637a68a6c3da25c8 and 9e342abf41a95e6711f04858ee3aa39343b66623; dedicated baseline lane #61 remains untouched.

A second current-head fixture review found that the older tests/downstream_tls_h2_wire.rs released its ephemeral traffic/metrics reservations immediately after discovering their addresses, leaving a TOCTOU window where unrelated local activity could bind a configured port and create a non-protocol RED. 5cdc84284f1eebde932d654c742ca4159ed54216 adopts the already-proven H2 fixture pattern: both TcpListener reservations remain live until immediately before gateway spawn. Current exact a15404e7822a4e751b2f363f67a7bebf1a837fec records this repair in cutover TRACEABILITY. The H2/TLS, cleartext-origin proxy and exact metric oracles are unchanged.

Current exact a15404e... independently passed CI 34604676321, Supply Chain 34604676279, and PgErd bounded-origin capacity 34604676389. CI includes formatting, compile/test, strict Clippy, warnings-denied rustdoc, complete owned-production coverage, load-contract and rootless/read-only OCI runtime. Fresh review-thread enumeration is empty. Exact-current technical COMMENT review 5179290159 records scope/evidence only and is not independent approval. The PR is Ready for review, open and mergeable. Organization rules still require a ruleset-valid independent approving review plus central required workflows before protected integration; no bypass, protected merge, release, canary, cutover or legacy-removal credit is claimed.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 80aa5711-d2e1-4951-b606-93801e44b711

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current technical review for a15404e7822a4e751b2f363f67a7bebf1a837fec: CI 34604676321, Supply Chain 34604676279, and PgErd bounded-origin capacity 34604676389 are terminal SUCCESS. Current branch still points to this exact head; review-thread enumeration is empty. The coverage RCA is repaired by 2e267403... without telemetry semantic changes, canonical ARCHITECTURE/TRD are code-current, and 5cdc842... removes the H2 traffic/metrics ephemeral-port TOCTOU by holding both listener reservations until spawn without changing the H2/TLS/origin/metric oracles. Effective scope remains rollback/canary transport telemetry plus its executable acceptance and documentation. Product auth/business logic, Keyverse, Wardnet/EgressWeave, certificate lifecycle, generic client-IP/trusted-hop authority, supplier framing roots, deployment identity, actual canary/rollback/cutover, and legacy removal remain outside this increment. This COMMENT is technical evidence only and is not an independent approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request priority: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant