atenet: pick up refreshed pod certificate - #531
Conversation
3eff542 to
f314e0b
Compare
| }, | ||
| // By specifying WatchedDirectory, we tell envoy to watch changes to the mounted pod certificate file. | ||
| // See documentation in https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/v3#:~:text=This%20only%20applies%20when%20a%20%E2%80%9CTlsCertificate%E2%80%9C%20is%20delivered%20by%20SDS | ||
| WatchedDirectory: &corev3.WatchedDirectory{ |
There was a problem hiding this comment.
🤖 question 🟢 – Was the reload verified end to end, rather than just the config shape? The tests pin the SDS wiring well, but nothing here exercises Envoy actually re-reading the bundle, and the failure mode is silent until the cert expires a day later. Forcing an early rotation (or touching the projected volume and watching Envoy's secret stats / admin /certs) would confirm the watch fires for a podCertificate volume specifically.
There was a problem hiding this comment.
I only verified it by deploying atenet locally and forcing kubelet to resign the cert. 🤖 was right that the unit tests here only checks the expected config is delivered. I think to test envoy is actually re-reading the bundle, we would need to set up an e2e test with real envoy instance, and modify the podcertificate controller to issue certs with very short lifespan.
… in tests. Address review feedback that the SDS+WatchedDirectory change was only pinned at the config-shape level. Adds two tests that stand in for Envoy: - TestXdsServer_ServesSecretOverSds subscribes over a real StreamSecrets gRPC stream and asserts the secret received on the wire, covering the SDS service registration in Serve. - TestTlsSecret_WatchFiresOnProjectedVolumeRotation lays out a kubelet-faithful projected volume, rotates it with the AtomicWriter ..data symlink swap (never writing the bundle in place), and asserts the swap event lands inside the secret's WatchedDirectory and that the secret's certificate filename resolves to the new bundle afterwards. fsnotify moves from indirect to direct in go.mod (already vendored).
|
Verified that envoy is picking up the refreshed cert. Test logs in https://github.com/yufan-su/substrate/blob/router-cert-sds-reload/pr-531-local-verification.md |
4 participants
Otherwise, envoy's serving certificate expires after 24hours.