Skip to content

fix: handle config-reloader secret watch tombstones#2303

Merged
AndrewChubatiuk merged 2 commits into
VictoriaMetrics:masterfrom
immanuwell:fix/config-reloader-secret-tombstone
Jun 17, 2026
Merged

fix: handle config-reloader secret watch tombstones#2303
AndrewChubatiuk merged 2 commits into
VictoriaMetrics:masterfrom
immanuwell:fix/config-reloader-secret-tombstone

Conversation

@immanuwell

@immanuwell immanuwell commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

this fixes a small crash in config-reloader.

when the secret informer misses a delete event and catches it on relist, client-go can send cache.DeletedFinalStateUnknown to DeleteFunc. current code does obj.(*corev1.Secret) and panics, so the reloader falls over.

fix is pretty small:

  • unwrap tombstones before queueing secret events
  • keep the normal *corev1.Secret path the same
  • add a unit test for raw secret and tombstone payloads

repro:

  1. run the secret informer in cmd/config-reloader
  2. let the watch close and miss a Secret delete
  3. on the next relist, client-go delivers cache.DeletedFinalStateUnknown
  4. current DeleteFunc type asserts to *corev1.Secret and crashes

checks:

  • go test ./cmd/config-reloader/...
  • make test

Summary by cubic

Fix a crash in config-reloader when the secret watcher receives tombstone events. We now unwrap cache.DeletedFinalStateUnknown instead of asserting directly to *corev1.Secret.

  • Bug Fixes
    • Added secretFromEvent to handle *corev1.Secret and tombstones (value and pointer) from client-go.
    • Updated Add/Update/Delete handlers to use secretFromEvent and log unexpected types.
    • Added unit tests for raw secret, tombstone variants, and unexpected types.
    • Updated docs/CHANGELOG.md to note the fix.

Written for commit afa7bed. Summary will update on new commits.

Review in cubic

Signed-off-by: immanuwell <pchpr.00@list.ru>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Re-trigger cubic

@AndrewChubatiuk AndrewChubatiuk enabled auto-merge (squash) June 17, 2026 19:01
@AndrewChubatiuk AndrewChubatiuk merged commit d898214 into VictoriaMetrics:master Jun 17, 2026
4 of 5 checks passed
@AndrewChubatiuk

Copy link
Copy Markdown
Contributor

@immanuwell thanks for contribution!

AndrewChubatiuk added a commit that referenced this pull request Jun 18, 2026
* fix: handle config-reloader secret watch tombstones

Signed-off-by: immanuwell <pchpr.00@list.ru>

* added changelog

---------

Signed-off-by: immanuwell <pchpr.00@list.ru>
Co-authored-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
@cubic-dev-ai cubic-dev-ai Bot mentioned this pull request Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants