Replace CSI snapshot handle polling with watch when available - #10249
Replace CSI snapshot handle polling with watch when available#10249Pragati5-DEBUG wants to merge 2 commits into
Conversation
👷 Deploy request for velero pending review.Visit the deploys page to approve it
|
f20c700 to
bd0f96e
Compare
Use Kubernetes watch to wait for VolumeSnapshotContent snapshot handle readiness when a WithWatch client is available, falling back to polling only for non-timeout watch failures. Preserve VSS early frequent polling via CSI_SNAPSHOT_EARLY_FREQUENT_POLLING, keep timeout errors from triggering a second poll wait, and add watch-path unit tests. Fixes velero-io#9956 Signed-off-by: Pragati <Pragati5-DEBUG@users.noreply.github.com>
Signed-off-by: Pragati <Pragati5-DEBUG@users.noreply.github.com>
6a47255 to
7798278
Compare
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
0f2a073 to
286ae73
Compare
|
|
||
| // WaitUntilVSCHandleIsReady returns the VolumeSnapshotContent | ||
| // object associated with the volumesnapshot. | ||
| func WaitUntilVSCHandleIsReady( |
There was a problem hiding this comment.
This introduces a critical timeout bug when falling back from Watch to Poll.
Both waitUntilVSCHandleIsReadyWithWatch & waitUntilVSCHandleIsReadyWithPoll create a brand new context.WithTimeout(context.Background(), csiSnapshotTimeout).
If the API closes the watch channel after 5 min, we will fall back to polling, which then starts a brand new 10 min timeout from zero. It will silently breaking the csiSnapshotTimeout contract.
Fix :- WaitUntilVSCHandleIsReady should create a single context.WithTimeout at the top level &
pass the resulting ctx context.Context down into the helpers, instead of passing the duration and recreating the context.
|
@sseago Could you take a look at this PR when you are availble? |
3 participants
Summary
Replace CSI VolumeSnapshotContent handle readiness polling with Kubernetes watch when a WithWatch client is available, reducing apiserver GET traffic during CSI backups.
WaitUntilVSCHandleIsReady now:
Tries a watch-based wait on VolumeSnapshot, then VolumeSnapshotContent
Falls back to polling if watch is unavailable or fails (except on timeout)
Preserves existing VSS behavior via CSI_SNAPSHOT_EARLY_FREQUENT_POLLING (1s polling for the first 10s, then 5s intervals)
The PVC backup item action is updated to use KubebuilderWatchClient() so the watch path is used in production.
Does your change fix a particular issue?
Fixes #9956
Changes
Refactor WaitUntilVSCHandleIsReady into tryPopulateVSCHandle, watchUntilVSCHandleReady, and poll/watch wrappers
Switch pvcBackupItemAction client from crclient.Client to crclient.WithWatch
Update unit tests to use NewFakeControllerRuntimeWatchClient
Testing
Checklist
Accepted the DCO. Commits without the DCO will delay acceptance.
Created a changelog file (make new-changelog) or comment /kind changelog-not-required on this PR.
Updated the corresponding documentation in site/content/docs/main.