Skip to content

Improve droplet upload draining - #245

Merged
kathap merged 2 commits into
mainfrom
improve-droplet-upload-draining
Sep 16, 2025
Merged

Improve droplet upload draining#245
kathap merged 2 commits into
mainfrom
improve-droplet-upload-draining

Conversation

@kathap

@kathap kathap commented Jul 18, 2025

Copy link
Copy Markdown
Contributor
  • In main.go signal handler:
    • Refactor main into smaller funcs: createShutdownSignal, configureServers, waitForDrainingToFinish for clearer flow and easier testing.
    • Introduce -shutdownTimeoutInMinutes flag (default 15m) to bound graceful shutdown time.
    • Switch shutdown strategy to Ifrit-only: send os.Interrupt to the group to stop accepting new conns, then wait up to the timeout for in‑flight uploads to complete by racing uploadWaitGroup.Wait() against timeout.
    • Remove direct http.Server management and Server.Shutdown calls; initializeServer now returns an ifrit.Runner only.
    • Keep existing TLS/non-TLS behavior and debug server; preserve TLS12 + cipher suite settings.
    • Replace deprecated ioutil.ReadFile with os.ReadFile.

  • In upload_droplet.ServeHTTP:
    • Retain existing logging and WaitGroup tracking for in-flight requests.
    • replace deprecated CloseNotifier with context-based cancellation:
    • Enhance per-request flow, introduce context.WithTimeout(r.Context(), timeout) to produce a single ctx.Done() channel that:
    • Cancels when the client disconnects (r.Context()).
    • Cancels when the configured timeout elapses.

This ensures that ongoing droplet uploads finish cleanly while preventing new requests during shutdown.

Related PRs:
cloudfoundry/capi-release#562

Related issue:
cloudfoundry/cloud_controller_ng#4351

@kathap
kathap marked this pull request as draft July 18, 2025 07:24
@kathap
kathap force-pushed the improve-droplet-upload-draining branch 2 times, most recently from 07c903e to b5eddc1 Compare July 21, 2025 13:06
@kathap
kathap marked this pull request as ready for review July 24, 2025 07:05
@kathap
kathap marked this pull request as draft July 29, 2025 14:54
@kathap
kathap marked this pull request as ready for review July 30, 2025 14:21

@jochenehret jochenehret left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general, but I still had some questions...

Comment thread cmd/cc-uploader/main.go
Comment thread cmd/cc-uploader/main.go
Comment thread cmd/cc-uploader/main.go Outdated
Comment thread cmd/cc-uploader/main.go Outdated
@kathap
kathap marked this pull request as draft August 14, 2025 08:01
@kathap
kathap marked this pull request as ready for review August 15, 2025 09:39
@kathap
kathap force-pushed the improve-droplet-upload-draining branch from fceb7cf to a5b111a Compare August 15, 2025 14:23

@peanball peanball left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments on the Go code, as requested. I've primarily looked at the changes and new code, not at the existing structure/usage patterns.

Comment thread cmd/cc-uploader/main.go Outdated
Comment thread cmd/cc-uploader/main.go Outdated
Comment thread cmd/cc-uploader/main.go Outdated
Comment thread cmd/cc-uploader/main.go
Comment thread cmd/cc-uploader/main.go
Comment thread cmd/cc-uploader/main.go
Comment thread cmd/cc-uploader/main.go
In main.go signal handler:
• Refactor main into smaller funcs: createShutdownSignal, configureServers, waitForDrainingToFinish for clearer flow and easier testing.
• Introduce -shutdownTimeoutInMinutes flag (default 15m) to bound graceful shutdown time.
• Switch shutdown strategy to Ifrit-only: send os.Interrupt to the group to stop accepting new conns, then wait up to the timeout for in‑flight uploads to complete by racing uploadWaitGroup.Wait() against timeout.
• Remove direct http.Server management and Server.Shutdown calls; initializeServer now returns an ifrit.Runner only.
• Keep existing TLS/non-TLS behavior and debug server; preserve TLS12 + cipher suite settings.
• Replace deprecated ioutil.ReadFile with os.ReadFile.

In upload_droplet.ServeHTTP:
• Retain existing logging and WaitGroup tracking for in-flight requests.
• replace deprecated CloseNotifier with context-based cancellation:
• Enhance per-request flow, introduce context.WithTimeout(r.Context(), timeout) to produce a single ctx.Done() channel that:
• Cancels when the client disconnects (r.Context()).
• Cancels when the configured timeout elapses.

Co-Authored-By: serdarozerr <serdarozer14@gmail.com>
@kathap
kathap force-pushed the improve-droplet-upload-draining branch from 3fd0a4c to 8b2b068 Compare September 10, 2025 19:03
jochenehret
jochenehret previously approved these changes Sep 15, 2025
Comment thread cmd/cc-uploader/main.go Outdated
Comment thread cmd/cc-uploader/main.go Outdated
Comment thread cmd/cc-uploader/main.go Outdated
@kathap
kathap merged commit ac7cc63 into main Sep 16, 2025
2 checks passed
ari-wg-gitbot added a commit to cloudfoundry/capi-release that referenced this pull request Sep 16, 2025
Changes in code.cloudfoundry.org/cc-uploader:

- Improve droplet upload draining
    PR: cloudfoundry/cc-uploader#245
    Author: Katharina Przybill <30441792+kathap@users.noreply.github.com>
@moleske
moleske deleted the improve-droplet-upload-draining branch September 16, 2025 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants