Skip to content

feat(vllm-performance): Added logic for K8s environments garbage collection - #1299

Open
christian-pinto wants to merge 2 commits into
mainfrom
cp-gc-envs
Open

feat(vllm-performance): Added logic for K8s environments garbage collection#1299
christian-pinto wants to merge 2 commits into
mainfrom
cp-gc-envs

Conversation

@christian-pinto

@christian-pinto christian-pinto commented Jul 28, 2026

Copy link
Copy Markdown
Member

Summary

Adds idle-environment garbage collection (GC) to the vllm_performance actuator. Previously, Kubernetes deployments released by an experiment were kept alive indefinitely until they were needed again or cleaned up manually. This PR introduces a configurable TTL-based free pool and a background GC loop that automatically deletes stale deployments, with an optional force-delete escalation path for stuck environments.

High-Level Changes

  • Free environment pool with TTL — finished deployments are returned to a pool and reused for compatible experiments. A configurable free_environment_ttl (default 300 s) controls how long an idle deployment may live before GC deletes it. Setting TTL to 0 disables the pool and deletes immediately on release.
  • Background GC loop — a new async loop runs periodically, deleting stale free environments and confirming that previously deleted deployments have actually disappeared from Kubernetes.
  • Force-delete escalation — two new parameters (gc_force_delete, gc_force_delete_threshold) allow automatic force-deletion (clearing finalizers, grace_period=0) of deployments that remain stuck in Kubernetes after repeated GC checks.
  • New K8s operationComponentsManager gains a force_delete_environment method that deletes the service, clears pod finalizers, and force-deletes pods and the deployment.
  • New parametersfree_environment_ttl, gc_force_delete, and gc_force_delete_threshold added to VLLMPerformanceTestParameters with validation.
  • Documentation — user guide updated with a new "Environment free pool and garbage collection" section, parameter descriptions, and configuration examples.
  • Tests — new test file covers the GC loop, TTL behaviour, force-delete escalation, and K8s error handling.

Impact

  • Cluster resource management — idle deployments are now automatically cleaned up, reducing unnecessary cluster resource consumption.
  • Backwards compatible — existing configurations are unaffected; defaults preserve the previous behaviour of keeping deployments alive (TTL=300 s), though deployments will now eventually be garbage-collected rather than held indefinitely.
  • Plugin version bump1.10.3 → 1.10.4 (patch release).

This PR fixes #273

Comment thread docs/user-guide/actuators/vllm-performance.md Outdated
Comment thread docs/user-guide/actuators/vllm-performance.md
Comment thread docs/user-guide/actuators/vllm-performance.md
Comment thread plugins/actuators/vllm_performance/VERSION Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(vllm_performance): garbage collect stray deployments

2 participants