Skip to content

feat(controller): add GPUQuota validating webhook for InferenceService#1118

Merged
Defilan merged 1 commit into
defilantech:mainfrom
Defilan:foreman/revise-1095-r1/issue-1095
Jul 14, 2026
Merged

feat(controller): add GPUQuota validating webhook for InferenceService#1118
Defilan merged 1 commit into
defilantech:mainfrom
Defilan:foreman/revise-1095-r1/issue-1095

Conversation

@Defilan

@Defilan Defilan commented Jul 14, 2026

Copy link
Copy Markdown
Member

What

Adds the GPUQuota validating webhook for InferenceService (internal/controller/inferenceservice_quota_webhook.go), the generated config/webhook manifest and RBAC, and fake-client unit tests. This is the enforcement half of the multi-tenancy epic (#416): it rejects an InferenceService whose admission would exceed an applicable GPUQuota.

InferenceServiceQuotaValidator implements admission.Validator[*InferenceService] (create + update; delete is a no-op). For each GPUQuota whose scope covers the incoming InferenceService's namespace, it computes current usage, builds the incoming request, calls the pure quota.Decide (#1094), and denies with the returned reason if any quota rejects. failurePolicy: fail, sideEffects: None.

Why

Fixes #1095

Types (#1092) and the decision function (#1094) are merged; this is the piece that actually enforces a quota at admission. It calls Decide so the allow/deny rules stay in one place.

How

Usage is computed fresh at admission (list InferenceServices in the quota's scope, sum resources.gpu * replicas, excluding the incoming object so an update does not double-count), so the decision does not depend on the status reconciler (#1117) having run. Selector scope uses metav1.LabelSelectorAsSelector (honors matchLabels + matchExpressions).

Documented deferrals (consistent with the rest of the epic):

  • costBudgetBreached is always false for now: InferCost TokenBudget integration is a follow-up. Commented at the call site.
  • The webhook does not update GPUQuota.Status.AdmissionDenials: a validating webhook is sideEffects: None, so the denial counter is deferred (a metric or a reconciler-observed counter). Commented.
  • Incoming.VRAMBytes is 0: InferenceService exposes no VRAM field (mirrors [FEATURE] GPUQuota status reconciler (#416 story 2) #1093).

Testing / verification

Verified locally in an isolated worktree:

  • go build ./..., go vet ./internal/controller/... ./cmd/... clean
  • go test ./internal/controller/ fake-client unit tests pass: in-quota admit; over-gpuCount deny (with reason); priority-floor deny; a quota that does not cover the namespace is ignored; an update that stays within quota is admitted; and gpu:2 * replicas:3 = 6 counts correctly
  • make manifests generate chart-crds then git status stable: the webhook manifest and RBAC role are regenerated and in sync
  • gofmt clean; golangci-lint --new-from-rev 0 new issues
  • the full controller envtest suite (make test) passed in the clean-room gate

Notes for the reviewer

AI assistance disclosure

Assisted-by: Foreman, the project's local agentic coding harness (coder model qwopus3.6-35B on-prem). Foreman generated the webhook and tests; a review pass I directed found that three of its own "over-gpuCount" tests asserted a deny at exactly-at-cap (which Decide correctly admits) and that gpuCount ignored replicas. Those were fed back as a revision cycle that corrected the tests and the replicas math and added a gpu*replicas case. I reviewed the result and ran the verification above. I own this change and the review conversation; per AGENTS.md the commit carries no attribution trailer and the DCO sign-off is mine.

Checklist

  • Tests added/updated
  • Tests pass locally (go test ./internal/controller/, incl. the full envtest suite in the gate)
  • Lint clean for this change (golangci-lint --new-from-rev 0 new issues; gofmt clean)
  • Commit messages follow conventional commits
  • Commit is signed off (git commit -s) per DCO
  • AI assistance is disclosed above
  • Documentation updated (n/a: internal webhook; user-facing docs land with the Helm toggle in [FEATURE] Helm multitenancy toggle + tenant RBAC (#416 story 5) #1096)

@Defilan Defilan added kind/feature New feature or request component/controller Related to the operator controller area/multi-tenancy Multi-tenancy and resource isolation area/security Security-related issues labels Jul 14, 2026
@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 62.33766% with 29 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ernal/controller/inferenceservice_quota_webhook.go 65.75% 17 Missing and 8 partials ⚠️
cmd/main.go 0.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Defilan Defilan force-pushed the foreman/revise-1095-r1/issue-1095 branch from d4159eb to 2d12716 Compare July 14, 2026 01:23
…cation

Fix 1: Three tests expected DENY when incoming GPU equals the quota cap
(0+4=4, cap=4), but quota.Decide uses an exclusive > comparison:
current+incoming > cap. Since 4 is NOT > 4, the webhook correctly
allows. Changed the three deny tests to genuinely exceed the cap by
setting incoming GPU to 5 (5 > 4 -> deny).

Fix 2: gpuCount returned only isvc.Spec.Resources.GPU, ignoring
replicas. Total GPU usage is gpu-per-pod * replicas (default 1).
Added replica multiplication and two regression tests proving
gpu:2 * replicas:3 = 6 exceeds a quota cap of 5.

Fixes defilantech#1095

Signed-off-by: Foreman Bot <chris@mahercode.io>
@Defilan Defilan force-pushed the foreman/revise-1095-r1/issue-1095 branch from 2d12716 to a7cf2fb Compare July 14, 2026 04:06
@Defilan Defilan merged commit 26804bd into defilantech:main Jul 14, 2026
23 checks passed
@github-actions github-actions Bot mentioned this pull request Jul 14, 2026
eleboucher pushed a commit to eleboucher/homelab that referenced this pull request Jul 14, 2026
…5 ➔ 0.9.6) (#1591)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/defilantech/charts/llmkube](https://github.com/defilantech/LLMKube) | patch | `0.9.5` → `0.9.6` |

---

### Release Notes

<details>
<summary>defilantech/LLMKube (ghcr.io/defilantech/charts/llmkube)</summary>

### [`v0.9.6`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#096-2026-07-14)

[Compare Source](defilantech/LLMKube@v0.9.5...v0.9.6)

##### Features

- **api:** add GPUQuota CRD types for multi-tenant GPU governance ([#&#8203;1101](defilantech/LLMKube#1101)) ([5dd867a](defilantech/LLMKube@5dd867a))
- **controller:** add GPUQuota status reconciler ([#&#8203;1117](defilantech/LLMKube#1117)) ([1d583e4](defilantech/LLMKube@1d583e4))
- **controller:** add GPUQuota validating webhook for InferenceService ([#&#8203;1118](defilantech/LLMKube#1118)) ([26804bd](defilantech/LLMKube@26804bd))
- **controller:** add s3:// model source via curl --aws-sigv4 ([#&#8203;1098](defilantech/LLMKube#1098)) ([#&#8203;1125](defilantech/LLMKube#1125)) ([ed35142](defilantech/LLMKube@ed35142))
- **foreman:** preserve a coder's gate-failed branch instead of discarding it ([#&#8203;1115](defilantech/LLMKube#1115)) ([bc39c77](defilantech/LLMKube@bc39c77))
- **helm:** gate InferenceService quota webhook and tenant RBAC behind multitenancy toggle ([#&#8203;1122](defilantech/LLMKube#1122)) ([26a994e](defilantech/LLMKube@26a994e))
- **quota:** add GPUQuota admission decision function ([#&#8203;1107](defilantech/LLMKube#1107)) ([4897093](defilantech/LLMKube@4897093))
- **runtime:** close out the SGLang kitchen-sink ([#&#8203;1060](defilantech/LLMKube#1060)): minor flags, accept thresholds, typed LoRA adapters, LoRAAdapter CRD ([#&#8203;1103](defilantech/LLMKube#1103)) ([8edf8bd](defilantech/LLMKube@8edf8bd))

##### Bug Fixes

- **foreman:** honor GateProfile source extensions in scope-overlap issue-ref extraction ([#&#8203;1120](defilantech/LLMKube#1120)) ([0c20431](defilantech/LLMKube@0c20431))

##### Documentation

- **samples:** add air-gapped local file-path model source example ([#&#8203;1099](defilantech/LLMKube#1099)) ([#&#8203;1124](defilantech/LLMKube#1124)) ([913ff3d](defilantech/LLMKube@913ff3d))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/1591
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/multi-tenancy Multi-tenancy and resource isolation area/security Security-related issues component/controller Related to the operator controller kind/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] GPUQuota validating webhook wiring (#416 story 4)

1 participant