Skip to content

feat(compute)!: distinguish hard limits from sampled budgets - #5119

Open
shiju-nv wants to merge 3 commits into
NVIDIA:mainfrom
shiju-nv:feat/5111-resource-enforcement/shiju-nv
Open

shiju-nv wants to merge 3 commits into
NVIDIA:mainfrom
shiju-nv:feat/5111-resource-enforcement/shiju-nv

Conversation

@shiju-nv

@shiju-nv shiju-nv commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Show whether a backend enforces a hard limit, reserves resources through a scheduler, or uses a watchdog that checks usage periodically and stops the workload when a budget is exceeded. Add explicit CPU-time and resident-memory budgets, and reject unsupported guarantees before creating resources or changing a retained launch.

Related Issue

Closes #5111. Part of #5114. Coordinates with typed-resource PR #3012; that migration remains separate.

Changes

  • Describe enforcement scope, sampling interval, CPU-time resolution and overshoot. Sampled budgets do not satisfy hard CPU-rate or memory limits.
  • Preserve budgets through templates, CLI and supported SDKs. Clients check public Health acknowledgement before budget-bearing writes; the gateway and driver admit enforcement separately.
  • Recheck current driver guarantees before create, explicit start/resume, automatic restart and startup recovery changes.
  • Use sampled-budget tag 4 in public and compute resource messages, leaving CPU 2 and memory 3 available for feat(api)!: model CPU and memory as typed compute resource requirements #3012. Coordinate the typed CPU/memory migration with that PR's owner.
  • Keep feat(api)!: model CPU and memory as typed compute resource requirements #3012's typed workload migration separately owned. That migration must retain legacy sampled-budget field 4 and copy it into the typed message.

Release Notes

Source-breaking API addition for the next minor release only. Complete raw Rust resource and Health messages need the new fields or defaults; complete TypeScript Health values need supportsSampledResourceBudget: false or a message builder. Upgrade gateway writers together. Maintainers must seed and qualify the minor train before landing; do not merge or backport this into the active patch train. No Release-As override is introduced.

Testing

  • Checks appropriate to the affected code and behavior pass
  • Unit tests added/updated (if applicable)
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@shiju-nv shiju-nv added the test:e2e Requires end-to-end coverage label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

Label test:e2e applied for 9f85846. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. rust-sdk — No explicit risk acceptance found

SandboxSpec gains sampled_budget, while SDK-exposed HealthResponse and resource structs gain fields that existing complete Rust literals omit.

  • Impact: Downstream complete literals and exhaustive patterns stop compiling, including mock/server HealthResponse constructors; client calls using defaults remain source-compatible.
  • Action: Defer these fields to a separately qualified minor train and document sampled_budget: None, default-based literals, and non-exhaustive patterns as applicable.
  • PR: feat(compute)!: distinguish hard limits from sampled budgets #5119

2. typescript-sdk — No explicit risk acceptance found

Raw HealthResponse previously required only the baseline message fields and now additionally requires supportsSampledResourceBudget.

  • Impact: Existing typed HealthResponse literals and mock responses no longer type-check, although callers using generated message builders remain compatible.
  • Action: Preserve the existing raw HealthResponse shape for v0.1.7 and defer the field to a qualified minor train with the documented false-default or builder migration.
  • PR: feat(compute)!: distinguish hard limits from sampled budgets #5119

3. api-behavior — No explicit risk acceptance found

The VM driver previously accepted and ignored portable CPU/memory resources but now rejects them with FailedPrecondition.

  • Impact: Existing resource-bearing VM templates cannot create workloads, and persisted VM sandboxes with those fields cannot start or resume after upgrading.
  • Action: Retain VM no-op resource handling for v0.1.7 and defer rejection until a minor train provides template and persisted-sandbox migration guidance.
  • PR: feat(compute)!: distinguish hard limits from sampled budgets #5119
Review details
  • Execution: complete
  • Candidate: PR #5119 (2c5a884fa22d75d7ff1fcb0a958cf2482cf68876)
  • Baseline: c92c2c7983b1307715f6a922366dd3759e41b35b (c92c2c7983b1307715f6a922366dd3759e41b35b)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@shiju-nv
shiju-nv force-pushed the feat/5111-resource-enforcement/shiju-nv branch from 9f85846 to 0e01b18 Compare November 2, 2026 14:46
Resource requests need to say whether a driver provides a hard limit, a scheduler request or termination after sampled excess. This reports those guarantees and adds explicit CPU-time and resident-memory budgets. Unsupported guarantees are rejected before provisioning or retained-launch state changes.

Signed-off-by: Shiju <shiju@nvidia.com>
@shiju-nv
shiju-nv force-pushed the feat/5111-resource-enforcement/shiju-nv branch from 0e01b18 to ea204f9 Compare November 3, 2026 11:47
shiju-nv and others added 2 commits November 4, 2026 12:41
get.k3s.io serves the latest install.sh from k3s master, so the pinned
checksum broke when upstream changed the script. Download install.sh from
the release tag matching k3s_version instead and pin its checksum.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Shiju <shiju@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Distinguish hard CPU and memory limits from sampled resource budgets

2 participants