[Epic]: Component upgrade safety — make version transitions machine-readable

[lockwobr]

Goal

Make "is this component version transition safe?" a machine-readable question with a machine-readable answer, attached to the artifact that encodes the version change.

ADR-021 Component Upgrade Safety (#2343, merged) decides the design. This epic tracks building it.

The motivating case is real and recent. #2333 moved nvsentinel from v1.9.0 to v1.20.0, skipping eleven minor versions, typed Build/CI/tooling with the breaking-change box unchecked. Nothing claimed the upgrade was safe, because nothing asks: a pin says what a new deployment gets, and UAT only ever provisions fresh clusters (provision → CUJ → teardown), so no lane has exercised an upgrade. The jump may well be fine. Nobody recorded an answer.

Acceptance bar: a transition record exists for every pinned component version, aicr upgrade-check reports a verdict for both artifact-to-artifact and cluster-to-artifact comparisons, the coverage gate fails a pin bump that does not touch its record, and a safe verdict cannot be authored without naming what verified it.

Current state

Verified against main at filing.

Piece	State
ADR-021	Merged (30ed651dd)
tools/upgrade-matrix + docs/user/component-version-matrix.md	Landed. Best effort, wired to no gate
recipes/upgrades/	Does not exist. Zero records authored
aicr upgrade-check	Does not exist
Coverage gate	Does not exist. 34 components carry defaultVersion
ownsCRDs	Consumed by one deployer (pkg/bundler/deployer/flux/flux.go); CRDs stay at day-one schema on the other four
Generated wrapper Chart.yaml	Still hardcodes version: 0.1.0, so online mode cannot read a payload version
Helm SDK	Not vendored (vendor/helm.sh absent)

What the matrix tool already measured: 19 transitions across the last 8 releases, 7 needing extra scrutiny (chart-identity change, backwards move, or ≥3 major/minor apart). Roughly a third of real transitions are not ordinary one-version bumps.

Phases

Phase 1 — prerequisites. Independent of each other and of everything below; can run in parallel.

1. Close the ownsCRDs deployer gap (ADR Decision 8). First, because it is a prerequisite for a verdict meaning what it says: a safe transition whose CRDs changed is not safe on a deployer that leaves CRDs at day-one schema, which today is four of five.
2. Wrapper chart versioning (Decision 7). Stamp the AICR version into version: and the payload version into aicr.run/component-version; normalize dev builds to 0.0.0-dev or make dev-env breaks.

Phase 2 — the offline check. This is the whole feature for CI and GitOps.

3. Transition record schema and loader. recipes/upgrades/<component>.yaml, the upgrades.file registry field, semver range matching with strict directionality, the forward-reach ban, the gap check, and a loader that fails closed on an unrecognized apiVersion rather than degrading to unknown.
4. Offline upgrade-check. --from/--to over recipes and bundles, --to optional (re-resolve the artifact's own criteria against current pins), table and JSON output, non-zero exit by default via --fail-on-error.
5. Bundle rendering. Records render into the bundle README filtered to the bundle's deployer.

Phase 3 — cluster-aware.

6. Vendor helm.sh/helm/v4. Lands in make scan, api-diff, the licence allowlist and the vendor tree. Worth landing separately so Phase 2 is not blocked on it.
7. Online mode. Helm release inventory read path, plus the at-risk scan for unmanaged resources.
8. -premigrate folder emission (Decision 4). Only once a real migration needs a hook; the nodewright Helm-adoption case is the first candidate.

Phase 4 — keeping it honest.

9. KWOK upgrade/rollback lane. Synthetic fixture component, per-PR speed.
10. UAT upgrade/rollback lane (Decision 9). Release-to-release; safe transitions are the priority.
11. Coverage gate and allowlist (Decision 10). Lands with the first real records so the allowlist shrinks rather than sitting static.
12. Authoring workflow (Decision 11). CLAUDE.md pin-bump rule gains a transition-record clause (mirrored to AGENTS.md), checklist lands in docs/contributor/, upgrade-supportability becomes a registry admission criterion.

Known gaps carried forward

Recorded in the ADR as limits of the design rather than oversights:

• Chart-identity changes break semver comparison. Two of the last 19 transitions changed chart repository or name (nvidia-dra-driver-gpu → registry.k8s.io in feat(recipes): migrate nvidia-dra-driver-gpu to registry.k8s.io v0.4.0 #1285; nodewright-operator between v0.15.1 and v0.17.0), so the versions are not comparable and range matching reads them as downgrades. No field expresses this yet.
• No pinning gate covers Kustomize. bom-pinning-check is Helm-only, so a mutable Kustomize defaultTag is unchecked. Probably an ADR-006 amendment.

Verification debts

Claims the ADR makes that nobody has checked. Each is cheap and worth closing early, because Decisions 1, 3 and 4 rest on the first one.

• Helm 3 ownership-adoption behaviour against nodewright's mirror controller.
• Helm v4's dependency tree against the licence allowlist (go-licenses check). MPL-2.0 is cleared only via ten per-import-path ignores.
• The Kustomize limb of Decision 4 is reasoned, not demonstrated: the registry has zero Kustomize components.

[mchmarny]

Roadmap reconciliation: confirmed outside v1. Full machine-readable upgrade lifecycle safety remains a post-v1 priority, not a release exit gate.