A Go CLI that automates the OpenShift 4 cluster upgrade pre-checks described
in Red Hat KCS solution
7004992. It connects directly
to your cluster's Kubernetes API (no oc binary required), runs 20+ checks
concurrently, and prints a PASS/WARN/FAIL/SKIP/INFO report you can review
before scheduling an upgrade. Every WARN/FAIL result comes with a Solution:
line telling you what to actually do about it, not just what was found.
Nearly everything KCS 7004992 calls out is covered by a real, live check
against cluster state — including several items (FeatureGate status, cgroup
mode, control-plane node labels, mirrored-registry detection, deprecated
catalog usage, reserved monitoring labels) that aren't literally spelled out
as oc get commands in the article but are checkable via the API and matter
just as much. Only a small, clearly-labeled reminders check remains for
the handful of items that are inherently human/process actions no API query
can verify (opening a proactive support case, reading release notes, HAProxy
response-header behavior, LDAP TLS negotiation, and similar).
It is read-only: every check only lists/gets/watches cluster resources. Nothing is created, modified, or deleted.
The KCS article lays out a long list of things to check before every OpenShift
minor/patch upgrade — node health, ClusterOperators, MachineConfigPools,
PVs/PVCs, pending CSRs, PodDisruptionBudgets, firing alerts, the supported
update path, soon-to-be-removed API usage, and more. Doing this by hand with
a dozen oc get/oc describe commands every time is slow and easy to get
wrong. This tool runs the whole list in a few seconds and gives you a single
report to attach to your change record or proactive support case.
Requires Go 1.22+.
git clone https://github.com/rnandanw/ocp-upgrade-precheck.git
cd ocp-upgrade-precheck
go build -o ocp-upgrade-precheck .Or install directly:
go install github.com/rnandanw/ocp-upgrade-precheck@latestThe tool uses standard kubeconfig resolution: --kubeconfig flag, then
$KUBECONFIG, then ~/.kube/config, then in-cluster config if running as a
pod. Make sure you're logged in (oc login / oc whoami works) before
running it.
# Run every check against the current context
./ocp-upgrade-precheck
# Check readiness for a specific target version (enables update-path,
# removed-API, cgroup-mode, monitoring-label, and reminder checks that are
# version-gated)
./ocp-upgrade-precheck --target-version 4.16.10
# Use a specific kubeconfig/context
./ocp-upgrade-precheck --kubeconfig ~/clusters/prod.kubeconfig --context prod
# Also write a machine-readable report, e.g. to attach to a support case
./ocp-upgrade-precheck --target-version 4.16.10 --json report.json
# Run only a subset of checks
./ocp-upgrade-precheck --only nodes,clusteroperators,mcp
# Skip a noisy/unreachable check, e.g. no network route to Alertmanager
./ocp-upgrade-precheck --skip alertmanager
# List every check name (for use with --only / --skip)
./ocp-upgrade-precheck --list-checks| Flag | Default | Description |
|---|---|---|
--kubeconfig |
(auto) | Path to kubeconfig. Falls back to $KUBECONFIG, then ~/.kube/config, then in-cluster config. |
--context |
(current) | kubeconfig context to use. |
--target-version |
(none) | Target OpenShift version, e.g. 4.16.10. Enables/adjusts the updatepath, removedapi, cgroupmode, monitoringlabels, operatorcompat, and reminders checks. |
--json |
(none) | Also write a machine-readable JSON report to this path. |
--no-color |
false |
Disable ANSI color in terminal output. |
--only |
(none) | Comma-separated check names to run exclusively. |
--skip |
(none) | Comma-separated check names to skip. |
--timeout |
30s |
Per-check timeout. |
--list-checks |
false |
Print all check names and exit. |
| Code | Meaning |
|---|---|
0 |
Every check is PASS/WARN/SKIP/INFO. |
1 |
At least one check FAILed. |
2 |
Setup error (bad flags, couldn't connect to the cluster). |
OpenShift 4 Upgrade Pre-Check Report
Reference: https://access.redhat.com/solutions/7004992
Cluster: https://api.example.com:6443
Target version: 4.16.10
Generated: 2026-08-03 09:12:41 UTC
== Checks ==
[FAIL] clusteroperators 3 cluster operator(s) checked
- storage: Degraded=True (VolumeSnapshotClass reconciliation failing)
Solution: Run `oc get co <name> -o yaml` and `oc adm must-gather` to investigate the Degraded/unavailable operator before upgrading -- a Degraded operator can abort the upgrade partway through.
[PASS] csr 4 CSR(s) checked, 0 pending
[PASS] etcd etcd ClusterOperator healthy
- NodeInstallerProgressing: 3 nodes are at revision 7
[WARN] events 12 Warning event(s) found cluster-wide
Solution: Scan for repeating events on the same object (e.g. FailedScheduling, BackOff, ProbeError) -- those indicate an ongoing problem worth fixing before upgrading, as opposed to a one-off event from routine pod churn.
[PASS] featuregate FeatureSet is Default (no non-default gates enabled)
[PASS] mcp 2 MachineConfigPool(s), 6 node(s) checked
[PASS] nodes 6 node(s) checked
[INFO] operatorcompat 6 installed ClusterServiceVersion(s) (copies across namespaces excluded), 4 Subscription(s) found
- cross-check each operator's compatibility with target OpenShift 4.16.10 using the Operator Update Information Checker
[WARN] pdb 3 PodDisruptionBudget(s) found, manual review recommended (must-gather does not capture these fully)
- openshift-image-registry/image-registry: currentHealthy=0 desiredHealthy=1 disruptionsAllowed=0 -- may block node drains during upgrade
Solution: Scale up the owning workload's replicas, or relax `minAvailable`/`maxUnavailable`, so `disruptionsAllowed` is >0 -- a PDB with zero allowed disruptions can stall or block node drains during the upgrade.
[WARN] pods 142 pod(s) checked, 2 not Running/Completed/Succeeded
- openshift-marketplace/redhat-operators-abcde: ImagePullBackOff
Solution: Run `oc logs`/`oc describe pod` on each listed pod. Stale one-shot pods (old Builds, old installer retries) are usually safe to `oc delete`; CrashLoopBackOff/ImagePullBackOff pods should be root-caused before upgrading since the upgrade will restart many pods cluster-wide.
[PASS] pvpvc 14 PV(s), 14 PVC(s) checked
[INFO] reminders items that require manual verification (not automatable via the cluster API)
- Open a proactive Red Hat support case ahead of the maintenance window with: ...
- Review the release notes for OpenShift 4.16.10: https://docs.openshift.com/container-platform/4.16/release_notes/ocp-4.16-release-notes.html
- [4.14] HAProxy was updated to 2.6; check for duplicate HTTP headers in application responses, which are now rejected.
- [4.15] Verify ServiceAccount token secrets exist as expected for any workload that depends on them.
[PASS] removedapi 38 apirequestcount(s) checked against target 4.16.10
[SKIP] resources insufficient RBAC permissions to run this check
[PASS] thirdparty 4 Subscription(s) checked, 0 from a non-Red-Hat-first-party catalog
[PASS] updatepath 4.16.10 is a directly available update from 4.15.22 on channel stable-4.16
Summary: pass=10 warn=3 fail=1 skip=1 info=2
RESULT: FAIL - resolve the failed checks above before upgrading.
Every WARN/FAIL result includes a Solution: line (omitted below for brevity — see the source for the exact wording) telling you what to actually do about the finding.
| Check | KCS 7004992 item | What it validates | Result semantics |
|---|---|---|---|
nodes |
Node status | Every node's Ready condition and Unschedulable flag |
FAIL on any NotReady/SchedulingDisabled node |
clusteroperators |
Cluster Operators | Every ClusterOperator's Available/Degraded conditions |
FAIL on any non-Available or Degraded operator |
pvpvc |
PV/PVC health | PersistentVolume/PersistentVolumeClaim phases | FAIL on Failed/Pending PVs or Pending/stuck-Terminating PVCs |
mcp |
MachineConfigPools | Every node matches an MCP nodeSelector; no MCP paused; machineCount == readyMachineCount; degradedMachineCount == 0 |
FAIL on any mismatch |
resources |
Resource allocation | Requested CPU/memory per node vs. allocatable capacity | WARN ≥80%, FAIL ≥95% of either resource |
pods |
Pod health | Pods not Running/Succeeded/Completed (including CrashLoopBackOff/ImagePullBackOff masked behind Phase: Running) |
WARN (pod churn is common and not always upgrade-blocking) |
etcd |
etcd health | etcd ClusterOperator's Available/Degraded conditions (best-effort — does not replace an etcdctl member-health check) |
FAIL if Degraded, SKIP if inconclusive |
csr |
Pending CSRs | CertificateSigningRequests with no Approved/Denied condition | WARN on any pending CSR |
pdb |
Pod Disruption Budgets | Every PDB's currentHealthy/desiredHealthy/disruptionsAllowed |
WARN on any PDB that could block a node drain; always lists all PDBs |
alertmanager |
Alertmanager firing alerts | Active Warning/Critical alerts via the in-cluster Alertmanager route (best-effort; needs a bearer-token kubeconfig and network access to the route) | FAIL on critical, WARN on warning, SKIP if unreachable |
events |
Warning events | Cluster-wide type=Warning events, most recent first |
WARN if any exist (informational) |
updatepath |
Update path verification | Whether --target-version is in the ClusterVersion's availableUpdates/conditionalUpdates |
FAIL if not reachable from the current channel, WARN if conditional with unresolved risk |
removedapi |
Removed API check | apirequestcounts.apiserver.openshift.io usage for APIs removed at or before the target minor version, excluding system:* traffic |
FAIL if any non-system consumer used a to-be-removed API in the last 24h |
operatorcompat |
Operator compatibility | Lists installed ClusterServiceVersions/Subscriptions (OLM's per-namespace "copied CSV" duplicates filtered out) | Always INFO — the actual compatibility verdict requires the Operator Update Information Checker |
featuregate |
(not in KCS; general upgrade blocker) | Whether a non-default FeatureSet (e.g. TechPreviewNoUpgrade) is enabled on featuregates.config.openshift.io/cluster |
FAIL if non-default — most such gates block or can't be reverted from |
etcdbackup |
etcd backup | Presence of the Tech Preview automated-backup CRDs (Backup, EtcdBackup) |
INFO if found, WARN otherwise — always reminds to take a fresh manual backup, since no API query can confirm one was taken at the right moment |
disconnected |
Disconnected cluster considerations | Presence of ImageDigestMirrorSet/ImageContentSourcePolicy objects |
PASS if none found, INFO with a mirroring/CVO/OSUS reminder if found |
thirdparty |
Third-party software compatibility | Subscriptions sourced from a catalog other than redhat-operators |
WARN listing non-first-party operators to confirm with the vendor |
cgroupmode |
4.19 cgroup v2 note | spec.cgroupMode on nodes.config.openshift.io/cluster |
FAIL if v1 and target ≥4.19, WARN if v1 or unset otherwise |
gatewayapi |
4.19 Gateway API CRD note | Presence of GatewayClass objects |
PASS if none, INFO listing them if present |
marketplace |
4.20 Red Hat Marketplace deprecation | Subscriptions/CatalogSource referencing redhat-marketplace |
WARN if in use |
monitoringlabels |
4.20 reserved cluster label note |
externalLabels.cluster in the platform/user-workload monitoring ConfigMaps |
WARN if set and target ≥4.20, INFO otherwise |
controlplanelabels |
4.19 control-plane label note | Every node with the legacy node-role.kubernetes.io/master label also has node-role.kubernetes.io/control-plane |
WARN/FAIL on mismatches |
reminders |
Proactive case, release notes, HAProxy/SA-token/LDAP-TLS/cert-policy notes | N/A — these depend on application behavior, external LDAP servers, or process actions no API query can verify | Always INFO; filtered cumulatively by --target-version |
Checks requiring a CRD that isn't installed, or RBAC that isn't granted, degrade to SKIP instead of failing the whole run. gatewayapi and disconnected are exceptions — a missing CRD there means the feature genuinely isn't in use, so it's a legitimate PASS.
Read access to most built-in resources (nodes, pods, PVs/PVCs, ConfigMaps,
events, CSRs, PDBs) plus the OpenShift/OLM custom resources this tool queries
(clusterversions, clusteroperators, machineconfigpools,
apirequestcounts, clusterserviceversions, subscriptions,
catalogsources, routes, featuregates, nodes.config.openshift.io,
imagedigestmirrorsets, imagecontentsourcepolicies, backups,
etcdbackups, gatewayclasses). The built-in cluster-reader ClusterRole
covers all of this. Running with less access is fine — any check that hits a
missing CRD or an RBAC wall reports SKIP instead of failing the run.
main.go CLI flags and orchestration
internal/client/ kubeconfig/in-cluster bootstrap (typed + dynamic clients)
internal/checks/ one file per check, run concurrently with panic/timeout isolation
internal/report/ Result/Report types (incl. Solution), colored terminal printer, JSON export
OpenShift/OLM custom resources are accessed via k8s.io/client-go/dynamic +
unstructured rather than openshift/client-go, keeping the dependency
graph to just k8s.io/{api,apimachinery,client-go}.
go build ./...
go vet ./...
gofmt -l .There's no unit test suite yet — verification is currently build/vet/fmt
plus running the binary against a real cluster (--list-checks and --help
don't need one; every other check does).
This tool is a convenience aid, not a substitute for reading KCS 7004992 itself or opening a proactive support case with Red Hat before a production upgrade. It is not an official Red Hat product.