Fix noschedule rules for HyperShift Hosted Control Plane

[yuumasato]

Description:

This re-opens the work from #13699 (closed for inactivity) by cherry-picking
@sluetze's commit. It makes two control-plane rules behave correctly on
HyperShift Hosted Control Plane (HCP) clusters:

• master_taint_noschedule: the previous jqfilter iterated
  .spec.taints[] unconditionally, so on hosted clusters — where worker nodes
  have no taints — the api-checks resource filtering errored out
  (couldn't filter '{"kind":"NodeList",...}') and aborted the whole scan. The
  filter is rewritten to use .spec.taints[]? and collect the names of tainted
  master nodes into a list, which evaluates safely on both HCP and regular
  clusters. The rule is also gated with platform: not ocp4-on-hypershift-hosted.
• control_plane_nodes_in_three_zones: gated with
  platform: not ocp4-on-hypershift-hosted, since a hosted cluster's control
  plane runs in the management cluster and this check does not apply.

Rationale:

On HCP clusters these control-plane rules either crashed the scan
(master_taint_noschedule) or produced meaningless results. This makes them
HCP-safe and marks them not-applicable on hosted clusters.

Related:

• Complements CMP-4521: Fix HyperShift Hosted Cluster detection using controlPlaneTopology #15085 (CMP-4521); not a hard dependency. The
  master_taint_noschedule crash fix works independently of it, and the
  platform: not ocp4-on-hypershift-hosted gates added here are inert no-ops
  until that PR lands. CMP-4521: Fix HyperShift Hosted Cluster detection using controlPlaneTopology #15085 fixes the ocp4-on-hypershift-hosted CPE, which
  was structurally unreachable and never fired — as @sluetze documented in
  fix noschedule rules for hcp #13699, none of the platform: not ocp4-on-hypershift-hosted rules returned
  NOT-APPLICABLE on HCP before that fix. Once CMP-4521: Fix HyperShift Hosted Cluster detection using controlPlaneTopology #15085 merges, the gates here begin
  marking these rules NOT-APPLICABLE on hosted clusters. Either PR can merge
  first.
• Same class as CMP-4521's sibling bug
  CMP-4491 (jq Cannot iterate over null from strict array traversal, fixed by adding ? optional operators),
  which was addressed for the audit_error_alert_exists rule by another PR. The
  master_taint_noschedule change here applies the same optional-traversal fix
  pattern to node taints.

Original PR: #13699
Original author: @sluetze

[github-actions]

This datastream diff is auto generated by the check Compare DS/Generate Diff

Click here to see the full diff

Platform has been changed for rule 'xccdf_org.ssgproject.content_rule_control_plane_nodes_in_three_zones'
--- xccdf_org.ssgproject.content_rule_control_plane_nodes_in_three_zones
+++ xccdf_org.ssgproject.content_rule_control_plane_nodes_in_three_zones
@@ -1 +1 @@
-
+oval:ssg-installed_app_is_ocp4_on_hypershift_hosted:def:1

New content has different text for rule 'xccdf_org.ssgproject.content_rule_master_taint_noschedule'.
--- xccdf_org.ssgproject.content_rule_master_taint_noschedule
+++ xccdf_org.ssgproject.content_rule_master_taint_noschedule
@@ -16,9 +16,9 @@
 Therefore, you need to use a tool that can query the OCP API, retrieve the following:
 /api/v1/nodes
     API endpoint, filter with with the jq utility using the following filter
-    .items[] | select(.metadata.labels."node-role.kubernetes.io/master" == "" or .metadata.labels."node-role.kubernetes.io/control-plane" == "" ) | .spec.taints[] | select(.key == "node-role.kubernetes.io/master" and .effect == "NoSchedule")
+    [ .items[] | select(.spec.taints[]?.key == "node-role.kubernetes.io/master" and .spec.taints[]?.effect == "NoSchedule") | .metadata.name ]
     and persist it to the local
-    /api/v1/nodes#17a90876774be6b77ba144848cd6abef7db26fc4cfb1b71314faa3dcda45911c
+    /api/v1/nodes#5f689f16dda05c8639e331492dbe7a5cafed3a0d415f672b2ca6bb220d322870
     file.
 
 [reference]:

OCIL for rule 'xccdf_org.ssgproject.content_rule_master_taint_noschedule' differs.
--- ocil:ssg-master_taint_noschedule_ocil:questionnaire:1
+++ ocil:ssg-master_taint_noschedule_ocil:questionnaire:1
@@ -1,9 +1,9 @@
 Run the following command to see if control planes are schedulable
-$oc get --raw /api/v1/nodes | jq '.items[] | select(.metadata.labels."node-role.kubernetes.io/master" == "" or .metadata.labels."node-role.kubernetes.io/control-plane" == "" ) | .spec.taints[] | select(.key == "node-role.kubernetes.io/master" and .effect == "NoSchedule" )'
-for each master node, there should be an output of a key with the NoSchedule effect.
+$oc get --raw /api/v1/nodes | jq '[ .items[] | select(.spec.taints[]?.key == "node-role.kubernetes.io/master" and .spec.taints[]?.effect == "NoSchedule") | .metadata.name ]'
+for each non-schedulable master node, there should be the name in the output.
 
-By editing the cluster scheduler you can centrally configure the masters as schedulable or not
-by setting .spec.mastersSchedulable to true.
+By editing the cluster scheduler you can centrally configure the masters as not schedulable
+by setting .spec.mastersSchedulable to false.
 Use $oc edit schedulers.config.openshift.io cluster to configure the scheduling.
       Is it the case that Control Plane is schedulable?
       
Platform has been changed for rule 'xccdf_org.ssgproject.content_rule_master_taint_noschedule'
--- xccdf_org.ssgproject.content_rule_master_taint_noschedule
+++ xccdf_org.ssgproject.content_rule_master_taint_noschedule
@@ -1 +1 @@
-
+oval:ssg-installed_app_is_ocp4_on_hypershift_hosted:def:1

[taimurhafeez]

Tested on hypershift cluster 4.22.

oc get clusterversions.config.openshift.io 
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.22.13   True        False         62m     Cluster version is 4.22.13
oc get infrastructure cluster -o jsonpath='{.status.controlPlaneTopology}'
External

before fix, pod crash

oc get pods
NAME                                              READY   STATUS       RESTARTS        AGE
compliance-operator-79fcbf75f5-x9v9k              1/1     Running      1 (44m ago)     44m
ocp4-openshift-compliance-pp-5d89d457bb-m9fsh     1/1     Running      0               6m51s
pr15089-test-api-checks-pod                       0/2     Init:Error   6 (3m14s ago)   6m11s
pr15089-test-rs-7cf5f9c94b-cmtkb                  1/1     Running      0               6m11s
rhcos4-openshift-compliance-pp-78b44f98cb-cljwp   1/1     Running      0               43m

oc get pods -n openshift-compliance | grep api-checks
oc logs pr15089-test-api-checks-pod -n openshift-compliance -c api-resource-collector 2>&1 | grep FATAL
pr15089-test-api-checks-pod                       0/2     Init:CrashLoopBackOff   5 (117s ago)   4m54s
FATAL:Error fetching resources: couldn't filter '{"kind":"NodeList","apiVersion":"v1","metadata":{"resourceVersion":"17860"}...................

Scan results after fix

oc get ccr | grep -E 'three-zones|taint-noschedule'
pr15089-test-control-plane-nodes-in-three-zones                            FAIL     medium
pr15089-test-master-taint-noschedule                                       FAIL     medium

As #15085 is not landed yet, thus, not showing NOT-APPLICABLE result for these rules.