Skip to content

Added penetration tests important to Telco partners/customers#31283

Open
yogeshahiray wants to merge 8 commits into
openshift:mainfrom
yogeshahiray:penetration-tests
Open

Added penetration tests important to Telco partners/customers#31283
yogeshahiray wants to merge 8 commits into
openshift:mainfrom
yogeshahiray:penetration-tests

Conversation

@yogeshahiray

@yogeshahiray yogeshahiray commented Jun 11, 2026

Copy link
Copy Markdown

Added penetration tests which are very important to Telco partners/customers.

Summary by CodeRabbit

  • Tests
    • Added a comprehensive security penetration test suite for OpenShift clusters that scans for plaintext credentials in logs/configs, validates CNI SELinux labeling, detects SSH key material in Secrets, flags privileged pods in non-system namespaces, inspects sudoers and etcd encryption/TLS and file permissions, verifies monitoring/Prometheus rule posture and security operator presence, and checks registry auth posture, DB-like pods, cluster-admin bindings, NFS volumes, and external registry routes.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci Bot added ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 11, 2026
@openshift-ci

openshift-ci Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Hi @yogeshahiray. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 876476a5-8c89-4179-bfc7-21a88188cf58

📥 Commits

Reviewing files that changed from the base of the PR and between 506a2c4 and 4549d1c.

📒 Files selected for processing (1)
  • test/extended/security/penetration.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/extended/security/penetration.go

Walkthrough

Adds a Ginkgo test suite that runs node- and cluster-scoped security posture scans: password/log scans, CNI SELinux validation, secret/privileged pod checks, sudoers/etcd/Route inspections, monitoring/operator checks, and registry/PV/resource discovery.

Changes

Security Penetration Test Suite

Layer / File(s) Summary
Test Suite Structure and Definitions
test/extended/security/penetration.go
Ginkgo test suite with test cases wiring password/log scanning, SELinux validation, and combined security penetration checks.
Master log and config plaintext scan
test/extended/security/penetration.go
Helpers to run oc debug + grep across node/master log globs and selected YAML/config paths to find plaintext passwords (currently no passwords configured).
CNI discovery and SELinux checks
test/extended/security/penetration.go
Discover CNI directory on a representative node and verify SELinux contexts across nodes via ls -RZ.
Secrets and privileged pod scanning
test/extended/security/penetration.go
Scan all Secrets for SSH private-key-like keys and identify privileged pods outside allowed system namespaces.
Node sudoers detection
test/extended/security/penetration.go
Per-node listing of /etc/sudoers.d/ via debug sessions, filtering known debug/chroot artifacts and reporting unexpected entries.
etcd encryption, Routes, and directory permissions
test/extended/security/penetration.go
Verify APIServer encryption-at-rest is not identity, list Routes without TLS, scan master nodes for world-readable etcd/backup files, and validate /var/lib/etcd permissions with exclusions.
Operator, monitoring, and etcd TLS checks
test/extended/security/penetration.go
Discover security tooling operators via CSVs, fetch audit log profile, check monitoring pod health and PrometheusRules existence, and inspect etcds for TLS config.
Informational checks and registry security
test/extended/security/penetration.go
Count DB-like pods, cluster-admin ServiceAccount bindings, NFS-backed PVs, and read insecure registries from Image config; optionally fetch external image-registry Route host.

🎯 4 (Complex) | ⏱️ ~45 minutes


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 3 warnings, 1 inconclusive)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error penetration.go expectations embed foundInLogs/foundInYamls (%v) from grep -nl matches; matched lines may contain plaintext passwords, only partially redacted. Redact or avoid including grep/yaml match content in failure messages (e.g., omit output entirely or fully sanitize matched lines), and report only counts/paths.
Docstring Coverage ⚠️ Warning Docstring coverage is 47.62% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Structure And Quality ⚠️ Warning penetration.go has 20 o.Expect(err).NotTo(o.HaveOccurred()) without messages, 0 BeforeEach/AfterEach, and several It blocks combine multiple behaviors (e.g., log+YAML, etcd encryption+file perms). Use BeforeEach/AfterEach for setup/cleanup, split It blocks so each covers one behavior, and add descriptive messages to every Expect(err).NotTo(HaveOccurred()).
Microshift Test Compatibility ⚠️ Warning In penetration.go, unguarded tests include TestMonitoringStackHealthy (uses monitoring.coreos.com PrometheusRules + namespace openshift-monitoring) and TestNoUnexpectedPrivilegedPods (helper refere... MicroShift compatibility notice: add [apigroup:...] tags or guard with exutil.IsMicroShiftCluster()+g.Skip for TestMonitoringStackHealthy and TestNoUnexpectedPrivilegedPods, and avoid/conditionalize openshift-kube-* namespace references...
Title check ❓ Inconclusive The title 'Added penetration tests important to Telco partners/customers' is vague about what specific security posture checks or hardening validations are being introduced. Provide a more descriptive title that references the specific category of security checks (e.g., 'Add OpenShift security posture validation tests for Telco requirements' or similar) to clarify the scope of the 850+ lines of new test code.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed In penetration.go, all Ginkgo It/Describe titles are fixed string literals (no fmt.Sprintf/vars, node/ns names, IPs, or random suffixes); only assertions use dynamic values inside test bodies.
Single Node Openshift (Sno) Test Compatibility ✅ Passed penetration.go loops over nodes/master nodes but only asserts counts >0 (no >1/==2), and contains no affinity/topology spread, drain/reschedule, or leader-failover assumptions; no missing SNO guard...
Topology-Aware Scheduling Compatibility ✅ Passed Inspected test/extended/security/penetration.go: no pod scheduling constraints (anti-affinity/topologySpread/maxUnavailable/nodeSelector for placement); only node LabelSelectors and ControlPlaneTop...
Ote Binary Stdout Contract ✅ Passed penetration.go has no fmt.Print/Printf/Println, no klog/log.SetOutput, and no os.Stdout usage anywhere in the top-level suite initializer; only fmt.Sprintf-style string building.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed In test/extended/security/penetration.go, the new Ginkgo tests have 0 hardcoded IPv4 literals/CIDRs and no net.ParseIP/CIDR, localhost, or URL/curl/wget usage; they only run cluster-internal oc/kub...
No-Weak-Crypto ✅ Passed Scanned test/extended/security/penetration.go and other *.go files for MD5/SHA1/DES/RC4/3DES/Blowfish/ECB and crypto/subtle/bytes.Equal; none found.
Container-Privileges ✅ Passed In test/extended/security/penetration.go, there are no manifest-like fields for hostPID/hostNetwork/hostIPC/allowPrivilegeEscalation/SYS_ADMIN; only code counts existing privileged pods via Securit...
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from p0lyn0mial and sjenning June 11, 2026 03:15
@openshift-ci

openshift-ci Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yogeshahiray
Once this PR has been reviewed and has the lgtm label, please assign jogeo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Nitpick comments (1)
test/extended/security/penetration.go (1)

282-286: 💤 Low value

Consider making the regex more specific.

The pattern drwxr[A-Za-z0-9\s\.\-]+(/usr/[a-z0-9/]+) is quite permissive in the first part. While the captured path group [a-z0-9/]+ safely prevents injection, the leading pattern could match unintended output. Consider making it more specific to match only expected ls -ld permission strings.

📝 Example of more specific pattern
-	re := regexp.MustCompile(`drwxr[A-Za-z0-9\s\.\-]+(/usr/[a-z0-9/]+)`)
+	// Match: drwxr-xr-x. <number> <user> <group> ... /usr/...
+	re := regexp.MustCompile(`^drwxr-?x?r-?x?[.-]\s+\d+\s+\S+\s+\S+\s+.*?(/usr/libexec/cni|/opt/cni)`)

This more precisely matches the ls -ld output format and explicitly looks for the expected CNI paths.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 282 - 286, The current
regex assigned to re is too permissive; narrow it to match a precise ls -ld
permission line and restrict the captured path to expected CNI locations. Update
the pattern used by re (the string `drwxr[A-Za-z0-9\s\.\-]+(/usr/[a-z0-9/]+)`)
to explicitly match permission bits (e.g., ^d[rwx-]{9}), whitespace-separated
columns (inode/owner/group/size/date), then capture only known /usr subpaths
(for example /usr/bin, /usr/lib, /usr/libexec, etc.) with a conservative
character class for the path; keep the code flow using matches :=
re.FindStringSubmatch(output) and the existing conditional that returns
matches[1] when present.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/security/penetration.go`:
- Around line 234-236: The testPasswords slice is empty so the YAML-password
scanning never runs; populate testPasswords with actual values (either load from
the cluster configuration or supply a sensible default list of common/test
passwords) and reuse it in the YAML check function (the slice named
testPasswords used in the YAML scan) and in checkLogsForPasswords so both checks
operate on the same non-empty password list; update the code that builds
testPasswords and ensure both the YAML scanner and checkLogsForPasswords
reference that populated slice.
- Around line 192-194: The test currently defines an empty slice testPasswords
which prevents the nested verification loops in the penetration test from
running; either populate testPasswords with representative password strings used
in the cluster (e.g., common service/account passwords or test fixtures) so the
loops in the test exercise log-scanning, or explicitly skip the test when it’s
not configured by calling e2eskipper.Skipf (or the existing skip helper) early
in the test; update the code paths around testPasswords and the test function in
test/extended/security/penetration.go (the loops that scan logs) to use the
populated slice or to return after Skipf so the test no longer silently passes.
- Around line 627-643: The current verifyEtcdUsesTLS logic inspects etcd.Object
by converting spec and status maps to strings (fmt.Sprintf("%v", spec/status))
and substring-searching for "cert"/"tls", which is fragile; replace this with
structured checks: retrieve spec and status via unstructured.NestedMap (as you
already do), then explicitly inspect known TLS-related keys (e.g., spec["tls"],
spec["clientTLS"], spec["serverTLS"], spec["peerTLS"], spec["backup"] or the
etcd-operator fields like "tls", "tlsClientConfig", "certFile", "keyFile",
"caFile") and iterate nested maps/slices to detect boolean flags or presence of
certificate file fields; update verifyEtcdUsesTLS to return true only when those
concrete fields exist/are enabled rather than on loose substring matches of
fmt.Sprintf output.
- Line 54: Before calling findCNIPath(oc, nodes.Items[0].Name) add a defensive
check that nodes.Items is non-empty; if len(nodes.Items) == 0, handle the case
(return an error from the surrounding test function or log and fail the test)
instead of indexing [0]. Locate the call to findCNIPath and the variable nodes
in the same test function in test/extended/security/penetration.go and add the
guard so the code never dereferences nodes.Items[0] when the node list is empty.
- Around line 247-253: The code builds a shell command via fmt.Sprintf("grep -nl
'%s' %s") (cmd) and then runs it through "/bin/bash -c", which allows command
injection; change the call in this block so you do not invoke a shell or
interpolate pwd into a single-quoted string. Replace the Sprintf/"/bin/bash -c"
approach by passing grep and its arguments directly to
oc.AsAdmin().Run("debug").Args (e.g., use "/bin/grep", "-nl", pwd, yamlPath) so
pwd and yamlPath are separate Args instead of being concatenated into cmd;
update the code around the cmd variable and the oc.AsAdmin().Run("debug").Args
invocation to use these explicit args.
- Around line 91-92: The shared ctx := context.Background() declared at the
Describe-level is reused across multiple It test cases (the It blocks such as
"TestEtcdBackupEncryptionAndRestriction
[apigroup:config.openshift.io][apigroup:operator.openshift.io]" and the
subsequent It blocks) which breaks test isolation; remove the top-level ctx
declaration and instead add a fresh ctx := context.Background() as the first
statement inside each It block that currently references ctx (all It blocks
between lines 92-181), ensuring each test case obtains its own context.
- Around line 211-217: The grep command is vulnerable to shell injection because
pwd is interpolated into a bash -c string; update the
oc.AsAdmin().Run("debug").Args(...).Output() invocation that builds cmd so it
does not pass an interpolated shell string. Fix by invoking grep without using
"/bin/bash -c" and pass the pattern and logPath as separate arguments (use pwd
and logPath as raw args), or if shell usage is unavoidable, escape/quote pwd
with a robust shell-escaping utility (e.g., use a library that safely quotes
shell arguments) before formatting into the command; ensure the change targets
the code building cmd and the oc.AsAdmin().Run(...).Args call that executes it.

---

Nitpick comments:
In `@test/extended/security/penetration.go`:
- Around line 282-286: The current regex assigned to re is too permissive;
narrow it to match a precise ls -ld permission line and restrict the captured
path to expected CNI locations. Update the pattern used by re (the string
`drwxr[A-Za-z0-9\s\.\-]+(/usr/[a-z0-9/]+)`) to explicitly match permission bits
(e.g., ^d[rwx-]{9}), whitespace-separated columns (inode/owner/group/size/date),
then capture only known /usr subpaths (for example /usr/bin, /usr/lib,
/usr/libexec, etc.) with a conservative character class for the path; keep the
code flow using matches := re.FindStringSubmatch(output) and the existing
conditional that returns matches[1] when present.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 7cc44443-f610-468e-a69f-5d3eae92b3dc

📥 Commits

Reviewing files that changed from the base of the PR and between af43b92 and c234f4c.

📒 Files selected for processing (1)
  • test/extended/security/penetration.go

Comment thread test/extended/security/penetration.go
Comment thread test/extended/security/penetration.go Outdated
Comment thread test/extended/security/penetration.go
Comment thread test/extended/security/penetration.go Outdated
Comment thread test/extended/security/penetration.go
Comment thread test/extended/security/penetration.go Outdated
Comment thread test/extended/security/penetration.go Outdated
@neisw

neisw commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

/test e2e-aws-ovn-microshift

@neisw

neisw commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

@openshift-ci

openshift-ci Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

@neisw: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/39651d60-6590-11f1-8635-10ef9a19d878-0

Comment thread test/extended/security/penetration.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (5)
test/extended/security/penetration.go (5)

124-131: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Sampling only masterNodes.Items[0] leaves master-local etcd permission issues undetected.

Both specs validate a single control-plane node, but backup artifacts and /var/lib/etcd permissions can drift per node. If the first master is clean and another is not, these tests still pass. Iterate over all listed masters before asserting success.

Also applies to: 157-164

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 124 - 131, The test only
checks masterNodes.Items[0] which can miss per-node etcd permission issues;
update the code that calls findWorldReadableCriticalEtcdFiles(oc,
masterNodes.Items[0].Name) to iterate over all masterNodes.Items, call
findWorldReadableCriticalEtcdFiles for each node (using
masterNodes.Items[i].Name), collect/merge any returned critical files, and
assert the aggregated result is empty (or assert per-node emptiness) using the
existing o.Expect checks; apply the same iteration/fix to the duplicate check
that uses findWorldReadableCriticalEtcdFiles later in the file.

328-350: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

findCNIPath can skip valid nodes and can never correctly return /opt/cni.

ls -ld /opt/cni /usr/libexec/cni exits non-zero when either path is missing, so a node with only one valid CNI location hits Line 336 and the test skips. Even when parsing succeeds, the regex only matches /usr/..., and the fallback hardcodes /usr/libexec/cni, so /opt/cni is effectively unreachable. Probe each candidate independently and return the one that actually exists.


252-279: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Literal *.log / *.yaml arguments mean most of these grep scans never touch real files.

Passing /var/log/containers/*.log or /etc/kubernetes/manifests/*.yaml as raw args to /bin/grep does not expand the glob, so grep sees a literal filename and returns an error. The current err == nil gate then treats that execution failure as “nothing found”, which is another false-clean result. Enumerate matching files first and search them with literal matching (grep -F --), rather than handing wildcard patterns straight to grep.

Also applies to: 298-319

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 252 - 279, The grep calls
loop over logPaths (e.g., "/var/log/containers/*.log") with
oc.AsAdmin().Run("debug") but pass unexpanded globs to /bin/grep, so grep sees
literal filenames and errors are treated as "not found"; fix by enumerating
matching files on the node before grepping (e.g., run a safe listing like ls -1
or find via oc debug for each pattern), then invoke /bin/grep -F -- <pwd>
<eachFoundFile> (literal matching and explicit file args) and append to
foundPasswords only when grep succeeds; apply the same change to the similar
block referenced at 298-319.

466-475: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

A node that cannot be inspected is currently treated as “no unexpected sudoers files”.

continue on the debug-command error drops that node from the result set, so this spec can pass even when one of the nodes was never checked. Surface the error, or at least record the node as an inspection failure instead of silently skipping it. As per coding guidelines, Go code should never ignore error returns.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 466 - 475, The code
currently ignores errors from oc.AsAdmin().Run("debug").Args(...).Output() (the
block using node.Name and cmd) by doing if err != nil { continue }, which
silently skips nodes; fix this by surfacing the error or recording the node as
an inspection failure instead of continuing. Replace the continue with error
handling that logs or returns the error (include node.Name and err in the
message) or append node.Name to a failures slice (e.g., failedNodes) so the spec
can fail or assert on failedNodes after the loop; ensure you handle the Output()
error from oc.AsAdmin().Run("debug").Args(...) rather than ignoring it.

Source: Coding guidelines


447-455: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Privileged init and ephemeral containers bypass this check.

This only inspects pod.Spec.Containers. A privileged initContainer or ephemeralContainer is just as relevant for a penetration test and will currently be missed, so the spec can pass while a namespace still runs privileged workload.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 447 - 455, The current
check only iterates pod.Spec.Containers and misses privileged init or ephemeral
containers; update the logic that builds privilegedPods to also iterate
pod.Spec.InitContainers and pod.Spec.EphemeralContainers and perform the same
nil and *SecurityContext.Privileged checks for each element, appending
fmt.Sprintf("%s/%s", pod.Namespace, pod.Name) to privilegedPods and breaking
once any privileged container (regular, init, or ephemeral) is found; ensure you
reference the same variable names (pod, privilegedPods) and mirror the existing
condition used for pod.Spec.Containers.
♻️ Duplicate comments (1)
test/extended/security/penetration.go (1)

243-249: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

These plaintext-password tests still pass without scanning anything.

Both helpers return an empty result when testPasswords is empty, and the callers assert that emptiness as success. That makes the suite report a clean result even though no password exposure check ran. Either load real inputs or explicitly Skipf/fail the spec until the source of passwords exists.

Also applies to: 289-295

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 243 - 249, The
plaintext-password check currently treats an empty testPasswords slice as a
successful pass; update the logic in the penetration test to not silently
succeed when no inputs exist: in the block where testPasswords is declared and
where the helper returns early (the code around the testPasswords variable and
the function that returns foundPasswords), if testPasswords is empty call
t.Skipf (or t.Fatalf if you prefer a hard failure) with an explanatory message
indicating passwords are not configured, or load real test inputs from cluster
config; apply the same change to the similar block around lines 289-295 so the
spec is skipped/failed instead of passing when no passwords are provided,
referencing the testPasswords variable and the helper that returns
foundPasswords.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@test/extended/security/penetration.go`:
- Around line 124-131: The test only checks masterNodes.Items[0] which can miss
per-node etcd permission issues; update the code that calls
findWorldReadableCriticalEtcdFiles(oc, masterNodes.Items[0].Name) to iterate
over all masterNodes.Items, call findWorldReadableCriticalEtcdFiles for each
node (using masterNodes.Items[i].Name), collect/merge any returned critical
files, and assert the aggregated result is empty (or assert per-node emptiness)
using the existing o.Expect checks; apply the same iteration/fix to the
duplicate check that uses findWorldReadableCriticalEtcdFiles later in the file.
- Around line 252-279: The grep calls loop over logPaths (e.g.,
"/var/log/containers/*.log") with oc.AsAdmin().Run("debug") but pass unexpanded
globs to /bin/grep, so grep sees literal filenames and errors are treated as
"not found"; fix by enumerating matching files on the node before grepping
(e.g., run a safe listing like ls -1 or find via oc debug for each pattern),
then invoke /bin/grep -F -- <pwd> <eachFoundFile> (literal matching and explicit
file args) and append to foundPasswords only when grep succeeds; apply the same
change to the similar block referenced at 298-319.
- Around line 466-475: The code currently ignores errors from
oc.AsAdmin().Run("debug").Args(...).Output() (the block using node.Name and cmd)
by doing if err != nil { continue }, which silently skips nodes; fix this by
surfacing the error or recording the node as an inspection failure instead of
continuing. Replace the continue with error handling that logs or returns the
error (include node.Name and err in the message) or append node.Name to a
failures slice (e.g., failedNodes) so the spec can fail or assert on failedNodes
after the loop; ensure you handle the Output() error from
oc.AsAdmin().Run("debug").Args(...) rather than ignoring it.
- Around line 447-455: The current check only iterates pod.Spec.Containers and
misses privileged init or ephemeral containers; update the logic that builds
privilegedPods to also iterate pod.Spec.InitContainers and
pod.Spec.EphemeralContainers and perform the same nil and
*SecurityContext.Privileged checks for each element, appending
fmt.Sprintf("%s/%s", pod.Namespace, pod.Name) to privilegedPods and breaking
once any privileged container (regular, init, or ephemeral) is found; ensure you
reference the same variable names (pod, privilegedPods) and mirror the existing
condition used for pod.Spec.Containers.

---

Duplicate comments:
In `@test/extended/security/penetration.go`:
- Around line 243-249: The plaintext-password check currently treats an empty
testPasswords slice as a successful pass; update the logic in the penetration
test to not silently succeed when no inputs exist: in the block where
testPasswords is declared and where the helper returns early (the code around
the testPasswords variable and the function that returns foundPasswords), if
testPasswords is empty call t.Skipf (or t.Fatalf if you prefer a hard failure)
with an explanatory message indicating passwords are not configured, or load
real test inputs from cluster config; apply the same change to the similar block
around lines 289-295 so the spec is skipped/failed instead of passing when no
passwords are provided, referencing the testPasswords variable and the helper
that returns foundPasswords.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4881a8d4-aec9-48cc-ab99-4e0c9b6f870b

📥 Commits

Reviewing files that changed from the base of the PR and between 94ae41c and 1d9d9e3.

📒 Files selected for processing (1)
  • test/extended/security/penetration.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
test/extended/security/penetration.go (3)

267-272: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Restore wildcard expansion and literal matching in the password scans.

Passing *.log and *.yaml paths as raw grep arguments means those wildcards are never expanded, so these checks usually return exit code 2 and silently miss the files they are supposed to scan. grep is also treating pwd as a regex, so passwords containing regex metacharacters can produce false positives or misses. Expand the file list separately and search with grep -F -- so the scans stay both safe and functional.

Suggested fix
-				output, err := oc.AsAdmin().Run("debug").Args(
-					fmt.Sprintf("node/%s", node.Name),
-					"--",
-					"/bin/grep", "-nl", pwd, logPath,
-				).Output()
+				output, err := oc.AsAdmin().Run("debug").Args(
+					fmt.Sprintf("node/%s", node.Name),
+					"--",
+					"/bin/sh", "-c",
+					`for f in $1; do [ -e "$f" ] && /bin/grep -F -nl -- "$2" "$f"; done`,
+					"_", logPath, pwd,
+				).Output()

Apply the same pattern to the YAML scan as well.

As per coding guidelines, "Command: no shell=True, os.system, or backtick exec with user input".

Also applies to: 307-312

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 267 - 272, The grep
invocation in oc.AsAdmin().Run("debug").Args (where you pass pwd and logPath) is
feeding unexpanded wildcards and a regex pattern to grep; change it to first
expand the target file globs (e.g., expand logPath like "*.log" and the YAML
glob into an explicit list of files) on the node before invoking grep, then call
grep with fixed-string mode and explicit end-of-options (use "-F" and "--") so
the password string (pwd) is treated literally; update the same pattern for the
YAML scan too (the other oc.AsAdmin().Run("debug").Args site mentioned) so both
scans expand globs and use grep -F -- with expanded file lists.

Source: Coding guidelines


725-731: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

countDatabasePods can count the same pod more than once.

The break only exits the dbImages loop. If a pod has two matching containers, count is incremented twice even though the helper comment and log message both describe a pod count. Use a labeled continue or a per-pod boolean so each pod contributes at most once.

Suggested fix
-	for _, pod := range pods.Items {
+podLoop:
+	for _, pod := range pods.Items {
 		for _, container := range pod.Spec.Containers {
 			lowerImage := strings.ToLower(container.Image)
 			for _, dbType := range dbImages {
 				if strings.Contains(lowerImage, dbType) {
 					count++
-					break // Count each pod only once
+					continue podLoop
 				}
 			}
 		}
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 725 - 731, The loop in
countDatabasePods is incrementing count once per matching container because the
break only exits the dbImages loop; change the logic so each pod increments
count at most once by introducing a per-pod boolean (e.g., found) that is set
when any container image matches dbImages and then break out of the container
loop or use a labeled continue to skip to the next pod; update the use of
pods.Items, pod.Spec.Containers, dbImages and count accordingly so the helper
truly counts pods, not containers.

451-456: ⚠️ Potential issue | 🟠 Major

Privileged-pod test misses privileged init/ephemeral containers

countPrivilegedPodsInUserNamespaces only checks pod.Spec.Containers[*].SecurityContext.Privileged (lines 451-458), so pods with privileged initContainers or ephemeralContainers can evade TestNoUnexpectedPrivilegedPods. Update the helper to scan those container types as well, while still counting each pod only once.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/security/penetration.go` around lines 451 - 456, The helper
countPrivilegedPodsInUserNamespaces currently only inspects pod.Spec.Containers
for SecurityContext.Privileged; update it to also iterate
pod.Spec.InitContainers and pod.Spec.EphemeralContainers and check each
container's SecurityContext != nil and SecurityContext.Privileged != nil &&
*SecurityContext.Privileged, incrementing count and breaking out once per pod
(same behavior as for pod.Spec.Containers) so pods with privileged init or
ephemeral containers are counted exactly once; reuse the existing loop/break
pattern and nil checks to avoid panics.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/security/penetration.go`:
- Around line 194-200: The tests named TestNoUnprotectedDatabasePods,
TestNoUnexpectedClusterAdminServiceAccounts, and TestNoNFSVolumesRisk currently
only log via g.By(...) when a non-zero risk is detected, so CI doesn't fail;
replace the informational g.By(...) calls with explicit assertions (e.g., use
g.Expect(dbPodCount).To(BeZero(), "found %d database pod(s) - verify credentials
use Secrets", dbPodCount)) so the test fails when risk is detected; update the
occurrences around TestNoUnprotectedDatabasePods (calling countDatabasePods),
TestNoUnexpectedClusterAdminServiceAccounts (calling countClusterAdminSAs or
similar), and TestNoNFSVolumesRisk (calling countNFSVolumes or similar) to
assert counts are zero (or rename tests to make them discovery-only if you
prefer a non-failing check).

---

Outside diff comments:
In `@test/extended/security/penetration.go`:
- Around line 267-272: The grep invocation in oc.AsAdmin().Run("debug").Args
(where you pass pwd and logPath) is feeding unexpanded wildcards and a regex
pattern to grep; change it to first expand the target file globs (e.g., expand
logPath like "*.log" and the YAML glob into an explicit list of files) on the
node before invoking grep, then call grep with fixed-string mode and explicit
end-of-options (use "-F" and "--") so the password string (pwd) is treated
literally; update the same pattern for the YAML scan too (the other
oc.AsAdmin().Run("debug").Args site mentioned) so both scans expand globs and
use grep -F -- with expanded file lists.
- Around line 725-731: The loop in countDatabasePods is incrementing count once
per matching container because the break only exits the dbImages loop; change
the logic so each pod increments count at most once by introducing a per-pod
boolean (e.g., found) that is set when any container image matches dbImages and
then break out of the container loop or use a labeled continue to skip to the
next pod; update the use of pods.Items, pod.Spec.Containers, dbImages and count
accordingly so the helper truly counts pods, not containers.
- Around line 451-456: The helper countPrivilegedPodsInUserNamespaces currently
only inspects pod.Spec.Containers for SecurityContext.Privileged; update it to
also iterate pod.Spec.InitContainers and pod.Spec.EphemeralContainers and check
each container's SecurityContext != nil and SecurityContext.Privileged != nil &&
*SecurityContext.Privileged, incrementing count and breaking out once per pod
(same behavior as for pod.Spec.Containers) so pods with privileged init or
ephemeral containers are counted exactly once; reuse the existing loop/break
pattern and nil checks to avoid panics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 275ca5b4-bce4-4c29-86bf-7965200fb8e3

📥 Commits

Reviewing files that changed from the base of the PR and between 1d9d9e3 and 26324d3.

📒 Files selected for processing (1)
  • test/extended/security/penetration.go

Comment thread test/extended/security/penetration.go Outdated
@yogeshahiray yogeshahiray requested a review from neisw June 11, 2026 14:03
@yogeshahiray

Copy link
Copy Markdown
Author

I have addressed all the comments

@sosiouxme sosiouxme changed the title Added penetrstion tests important to Telco partners/customers Added penetration tests important to Telco partners/customers Jun 11, 2026
@sosiouxme

Copy link
Copy Markdown
Member

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 12, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci

openshift-ci Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

@yogeshahiray: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-microshift 4549d1c link true /test e2e-aws-ovn-microshift
ci/prow/e2e-vsphere-ovn-upi 4549d1c link true /test e2e-vsphere-ovn-upi
ci/prow/e2e-aws-ovn-fips 4549d1c link true /test e2e-aws-ovn-fips
ci/prow/e2e-gcp-ovn 4549d1c link true /test e2e-gcp-ovn
ci/prow/e2e-vsphere-ovn 4549d1c link true /test e2e-vsphere-ovn
ci/prow/e2e-metal-ipi-ovn-ipv6 4549d1c link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt

openshift-trt Bot commented Jun 12, 2026

Copy link
Copy Markdown

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: 4549d1c

Job Name New Test Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestMonitoringStackHealthy [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedPrivilegedPods [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestProperNodeSudoConfiguration [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestAllRoutesUseTLS [apigroup:route.openshift.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestEtcdBackupEncryptionAndRestriction [apigroup:config.openshift.io][apigroup:operator.openshift.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedClusterAdminServiceAccounts [apigroup:rbac.authorization.k8s.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedPrivilegedPods [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestProperNodeSudoConfiguration [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestSecurityToolingInstalled [apigroup:operators.coreos.com] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestAllRoutesUseTLS [apigroup:route.openshift.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestEtcdBackupEncryptionAndRestriction [apigroup:config.openshift.io][apigroup:operator.openshift.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoNFSVolumesRisk [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedClusterAdminServiceAccounts [apigroup:rbac.authorization.k8s.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedPrivilegedPods [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestProperNodeSudoConfiguration [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestSecurityToolingInstalled [apigroup:operators.coreos.com] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestEtcdBackupEncryptionAndRestriction [apigroup:config.openshift.io][apigroup:operator.openshift.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedClusterAdminServiceAccounts [apigroup:rbac.authorization.k8s.io] [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedPrivilegedPods [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn High - "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestProperNodeSudoConfiguration [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
(...showing 20 of 26 rows)

New tests seen in this PR at sha: 4549d1c

  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestAllRoutesUseTLS [apigroup:route.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 2, Fail: 2, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestContainerRegistryAuthentication [apigroup:config.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 4, Fail: 0, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestEtcdBackupEncryptionAndRestriction [apigroup:config.openshift.io][apigroup:operator.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 0, Fail: 4, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestEtcdDirectoryPermissions [apigroup:operator.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 4, Fail: 0, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestMonitoringStackHealthy [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 4, Fail: 1, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNetworkTrafficEncrypted [apigroup:operator.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 4, Fail: 0, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoNFSVolumesRisk [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 4, Fail: 1, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoSSHKeysInUnexpectedSecrets [apigroup:security.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 4, Fail: 0, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedClusterAdminServiceAccounts [apigroup:rbac.authorization.k8s.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 0, Fail: 4, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnexpectedPrivilegedPods [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 0, Fail: 5, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestNoUnprotectedDatabasePods [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 5, Fail: 0, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestProperNodeSudoConfiguration [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 0, Fail: 5, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] Security Penetration Tests TestSecurityToolingInstalled [apigroup:operators.coreos.com] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 0, Fail: 4, Flake: 0]
  • "[sig-auth][Feature:SecurityPenetration] TestNoPasswordExposedInLogFiles [apigroup:config.openshift.io] [Suite:openshift/conformance/parallel]" [Total: 4, Pass: 4, Fail: 0, Flake: 0]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Indicates a non-member PR verified by an org member that is safe to test. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants