Skip to content

Align tests and schemas#775

Open
hamistao wants to merge 5 commits into
rancher:mainfrom
hamistao:improve_test_schemas
Open

Align tests and schemas#775
hamistao wants to merge 5 commits into
rancher:mainfrom
hamistao:improve_test_schemas

Conversation

@hamistao

Copy link
Copy Markdown
Contributor

As part of a personal Goal, I experimented with using AI to identify discrepancies between our automated tests and their schemas and this PR is a result of trying to fix the identified discrepancies.

The more convoluted fixes here would be:

  • The helm install commands from TestAmbientInstallation and TestGatewayStandaloneInstallation schemas were updated to match the one used on the tests.
  • TestLoadBalancer and TestLoadBalanceScaleAndUpgrade were updated to test the connectivity to the actual Load Balancers instead of the Node port

@hamistao
hamistao requested a review from a team as a code owner July 22, 2026 16:57
@hamistao
hamistao requested a review from Copilot July 22, 2026 16:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to align PIT schemas with existing automated test behavior and to adjust load balancer connectivity tests to validate connectivity against the actual LoadBalancer endpoint rather than NodePort, reducing schema/test drift and improving real-world coverage.

Changes:

  • Refactors workload deployment test table setup to remove redundant per-test create function wiring.
  • Updates PIT schema steps/commands/positions across multiple suites to better match test execution.
  • Updates LoadBalancer tests to validate connectivity via the LoadBalancer service address and introduces a new helper for that validation.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
validation/workloads/workload_test.go Simplifies deployment test case struct by removing redundant create function field.
validation/workloads/schemas/pit_schemas.yaml Adjusts workload PIT schema formatting and scale-step definitions to match test behavior.
validation/upgrade/schemas/pit_schemas.yaml Updates PIT schema formatting and step position numbering for upgrade workload cases.
validation/networking/connectivity/port_test.go Switches LoadBalancer connectivity checks from NodePort to LoadBalancer service endpoint and simplifies test structure.
validation/longhorn/schemas/pit_schemas.yaml Updates Longhorn PIT schema steps to reflect revised replica/scale expectations (with a noted grammar fix needed).
validation/fleet/schemas/pit_schemas.yaml Updates expected namespace wording in Fleet PIT schema to match test expectation.
validation/charts/schemas/pit_schemas.yaml Updates chart PIT schema commands/details (alerting namespace, neuvector chart list) to match tests.
validation/charts/appco/schemas/pit_schemas.yaml Updates Istio AppCo standalone install schema command (currently diverges from test behavior regarding imagePullSecrets).
actions/workloads/deployment/verify.go Fixes replica update logging to reflect the computed target replica count.
actions/networking/verify.go Adds LoadBalancer connectivity helper (currently contains correctness/compilation issues that need fixes).

Comment thread actions/networking/verify.go
Comment thread actions/networking/verify.go
Comment thread validation/longhorn/schemas/pit_schemas.yaml Outdated
Comment thread validation/charts/appco/schemas/pit_schemas.yaml Outdated
Comment thread actions/networking/verify.go

k8sService := &corev1.Service{}
err = steveV1.ConvertToK8sType(service, k8sService)
require.NoError(t, err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually just return the error from the action and verify it in the test. As an example: https://github.com/hamistao/tests/blob/a1642af8087366191e6d61b758e7cf7ea600028e/actions/networking/verify.go#L49

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting that I remove t *testing.T from the parameters? I don't think it makes sense to take in the parameter and not use it here. The example you gave doesn't include such a parameter thus it makes sense for it to return the error.

I decided to include this parameter here so we could have the IP and port in the logging as well, which I think is nice.

@hamistao hamistao Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can use logrus here instead if you prefer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen the t *testing.T, yes you can remove it and use the logrus

hamistao added 4 commits July 22, 2026 14:58
Signed-off-by: hamistao <pedro.ribeiro@suse.com>
Signed-off-by: hamistao <pedro.ribeiro@suse.com>
Signed-off-by: hamistao <pedro.ribeiro@suse.com>
Signed-off-by: hamistao <pedro.ribeiro@suse.com>
@hamistao
hamistao force-pushed the improve_test_schemas branch from a1642af to dbe39dc Compare July 22, 2026 18:02
Signed-off-by: hamistao <pedro.ribeiro@suse.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants