Repository navigation
test: add deterministic OSAC SP simulator - #67
Draft
testetson22 wants to merge 53 commits into
Draft
testetson22 wants to merge 53 commits into
testetson22 wants to merge 53 commits into
Conversation
Signed-off-by: Thomas Stetson <tstetson@redhat.com>
…E tests - Add `scripts/deploy-osac-backend.sh` and Makefile targets to deploy/teardown the backend on OCP, plus macOS `port-forward-osac` helpers. - Integrate `--deploy-osac-backend` into `deploy-dcm.sh`: auto-detects backend credentials, injects TLS and Darwin port-forward compose overlays, tears down with `--tear-down`, and patches profile-gated `depends_on` for podman-compose compatibility. - Activate the `environment-agent` profile so the merged agent service starts under podman-compose, and fix the OSAC SP registration URL path. - Harden OSAC SP E2E tests: exact provider registration assertions, AEP-132 `results` wrapper, stricter CloudEvent envelope checks, 502 skip handling, RFC 9457 "Bad Request" title, and idempotent create returns original resource. - Document setup in CLAUDE.md and Cursor prompts; ignore `deploy/` artifacts. Signed-off-by: Thomas Stetson <tstetson@redhat.com>
…t skipped coverage - Update the CLI provider test suite to target `/agents`, rename provider references to agent, and document the CLI + assertion blockers (FLPATH-4895) keeping the suite pending (`PDescribe`). - Implement the previously pending RFC 7807 contract test for 422/406 provider-error responses by stopping all providers. - Implement sequential failover orphan cleanup and deferred-deletion queue checks, tracking superseded resource IDs across rehydrations. - Update `restartSPRM()` to restart the merged `environment-agent` container. - Annotate the pending OSAC `max_page_size > 100` test with the known backend gap and Jira references (FLPATH-4459/4463). Signed-off-by: Thomas Stetson <tstetson@redhat.com>
…y for Phase 2 - Strengthen cluster/VM API assertions: RFC 9457 problem details for missing and invalid fields, explicit non-null `results` array checks, and table-driven boundary tests for storage and query parameters. - Align health/status tests with observed SP behavior: detail required when degraded, accept "PROGRESSING" cluster status, and expect CloudEvent source prefix `dcm/providers/osac-sp`. - Capture `spec` in cluster/VM structs to enable persisted-field round-trip checks. - Extend `scripts/deploy-osac-backend.sh` with optional Phase 2 components (osac-operator, BMFO, aap-mock, fixtures), upstream manifest pinning, `--skip-phase2`, and safer teardown of cluster-scoped/APIService resources. - Document test-credential provenance and gRPC TLS CA-cert troubleshooting in CLAUDE.md and Cursor prompts. Signed-off-by: Thomas Stetson <tstetson@redhat.com>
Update deployment scripts to initialize Keycloak roles and improve trap handling. Add new test utilities to query resources across all namespaces. Update documentation and test plans to explicitly define the scope and limitations of the opt-in Tier B++ orchestration-boundary testing suite relative to real Tier C infrastructure requirements.
…fixes Introduce `scripts/osac-ca.sh` to centralize OSAC CA cert validation logic. Update `deploy-dcm.sh` and `deploy-osac-backend.sh` to validate the CA file exists, is readable, non-empty, and contains valid PEM X.509 content before deployment proceeds.
…t deployment topologies (SNO, Compact, Standard) impact Tier B++ and Tier C test coverage. Define resource limits for each profile, specifically addressing AAP sizing, ODF memory requirements, and HPP data persistence risks during node failures.
…tion and test harness Update `deploy-dcm.sh` to support a combined workflow for deploying the OSAC fulfillment-service backend and starting macOS port-forwards in a single invocation. Add `deploy-osac-aap.sh` to manage disposable AAP Controller/Gateway integration, enabling Tier B++ testing via `--osac-aap-mode real`. Introduce `normalize-fulfillment-token-issuer.sh` to correctly align token issuers with the external TLS Route, resolving JWKS discovery issues. Enhance `osac-port-forward.sh` with an `--ensure` mode for idempotent start/reuse behavior. Add `.github/workflows/validate-scripts.yaml` job and `tests/test-osac-deploy-wiring.sh` to validate these integration points.
…P-030; add templateID validation, provisioning job checks in dispatch test; wire delete-template support and external IP CRDs into turnkey deployment with RBAC and wiring verification
… projection Introduce a `delete-delayed` scenario for the OSAC simulator to test graceful handling of resources in deletion transit. This adds a `filterDeleted` interceptor that hides items from list responses once their deletion count threshold is met, ensuring the SP contract tests reflect eventual consistency. Additionally, add E2E assertions for the `vm-running` scenario to verify deterministic IP projection (internal and external) aligns with the simulator's fixed test addresses. Update documentation and Makefile to reflect these new testing modes and clarify Tier C resolution status.
…ier-b-dispatch # Conflicts: # tests/run-e2e.sh
…imulator # Conflicts: # tests/run-e2e.sh
testetson22
added a commit
that referenced
this pull request
Oct 27, 2026
## Summary Establishes the OSAC Service Provider contract-test foundation and real fulfillment-service E2E path. ## Scope - Cluster and VM API validation. - Provider registration and health checks. - CRUD and idempotency behavior. - Status projection and CloudEvents/NATS coverage. - Real fulfillment-service backend fixtures and OSAC SP Compose wiring. - JUnit-compatible SP test targets. - Initial backend deployment safety and ownership checks. - Clear separation between SP contract validation and downstream Agent, BMFO, AAP, BMC, HostedCluster, and infrastructure behavior. ## Shared Harness Impact This branch includes shared environment-agent/SPRM migration compatibility and rehydration contract/failover test updates required by the current E2E architecture. These tests are not OSAC-specific, but are included because the OSAC SP integration depends on the same provider-registration, restart, and E2E helper paths. ## Validation - E2E suite compiles. - SP contract tests run against the real fulfillment-service backend. - Provider registration, health, CRUD/idempotency, status, and CloudEvent coverage are included. - Rehydration/auth/network conflict-sensitive suites compile and retain their intended skip/enable gates. - Known product issue: [FLPATH-4945](https://redhat.atlassian.net/browse/FLPATH-4945) is reproduced by the enabled `max_page_size > 100` assertion. The SP currently returns HTTP 200 instead of the required HTTP 400. - CI compiles and validates the E2E suite and scripts; it does not execute live OCP or AAP deployment. ## Review Boundary This PR validates the OSAC SP contract and its real fulfillment-service integration. It does not claim Agent registration, BMFO/BMI/BMH allocation, Ironic/BMC provisioning, AAP playbook correctness, HostedCluster/NodePool readiness, VM networking, or physical infrastructure lifecycle. ## Stack - PR #66: Tier B++ dispatch and real-AAP integration. - PR #67: deterministic OSAC SP simulator and state/error coverage. Both dependent PRs remain draft. --------- Signed-off-by: Thomas Stetson <tstetson@redhat.com>
…ier-b-dispatch # Conflicts: # .cursor/prompts/deploy-osac-backend.md # CLAUDE.md # Makefile # scripts/deploy-dcm.sh # scripts/deploy-osac-backend.sh # scripts/osac-port-forward.sh # test-plans/OSAC-tier-c-test-plan.md # tests/compose-osac-sp-tls.yaml # tests/osac-backend/keycloak.yaml # tests/osac-backend/realm.json
…H-4788-simulator # Conflicts: # README.md # scripts/deploy-dcm.sh # test-plans/OSAC-tier-c-test-plan.md # tests/e2e/sp_osac_cluster_api_test.go # tests/e2e/sp_osac_status_test.go
Signed-off-by: Thomas Stetson <tstetson@redhat.com>
…H-4788-simulator # Conflicts: # tests/e2e/sp_osac_dispatch_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds deterministic fulfillment-service scenarios for OSAC SP status, error, deletion, VM, and kubeconfig mapping on top of the Tier B++ integration stack.
Scope
ready,failed,backend-unavailable,backend-timeout,delete-delayed, andvm-running.Shared Harness Impact
This PR inherits the environment-agent migration and shared E2E harness changes from PR #65. Any rehydration/provider test impact is through those shared helpers, not through simulator behavior.
Validation
Review Boundary
The simulator validates SP-visible state/error mapping. It does not replace Agent, BMFO, AAP, BMC, HostedCluster, or physical infrastructure testing.
Stack
Depends on PR #66 and PR #65. This PR is intentionally cumulative because parent fork branches cannot be upstream base branches. It remains draft until the parent PRs are reviewed.