diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..cc03b01a9 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,85 @@ +# Cluster Kube Scheduler Operator + +A static pod operator that manages the lifecycle of `kube-scheduler` on OpenShift control plane nodes. Built on the [library-go](https://github.com/openshift/library-go) static pod operator framework, it observes cluster configuration and reconciles the target kube-scheduler config into static pod manifests. Installed by the [Cluster Version Operator](https://github.com/openshift/cluster-version-operator) (CVO). + +See [ARCHITECTURE.md](ARCHITECTURE.md) for the full design and data flow. + +## Build and Test + +```bash +make build # Build all binaries (operator + OTE test runner) +make test # Unit tests (./pkg/... ./cmd/... ./bindata/...) +make verify # Formatting, vetting, golang version checks +make e2e # E2E operator tests (1h timeout) +make test-e2e-preferred-host # E2E preferred-host tests (1h timeout) +``` + +Go version: see `go.mod`. + +## Project Structure + +| Directory | Purpose | +|-----------|---------| +| `bindata/` | Embedded assets: default config, static pod template, alerts, RBAC, bootstrap manifests | +| `cmd/cluster-kube-scheduler-operator/` | Operator binary entry point (operator, installer, pruner) | +| `cmd/cluster-kube-scheduler-operator-tests-ext/` | OpenShift Tests Extension (OTE) test runner entry point | +| `cmd/render/` | Bootstrap manifest renderer for cluster installation | +| `manifests/` | CVO deployment manifests (namespace, deployment, RBAC, ServiceMonitors) | +| `pkg/operator/configmetrics/` | Prometheus metrics derived from the Scheduler CR | +| `pkg/operator/configobservation/` | Configuration observers — watch cluster state to produce operand config | +| `pkg/operator/operatorclient/` | Namespace constants and operator client interfaces | +| `pkg/operator/resourcesynccontroller/` | Syncs ConfigMaps/Secrets between namespaces | +| `pkg/operator/starter.go` | Operator initialization — creates clients, informers, and starts all controllers | +| `pkg/operator/targetconfigcontroller/` | Renders observed config + defaults into kube-scheduler ConfigMaps | +| `test/e2e/` | E2E test suite (operator behavior) | +| `test/e2e-preferred-host/` | E2E test suite (preferred-host kube-apiserver communication) | +| `test/library/` | Shared test utilities | + +## Controller Pattern + +Controllers use the library-go `factory.Controller` base. Each controller has a `sync(ctx, syncContext)` method called by the framework on informer events or periodic resyncs. The operator wires them in `pkg/operator/starter.go` via `RunOperator()`. + +Config observers follow a specific pattern: each observer function receives the existing config and returns `(observedConfig, errors)`. Observers are registered in `pkg/operator/configobservation/configobservercontroller/observe_config_controller.go`. + +## Key Conventions + +- **Namespaces:** `openshift-kube-scheduler-operator` (operator), `openshift-kube-scheduler` (operand), `openshift-config` (user config), `openshift-config-managed` (platform config). Constants in `pkg/operator/operatorclient/interfaces.go`. +- **Logging:** `k8s.io/klog/v2` with verbosity levels +- **Error handling:** wrap with `fmt.Errorf("context: %w", err)` +- **Feature gates:** controllers that depend on feature gates use `FeatureGateAccessor` from library-go; wait for gates before starting + +## What Not To Do + +These are specific mistakes that are easy to make in this codebase — read before changing anything: + +- **Don't modify `vendor/` directly** — use `go mod tidy && go mod vendor`. Manual edits to `vendor/` will be overwritten and break reproducibility. + +- **Don't edit `bindata/assets.go`** — this file uses Go's `embed` directive and is auto-generated. Update the asset files under `bindata/assets/` and `bindata/bootkube/`; never edit the embed wrapper itself. + +- **Don't add a config observer without registering it** — new observer functions must be added to the observer list in `pkg/operator/configobservation/configobservercontroller/observe_config_controller.go`. A function that is never registered will silently do nothing. + +- **Don't bypass the static pod upgrade protocol** — the installer/revision/pruner controllers in library-go manage rollout ordering and availability guarantees. Do not apply static pod changes directly to nodes or skip the revision mechanism. + +- **Don't use direct API calls in config observers** — use the listers available in `configobservation.Listers`. Direct `client.Get()` or `client.List()` calls in observer functions add latency and load, and break the informer-driven design. + +- **Don't fix library-go bugs here** — if a bug is in library-go, fix it upstream in [library-go](https://github.com/openshift/library-go) and vendor the fix. Workarounds in this repo create divergence that makes future rebases harder. + +- **Don't rename namespace constants** without a full grep of all usages — `OperatorNamespace`, `TargetNamespace`, etc. in `pkg/operator/operatorclient/interfaces.go` are used throughout the codebase and in static asset templates under `bindata/`. + +- **Don't assume feature gates are available at startup** — `featureGateAccessor.InitialFeatureGatesObserved()` must be awaited (with a timeout) before accessing feature gates. The operator already does this in `starter.go`; don't bypass or reorder it. + +## Contributing + +See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines. Key rules: + +- Do not modify files under `vendor/`. Use `go mod tidy && go mod vendor`. +- `bindata/assets.go` uses Go's `embed` directive to embed asset files — update the embedded files, not this file. +- Write unit tests for every change. E2E tests for significant features. +- Backwards compatibility matters — deprecate before removing. +- Before modifying the operator API, ensure there is a corresponding enhancement proposal in [openshift/enhancements](https://github.com/openshift/enhancements). API changes require design review and approval. + +## Testing + +- **Unit tests:** co-located `*_test.go` files, table-driven, `go test ./pkg/... ./cmd/... ./bindata/...` +- **E2E tests:** suites under `test/e2e/` and `test/e2e-preferred-host/`, each with its own Makefile target. Tests use the Ginkgo v2 framework via the OTE framework. +- **OTE framework:** `cluster-kube-scheduler-operator-tests-ext` binary. See [CONTRIBUTING.md](CONTRIBUTING.md#openshift-tests-extension-ote) for usage. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 000000000..136498a75 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,110 @@ +# Architecture + +## Overview + +The cluster-kube-scheduler-operator is a static pod operator that manages the `kube-scheduler` on OpenShift control plane nodes. It is deployed by the Cluster Version Operator (CVO) and uses the [library-go](https://github.com/openshift/library-go) static pod operator framework. + +The operator's primary responsibilities: +- Observe cluster configuration from multiple sources and synthesize kube-scheduler config +- Manage kube-scheduler static pods across control plane nodes (install, revision, prune) +- Report status via the `ClusterOperator/kube-scheduler` resource + +## Data Flow + +```text + config.openshift.io resources + (Scheduler, APIServer) + │ + ▼ + ┌───────────────────────────────────────────────────┐ + │ Config Observer Controllers │ + │ (observe external state, produce observedConfig) │ + └──────────────────────┬────────────────────────────┘ + │ observedConfig (sparse JSON) + ▼ + ┌───────────────────────────────────────────────────┐ + │ Target Config Controller │ + │ (merge defaults + observedConfig + overrides │ + │ → render ConfigMaps in target namespace) │ + └──────────────────────┬────────────────────────────┘ + │ ConfigMaps + ▼ + ┌───────────────────────────────────────────────────┐ + │ Static Pod Controllers (library-go) │ + │ Installer → Revision Controller → Pruner │ + │ (roll out new revisions to each control plane │ + │ node as static pod manifests) │ + └──────────────────────┬────────────────────────────┘ + │ + ▼ + kube-scheduler static pods + (one per control plane node) +``` + +## Operator Startup + +Entry point: `cmd/cluster-kube-scheduler-operator/main.go` → `pkg/operator/starter.go:RunOperator()`. + +Startup sequence: +1. Create clients (Kubernetes, config, operator) +2. Create informers for watched namespaces (see [Namespaces](#namespaces)) +3. Initialize feature gates via `FeatureGateAccessor` and wait for observation (1-minute timeout) +4. Create and start all controllers concurrently +5. Block until context cancellation + +## Namespaces + +| Namespace | Constant | Purpose | +|-----------|----------|---------| +| `openshift-config` | `GlobalUserSpecifiedConfigNamespace` | User-provided configuration (policy configmaps) | +| `openshift-config-managed` | `GlobalMachineSpecifiedConfigNamespace` | Platform-managed configuration | +| `openshift-kube-scheduler-operator` | `OperatorNamespace` | Operator deployment and its resources | +| `openshift-kube-scheduler` | `TargetNamespace` | Operand: kube-scheduler pods and config | + +The `ResourceSyncController` copies ConfigMaps and Secrets between these namespaces as needed (e.g. policy configmap from `openshift-config` to the target namespace). + +## Static Pod Management + +The operator uses library-go's `staticpod.NewBuilder()` to manage kube-scheduler static pods. This framework provides: + +- **Installer controller** — creates new static pod revisions on each control plane node. Uses a custom installer command (`cluster-kube-scheduler-operator installer`). +- **Revision controller** — tracks revisions of ConfigMaps and Secrets. When any revisioned resource changes, a new revision is created. The first ConfigMap in the list (`kube-scheduler-pod`) contains the static pod manifest template. +- **Pruner** — removes old static pod revisions to free disk space. +- **PDB guard** — ensures availability during upgrades (only on multi-node clusters; disabled for single-node). + +Resources are split into two categories: +- **Revisioned** — ConfigMaps and Secrets declared in `deploymentConfigMaps`/`deploymentSecrets` in `starter.go`. Any change to these triggers a new static pod revision. The first ConfigMap in the list contains the static pod manifest template. +- **Unrevisioned certs** — Secrets declared in `CertSecrets` in `starter.go`, managed by `WithUnrevisionedCerts`. These are updated in-place without triggering a new revision. + +## Configuration Observers + +Configuration observers watch external cluster resources and produce a sparse JSON config (`observedConfig`) that gets merged into the kube-scheduler configuration. Each observer function receives the existing config and returns `(observedConfig, errors)`. + +Observers are registered in `pkg/operator/configobservation/configobservercontroller/observe_config_controller.go`: + +| Package | Watches | Effect | +|---------|---------|--------| +| `scheduler/` | `Scheduler` CR | Syncs `policy-configmap` from `openshift-config` to `openshift-kube-scheduler` if `.spec.policy.name` is set | +| *(library-go)* `apiserver` | `APIServer` CR | TLS security profile | + +## Target Config Controller + +`pkg/operator/targetconfigcontroller/` takes the merged configuration (defaults + observedConfig + unsupportedConfigOverrides) and renders it into concrete resources in the target namespace: + +- `config` ConfigMap — the main kube-scheduler configuration, built by merging `bindata/assets/config/defaultconfig.yaml` with the observed config and applying the scheduling profile +- `kube-scheduler-pod` ConfigMap — the static pod manifest template + +The target config controller handles scheduling profiles (`LowNodeUtilization`, `HighNodeUtilization`, `NoScoring`) and passes cluster feature gates as `--feature-gates` flags to the scheduler binary. + +## Render Command + +`cmd/render/` is a bootstrap manifest renderer used during cluster installation. It takes installer-provided inputs and renders the initial set of manifests needed to bootstrap kube-scheduler before the operator is running. The templates live in `bindata/bootkube/`. + +## Other Controllers + +| Controller | Purpose | +|-----------|---------| +| `StaticResourceController` | Applies static manifests from `bindata/` (namespace, network policies, RBAC, service, ServiceAccount, kubeconfig) | +| `ClusterOperatorStatus` | Reports operator status, versions, and related objects to `ClusterOperator/kube-scheduler` | +| `ResourceSyncController` | Syncs ConfigMaps/Secrets between namespaces (e.g. policy configmap) | +| `ConfigMetrics` | Registers Prometheus metrics derived from the `Scheduler` CR (`cluster_master_schedulable`, `cluster_legacy_scheduler_policy`, `cluster_default_node_selector`) | diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..43c994c2d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..a58dc6c34 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,218 @@ +# Contributing to the OpenShift Control Plane Components/Repositories + +This document serves as a guide for contributing to the OpenShift components/repositories +that the OpenShift Control Plane group is responsible for maintaining. + +This document is explicitly for contributions to individual component repositories and not for high-level +feature proposals within OpenShift. + +Feature proposals should follow the OpenShift Enhancement Proposal process outlined in https://github.com/openshift/enhancements/blob/master/dev-guide/feature-zero-to-hero.md#openshift-feature-development-zero-to-hero-guide. +If you are looking for a review on an OpenShift Enhancement Proposal that involves changes to components +maintained by the control plane group, please request a review in the [`#forum-ocp-apiserver`](https://redhat.enterprise.slack.com/archives/CB48XQ4KZ) Slack channel. Requests for review may be redirected to more appropriate and/or more focused channels for discussion. + +This document contains the following sections: + +- [Code conventions](#code-conventions) - A collection of guidelines, style suggestions, and tips for writing code. +- [Testing guidelines](#testing-guidelines) - Guidelines and expectations for testing of contributions. +- [Pull Request process/guidelines](#pull-request-process-and-guidelines) - Guidelines and expectations of pull requests containing contributions. +- [Review expectations](#review-expectations) - Guidelines and expectations for requesting reviews and interacting with reviewers. + +## Code Conventions + +We largely follow the [Kubernetes Code Conventions](https://github.com/kubernetes/community/blob/main/contributors/guide/coding-conventions.md#code-conventions). + +Review both the Kubernetes Code Conventions and the ones specified here. +There will be some overlap. If any conventions are at odds with one another, prefer the conventions explicitly documented here. + +### Bash + +- Follow the [shell styleguide](https://google.github.io/styleguide/shellguide.html). +- Use [`shellcheck`](https://github.com/koalaman/shellcheck) to identify common mistakes or caveats. +- Ensure that all scripts run consistently across Linux and macOS. + +### Golang (Go) + +- Review [Effective Go](https://go.dev/doc/effective_go). +- Review common [Go Code Review Comments](https://go.dev/wiki/CodeReviewComments). +- Review and avoid [Go Landmines](https://gist.github.com/lavalamp/4bd23295a9f32706a48f) +- Comment your code following the [Go comment conventions](https://go.dev/doc/comment). + - Comments should be meaningful and add context and/or explain choices that cannot be expressed through clear code. + - All exported types, functions, and methods must have descriptive comments. + - All unexported types, functions, and methods should have descriptive comments. +- When adding command-line flags, use dashes/hyphens (`-`) and not underscores (`_`). +- Naming + - Please consider package name when selecting an interface name, and avoid redundancy. For example, `storage.Interface` is better than `storage.StorageInterface`. + - Do not use uppercase characters, underscores, or dashes in package names. + - Please consider parent directory name when choosing a package name. For example, `pkg/controllers/autoscaler/foo.go` should say `package autoscaler` not `package autoscalercontroller`. + - Unless there's a good reason, the package foo line should match the name of the directory in which the .go file exists. + - Importers can use a different name if they need to disambiguate. + - Locks should be called `lock` and should never be embedded (always `lock sync.Mutex`). When multiple locks are present, give each lock a distinct name following Go conventions: `stateLock`, `mapLock` etc. +- Error handling + - Wrap errors with meaningful context before returning or logging them. +- When logging, follow the [Kubernetes Logging Conventions](https://github.com/kubernetes/community/blob/main/contributors/devel/sig-instrumentation/logging.md). +- When patching OpenShift-maintained forks of "upstream" repositories, patches should be as small as reasonably possible and should minimize touch points with code that is likely to change and impact the rebasing process. +- Dependencies must be vendored. When making changes to dependencies, ensure you've run `go mod tidy` and `go mod vendor`. + +### General + +Regardless of the programming language, make sure to take the following into consideration: +- Keep readability / maintainability in mind when writing code. + - Clever code and abstractions are often harder to reason about after the fact. Keep clever code and abstractions to the minimum necessary to accomplish the end-goal. +- Do not reinvent the wheel. Where possible, use existing standard library or vendored library functionality. If you are adding a net-new dependency, stop and think if you _really_ need to add the new dependency to achieve your goals. +- When writing tests, focus on testing the functional behaviors your code exercises. Avoid writing tests that are testing that the standard library works as expected or is trivial. Do not write tests just for line coverage. + +### Directory and File Conventions + +- Avoid package sprawl. Find an appropriate subdirectory for new packages. + - Libraries with no appropriate home belong in new package subdirectories of `pkg/util`. +- Avoid general utility packages. Packages called "util" are suspect. Instead, derive a name that describes your desired function. For example, the utility functions dealing with waiting for operations are in the `wait` package and include functionality like `Poll`. The full name is `wait.Poll`. +- All filenames should be lowercase. +- Go source files and directories use underscores, not dashes. + - Package directories should generally avoid using separators as much as possible. When package names are multiple words, they usually should be in nested subdirectories. + +## Testing Guidelines + +These are high-level testing guidelines. Where individual component repositories may have +additional testing guidelines to follow when making contributions. + +- All changes must include unit test additions/changes. + - Exceptions are at reviewer/approver discretion. +- Table-driven unit tests are preferred for testing multiple scenarios/inputs. For an example, see https://github.com/openshift/cluster-authentication-operator/blob/a493799952e9b6838021ccc7d15d3d37d7ad3508/pkg/controllers/externaloidc/externaloidc_controller_test.go#L108 . +- Unit tests must pass on all platforms (at the very least, Linux + macOS). +- Significant features should come with integration and/or end-to-end (e2e) tests where appropriate. + - End-to-end tests _may_ be scoped as a separate work item when the end-to-end tests for the component must be added to the openshift/origin repository instead of the component repository. Adding e2e tests to the component repository is preferred where possible. It is up to reviewer/approver discretion whether a contribution can be merged without end-to-end tests being implemented. +- Do not expect an asynchronous thing to happen immediately. Do not wait for one second and expect a pod to be running. Wait and retry instead. + + +If necessary, manual integration testing can be done by creating a cluster using the [`Cluster Bot` Slack App](https://redhat.enterprise.slack.com/archives/D03KX7M1CRJ). +Once you have a cluster created, you can follow some of the instructions in https://github.com/openshift/enhancements/blob/master/dev-guide/operators.md for guidance on how to +build component images and modify cluster-operators to deploy those images. + +Most component repos have existing tooling to run unit tests. Check for Makefiles and shell scripts that might run the unit tests. If none exist, you should be able to use standard testing tooling like `go test ./...` to run all tests for the project. https://pkg.go.dev/cmd/go/internal/test is a good reference for how the `go test` command works. + +## Pull Request Process and Guidelines + +This section assumes that you have a functional understanding of `git` and how to create a pull request on GitHub. + +If you do not, start with [GitHub's "Getting Started" guide](https://docs.github.com/en/get-started/start-your-journey). + +### Prerequisites + +Before you commit any changes or create any pull requests, you must adhere to OpenShift contribution policies. +Currently, that means enabling commit signature verification. + +See https://docs.google.com/document/d/1184EPSGunUkcSQYUK8T4a6iyawwi6f2zxdbB2jtG9nQ/edit?usp=sharing for more details on +how to adhere to the commit signature verification policy of OpenShift. + +### Creating a Pull Request + +When creating a pull request, include the following: + +- A brief, but descriptive, title. + - All pull requests _should_ link to a Jira ticket associated with the work. There is automation that performs this linking when prefixing the title with the Jira ticket identifier like: `CNTRLPLANE-XXXX: my pull request title`. For pull requests that have no Jira ticket associated with it, you can prefix it with `NO-JIRA:` to signal that there is not a Jira ticket associated with it. +- A useful description of the changes being made and why they are important. Include links to supporting documents and any additional context that reviewers may need. + +### CI / CD + +For CI/CD, OpenShift uses Prow to run various checks. This can include unit tests, e2e tests, linters, etc. + +The jobs configured for each repository are in https://github.com/openshift/release/tree/main/ci-operator/config/openshift . If you find yourself needing to add additional jobs, review the documentation at https://docs.ci.openshift.org/how-tos/contributing-openshift-release/ . + +There are often a mixture of required and optional checks as well as merge criteria that must be met before a pull request can merge. +When any of these checks fail, the GitHub Prow bot will leave a comment on the PR with links to the run of that check that failed. + +As the PR author, it is your responsibility to evaluate the failed checks and determine if there are any changes necessary to pass the checks. +If you suspect that the check failure was a flake, you can trigger retests by commenting `/retest` (or `/retest-required` for retesting only the required checks) on the PR. + +### Verifying your changes / Creating an OpenShift cluster from a PR + +As part of merging a PR, there is a requirement to verify that the changes you've made are working as expected using the `/verified` comment command. + +While there are a lot of scenarios where the existing CI/CD checks may be sufficient to verify your changes are working (and can be denoted by commenting `/verified by ci`), +there may be scenarios where manual verification is required. + +You can use the `Cluster Bot` Slack App to create a cluster from a PR by sending it a message in the format of `launch ${OCP_VERSION},${PR_LINK} ${PLATFORM},${VARIANT}`. +As an example, `launch 4.23,https://github.com/openshift/cluster-kube-scheduler-operator/pull/123 aws` would launch an OpenShift 4.23 cluster with the changes made in that PR running on AWS. +For more information on what `Cluster Bot` can do, you can send it a message saying `help` and it will respond with additional documentation on how it can be used. + +Once you've verified your changes work as expected, you can mark the PR as verified by commenting `/verified by @{your_github_handle}` on the PR. + +### Additional Resources + +For more information regarding more general OpenShift pull request processes, the following resources are helpful: + +- https://docs.ci.openshift.org/architecture/jira +- https://docs.ci.openshift.org/ +- https://steps.ci.openshift.org/ + +## Review Expectations + +### Requesting a review + +If you are not a member of the OpenShift control plane team and you need a review on a PR, post it in the [#forum-ocp-apiserver](https://redhat.enterprise.slack.com/archives/CB48XQ4KZ) Slack channel or +reach out to folks outlined in the OWNERS file directly. + +If you are a member of the OpenShift control plane team, reviews should come from your feature team. In the event your feature team does not have someone that can approve +a PR, post it in the [#control-plane](https://redhat.enterprise.slack.com/archives/CC3CZCQHM) Slack channel. + +OpenShift uses AI code review tools as part of the code review process. +Before requesting a review, address all feedback from the code review agent(s). +It is up to your discretion as the contributor how you would like to address that feedback. +Responding with an explanation as to why you are not going to take action on a comment made +by the agent is an acceptable way to "address" its feedback. + +### Interacting with reviewers + +When interacting with reviewers/approvers: + +- Be professional. +- Be respectful of differing opinions, viewpoints, and experiences. +- Gracefully give and receive constructive feedback. +- Focus on what is best for the product/organization, not just us as individuals. + +A special note on the usage of AI - to respect the time of those that are reviewing your contribution, please do not use AI to respond to review comments. + +## OpenShift Tests Extension (OTE) + +This repository is compatible with the [OpenShift Tests Extension (OTE)](https://github.com/openshift-eng/openshift-tests-extension) framework. +The test binary and suite definitions live in [`cmd/cluster-kube-scheduler-operator-tests-ext`](cmd/cluster-kube-scheduler-operator-tests-ext/main.go). + +### Building the test binary + +```bash +make build +``` + +### Available Test Suites + +- **`openshift/cluster-kube-scheduler-operator/operator/serial`** — Tests that modify cluster-scoped resources (Scheduler CR) and must run sequentially (Parallelism: 1) +- **`openshift/cluster-kube-scheduler-operator/operator/parallel`** — Tests that are isolated and can run concurrently (Parallelism: 10) +- **`openshift/cluster-kube-scheduler-operator/preferred-host/serial`** — Tests that verify KubeScheduler communication with kube-apiserver over preferred host (Parallelism: 1) + +### Running test suites and tests + +```bash +# Run a suite +./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/serial + +# Run a suite with concurrency limited to 1 (serial execution) +./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/serial --max-parallel-tests=1 + +# Run a specific test +./cluster-kube-scheduler-operator-tests-ext run-test "test-name" + +# Run with JUnit output +./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/serial --junit-path=/tmp/junit.xml +``` + +### Listing available tests and suites + +```bash +# List all test suites +./cluster-kube-scheduler-operator-tests-ext list suites + +# List tests in a suite +./cluster-kube-scheduler-operator-tests-ext list tests --suite=openshift/cluster-kube-scheduler-operator/operator/serial +``` + +For more information about the OTE framework, see the [openshift-tests-extension documentation](https://github.com/openshift-eng/openshift-tests-extension). diff --git a/README.md b/README.md index c930ae636..65ddf086d 100644 --- a/README.md +++ b/README.md @@ -308,60 +308,8 @@ I0105 03:32:31.936642 1 dumper.go:52] "Dump of cached NodeInfo" nodes=< ## Tests -This repository is compatible with the [OpenShift Tests Extension (OTE)](https://github.com/openshift-eng/openshift-tests-extension) framework. +See [CONTRIBUTING.md#openshift-tests-extension-ote](CONTRIBUTING.md#openshift-tests-extension-ote) for how to build and run the test binary. -### Building the test binary +## Contributing -```bash -make build -``` - -### Available Test Suites - -The e2e tests are organized into multiple suites based on their execution requirements: - -- **`openshift/cluster-kube-scheduler-operator/operator/serial`** - Tests that modify cluster-scoped resources (Scheduler CR) and must run sequentially (Parallelism: 1) -- **`openshift/cluster-kube-scheduler-operator/operator/parallel`** - Tests that are isolated and can run concurrently (Parallelism: 10) -- **`openshift/cluster-kube-scheduler-operator/preferred-host/serial`** - Tests that verify KubeScheduler communication with kube-apiserver over preferred host (Parallelism: 1) - -### Running test suites and tests - -```bash -# Run serial test suite (tests that modify cluster-scoped resources) -./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/serial --max-parallel-tests=1 - -# Run parallel test suite (isolated tests) -./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/parallel - -# Run preferred-host test suite -./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/preferred-host/serial --max-parallel-tests=1 - -# Run a specific test by name -./cluster-kube-scheduler-operator-tests-ext run-test "[sig-scheduling] kube scheduler operator [Operator][Serial] should create configmap when scheduler CR is updated" - -# Run with JUnit output -./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/serial --junit-path /tmp/junit-serial.xml - -# Run with verbose logging -./cluster-kube-scheduler-operator-tests-ext run-suite openshift/cluster-kube-scheduler-operator/operator/serial -v 4 -``` - -### Listing available tests and suites - -```bash -# List all test suites -./cluster-kube-scheduler-operator-tests-ext list suites - -# List all available tests -./cluster-kube-scheduler-operator-tests-ext list tests -``` - -### Test Categorization - -Tests are tagged with the following labels to determine their execution suite: - -- **`[Operator][Serial]`** - Tests that modify shared cluster resources and must run sequentially -- **`[Operator][Parallel]`** - Tests that are isolated and can run concurrently -- **`[PreferredHost][Serial]`** - Tests that verify KubeScheduler communication with kube-apiserver over preferred host - -For more information about the OTE framework, see the [openshift-tests-extension documentation](https://github.com/openshift-eng/openshift-tests-extension). +See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines, code conventions, testing requirements, and the pull request process.