From 67509a5a703526716e0f70a4586284f0d743ab0f Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Wed, 21 Oct 2026 18:29:01 +0000 Subject: [PATCH 1/6] docs: plan the GitBranch and KubeProvider API break Co-Authored-By: Claude Opus 5.5 (1M context) --- .../design/gitbranch-and-kubeprovider-plan.md | 346 ++++++++++++++++++ 1 file changed, 346 insertions(+) create mode 100644 docs/design/gitbranch-and-kubeprovider-plan.md diff --git a/docs/design/gitbranch-and-kubeprovider-plan.md b/docs/design/gitbranch-and-kubeprovider-plan.md new file mode 100644 index 00000000..d05ce1a1 --- /dev/null +++ b/docs/design/gitbranch-and-kubeprovider-plan.md @@ -0,0 +1,346 @@ +# GitBranch, KubeProvider, and a phased API break + +Status: **plan**, 2026-10-21. Nothing here is built. The current contract stays in +[architecture.md](../architecture.md) and [configuration.md](../configuration.md) until each phase +ships. [ideal-shared-stuff.md](ideal-shared-stuff.md) is the analysis behind this plan; this page +records what we decided from it and in what order we build it. + +The plan breaks the API on purpose, more than once. There are no production users to carry across, +so each phase replaces a shape outright instead of keeping a deprecated twin beside it. A removed +field stays in the schema only so that setting it is refused by name, the pattern the API already +uses for `qps`, `burst`, and `allowAnySourceNamespace`. + +## Decisions + +| Decision | Choice | +|---|---| +| The repository branch gets its own kind | `GitBranch`, namespaced, referenced as `gitBranchRef` | +| A `GitBranch` names its credential | `gitProviderRef` in its own namespace only | +| A `GitTarget` names its branch | `gitBranchRef` is required; `gitProviderRef`, `branch`, `parentBranch` leave the target | +| Who may write a branch | One `GitBranch` per repository branch, and it must prove push access to claim it | +| Changing a branch's identity | `gitProviderRef`, `branch`, and `parentBranch` are immutable; replace the object instead | +| Deleting a `GitBranch` | Never deletes the branch on the remote | +| The Kubernetes connection | `KubeProvider`, namespaced, replacing `ClusterProvider` | +| Its credential | A flat `secretRef` to a kubeconfig Secret in the provider's namespace | +| The operator's own identity | An explicit `inCluster: {}` arm, honored only where the installation authorizes it | +| References between kinds | Explicit everywhere; no `default` name, no fallback | + +## The API after phase 2 + +```text + GitProvider (credential, allowedBranches, accessFrom) + ^ + | gitProviderRef (same namespace) + | +GitTarget --gitBranchRef--> GitBranch (repository branch: one per repo+branch, one branch worker) + | + +--kubeProviderRef--> KubeProvider (source cluster: kubeconfig or inCluster, accessFrom) + | + +--applyTriggerRef--> FluxTrigger / ArgoCDTrigger --kubeProviderRef--> KubeProvider +``` + +A `GitTarget` is a folder: what it mirrors, where in the branch it writes, how it places, prunes, +encrypts, and commits, and which applier it holds. Everything that is one fact per repository +branch lives on the `GitBranch`. Everything that is one fact per credential lives on the provider. + +| Field or status | Moves to | Why | +|---|---|---| +| `GitTarget.spec.gitProviderRef`, `.branch`, `.parentBranch` | `GitBranch.spec` | Two targets can no longer disagree about one branch | +| `GitTarget.status.remote` | `GitBranch.status.remote` | It describes the branch, so N targets carried N copies | +| `GitProvider.status.branches` | Removed | Listing `GitBranch` objects answers the same question | +| `GitProvider.spec.commit` (committer, signing) | Stays | The signing key belongs to the account the credential is for | +| `GitProvider.spec.allowedBranches` | Stays | It is the ceiling every `GitBranch` must fit under | +| `GitTarget.spec.commit.window`, `.commit.message`, `.suspend` | Stays | They describe the folder | + +## Phase 0: merge the shared `GitProvider` + +PR #428 keys the branch worker by repository and branch, adds `GitProvider.spec.accessFrom`, and +lets a `GitTarget` name a `GitProvider` in another namespace. Phase 1 builds on all three. Its one +known gap is that the oldest `GitTarget` decides which credential owns a branch, so any namespace +can squat a repository branch without a credential. Phase 1 closes that, so do not cut a release +from main between the two unless that gap is acceptable for it. + +## Phase 1: `GitBranch` + +### Shape + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: customer-main + namespace: platform-connections +spec: + gitProviderRef: + name: customer-config # a GitProvider in this namespace + branch: main # must match the provider's allowedBranches + parentBranch: develop # optional; only used to create a missing branch +status: + observedGeneration: 3 + conditions: [] # Ready, Reconciling, Stalled + repository: git.example.com/acme/config # normalized; half of the claim key + remote: + commit: 4f1c... + lastVerifiedAt: "2026-10-21T09:12:00Z" + verifiedBy: Push + parent: {} # unchanged shape, moved from GitTarget + gitTargetCount: 2 +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: settings + namespace: tenant-a +spec: + gitBranchRef: + name: customer-main + namespace: platform-connections # omitted means the target's own namespace + path: tenants/a +``` + +The schema requires `gitProviderRef.name` and `branch`. CEL keeps all three spec fields immutable +(`self == oldSelf`), the rule `GitTarget` applies to these fields today. On the target, +`gitBranchRef` becomes immutable the same way, and `path` stays immutable. + +`gitProviderRef` is a `LocalObjectReference`. Creating a `GitBranch` therefore requires the right to +create objects in the namespace that holds the credential, and that is what makes a branch claim +an act of the credential's owner. A `GitBranch` has no `accessFrom` of its own: a `GitTarget` in +another namespace may reference it when the branch's `GitProvider` admits the target's namespace. +One policy governs both hops, so it cannot be split across two objects that disagree. + +### Who owns a repository branch + +The claim key is the provider's repository URL, as `internal/repourl` normalizes it, plus the +branch. The `GitBranch` UID holds the claim, so a recreated object with the same name starts +without it. + +1. A `GitBranch` becomes a candidate once its `GitProvider` is ready and its credential opens a + receive-pack session on the remote. On GitHub and GitLab that requires push permission, so a + provider with a missing or wrong Secret, or none on a public repository, never becomes a + candidate. On a host that advertises receive-pack anonymously this proof degrades to + reachability; the upgrade notes say so. +2. Among the candidates for one key, the oldest by `creationTimestamp` (then UID) holds the claim + and gets the branch worker. Every other `GitBranch` for that key reports `Stalled=True`, reason + `BranchConflict`, with a message naming the holder. +3. The holder keeps the claim through a later credential or network failure. It reports + `Ready=False` with the failure and does not hand the branch to another `GitBranch`. +4. Admission refuses a new `GitBranch` whose key a holder already claims. It reads `GitBranch` + status only and makes no Git call. An unproven `GitBranch` claims nothing, so creating one first + cannot block anyone at admission. +5. The claim lives in the manager's memory. After a restart it is recomputed by rules 1 and 2. A + persisted claim, and with it a holder that survives a restart while its credential is broken, + comes with the HA work, not here. + +This replaces `internal/git/branch_owner.go`. Its three callers (the reconciler's conflict check, +the worker sweep, and `EnsureWorker`) ask the claim instead of electing the oldest `GitTarget`. +`git.BranchKey` stays as it is. The worker's credential, signing key, and metric labels come from +the claiming `GitBranch`'s provider, so `provider_namespace` and `provider_name` keep their +meaning for existing dashboards. + +### References are checked in status, not at admission + +Admission refuses what is wrong with the object itself: a missing required field, a removed field, +an immutable field changed, and a duplicate claim from rule 4. It does not refuse a `GitTarget` +whose `GitBranch` does not exist yet. Flux and Argo CD apply a folder of manifests in one pass, in +no guaranteed order, and an admission rule that demands dependency order breaks the most common +way this API is installed. A missing or unauthorized reference is a status condition instead: + +| Object | Reason | When | +|---|---|---| +| `GitTarget` | `GitBranchNotFound` | The referenced `GitBranch` does not exist | +| `GitTarget` | `GitBranchNotReady` | It exists but holds no claim or is failing | +| `GitTarget` | `GitTargetNamespaceNotAuthorized` | The branch's `GitProvider` does not admit the target's namespace | +| `GitBranch` | `GitProviderNotFound`, `GitProviderNotReady` | Moved from `GitTarget` | +| `GitBranch` | `BranchNotAllowed` | `branch` is outside the provider's `allowedBranches`; moved from `GitTarget` | +| `GitBranch` | `ParentBranchNotFound`, `RecoveringParentBranch` | Moved from `GitTarget`, with their rules unchanged | +| `GitBranch` | `BranchConflict` | Another `GitBranch` holds the key (new) | + +This departs from the strict-admission section of `ideal-shared-stuff.md`, deliberately. +`BranchConflict` is a new reason; the others exist today and change only which kind reports them. + +### Lifecycle + +A `GitBranch` carries a finalizer. Deletion stops new intake for every target on the branch. The +worker publishes or refuses each pending write, and each attached `CommitRequest` gets its outcome. +Then the finalizer is removed and the claim released. Nothing deletes the branch on the remote. + +Deleting a `GitBranch` that targets still reference is allowed; they report `GitBranchNotFound` +afterwards. A new `GitBranch` with the same name is a new UID and cannot pick up the old one's +pending work. + +### Removed from `GitTarget` + +`spec.gitProviderRef`, `spec.branch`, `spec.parentBranch`, and `status.remote` leave `GitTarget`. +The three spec fields stay in the schema only to be refused by name, with a message that names +`gitBranchRef`. `GitProvider.status.branches` is removed outright: it is status, so there is no +stored value for a user to be refused on. + +### Done when + +- Two `GitTarget`s in two namespaces reference one `GitBranch` and publish through one worker with + one credential, without copying the Secret. +- A `GitBranch` with a bad Secret created first does not stop a valid one created later from + claiming the branch. +- A second `GitBranch` for a claimed repository branch is refused at admission, also when it names + another `GitProvider` for the same repository. +- Two `GitBranch` objects for one key that both pass admission (a create race) never run two + workers; the younger reports `BranchConflict`. +- Setting a removed `GitTarget` field is refused with a message naming `gitBranchRef`. +- Deleting a `GitBranch` with pending writes gives each one an outcome before the finalizer goes. +- The parent-branch e2e specs pass with `parentBranch` on the `GitBranch`. +- The chart's quick start renders a `GitBranch`. + +## Phase 2: `KubeProvider` + +### Shape + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: KubeProvider +metadata: + name: workspace-a + namespace: platform-connections +spec: + secretRef: + name: workspace-a-kubeconfig # a kubeconfig Secret in this namespace + key: value # optional; default "value", then "value.yaml" + accessFrom: + selector: + matchLabels: + platform.example.com/tenant: "true" + allowCrossNamespace: + sources: false + appliers: false + client: + qps: 20 + burst: 30 + attribution: + auditRoute: workspace-a +--- +apiVersion: configbutler.ai/v1alpha3 +kind: KubeProvider +metadata: + name: local + namespace: gitops-reverser +spec: + inCluster: {} +``` + +The schema requires exactly one of `secretRef` and `inCluster`. `client` is refused beside +`inCluster`; today it is silently ignored there. The kubeconfig safety checks (exec auth, insecure +TLS, file references) and their reasons stay as they are, because they describe the Secret's +contents. + +`KubeProvider` is a new kind, not `ClusterProvider` changing scope. Kubernetes cannot change an +existing CRD's scope in place, so a rename is also the cheapest way to make the break: the +`ClusterProvider` CRD is removed and `KubeProvider` is added beside it. Rule 1 of +[definitions.md](../definitions.md) then renames every `ClusterProvider*` reason and identifier, +with no alias left behind. + +### Behavior that changes + +- **Secrets resolve in the provider's namespace.** Today they resolve in the operator's namespace. + A tenant's connection keeps its credential with the tenant. +- **`inCluster` needs the installation's consent.** An installation flag lists the exact + `namespace/name` of each `KubeProvider` allowed to use the operator's own identity. The chart sets + it for the provider it creates. A `KubeProvider` with `inCluster` that is not on the list reports + `Stalled=True`, reason `InClusterNotAuthorized`, and is never used. There is no path from an + omitted, malformed, or forbidden credential to the operator's identity. +- **The own namespace is admitted.** Like `GitProvider`, a `KubeProvider` admits consumers in its + own namespace, and `accessFrom` admits the others. Today an empty `ClusterProvider.accessFrom` + admits nobody, because a cluster-scoped object has no own namespace. +- **References are required.** `kubeProviderRef` is a required `NamespacedObjectReference` on + `GitTarget`, `FluxTrigger`, and `ArgoCDTrigger`; an omitted namespace means the referrer's own. + The provider named `default` that an omitted `clusterProviderRef` selects today goes away. +- **Audit routes cannot be borrowed across namespaces.** An omitted `auditRoute` defaults to + `.`, which cannot collide because a namespace name has no dots. An explicit + route may be shared by providers in one namespace (several connections to one API server) and is + refused at admission when a provider in another namespace already carries it. + +The `allowCrossNamespace` grants keep today's meaning. Narrower per-tenant grants are a later +design. + +### Done when + +- A tenant namespace mirrors a remote cluster through its own `KubeProvider` and Secret, and a + second tenant cannot reference it without `accessFrom`. +- A `KubeProvider` with `inCluster` outside the authorized list never reaches the API server with + the operator's identity. +- Two `KubeProvider`s with one name in two namespaces keep separate watches, caches, and + attribution facts, also after one is deleted and recreated. +- A remote-source `GitTarget` with a local Flux applier (gitops-api ask #34) still exercises the + apply window end to end. +- An explicit audit route already used in another namespace is refused at admission. + +## Phase 3: one receiver for many repositories + +Gitops-api ask #30: a GitHub App has one webhook endpoint, and today every `GitProvider` carries its +own receiver with its own token. Replace `GitProvider.spec.receiver` with a namespaced +`GitReceiver` and a reference to it: + +```yaml +kind: GitReceiver +metadata: + name: github-app + namespace: platform-connections +spec: + type: github + secretRef: + name: github-app-webhook-token + accessFrom: + selector: {} +--- +kind: GitProvider +metadata: + name: customer-config + namespace: platform-connections +spec: + url: https://git.example.com/acme/config + receiverRef: + name: github-app +``` + +A delivery authenticates against the receiver's token, resolves the repository and branch from the +payload, and wakes each claiming `GitBranch` whose provider references that receiver, once. A +delivery only wakes; the worker reads the remote with the provider's credential to find out what +happened. Unknown repositories are counted and ignored. A self-service team gets its own endpoint by +creating a `GitReceiver` in its own namespace, so there is one mechanism, not two. + +The name avoids Flux's `Receiver` kind, which sits in the same clusters. + +## Not in these phases + +Each of these needs its own design when it is next on the list. None blocks phases 1 to 3. + +- **Folder and branch grants per namespace.** Until they exist, every namespace a `GitProvider` + admits may write any folder on any allowed branch. That suits trusted teams; it is not isolation + between tenants, and the docs must say so. +- **Two installations on one control cluster** (ask #28): disjoint namespace sets per installation, + with the cache, webhook, and status-writer consequences. +- **Durable pending writes and HA**: a persisted claim, fencing, and saves that survive a restart. + See the [HA plan](../future/ha-gittarget-distribution-plan.md). +- **Repository aliases**: URLs the normalizer cannot prove equal, such as vanity domains. +- **A branch-wide suspend**: target `suspend` covers today's need, and adding it later breaks + nothing. +- **The observability asks #31 to #33**: they are not about sharing and do not need this work. + +## Open questions per phase + +Phase 1: + +- Should `GitBranch` get printer columns for provider, branch, commit, and the holder's readiness? + [definitions.md](../definitions.md) rule 8 governs what a dependency projection may show. +- Is the receive-pack proof enough on the Git hosts the e2e suite covers, or does one of them need a + documented exception? + +Phase 2: + +- The flag name and chart value for the `inCluster` list, per + [config-flag-conventions.md](../config-flag-conventions.md). +- Whether the chart's local provider keeps the name `default` (less churn in examples) or becomes + `local` (no hint of a fallback). + +Phase 3: + +- Whether `GitReceiver` needs `accessFrom` at all, or whether only providers in its own namespace + may reference it. From b3d58ed65d32981eec2f4f00cac09ef74f550fe2 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Wed, 21 Oct 2026 18:36:11 +0000 Subject: [PATCH 2/6] feat(api): add the GitBranch kind GitBranch is one branch of a repository, written through a GitProvider in its own namespace. GitTarget gains spec.gitBranchRef. Nothing reads either yet; gitProviderRef and branch are optional until they leave the target. Co-Authored-By: Claude Opus 5.5 (1M context) --- PROJECT | 8 + api/v1alpha3/examples_test.go | 2 + api/v1alpha3/gitbranch_types.go | 211 +++++++++++++ api/v1alpha3/gittarget_types.go | 116 ++------ api/v1alpha3/status_accessors.go | 6 + api/v1alpha3/zz_generated.deepcopy.go | 187 +++++++++--- .../bases/configbutler.ai_gitbranches.yaml | 280 ++++++++++++++++++ .../crd/bases/configbutler.ai_gittargets.yaml | 57 ++-- config/crd/kustomization.yaml | 1 + docs/configuration.md | 15 +- hack/crdfields/fields.yaml | 25 ++ .../gittarget_parent_branch_test.go | 12 +- internal/controller/gittarget_remote.go | 12 +- internal/controller/gittarget_remote_test.go | 10 +- .../controller/renamed_required_field_test.go | 258 ---------------- 15 files changed, 778 insertions(+), 422 deletions(-) create mode 100644 api/v1alpha3/gitbranch_types.go create mode 100644 config/crd/bases/configbutler.ai_gitbranches.yaml delete mode 100644 internal/controller/renamed_required_field_test.go diff --git a/PROJECT b/PROJECT index bfcca565..62e4671e 100644 --- a/PROJECT +++ b/PROJECT @@ -25,6 +25,14 @@ resources: kind: GitTarget path: github.com/ConfigButler/gitops-reverser/api/v1alpha3 version: v1alpha3 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: configbutler.ai + kind: GitBranch + path: github.com/ConfigButler/gitops-reverser/api/v1alpha3 + version: v1alpha3 - api: crdVersion: v1 namespaced: true diff --git a/api/v1alpha3/examples_test.go b/api/v1alpha3/examples_test.go index a90ef29b..525483fd 100644 --- a/api/v1alpha3/examples_test.go +++ b/api/v1alpha3/examples_test.go @@ -124,6 +124,8 @@ func newExampleObject(kind string) any { switch kind { case "GitProvider": return &GitProvider{} + case "GitBranch": + return &GitBranch{} case "GitTarget": return &GitTarget{} case "ClusterProvider": diff --git a/api/v1alpha3/gitbranch_types.go b/api/v1alpha3/gitbranch_types.go new file mode 100644 index 00000000..9864340f --- /dev/null +++ b/api/v1alpha3/gitbranch_types.go @@ -0,0 +1,211 @@ +// SPDX-License-Identifier: Apache-2.0 + +package v1alpha3 + +import ( + meta "github.com/fluxcd/pkg/apis/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" +) + +// GitBranchSpec defines the desired state of GitBranch. +// +// Every field is immutable: together they are the repository branch this object claims, and a +// claim that could be repointed would move a branch worker's checkout, credential and pending +// writes to another branch under the GitTargets that reference it. Delete and recreate instead. +// +kubebuilder:validation:XValidation:rule="self.gitProviderRef == oldSelf.gitProviderRef",message="spec.gitProviderRef is immutable; delete and recreate the GitBranch to change it" +// +kubebuilder:validation:XValidation:rule="self.branch == oldSelf.branch",message="spec.branch is immutable; delete and recreate the GitBranch to change it" +// +kubebuilder:validation:XValidation:rule="has(self.parentBranch) == has(oldSelf.parentBranch) && (!has(self.parentBranch) || self.parentBranch == oldSelf.parentBranch)",message="spec.parentBranch is immutable; delete and recreate the GitBranch to change it" +type GitBranchSpec struct { + // A LocalObjectReference on purpose: creating a GitBranch then needs the right to create objects + // in the namespace that holds the credential, which is what makes a branch claim an act of the + // credential's owner. Other namespaces reach the branch through the GitProvider's accessFrom. + + // GitProviderRef names the GitProvider, in this GitBranch's namespace, whose repository and + // credential this branch is written with. GitTargets in other namespaces may reference this + // GitBranch while that GitProvider's spec.accessFrom admits their namespace. + // Immutable: delete and recreate the GitBranch to change it. + // +required + // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.gitProviderRef.name must not be empty" + GitProviderRef meta.LocalObjectReference `json:"gitProviderRef"` + + // Branch is the branch on the remote that this GitBranch writes. It must match one of the + // GitProvider's allowedBranches. Only one GitBranch writes a repository branch: the oldest one + // whose credential proves it can open a push session holds it, and any other GitBranch for the + // same repository and branch reports Stalled with reason BranchConflict. + // Immutable: delete and recreate the GitBranch to change it. + // +required + // +kubebuilder:validation:MinLength=1 + Branch string `json:"branch"` + + // Why "parent" and not base/source/revision: docs/definitions.md reserves those words, and the + // field is flat because it is one fact. It decides what history a new write branch gets, and a + // parent changed under a worker holding writes is the one case that needed its own replay, so it + // is immutable. It can never hold a commit or tag: a pinned starting point would be a sibling. + + // ParentBranch is the branch spec.branch is created from, and the branch compared against while + // spec.branch does not exist on the remote. Omitted: the branch the remote's HEAD points at (its + // default branch), or a branch with no history when the repository is empty. When set, it must + // exist on the remote for spec.branch to be created; otherwise this GitBranch reports + // ParentBranchNotFound and nothing is written. Once spec.branch exists, it is used as it is and + // the parent is not followed. It is only read, so it does not need to be in the GitProvider's + // allowedBranches. + // Immutable, including adding or removing it: delete and recreate the GitBranch to change it. + // +optional + // +kubebuilder:validation:MinLength=1 + ParentBranch string `json:"parentBranch,omitempty"` +} + +// GitBranchStatus defines the observed state of GitBranch. +type GitBranchStatus struct { + // ObservedGeneration is the latest generation observed by the controller. + // +optional + ObservedGeneration int64 `json:"observedGeneration,omitempty"` + + // Conditions report whether this GitBranch holds its repository branch and can write it: the + // Ready summary plus the kstatus Reconciling/Stalled pair. + // +optional + // +listType=map + // +listMapKey=type + // +patchMergeKey=type + // +patchStrategy=merge + Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` + + // Repository is the GitProvider's repository URL, normalized: host and path, with no scheme, + // user or ".git" suffix. Together with spec.branch it is the key only one GitBranch may hold, so + // two GitProviders spelling one repository differently still conflict. + // +optional + Repository string `json:"repository,omitempty"` + + // Remote is where this branch is on the Git remote, and when that was last proved. It is + // renewed by every confirmed look: a successful push as well as a fetch. Absent means nothing + // has proved anything yet: nothing has looked, or the GitProvider was recreated against a + // different repository and what was published no longer describes the one it names. + // +optional + Remote *GitBranchRemoteStatus `json:"remote,omitempty"` + + // GitTargetCount is how many GitTargets reference this GitBranch, from any namespace. More than + // one means they share its branch worker; each reports its own folder's health on its own + // conditions. + // +optional + GitTargetCount int32 `json:"gitTargetCount"` +} + +// GitBranchRemoteStatus is the answer to "where is my branch, and when did we last prove it". +// +// It is sampled when the GitBranch reconciles, from what its branch worker last proved about the +// branch, and written at most once a minute. The first observation, and a correction after the +// repository changed under the branch, are written at once. +type GitBranchRemoteStatus struct { + // Commit is the commit hash the branch is at on the remote. EMPTY means the branch is not on + // the remote at all, which is not an error: a branch does not exist without a commit, and a + // branch nothing has written to has nothing there yet. + // +optional + Commit string `json:"commit,omitempty"` + + // LastVerifiedAt is when the remote was last observed. + // +optional + LastVerifiedAt *metav1.Time `json:"lastVerifiedAt,omitempty"` + + // VerifiedBy is what proved it: `Push` means the server accepted a ref update of ours, so + // this commit is our own work; `Fetch` means we went and looked, and this is what was + // there. A `Fetch` next to a commit no publication of yours produced is how a foreign push + // to the branch is read off kubectl. + // +optional + // +kubebuilder:validation:Enum=Push;Fetch + VerifiedBy string `json:"verifiedBy,omitempty"` + + // Parent is what the branch would be created from, observed with it: present only while the + // remote does not carry the branch. Once the branch exists its parent is not followed, so this + // is omitted. + // +optional + Parent *GitBranchParentStatus `json:"parent,omitempty"` +} + +// GitBranchParentState is what a branch that does not exist yet would be created from. +// +kubebuilder:validation:Enum=Found;Missing;Unborn +type GitBranchParentState string + +const ( + // GitBranchParentFound means the parent branch exists; the first commit builds on its tip. + GitBranchParentFound GitBranchParentState = "Found" + // GitBranchParentMissing means the parent is not on the remote either, so the branch cannot be + // created; the GitBranch reports ParentBranchNotFound. The parent is spec.parentBranch, or, when + // that is omitted, the default branch of a repository that is not empty but whose HEAD names no + // branch it carries. + GitBranchParentMissing GitBranchParentState = "Missing" + // GitBranchParentUnborn means the repository is empty, with no refs at all (tags included), and + // no parent is configured, so the first commit starts the branch with no history. + GitBranchParentUnborn GitBranchParentState = "Unborn" +) + +// GitBranchParentStatus is the parent branch of a branch that does not exist yet. +type GitBranchParentStatus struct { + // State is what the branch would be created from: Found (the parent's tip), Missing (the parent + // is not on the remote; nothing is written) or Unborn (an empty repository, with no refs at + // all, and no parent configured; the first commit has no history). + // +required + State GitBranchParentState `json:"state"` + + // Branch is the parent branch: spec.parentBranch, or the branch the remote's HEAD names when + // that is omitted. Empty when State is Unborn, and when the remote names no default branch. + // +optional + Branch string `json:"branch,omitempty"` + + // Commit is the commit hash the parent branch is at on the remote: the commit the branch's + // first commit would build on. Set only when State is Found. + // +optional + Commit string `json:"commit,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="Provider",type=string,JSONPath=`.spec.gitProviderRef.name` +// +kubebuilder:printcolumn:name="Branch",type=string,JSONPath=`.spec.branch` +// +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.status.remote.commit` +// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` +// +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason`,priority=1 +// +kubebuilder:printcolumn:name="Repository",type=string,JSONPath=`.status.repository`,priority=1 +// +kubebuilder:printcolumn:name="Targets",type=integer,JSONPath=`.status.gitTargetCount`,priority=1 +// +kubebuilder:printcolumn:name="Verified",type=date,JSONPath=`.status.remote.lastVerifiedAt`,priority=1 +// +kubebuilder:printcolumn:name="Message",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].message`,priority=1 +// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` + +// GitBranch is one branch of a Git repository, written through a GitProvider's credential. It holds +// the branch for every GitTarget that references it: one branch worker, one credential, one +// checkout. Only one GitBranch may hold a repository branch at a time. +type GitBranch struct { + metav1.TypeMeta `json:",inline"` + + // metadata is a standard object metadata + // +optional + metav1.ObjectMeta `json:"metadata,omitempty,omitzero"` + + // spec defines the desired state of GitBranch + // +required + Spec GitBranchSpec `json:"spec"` + + // status defines the observed state of GitBranch + // +optional + Status GitBranchStatus `json:"status,omitempty,omitzero"` +} + +// GitProviderKey is the GitProvider this GitBranch writes through: spec.gitProviderRef, in this +// GitBranch's own namespace. +func (b *GitBranch) GitProviderKey() types.NamespacedName { + return types.NamespacedName{Namespace: b.Namespace, Name: b.Spec.GitProviderRef.Name} +} + +// +kubebuilder:object:root=true + +// GitBranchList contains a list of GitBranch. +type GitBranchList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + + Items []GitBranch `json:"items"` +} + +func init() { + SchemeBuilder.Register(&GitBranch{}, &GitBranchList{}) +} diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 6ec51758..9d09457f 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -10,7 +10,7 @@ import ( // GitTargetSpec defines the desired state of GitTarget. // -// The destination fields — gitProviderRef, branch, and path — are immutable. A +// The destination fields — gitBranchRef and path — are immutable. A // GitTarget materializes the watched resources at exactly one (provider, branch, // folder); changing where it writes would orphan the old materialization and require // migrating manifests between repositories/branches/folders. Instead of reconciling @@ -19,15 +19,9 @@ import ( // simple — a successful snapshot can never be silently invalidated by a destination // change. // -// The gitProviderRef rule is guarded on oldSelf rather than written as a plain equality, and the -// guard is a one-way migration door rather than a loosening. A GitTarget stored before this field -// was named gitProviderRef serves NO value for it (a field outside the structural schema is not -// served, measured in TestRenamedRequiredField_StoredObjectCanAdoptIt), so a plain equality would -// reject the very apply that migrates it and force a delete-and-recreate. Because the field is -// REQUIRED, it can never be absent on an object created from this release on, so the door is shut -// for everything except the objects it exists for. +// +kubebuilder:validation:XValidation:rule="has(self.gitBranchRef) == has(oldSelf.gitBranchRef) && (!has(self.gitBranchRef) || self.gitBranchRef == oldSelf.gitBranchRef)",message="spec.gitBranchRef is immutable; delete and recreate the GitTarget to change its destination" // +kubebuilder:validation:XValidation:rule="!has(oldSelf.gitProviderRef) || self.gitProviderRef == oldSelf.gitProviderRef",message="spec.gitProviderRef is immutable; delete and recreate the GitTarget to change its destination" -// +kubebuilder:validation:XValidation:rule="self.branch == oldSelf.branch",message="spec.branch is immutable; delete and recreate the GitTarget to change its destination" +// +kubebuilder:validation:XValidation:rule="has(self.branch) == has(oldSelf.branch) && (!has(self.branch) || self.branch == oldSelf.branch)",message="spec.branch is immutable; delete and recreate the GitTarget to change its destination" // +kubebuilder:validation:XValidation:rule="self.path == oldSelf.path",message="spec.path is immutable; delete and recreate the GitTarget to change its destination" // +kubebuilder:validation:XValidation:rule="has(self.parentBranch) == has(oldSelf.parentBranch) && (!has(self.parentBranch) || self.parentBranch == oldSelf.parentBranch)",message="spec.parentBranch is immutable; delete and recreate the GitTarget to change it" // @@ -38,21 +32,28 @@ import ( // +kubebuilder:validation:XValidation:rule="self.clusterProviderRef == oldSelf.clusterProviderRef",message="spec.clusterProviderRef is immutable; delete and recreate the GitTarget to change the cluster it mirrors" // +kubebuilder:validation:XValidation:rule="!has(self.onRefusal) || self.onRefusal != 'Reapply' || has(self.applyTriggerRef)",message="spec.onRefusal: Reapply needs spec.applyTriggerRef, the trigger asked to reapply" type GitTargetSpec struct { + // GitBranchRef names the GitBranch this target writes: the repository branch, and the credential + // it is written with. Namespace omitted means this GitTarget's own namespace. A GitBranch in + // another namespace is used only while its GitProvider's spec.accessFrom admits this GitTarget's + // namespace. Many GitTargets may reference one GitBranch; they share its branch worker. + // Immutable: delete and recreate the GitTarget to change its destination. + // +optional + // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.gitBranchRef.name must not be empty" + GitBranchRef *meta.NamespacedObjectReference `json:"gitBranchRef,omitempty"` + // GitProviderRef names the GitProvider that backs this target. Namespace omitted means this // GitTarget's own namespace. A GitProvider in another namespace is used only while its // spec.accessFrom admits this GitTarget's namespace. Many GitTargets may name the same // GitProvider, and every GitTarget writing one repository branch must name the same one. // Immutable: delete and recreate the GitTarget to change its destination. - // +required - // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.gitProviderRef.name must not be empty" - GitProviderRef meta.NamespacedObjectReference `json:"gitProviderRef"` + // +optional + GitProviderRef meta.NamespacedObjectReference `json:"gitProviderRef,omitzero"` // Branch to use for this target. // Must be one of the allowed branches in the provider. // Immutable: delete and recreate the GitTarget to change its destination. - // +required - // +kubebuilder:validation:MinLength=1 - Branch string `json:"branch"` + // +optional + Branch string `json:"branch,omitempty"` // Why "parent" and not base/source/revision: docs/definitions.md reserves those words, and the // field is flat because it is one fact. It is immutable like branch and path: it decides what @@ -446,76 +447,7 @@ type GitTargetStatus struct { // recreated against a different repository and what was published no longer describes the // one this target points at. // +optional - Remote *GitTargetRemoteStatus `json:"remote,omitempty"` -} - -// GitTargetRemoteStatus is the answer to "where is my branch, and when did we last prove it". -// -// It is sampled when the GitTarget reconciles, from what its branch worker last proved about the -// branch, and written at most once a minute. The first observation, and a correction after the -// repository changed under the target, are written at once. -type GitTargetRemoteStatus struct { - // Commit is the commit hash the branch is at on the remote. EMPTY means the branch is not on - // the remote at all, which is not an error: a branch does not exist without a commit, and a - // target that has never written has nothing there yet. - // +optional - Commit string `json:"commit,omitempty"` - - // LastVerifiedAt is when the remote was last observed. It answers "has anything looked", - // which placement.resolvedAtCommit deliberately does not: that one dates the resolution, so - // an old value there means the layout has not changed rather than that nothing has looked. - // +optional - LastVerifiedAt *metav1.Time `json:"lastVerifiedAt,omitempty"` - - // VerifiedBy is what proved it: `Push` means the server accepted a ref update of ours, so - // this commit is our own work; `Fetch` means we went and looked, and this is what was - // there. A `Fetch` next to a commit no publication of yours produced is how a foreign push - // to the branch is read off kubectl. - // +optional - // +kubebuilder:validation:Enum=Push;Fetch - VerifiedBy string `json:"verifiedBy,omitempty"` - - // Parent is what the write branch would be created from, observed with it: present only - // while the remote does not carry the write branch. Once the branch exists its parent is not - // followed, so this is omitted. - // +optional - Parent *GitTargetParentStatus `json:"parent,omitempty"` -} - -// GitTargetParentState is what a write branch that does not exist yet would be created from. -// +kubebuilder:validation:Enum=Found;Missing;Unborn -type GitTargetParentState string - -const ( - // GitTargetParentFound means the parent branch exists; the first commit builds on its tip. - GitTargetParentFound GitTargetParentState = "Found" - // GitTargetParentMissing means the parent is not on the remote either, so the write branch - // cannot be created; the GitTarget reports ParentBranchNotFound. The parent is spec.parentBranch, - // or, when that is omitted, the default branch of a repository that is not empty but whose HEAD - // names no branch it carries. - GitTargetParentMissing GitTargetParentState = "Missing" - // GitTargetParentUnborn means the repository is empty, with no refs at all (tags included), and - // no parent is configured, so the first commit starts the write branch with no history. - GitTargetParentUnborn GitTargetParentState = "Unborn" -) - -// GitTargetParentStatus is the parent branch of a write branch that does not exist yet. -type GitTargetParentStatus struct { - // State is what the write branch would be created from: Found (the parent's tip), Missing - // (the parent is not on the remote; nothing is written) or Unborn (an empty repository, with no - // refs at all, and no parent configured; the first commit has no history). - // +required - State GitTargetParentState `json:"state"` - - // Branch is the parent branch: spec.parentBranch, or the branch the remote's HEAD names when - // that is omitted. Empty when State is Unborn, and when the remote names no default branch. - // +optional - Branch string `json:"branch,omitempty"` - - // Commit is the commit hash the parent branch is at on the remote: the commit the write - // branch's first commit would build on. Set only when State is Found. - // +optional - Commit string `json:"commit,omitempty"` + Remote *GitBranchRemoteStatus `json:"remote,omitempty"` } // Two rules for anything added here. A field earns its place only if a reader cannot get it from @@ -684,6 +616,20 @@ func (g *GitTarget) GitProviderKey() types.NamespacedName { return types.NamespacedName{Namespace: namespace, Name: g.Spec.GitProviderRef.Name} } +// GitBranchKey is the GitBranch this GitTarget writes: spec.gitBranchRef, in this GitTarget's own +// namespace unless the reference names another one. An unset reference names nothing, and the zero +// key says exactly that. +func (g *GitTarget) GitBranchKey() types.NamespacedName { + if g.Spec.GitBranchRef == nil { + return types.NamespacedName{} + } + namespace := g.Spec.GitBranchRef.Namespace + if namespace == "" { + namespace = g.Namespace + } + return types.NamespacedName{Namespace: namespace, Name: g.Spec.GitBranchRef.Name} +} + // IsLocalSource reports whether this GitTarget references the "default" ClusterProvider, which the // watch data plane maps to its local cluster context. It is a NAME test, not a claim about the // physical cluster: a "default" provider may carry a kubeConfig. It only supplies the pre-discovery diff --git a/api/v1alpha3/status_accessors.go b/api/v1alpha3/status_accessors.go index bb3d04cb..0bfc2564 100644 --- a/api/v1alpha3/status_accessors.go +++ b/api/v1alpha3/status_accessors.go @@ -25,6 +25,12 @@ func (t *GitTarget) SetObservedGeneration(generation int64) { t.Status.ObservedG // StatusConditions returns the condition set to write through. func (t *GitTarget) StatusConditions() *[]metav1.Condition { return &t.Status.Conditions } +// SetObservedGeneration records the generation this object's status describes. +func (b *GitBranch) SetObservedGeneration(generation int64) { b.Status.ObservedGeneration = generation } + +// StatusConditions returns the condition set to write through. +func (b *GitBranch) StatusConditions() *[]metav1.Condition { return &b.Status.Conditions } + // SetObservedGeneration records the generation this object's status describes. func (p *GitProvider) SetObservedGeneration(generation int64) { p.Status.ObservedGeneration = generation diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index 744c9166..d01319ab 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -927,6 +927,147 @@ func (in *FluxTriggerStatus) DeepCopy() *FluxTriggerStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitBranch) DeepCopyInto(out *GitBranch) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + out.Spec = in.Spec + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitBranch. +func (in *GitBranch) DeepCopy() *GitBranch { + if in == nil { + return nil + } + out := new(GitBranch) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *GitBranch) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitBranchList) DeepCopyInto(out *GitBranchList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]GitBranch, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitBranchList. +func (in *GitBranchList) DeepCopy() *GitBranchList { + if in == nil { + return nil + } + out := new(GitBranchList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *GitBranchList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitBranchParentStatus) DeepCopyInto(out *GitBranchParentStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitBranchParentStatus. +func (in *GitBranchParentStatus) DeepCopy() *GitBranchParentStatus { + if in == nil { + return nil + } + out := new(GitBranchParentStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitBranchRemoteStatus) DeepCopyInto(out *GitBranchRemoteStatus) { + *out = *in + if in.LastVerifiedAt != nil { + in, out := &in.LastVerifiedAt, &out.LastVerifiedAt + *out = (*in).DeepCopy() + } + if in.Parent != nil { + in, out := &in.Parent, &out.Parent + *out = new(GitBranchParentStatus) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitBranchRemoteStatus. +func (in *GitBranchRemoteStatus) DeepCopy() *GitBranchRemoteStatus { + if in == nil { + return nil + } + out := new(GitBranchRemoteStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitBranchSpec) DeepCopyInto(out *GitBranchSpec) { + *out = *in + out.GitProviderRef = in.GitProviderRef +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitBranchSpec. +func (in *GitBranchSpec) DeepCopy() *GitBranchSpec { + if in == nil { + return nil + } + out := new(GitBranchSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitBranchStatus) DeepCopyInto(out *GitBranchStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Remote != nil { + in, out := &in.Remote, &out.Remote + *out = new(GitBranchRemoteStatus) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitBranchStatus. +func (in *GitBranchStatus) DeepCopy() *GitBranchStatus { + if in == nil { + return nil + } + out := new(GitBranchStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitProvider) DeepCopyInto(out *GitProvider) { *out = *in @@ -1177,21 +1318,6 @@ func (in *GitTargetList) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *GitTargetParentStatus) DeepCopyInto(out *GitTargetParentStatus) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetParentStatus. -func (in *GitTargetParentStatus) DeepCopy() *GitTargetParentStatus { - if in == nil { - return nil - } - out := new(GitTargetParentStatus) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetPlacementSpec) DeepCopyInto(out *GitTargetPlacementSpec) { *out = *in @@ -1238,30 +1364,6 @@ func (in *GitTargetPlacementStatus) DeepCopy() *GitTargetPlacementStatus { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *GitTargetRemoteStatus) DeepCopyInto(out *GitTargetRemoteStatus) { - *out = *in - if in.LastVerifiedAt != nil { - in, out := &in.LastVerifiedAt, &out.LastVerifiedAt - *out = (*in).DeepCopy() - } - if in.Parent != nil { - in, out := &in.Parent, &out.Parent - *out = new(GitTargetParentStatus) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetRemoteStatus. -func (in *GitTargetRemoteStatus) DeepCopy() *GitTargetRemoteStatus { - if in == nil { - return nil - } - out := new(GitTargetRemoteStatus) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetRetentionStatus) DeepCopyInto(out *GitTargetRetentionStatus) { *out = *in @@ -1284,6 +1386,11 @@ func (in *GitTargetRetentionStatus) DeepCopy() *GitTargetRetentionStatus { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = *in + if in.GitBranchRef != nil { + in, out := &in.GitBranchRef, &out.GitBranchRef + *out = new(meta.NamespacedObjectReference) + **out = **in + } out.GitProviderRef = in.GitProviderRef if in.Encryption != nil { in, out := &in.Encryption, &out.Encryption @@ -1359,7 +1466,7 @@ func (in *GitTargetStatus) DeepCopyInto(out *GitTargetStatus) { } if in.Remote != nil { in, out := &in.Remote, &out.Remote - *out = new(GitTargetRemoteStatus) + *out = new(GitBranchRemoteStatus) (*in).DeepCopyInto(*out) } } diff --git a/config/crd/bases/configbutler.ai_gitbranches.yaml b/config/crd/bases/configbutler.ai_gitbranches.yaml new file mode 100644 index 00000000..e2d06445 --- /dev/null +++ b/config/crd/bases/configbutler.ai_gitbranches.yaml @@ -0,0 +1,280 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.22.0 + name: gitbranches.configbutler.ai +spec: + group: configbutler.ai + names: + kind: GitBranch + listKind: GitBranchList + plural: gitbranches + singular: gitbranch + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.gitProviderRef.name + name: Provider + type: string + - jsonPath: .spec.branch + name: Branch + type: string + - jsonPath: .status.remote.commit + name: Commit + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].reason + name: Reason + priority: 1 + type: string + - jsonPath: .status.repository + name: Repository + priority: 1 + type: string + - jsonPath: .status.gitTargetCount + name: Targets + priority: 1 + type: integer + - jsonPath: .status.remote.lastVerifiedAt + name: Verified + priority: 1 + type: date + - jsonPath: .status.conditions[?(@.type=="Ready")].message + name: Message + priority: 1 + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha3 + schema: + openAPIV3Schema: + description: |- + GitBranch is one branch of a Git repository, written through a GitProvider's credential. It holds + the branch for every GitTarget that references it: one branch worker, one credential, one + checkout. Only one GitBranch may hold a repository branch at a time. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: spec defines the desired state of GitBranch + properties: + branch: + description: |- + Branch is the branch on the remote that this GitBranch writes. It must match one of the + GitProvider's allowedBranches. Only one GitBranch writes a repository branch: the oldest one + whose credential proves it can open a push session holds it, and any other GitBranch for the + same repository and branch reports Stalled with reason BranchConflict. + Immutable: delete and recreate the GitBranch to change it. + minLength: 1 + type: string + gitProviderRef: + description: |- + GitProviderRef names the GitProvider, in this GitBranch's namespace, whose repository and + credential this branch is written with. GitTargets in other namespaces may reference this + GitBranch while that GitProvider's spec.accessFrom admits their namespace. + Immutable: delete and recreate the GitBranch to change it. + properties: + name: + description: Name of the referent. + type: string + required: + - name + type: object + x-kubernetes-validations: + - message: spec.gitProviderRef.name must not be empty + rule: self.name != '' + parentBranch: + description: |- + ParentBranch is the branch spec.branch is created from, and the branch compared against while + spec.branch does not exist on the remote. Omitted: the branch the remote's HEAD points at (its + default branch), or a branch with no history when the repository is empty. When set, it must + exist on the remote for spec.branch to be created; otherwise this GitBranch reports + ParentBranchNotFound and nothing is written. Once spec.branch exists, it is used as it is and + the parent is not followed. It is only read, so it does not need to be in the GitProvider's + allowedBranches. + Immutable, including adding or removing it: delete and recreate the GitBranch to change it. + minLength: 1 + type: string + required: + - branch + - gitProviderRef + type: object + x-kubernetes-validations: + - message: spec.gitProviderRef is immutable; delete and recreate the GitBranch + to change it + rule: self.gitProviderRef == oldSelf.gitProviderRef + - message: spec.branch is immutable; delete and recreate the GitBranch + to change it + rule: self.branch == oldSelf.branch + - message: spec.parentBranch is immutable; delete and recreate the GitBranch + to change it + rule: has(self.parentBranch) == has(oldSelf.parentBranch) && (!has(self.parentBranch) + || self.parentBranch == oldSelf.parentBranch) + status: + description: status defines the observed state of GitBranch + properties: + conditions: + description: |- + Conditions report whether this GitBranch holds its repository branch and can write it: the + Ready summary plus the kstatus Reconciling/Stalled pair. + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gitTargetCount: + description: |- + GitTargetCount is how many GitTargets reference this GitBranch, from any namespace. More than + one means they share its branch worker; each reports its own folder's health on its own + conditions. + format: int32 + type: integer + observedGeneration: + description: ObservedGeneration is the latest generation observed + by the controller. + format: int64 + type: integer + remote: + description: |- + Remote is where this branch is on the Git remote, and when that was last proved. It is + renewed by every confirmed look: a successful push as well as a fetch. Absent means nothing + has proved anything yet: nothing has looked, or the GitProvider was recreated against a + different repository and what was published no longer describes the one it names. + properties: + commit: + description: |- + Commit is the commit hash the branch is at on the remote. EMPTY means the branch is not on + the remote at all, which is not an error: a branch does not exist without a commit, and a + branch nothing has written to has nothing there yet. + type: string + lastVerifiedAt: + description: LastVerifiedAt is when the remote was last observed. + format: date-time + type: string + parent: + description: |- + Parent is what the branch would be created from, observed with it: present only while the + remote does not carry the branch. Once the branch exists its parent is not followed, so this + is omitted. + properties: + branch: + description: |- + Branch is the parent branch: spec.parentBranch, or the branch the remote's HEAD names when + that is omitted. Empty when State is Unborn, and when the remote names no default branch. + type: string + commit: + description: |- + Commit is the commit hash the parent branch is at on the remote: the commit the branch's + first commit would build on. Set only when State is Found. + type: string + state: + description: |- + State is what the branch would be created from: Found (the parent's tip), Missing (the parent + is not on the remote; nothing is written) or Unborn (an empty repository, with no refs at + all, and no parent configured; the first commit has no history). + enum: + - Found + - Missing + - Unborn + type: string + required: + - state + type: object + verifiedBy: + description: |- + VerifiedBy is what proved it: `Push` means the server accepted a ref update of ours, so + this commit is our own work; `Fetch` means we went and looked, and this is what was + there. A `Fetch` next to a commit no publication of yours produced is how a foreign push + to the branch is read off kubectl. + enum: + - Push + - Fetch + type: string + type: object + repository: + description: |- + Repository is the GitProvider's repository URL, normalized: host and path, with no scheme, + user or ".git" suffix. Together with spec.branch it is the key only one GitBranch may hold, so + two GitProviders spelling one repository differently still conflict. + type: string + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index afc2f239..b724c713 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -144,7 +144,6 @@ spec: Branch to use for this target. Must be one of the allowed branches in the provider. Immutable: delete and recreate the GitTarget to change its destination. - minLength: 1 type: string clusterProviderRef: default: @@ -321,6 +320,27 @@ spec: required: - provider type: object + gitBranchRef: + description: |- + GitBranchRef names the GitBranch this target writes: the repository branch, and the credential + it is written with. Namespace omitted means this GitTarget's own namespace. A GitBranch in + another namespace is used only while its GitProvider's spec.accessFrom admits this GitTarget's + namespace. Many GitTargets may reference one GitBranch; they share its branch worker. + Immutable: delete and recreate the GitTarget to change its destination. + properties: + name: + description: Name of the referent. + type: string + namespace: + description: Namespace of the referent, when not specified it + acts as LocalObjectReference. + type: string + required: + - name + type: object + x-kubernetes-validations: + - message: spec.gitBranchRef.name must not be empty + rule: self.name != '' gitProviderRef: description: |- GitProviderRef names the GitProvider that backs this target. Namespace omitted means this @@ -339,9 +359,6 @@ spec: required: - name type: object - x-kubernetes-validations: - - message: spec.gitProviderRef.name must not be empty - rule: self.name != '' onRefusal: default: Ignore description: |- @@ -541,17 +558,20 @@ spec: resync; the writes suppressed while suspended are not replayed. type: boolean required: - - branch - - gitProviderRef - path type: object x-kubernetes-validations: + - message: spec.gitBranchRef is immutable; delete and recreate the GitTarget + to change its destination + rule: has(self.gitBranchRef) == has(oldSelf.gitBranchRef) && (!has(self.gitBranchRef) + || self.gitBranchRef == oldSelf.gitBranchRef) - message: spec.gitProviderRef is immutable; delete and recreate the GitTarget to change its destination rule: '!has(oldSelf.gitProviderRef) || self.gitProviderRef == oldSelf.gitProviderRef' - message: spec.branch is immutable; delete and recreate the GitTarget to change its destination - rule: self.branch == oldSelf.branch + rule: has(self.branch) == has(oldSelf.branch) && (!has(self.branch) + || self.branch == oldSelf.branch) - message: spec.path is immutable; delete and recreate the GitTarget to change its destination rule: self.path == oldSelf.path @@ -714,20 +734,17 @@ spec: description: |- Commit is the commit hash the branch is at on the remote. EMPTY means the branch is not on the remote at all, which is not an error: a branch does not exist without a commit, and a - target that has never written has nothing there yet. + branch nothing has written to has nothing there yet. type: string lastVerifiedAt: - description: |- - LastVerifiedAt is when the remote was last observed. It answers "has anything looked", - which placement.resolvedAtCommit deliberately does not: that one dates the resolution, so - an old value there means the layout has not changed rather than that nothing has looked. + description: LastVerifiedAt is when the remote was last observed. format: date-time type: string parent: description: |- - Parent is what the write branch would be created from, observed with it: present only - while the remote does not carry the write branch. Once the branch exists its parent is not - followed, so this is omitted. + Parent is what the branch would be created from, observed with it: present only while the + remote does not carry the branch. Once the branch exists its parent is not followed, so this + is omitted. properties: branch: description: |- @@ -736,14 +753,14 @@ spec: type: string commit: description: |- - Commit is the commit hash the parent branch is at on the remote: the commit the write - branch's first commit would build on. Set only when State is Found. + Commit is the commit hash the parent branch is at on the remote: the commit the branch's + first commit would build on. Set only when State is Found. type: string state: description: |- - State is what the write branch would be created from: Found (the parent's tip), Missing - (the parent is not on the remote; nothing is written) or Unborn (an empty repository, with no - refs at all, and no parent configured; the first commit has no history). + State is what the branch would be created from: Found (the parent's tip), Missing (the parent + is not on the remote; nothing is written) or Unborn (an empty repository, with no refs at + all, and no parent configured; the first commit has no history). enum: - Found - Missing diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 802252a2..a5e53d72 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -6,6 +6,7 @@ resources: - bases/configbutler.ai_clusterwatchrules.yaml - bases/configbutler.ai_commitrequests.yaml - bases/configbutler.ai_fluxtriggers.yaml + - bases/configbutler.ai_gitbranches.yaml - bases/configbutler.ai_gitproviders.yaml - bases/configbutler.ai_gittargets.yaml - bases/configbutler.ai_watchrules.yaml diff --git a/docs/configuration.md b/docs/configuration.md index 5b0f82f5..cd264eea 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -71,6 +71,16 @@ fields that need more than a line. | `commit.signing` | off | SSH commit signing. See [commit signing](#commit-signing) | | `receiver` | off | Push webhook endpoint, so the Git host can say somebody pushed. See [receiving push notifications](#receiving-push-notifications-specreceiver) | +### `GitBranch` (namespaced): one branch of the repository + +Every field is immutable: to change one, delete the `GitBranch` and create a new one. + +| Field | Default | What it does | +|---|---|---| +| `gitProviderRef` | **required** | The `GitProvider`, in this namespace, whose repository and credential write this branch | +| `branch` | **required** | Branch to write. Must be in the provider's `allowedBranches`. Only one `GitBranch` holds a repository branch | +| `parentBranch` | the remote's default branch | Branch a `branch` the remote does not have yet is created from. Only read, so it need not be allowed. See [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch) | + ### `ClusterProvider` (cluster-scoped): the source cluster | Field | Default | What it does | @@ -89,8 +99,9 @@ The destination fields are immutable: to move a target, delete it and create a n | Field | Default | What it does | |---|---|---| -| `gitProviderRef` | **required** | The `GitProvider` backing this target: its own namespace, or `namespace` names one whose `accessFrom` admits it. Every target on one repository branch names the same one | -| `branch` | **required** | Branch to write. Must be in the provider's `allowedBranches` | +| `gitBranchRef` | `gitProviderRef` and `branch` instead | The `GitBranch` this target writes: its own namespace, or `namespace` names one whose `GitProvider`'s `accessFrom` admits it | +| `gitProviderRef` | `gitBranchRef` instead | The `GitProvider` backing this target: its own namespace, or `namespace` names one whose `accessFrom` admits it. Every target on one repository branch names the same one | +| `branch` | `gitBranchRef` instead | Branch to write. Must be in the provider's `allowedBranches` | | `parentBranch` | the remote's default branch | Branch a `branch` the remote does not have yet is created from. Immutable. Only read, so it need not be allowed. See [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch) | | `path` | **required** | Folder within the repository. `.` targets the root; empty is rejected | | `clusterProviderRef` | `{"name":"default"}` | Source cluster to mirror from. The default names a `ClusterProvider` called `default` | diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index d57480a6..a772e139 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -61,6 +61,24 @@ kinds: Push webhook endpoint, so the Git host can say somebody pushed. See [receiving push notifications](#receiving-push-notifications-specreceiver) + - kind: GitBranch + title: "`GitBranch` (namespaced): one branch of the repository" + intro: "Every field is immutable: to change one, delete the `GitBranch` and create a new one." + fields: + - path: gitProviderRef + doc: >- + The `GitProvider`, in this namespace, whose repository and credential write this branch + - path: branch + doc: >- + Branch to write. Must be in the provider's `allowedBranches`. Only one `GitBranch` holds a + repository branch + - path: parentBranch + default: the remote's default branch + doc: >- + Branch a `branch` the remote does not have yet is created from. Only read, so it need not + be allowed. See + [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch) + - kind: ClusterProvider title: "`ClusterProvider` (cluster-scoped): the source cluster" fields: @@ -104,11 +122,18 @@ kinds: title: "`GitTarget` (namespaced): which branch and folder" intro: "The destination fields are immutable: to move a target, delete it and create a new one." fields: + - path: gitBranchRef + default: "`gitProviderRef` and `branch` instead" + doc: >- + The `GitBranch` this target writes: its own namespace, or `namespace` names one whose + `GitProvider`'s `accessFrom` admits it - path: gitProviderRef + default: "`gitBranchRef` instead" doc: >- The `GitProvider` backing this target: its own namespace, or `namespace` names one whose `accessFrom` admits it. Every target on one repository branch names the same one - path: branch + default: "`gitBranchRef` instead" doc: Branch to write. Must be in the provider's `allowedBranches` - path: parentBranch default: the remote's default branch diff --git a/internal/controller/gittarget_parent_branch_test.go b/internal/controller/gittarget_parent_branch_test.go index 5e219a66..974b8139 100644 --- a/internal/controller/gittarget_parent_branch_test.go +++ b/internal/controller/gittarget_parent_branch_test.go @@ -280,21 +280,21 @@ func TestParentBranchReadiness_IgnoresAnotherRepository(t *testing.T) { // a parent was named, and a moved parent is news even when the write branch is still absent. func TestParentStatusOf(t *testing.T) { absent := git.RemoteObservation{ParentState: git.ParentFound, ParentBranch: "main", ParentCommit: "aaaa"} - assert.Equal(t, &configbutleraiv1alpha3.GitTargetParentStatus{ - State: configbutleraiv1alpha3.GitTargetParentFound, Branch: "main", Commit: "aaaa", + assert.Equal(t, &configbutleraiv1alpha3.GitBranchParentStatus{ + State: configbutleraiv1alpha3.GitBranchParentFound, Branch: "main", Commit: "aaaa", }, parentStatusOf(absent)) assert.Nil(t, parentStatusOf(git.RemoteObservation{Commit: "bbbb"}), "an existing write branch does not follow its parent") assert.Nil(t, parentStatusOf(git.RemoteObservation{}), "an observation that names no parent state") - assert.Equal(t, &configbutleraiv1alpha3.GitTargetParentStatus{State: configbutleraiv1alpha3.GitTargetParentUnborn}, + assert.Equal(t, &configbutleraiv1alpha3.GitBranchParentStatus{State: configbutleraiv1alpha3.GitBranchParentUnborn}, parentStatusOf(git.RemoteObservation{ParentState: git.ParentUnborn}), "an empty repository") - assert.Equal(t, &configbutleraiv1alpha3.GitTargetParentStatus{ - State: configbutleraiv1alpha3.GitTargetParentMissing, Branch: "release", + assert.Equal(t, &configbutleraiv1alpha3.GitBranchParentStatus{ + State: configbutleraiv1alpha3.GitBranchParentMissing, Branch: "release", }, parentStatusOf(git.RemoteObservation{ParentState: git.ParentMissing, ParentBranch: "release"}), "a missing parent is named with no commit") at := metav1.NewTime(time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)) - published := &configbutleraiv1alpha3.GitTargetRemoteStatus{LastVerifiedAt: &at, Parent: parentStatusOf(absent)} + published := &configbutleraiv1alpha3.GitBranchRemoteStatus{LastVerifiedAt: &at, Parent: parentStatusOf(absent)} moved := published.DeepCopy() moved.Parent.Commit = "cccc" assert.True(t, remoteStatusIsNews(published, moved), "the parent moved") diff --git a/internal/controller/gittarget_remote.go b/internal/controller/gittarget_remote.go index bb389637..ebb2fb8c 100644 --- a/internal/controller/gittarget_remote.go +++ b/internal/controller/gittarget_remote.go @@ -143,7 +143,7 @@ func (r *GitTargetReconciler) publishRemote( // last thing anyone proved about it, and there is nothing better to replace it with. return 0 } - next := &configbutleraiv1alpha3.GitTargetRemoteStatus{ + next := &configbutleraiv1alpha3.GitBranchRemoteStatus{ Commit: observed.Commit, LastVerifiedAt: &metav1.Time{Time: observed.At}, VerifiedBy: string(observed.By), @@ -200,7 +200,7 @@ func differentRepository(a, b git.RepoIdentity) bool { // // News is not permission to write. It is the first of two tests; publishRemote applies the write // floor to whatever passes here. -func remoteStatusIsNews(published, next *configbutleraiv1alpha3.GitTargetRemoteStatus) bool { +func remoteStatusIsNews(published, next *configbutleraiv1alpha3.GitBranchRemoteStatus) bool { if published == nil || published.LastVerifiedAt == nil || next.LastVerifiedAt == nil { return true } @@ -214,18 +214,18 @@ func remoteStatusIsNews(published, next *configbutleraiv1alpha3.GitTargetRemoteS // parentStatusOf is the parent stanza of status.remote: present only while the observation found // the write branch absent, which is when it says what the branch would be created from. -func parentStatusOf(observed git.RemoteObservation) *configbutleraiv1alpha3.GitTargetParentStatus { +func parentStatusOf(observed git.RemoteObservation) *configbutleraiv1alpha3.GitBranchParentStatus { if observed.Commit != "" || observed.ParentState == "" { return nil } - return &configbutleraiv1alpha3.GitTargetParentStatus{ - State: configbutleraiv1alpha3.GitTargetParentState(observed.ParentState), + return &configbutleraiv1alpha3.GitBranchParentStatus{ + State: configbutleraiv1alpha3.GitBranchParentState(observed.ParentState), Branch: observed.ParentBranch, Commit: observed.ParentCommit, } } -func equalParentStatus(a, b *configbutleraiv1alpha3.GitTargetParentStatus) bool { +func equalParentStatus(a, b *configbutleraiv1alpha3.GitBranchParentStatus) bool { if a == nil || b == nil { return a == b } diff --git a/internal/controller/gittarget_remote_test.go b/internal/controller/gittarget_remote_test.go index 07857b25..2b3d2cea 100644 --- a/internal/controller/gittarget_remote_test.go +++ b/internal/controller/gittarget_remote_test.go @@ -199,11 +199,11 @@ func TestPublishRemote_AReProvedRevisionRefreshesTheClock(t *testing.T) { // reason, and this layer now agrees. func TestRemoteStatusIsNews_TheSameAnswerProvedASecondWayIsNot(t *testing.T) { at := metav1.NewTime(time.Now()) - published := &configbutleraiv1alpha3.GitTargetRemoteStatus{ + published := &configbutleraiv1alpha3.GitBranchRemoteStatus{ Commit: "aaaa", LastVerifiedAt: &at, VerifiedBy: "Push", } - assert.False(t, remoteStatusIsNews(published, &configbutleraiv1alpha3.GitTargetRemoteStatus{ + assert.False(t, remoteStatusIsNews(published, &configbutleraiv1alpha3.GitBranchRemoteStatus{ Commit: "aaaa", LastVerifiedAt: &at, VerifiedBy: "Fetch", })) } @@ -213,10 +213,10 @@ func TestRemoteStatusIsNews_TheSameAnswerProvedASecondWayIsNot(t *testing.T) { // age off at all. func TestRemoteStatusIsNews_AMissingTimestampIsAlwaysNews(t *testing.T) { at := metav1.NewTime(time.Now()) - withClock := &configbutleraiv1alpha3.GitTargetRemoteStatus{ + withClock := &configbutleraiv1alpha3.GitBranchRemoteStatus{ Commit: "aaaa", LastVerifiedAt: &at, VerifiedBy: "Fetch", } - without := &configbutleraiv1alpha3.GitTargetRemoteStatus{Commit: "aaaa", VerifiedBy: "Fetch"} + without := &configbutleraiv1alpha3.GitBranchRemoteStatus{Commit: "aaaa", VerifiedBy: "Fetch"} assert.True(t, remoteStatusIsNews(without, withClock)) assert.True(t, remoteStatusIsNews(withClock, without)) @@ -312,7 +312,7 @@ func TestPublishRemote_TheNewRepositoryRepopulatesTheStanza(t *testing.T) { func TestPublishRemote_ARestartPublishesAgainstTheObservationAlone(t *testing.T) { target := remoteTestTarget() at := metav1.NewTime(time.Now()) - target.Status.Remote = &configbutleraiv1alpha3.GitTargetRemoteStatus{ + target.Status.Remote = &configbutleraiv1alpha3.GitBranchRemoteStatus{ Commit: "aaaa", LastVerifiedAt: &at, VerifiedBy: "Push", } fresh := &GitTargetReconciler{} diff --git a/internal/controller/renamed_required_field_test.go b/internal/controller/renamed_required_field_test.go deleted file mode 100644 index dd2c2d7b..00000000 --- a/internal/controller/renamed_required_field_test.go +++ /dev/null @@ -1,258 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package controller - -import ( - "context" - "fmt" - "strings" - "testing" - "time" - - apiextv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" - "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/client-go/kubernetes/scheme" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/envtest" -) - -// TestRenamedRequiredField_StoredObjectCanAdoptIt is the measurement behind the guard on -// GitTarget's gitProviderRef immutability rule, and the reason renaming a required reference did -// not need a two-release migration. -// -// The hazard is real and this test reproduces it: a GitTarget stored before the rename serves NO -// value for the new name and no value for the old one either — a field outside the structural -// schema is not served, so the loss is immediate on the CRD upgrade rather than on the next write. -// The object is therefore missing a REQUIRED field, and the apply that fixes it is also the apply -// that sets an IMMUTABLE one. Written as a plain `self.x == oldSelf.x`, that rule rejects the -// migrating apply outright, because oldSelf has no such key: the user's only route would be -// delete-and-recreate. -// -// Guarding it as `!has(oldSelf.x) || self.x == oldSelf.x` opens a one-way door. It cannot loosen -// anything, because a required field can never be absent on an object created from this release -// on — which the last assertion here holds to. -// -// The schema surgery below stands in for the release: widen the served schema so an object can be -// stored without the field, then narrow it back to what ships. -func TestRenamedRequiredField_StoredObjectCanAdoptIt(t *testing.T) { - env := &envtest.Environment{ - CRDDirectoryPaths: []string{"../../config/crd/bases"}, - ErrorIfCRDPathMissing: true, - } - cfg, err := env.Start() - if err != nil { - t.Fatalf("start envtest control plane: %v", err) - } - t.Cleanup(func() { _ = env.Stop() }) - - sch := runtime.NewScheme() - if err := scheme.AddToScheme(sch); err != nil { - t.Fatal(err) - } - if err := apiextv1.AddToScheme(sch); err != nil { - t.Fatal(err) - } - c, err := client.New(cfg, client.Options{Scheme: sch}) - if err != nil { - t.Fatal(err) - } - ctx := context.Background() - - target := func(name string, withRef bool) *unstructured.Unstructured { - spec := map[string]any{"branch": "main", "path": "clusters/prod"} - if withRef { - spec["gitProviderRef"] = map[string]any{"name": "platform"} - } - return &unstructured.Unstructured{Object: map[string]any{ - "apiVersion": "configbutler.ai/v1alpha3", - "kind": "GitTarget", - "metadata": map[string]any{"name": name, "namespace": "default"}, - "spec": spec, - }} - } - - // Stand in for the PRE-rename release: the schema serves `providerRef` and does not know - // `gitProviderRef` at all. Storing under this schema is what makes the assertions below mean - // something — an object created without the new field under the SHIPPED schema would prove only - // that the guard works, not that the loss it exists for is real. - renameGitProviderRefTo(ctx, t, c, "providerRef") - legacy := &unstructured.Unstructured{Object: map[string]any{ - "apiVersion": "configbutler.ai/v1alpha3", - "kind": "GitTarget", - "metadata": map[string]any{"name": "stored-before-the-rename", "namespace": "default"}, - "spec": map[string]any{ - "providerRef": map[string]any{"name": "platform"}, - "branch": "main", - "path": "clusters/prod", - }, - }} - createUntilServed(ctx, t, c, legacy) - - // Ship the rename. - renameGitProviderRefTo(ctx, t, c, "gitProviderRef") - requireRenamedSchemaServed(ctx, t, c, target) - - // FIRST CLAIM: the old value is gone on READ, immediately, with no write in between. This is - // what makes the rename a stall rather than a silent mis-configuration, and it is the premise - // docs/UPGRADING.md's "re-apply every object" step rests on. - afterRename := requirePrunedOfBothSpellings(ctx, t, c, legacy.GetName()) - - // SECOND CLAIM: the migrating apply works. One update sets a required, immutable field for the - // first time, which is the shape the guard exists for. - if err := unstructured.SetNestedMap( - afterRename.Object, map[string]any{"name": "platform"}, "spec", "gitProviderRef"); err != nil { - t.Fatal(err) - } - if err := c.Update(ctx, afterRename); err != nil { - t.Fatalf("a stored object could not adopt the renamed required field in one apply: %v\n"+ - "Without this, upgrading across the rename would force delete-and-recreate on every "+ - "GitTarget. See the guard on spec.gitProviderRef in api/v1alpha3/gittarget_types.go.", err) - } - - // And the door is shut for everything created after the rename. - fresh := target("created-after-the-rename", true) - if err := c.Create(ctx, fresh); err != nil { - t.Fatalf("create under the shipped schema: %v", err) - } - if err := unstructured.SetNestedMap( - fresh.Object, map[string]any{"name": "somewhere-else"}, "spec", "gitProviderRef"); err != nil { - t.Fatal(err) - } - if err := c.Update(ctx, fresh); err == nil { - t.Fatal("gitProviderRef is no longer immutable: the migration guard has leaked into " + - "objects that were created with the field") - } -} - -// renameGitProviderRefTo rewrites the SERVED GitTarget schema so the provider reference is spelled -// `to` — property, required list and all — standing in for the release boundary the rename crosses. -// Crossing it in the test the way the release crosses it is the whole point: a field the schema no -// longer describes is a field the apiserver no longer serves. -func renameGitProviderRefTo(ctx context.Context, t *testing.T, c client.Client, to string) { - t.Helper() - const ( - oldName = "providerRef" - newName = "gitProviderRef" - ) - from := newName - if to == newName { - from = oldName - } - - var crd apiextv1.CustomResourceDefinition - if err := c.Get(ctx, client.ObjectKey{Name: "gittargets.configbutler.ai"}, &crd); err != nil { - t.Fatalf("get CRD: %v", err) - } - spec := crd.Spec.Versions[0].Schema.OpenAPIV3Schema.Properties["spec"] - if prop, ok := spec.Properties[from]; ok { - spec.Properties[to] = prop - delete(spec.Properties, from) - } - required := make([]string, 0, len(spec.Required)) - for _, r := range spec.Required { - if r == from { - r = to - } - required = append(required, r) - } - spec.Required = required - // The rules move with the field. A CEL rule is compiled against the schema, so leaving one - // naming a property that no longer exists makes the CRD itself invalid — which is a useful - // thing to have learned: the immutability rule and the field it guards cannot drift apart. - for i := range spec.XValidations { - spec.XValidations[i].Rule = strings.ReplaceAll(spec.XValidations[i].Rule, from, to) - spec.XValidations[i].Message = strings.ReplaceAll(spec.XValidations[i].Message, from, to) - } - crd.Spec.Versions[0].Schema.OpenAPIV3Schema.Properties["spec"] = spec - if err := c.Update(ctx, &crd); err != nil { - t.Fatalf("update CRD schema: %v", err) - } -} - -// createUntilServed retries a create until the apiserver is serving the schema change. A CRD schema -// update is not visible to the CR handler synchronously, so a single attempt is a flake. -func createUntilServed(ctx context.Context, t *testing.T, c client.Client, obj client.Object) { - t.Helper() - deadline := time.Now().Add(30 * time.Second) - var err error - for time.Now().Before(deadline) { - if err = c.Create(ctx, obj); err == nil { - return - } - obj.SetResourceVersion("") - time.Sleep(100 * time.Millisecond) - } - t.Fatalf("create never succeeded under the widened schema: %v", err) -} - -// requireRenamedSchemaServed blocks until the NARROWED schema is the one being served, proven by a -// create that must SUCCEED: one carrying only `gitProviderRef`. Under the widened schema that name -// is not described, so it is pruned and the create fails on the missing required `providerRef`; -// only once the rename is served does it get through. -// -// Waiting on a REJECTION instead is vacuous, and was the bug here: the schema surgery renames the -// required field rather than dropping it, so an object with no reference at all is refused under -// BOTH schemas. The very first probe therefore returned immediately — before the apiserver's CR -// handler had picked up the new schema — and the migration assertion then ran against whatever was -// still being served. On a fast machine the handler usually won the race and the test passed; under -// CI load it did not, and the stored object still served `providerRef`. -// -// A ref-less create must still be rejected once the rename IS served, or the object below could -// satisfy the schema without ever adopting the field. That claim is now checked after the wait, -// where it discriminates, instead of standing in for it. -func requireRenamedSchemaServed( - ctx context.Context, - t *testing.T, - c client.Client, - target func(string, bool) *unstructured.Unstructured, -) { - t.Helper() - deadline := time.Now().Add(30 * time.Second) - for i := 0; time.Now().Before(deadline); i++ { - probe := target(fmt.Sprintf("narrowing-probe-%d", i), true) - if err := c.Create(ctx, probe); err != nil { - time.Sleep(100 * time.Millisecond) - continue - } - _ = c.Delete(ctx, probe) - - refLess := target("ref-less-probe", false) - if err := c.Create(ctx, refLess); err == nil { - _ = c.Delete(ctx, refLess) - t.Fatal("gitProviderRef is not required under the shipped schema, so the migration " + - "assertion would be vacuous") - } - return - } - t.Fatal("the renamed schema was never served, so the migration assertion would be vacuous") -} - -// requirePrunedOfBothSpellings reads the stored object back and holds the premise of the migration: -// after the rename it serves NEITHER the old name (the schema no longer describes it) nor the new -// one (nothing has written it). It returns the object so the caller can attempt the migration. -func requirePrunedOfBothSpellings( - ctx context.Context, - t *testing.T, - c client.Client, - name string, -) *unstructured.Unstructured { - t.Helper() - obj := &unstructured.Unstructured{} - obj.SetAPIVersion("configbutler.ai/v1alpha3") - obj.SetKind("GitTarget") - if err := c.Get(ctx, client.ObjectKey{Name: name, Namespace: "default"}, obj); err != nil { - t.Fatalf("get stored object after the rename: %v", err) - } - spec, _, err := unstructured.NestedMap(obj.Object, "spec") - if err != nil { - t.Fatal(err) - } - if _, ok := spec["providerRef"]; ok { - t.Fatalf("the pruned field is still served, so this test is not reproducing the migration: %v", spec) - } - if _, ok := spec["gitProviderRef"]; ok { - t.Fatalf("the stored object gained the new field without being written: %v", spec) - } - return obj -} From 46cb09e76311af17198b19bd3719bd5b97eadfb7 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Wed, 21 Oct 2026 20:17:50 +0000 Subject: [PATCH 3/6] feat(git)!: claim a repository branch through GitBranch A GitBranch holds its repository branch once its GitProvider's credential opens a receive-pack session on the remote; the oldest such GitBranch holds it, keeps it through a later credential failure, and any other reports BranchConflict. The claim replaces oldest-GitTarget-wins in branch_owner.go, and EnsureWorker, the worker sweep and the GitTarget reconcile read it. A worker now writes for the claiming GitBranch and uses its GitProvider's credential, signing key and metric labels. GitTargets reach it through spec.gitBranchRef; one in another namespace is admitted by the GitBranch's GitProvider accessFrom. status.remote, ParentBranchNotFound and RecoveringParentBranch move to the GitBranch, and its finalizer stops the worker (pushing or refusing every pending write) before the claim goes. GitProvider.status.branches is no longer published. Co-Authored-By: Claude Opus 5.5 (1M context) --- cmd/main.go | 11 + config/rbac/role.yaml | 3 + internal/applier/dispatcher.go | 2 +- internal/applier/dispatcher_test.go | 17 +- internal/applier/references.go | 20 +- internal/authz/clusterprovider_admission.go | 38 +- ...on_test.go => gitbranch_admission_test.go} | 67 +- internal/authz/source_namespace_test.go | 3 +- .../controller/applytrigger_controller.go | 28 +- .../applytrigger_controller_test.go | 7 +- .../controller/applytrigger_holder_test.go | 2 +- internal/controller/applytrigger_rule_test.go | 16 +- .../controller/applytrigger_schema_test.go | 8 +- .../clusterwatchrule_admission_test.go | 7 +- .../controller/clusterwatchrule_controller.go | 86 +-- .../commitrequest_outcome_metric_test.go | 5 +- internal/controller/condition_helper_test.go | 4 +- .../controller/dependency_watches_test.go | 114 +-- .../controller/git_branch_helpers_test.go | 41 + internal/controller/gitbranch_controller.go | 441 +++++++++++ .../controller/gitbranch_controller_test.go | 356 +++++++++ ...ittarget_remote.go => gitbranch_remote.go} | 80 +- ...emote_test.go => gitbranch_remote_test.go} | 111 +-- internal/controller/gitprovider_branches.go | 76 -- .../controller/gitprovider_branches_test.go | 133 ---- internal/controller/gitprovider_controller.go | 13 +- .../controller/gitprovider_controller_test.go | 73 +- .../controller/gittarget_apply_trigger.go | 10 +- .../gittarget_apply_trigger_test.go | 40 +- internal/controller/gittarget_controller.go | 699 +++++++----------- .../controller/gittarget_controller_test.go | 499 +++---------- .../gittarget_controller_unit_test.go | 7 +- .../controller/gittarget_gitbranch_test.go | 222 ++++++ .../controller/gittarget_immutability_test.go | 114 ++- internal/controller/gittarget_layout_test.go | 2 +- .../gittarget_parent_branch_test.go | 302 -------- .../controller/gittarget_parent_recovery.go | 39 - .../gittarget_placement_validation_test.go | 11 +- internal/controller/gittarget_publication.go | 7 +- .../controller/gittarget_publication_test.go | 2 +- ...est.go => gittarget_shared_branch_test.go} | 59 +- .../controller/gittarget_source_cluster.go | 77 +- .../gittarget_source_cluster_test.go | 74 +- internal/controller/gittarget_status_test.go | 12 +- .../gittarget_worker_cleanup_test.go | 35 +- .../gittarget_worker_repoint_test.go | 71 +- .../controller/resource_condition_metrics.go | 3 + .../controller/status_event_severity_test.go | 2 +- internal/controller/stream_status_test.go | 4 +- internal/controller/suite_test.go | 6 +- .../superseded_fields_admission_test.go | 23 +- internal/controller/watchrule_controller.go | 77 +- .../controller/watchrule_controller_test.go | 20 +- .../controller/watchrule_source_namespace.go | 3 +- .../watchrule_source_namespace_test.go | 3 +- .../wiring/readiness_wiring_test.go | 5 +- internal/git/apply_recovery.go | 1 + internal/git/apply_request.go | 6 +- internal/git/apply_trigger.go | 3 +- internal/git/apply_trigger_test.go | 12 +- internal/git/branch_owner.go | 203 +++-- internal/git/branch_owner_test.go | 210 +++--- internal/git/branch_worker.go | 7 +- internal/git/branch_worker_loop_test.go | 5 +- .../git/branch_worker_queue_depth_test.go | 24 +- internal/git/branch_worker_split_test.go | 9 +- internal/git/branch_worker_test.go | 26 +- internal/git/claim_helpers_test.go | 36 + internal/git/credentials.go | 12 + internal/git/git_atomic_push.go | 24 +- internal/git/git_branch_ref.go | 42 ++ internal/git/git_branch_ref_test.go | 115 +++ internal/git/git_roundtrip_ledger_test.go | 10 +- internal/git/prove_push_session_test.go | 79 ++ internal/git/publication.go | 7 +- internal/git/publication_test.go | 24 +- internal/git/refusal_reapply_test.go | 11 +- internal/git/remote_report_test.go | 6 +- internal/git/secret_write_test.go | 7 +- internal/git/types.go | 18 +- internal/git/types_test.go | 24 +- internal/git/worker_branch_owner_test.go | 115 +-- internal/git/worker_manager.go | 312 ++++---- internal/git/worker_manager_test.go | 49 +- internal/git/worker_repo_identity_test.go | 38 +- .../git/write_boundary_precondition_test.go | 5 +- internal/rulestore/object_selector_test.go | 4 +- internal/rulestore/store.go | 70 +- internal/rulestore/store_test.go | 88 +-- internal/watch/api_resource_catalog_test.go | 2 +- internal/watch/bootstrap.go | 27 +- internal/watch/bootstrap_admission_test.go | 11 +- internal/watch/collection_overlap_test.go | 2 +- .../config_plane_split_review_fixes_test.go | 2 +- internal/watch/event_router.go | 21 +- internal/watch/event_router_test.go | 79 +- internal/watch/git_branch_helpers_test.go | 35 + internal/watch/manager_snapshot_test.go | 2 +- internal/watch/manager_startup_test.go | 10 +- internal/watch/object_selector_test.go | 8 +- .../watch/source_namespace_planning_test.go | 2 +- .../source_namespace_stream_summary_test.go | 2 +- internal/watch/source_namespace_test.go | 13 +- internal/watch/watched_type_helpers_test.go | 4 +- internal/watch/watched_type_metrics_test.go | 8 +- internal/watch/watched_type_resolver.go | 20 +- internal/watch/watched_type_resolver_test.go | 22 +- internal/watch/watchrule_compile.go | 8 +- 108 files changed, 3171 insertions(+), 2899 deletions(-) rename internal/authz/{gitprovider_admission_test.go => gitbranch_admission_test.go} (56%) create mode 100644 internal/controller/git_branch_helpers_test.go create mode 100644 internal/controller/gitbranch_controller.go create mode 100644 internal/controller/gitbranch_controller_test.go rename internal/controller/{gittarget_remote.go => gitbranch_remote.go} (70%) rename internal/controller/{gittarget_remote_test.go => gitbranch_remote_test.go} (88%) delete mode 100644 internal/controller/gitprovider_branches.go delete mode 100644 internal/controller/gitprovider_branches_test.go create mode 100644 internal/controller/gittarget_gitbranch_test.go delete mode 100644 internal/controller/gittarget_parent_branch_test.go delete mode 100644 internal/controller/gittarget_parent_recovery.go rename internal/controller/{gittarget_shared_provider_test.go => gittarget_shared_branch_test.go} (51%) create mode 100644 internal/git/claim_helpers_test.go create mode 100644 internal/git/git_branch_ref.go create mode 100644 internal/git/git_branch_ref_test.go create mode 100644 internal/git/prove_push_session_test.go create mode 100644 internal/watch/git_branch_helpers_test.go diff --git a/cmd/main.go b/cmd/main.go index 4ab6529b..6caba17a 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -355,6 +355,17 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "ClusterProvider") os.Exit(1) } + if err := (&controller.GitBranchReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + WorkerManager: workerManager, + SSHHostKeys: cfg.sshHostKeys, + CredentialPolicy: cfg.credentialPolicy, + Recorder: mgr.GetEventRecorderFor("gitbranch"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "GitBranch") + os.Exit(1) + } if err := (&controller.GitTargetReconciler{ Client: mgr.GetClient(), Scheme: mgr.GetScheme(), diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 5ee8f4ee..862c4882 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -59,6 +59,7 @@ rules: - clusterwatchrules/status - commitrequests/status - fluxtriggers/status + - gitbranches/status - gitproviders/status - gittargets/status - watchrules/status @@ -71,6 +72,7 @@ rules: resources: - clusterproviders - fluxtriggers + - gitbranches verbs: - get - list @@ -106,6 +108,7 @@ rules: - configbutler.ai resources: - fluxtriggers/finalizers + - gitbranches/finalizers - gitproviders/finalizers verbs: - update diff --git a/internal/applier/dispatcher.go b/internal/applier/dispatcher.go index ea793647..9b40f2d6 100644 --- a/internal/applier/dispatcher.go +++ b/internal/applier/dispatcher.go @@ -835,7 +835,7 @@ func holdingTrigger( // onBranch reports whether a GitTarget still names a trigger, on the request's branch. func onBranch(target *v1alpha3.GitTarget, req git.ApplyRequest) bool { - return target.Spec.ApplyTriggerRef != nil && git.WritesThrough(target, req.Provider, req.Branch.Branch) + return target.Spec.ApplyTriggerRef != nil && git.WritesThrough(target, req.GitBranch) } // delivered records a delivered request as an ApplyRequested Event on every GitTarget it was for, diff --git a/internal/applier/dispatcher_test.go b/internal/applier/dispatcher_test.go index 69a3ea1a..ec5d6506 100644 --- a/internal/applier/dispatcher_test.go +++ b/internal/applier/dispatcher_test.go @@ -44,6 +44,7 @@ const ( var ( testBranchKey = git.BranchKeyFor(testRepoURL, testBranch) testProvider = types.NamespacedName{Namespace: targetNS, Name: "platform"} + testGitBranch = types.NamespacedName{Namespace: targetNS, Name: "platform-" + testBranch} ) // Test-only annotations on a GitTarget, which newDispatcherFixture reads to create the trigger the @@ -61,9 +62,8 @@ func gitTarget(name string, kustomizationRef *fluxmeta.NamespacedObjectReference target := &v1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: targetNS, Name: name, UID: uid(name)}, Spec: v1alpha3.GitTargetSpec{ - GitProviderRef: fluxmeta.NamespacedObjectReference{Name: testProvider.Name}, - Branch: testBranch, - Path: name, + GitBranchRef: &fluxmeta.NamespacedObjectReference{Name: testGitBranch.Name}, + Path: name, }, } if cluster != "" { @@ -155,6 +155,12 @@ func controlPlane() []client.Object { ObjectMeta: metav1.ObjectMeta{Namespace: testProvider.Namespace, Name: testProvider.Name}, Spec: v1alpha3.GitProviderSpec{URL: testRepoURL}, }, + &v1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Namespace: testGitBranch.Namespace, Name: testGitBranch.Name}, + Spec: v1alpha3.GitBranchSpec{ + GitProviderRef: fluxmeta.LocalObjectReference{Name: testProvider.Name}, Branch: testBranch, + }, + }, clusterProvider(v1alpha3.DefaultClusterProviderName, targetNS), clusterProvider("prod-eu-1", targetNS), } @@ -186,7 +192,8 @@ var checkout = kustomizationApplier(v1alpha3.DefaultClusterProviderName, fluxSys // reason. func request(applier git.ApplierRef, reason v1alpha3.ApplyReason, names ...string) git.ApplyRequest { req := git.ApplyRequest{ - ID: 1, Branch: testBranchKey, Provider: testProvider, Applier: applier, Reason: reason, Commit: testCommit, + ID: 1, Branch: testBranchKey, Provider: testProvider, GitBranch: testGitBranch, Applier: applier, + Reason: reason, Commit: testCommit, } for _, name := range names { req.Targets = append(req.Targets, git.ApplyTarget{Ref: ref(name), UID: uid(name), Reason: reason}) @@ -558,7 +565,7 @@ func TestDispatcher_WithdrawnConsentAsksNothing(t *testing.T) { func TestDispatcher_ATargetThatChangedAsksNothing(t *testing.T) { for name, change := range map[string]func(*v1alpha3.GitTarget){ "it was deleted and recreated": func(g *v1alpha3.GitTarget) { g.UID = "uid-recreated" }, - "it moved to another branch": func(g *v1alpha3.GitTarget) { g.Spec.Branch = "release" }, + "it moved to another branch": func(g *v1alpha3.GitTarget) { g.Spec.GitBranchRef.Name = "platform-release" }, "it names another trigger": func(g *v1alpha3.GitTarget) { namesKustomization(g, fluxSystemNS, "other") }, "it names no trigger any more": func(g *v1alpha3.GitTarget) { g.Spec.ApplyTriggerRef = nil }, "its trigger is gone": func(g *v1alpha3.GitTarget) { diff --git a/internal/applier/references.go b/internal/applier/references.go index 89e6be30..bbe50abb 100644 --- a/internal/applier/references.go +++ b/internal/applier/references.go @@ -8,12 +8,12 @@ import ( "sort" "strings" - apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/authz" + "github.com/ConfigButler/gitops-reverser/internal/git" "github.com/ConfigButler/gitops-reverser/internal/repourl" ) @@ -178,20 +178,22 @@ func judgeReference( return ref, nil } - provider := &v1alpha3.GitProvider{} - key := target.GitProviderKey() - if err := reader.Get(ctx, key, provider); err != nil { - if !apierrors.IsNotFound(err) { - return ref, fmt.Errorf("read GitProvider %s: %w", key, err) - } - ref.State, ref.Message = ReferenceUndecided, fmt.Sprintf("GitProvider %s not found", key) + branch, provider, err := git.ResolveGitBranch(ctx, reader, target) + switch { + case err != nil: + return ref, err + case branch == nil: + ref.State, ref.Message = ReferenceUndecided, fmt.Sprintf("GitBranch %s not found", target.GitBranchKey()) + return ref, nil + case provider == nil: + ref.State, ref.Message = ReferenceUndecided, fmt.Sprintf("GitProvider %s not found", branch.GitProviderKey()) return ref, nil } if source == nil { ref.State, ref.Message = ReferenceUndecided, "the applier's source has not been read yet" return ref, nil } - want := Binding{RepositoryURL: provider.Spec.URL, Branch: target.Spec.Branch} + want := Binding{RepositoryURL: provider.Spec.URL, Branch: branch.Spec.Branch} if mismatch := bindingMismatch(trigger, source.URL, source.Revision, want); mismatch != "" { ref.State, ref.Reason, ref.Message = ReferenceRefused, ReasonSourceMismatch, mismatch return ref, nil diff --git a/internal/authz/clusterprovider_admission.go b/internal/authz/clusterprovider_admission.go index 076faa2b..08dcead3 100644 --- a/internal/authz/clusterprovider_admission.go +++ b/internal/authz/clusterprovider_admission.go @@ -59,7 +59,7 @@ type Decision struct { // GitTargetAdmitted reports whether target's namespace is admitted by the ClusterProvider that // target references, reading both the provider and the target's Namespace through reader, and by -// the GitProvider it writes through (gitProviderAdmitted). +// the GitProvider of the GitBranch it writes through (gitBranchAdmitted). // // It is evaluated on every reconcile rather than only at admission, so a policy TIGHTENED after a // GitTarget was created revokes it too. The two denial paths are deliberate and ordered: a missing @@ -84,29 +84,38 @@ func GitTargetAdmitted( if err != nil || !decision.Allowed { return decision, err } - return gitProviderAdmitted(ctx, reader, target) + return gitBranchAdmitted(ctx, reader, target) } -// gitProviderAdmitted reports whether target's namespace may write through the GitProvider it -// names. One in its own namespace always may. One in another namespace may only while its -// spec.accessFrom admits target's namespace, because the GitTarget then writes with that -// provider's credential. A GitProvider that does not exist admits nothing and denies nothing here: -// there is no credential to wield, and the Validated gate reports it as not found. -func gitProviderAdmitted( +// gitBranchAdmitted reports whether target's namespace may write through the GitBranch it +// references. One in its own namespace always may. One in another namespace may only while the +// GitBranch's GitProvider admits target's namespace in its spec.accessFrom, because the GitTarget +// then writes with that provider's credential: one policy governs both hops, so it cannot be split +// across two objects that disagree. A GitBranch or GitProvider that does not exist admits nothing +// and denies nothing here: there is no credential to wield, and their status reports them missing. +func gitBranchAdmitted( ctx context.Context, reader client.Reader, target *configv1alpha3.GitTarget, ) (Decision, error) { - key := target.GitProviderKey() - if key.Namespace == target.Namespace { + key := target.GitBranchKey() + if key.Name == "" || key.Namespace == target.Namespace { return Decision{Allowed: true}, nil } + branch := &configv1alpha3.GitBranch{} + if err := reader.Get(ctx, key, branch); err != nil { + if apierrors.IsNotFound(err) { + return Decision{Allowed: true}, nil + } + return Decision{}, fmt.Errorf("read GitBranch %s: %w", key, err) + } + providerKey := branch.GitProviderKey() provider := &configv1alpha3.GitProvider{} - if err := reader.Get(ctx, key, provider); err != nil { + if err := reader.Get(ctx, providerKey, provider); err != nil { if apierrors.IsNotFound(err) { return Decision{Allowed: true}, nil } - return Decision{}, fmt.Errorf("read GitProvider %s: %w", key, err) + return Decision{}, fmt.Errorf("read GitProvider %s: %w", providerKey, err) } nsLabels, err := namespaceLabels(ctx, reader, target.Namespace) if err != nil { @@ -116,14 +125,15 @@ func gitProviderAdmitted( if selErr != nil { return Decision{ Reason: ReasonNamespaceNotAuthorized, - Message: fmt.Sprintf("GitProvider %s accessFrom selector is invalid: %v", key, selErr), + Message: fmt.Sprintf("GitProvider %s accessFrom selector is invalid: %v", providerKey, selErr), }, nil } if !allowed { return Decision{ Reason: ReasonNamespaceNotAuthorized, Message: fmt.Sprintf( - "namespace %q is not permitted to reference GitProvider %s (spec.accessFrom)", target.Namespace, key), + "namespace %q is not permitted to reference GitBranch %s: its GitProvider %s does not admit it "+ + "(spec.accessFrom)", target.Namespace, key, providerKey), }, nil } return Decision{Allowed: true}, nil diff --git a/internal/authz/gitprovider_admission_test.go b/internal/authz/gitbranch_admission_test.go similarity index 56% rename from internal/authz/gitprovider_admission_test.go rename to internal/authz/gitbranch_admission_test.go index b09232d0..76da3a04 100644 --- a/internal/authz/gitprovider_admission_test.go +++ b/internal/authz/gitbranch_admission_test.go @@ -6,6 +6,7 @@ import ( "context" "testing" + "github.com/fluxcd/pkg/apis/meta" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -23,19 +24,29 @@ func sharedGitProvider(policy *configv1alpha3.NamespaceMatcher) *configv1alpha3. } } -// targetWriting is a GitTarget in testNS writing through the GitProvider namespace/name; an empty +// sharedGitBranch is the GitBranch platform/main, written with the GitProvider platform/provider. +func sharedGitBranch(provider string) *configv1alpha3.GitBranch { + return &configv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Namespace: "platform", Name: "main"}, + Spec: configv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: provider}, Branch: "main", + }, + } +} + +// targetWriting is a GitTarget in testNS writing through the GitBranch namespace/name; an empty // namespace means its own. func targetWriting(namespace, name string) *configv1alpha3.GitTarget { target := targetIn("") - target.Spec.GitProviderRef.Name = name - target.Spec.GitProviderRef.Namespace = namespace + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: name, Namespace: namespace} return target } -// TestGitTargetAdmitted_GitProviderAccessFrom: a GitTarget writes with its GitProvider's credential, -// so one in another namespace is used only while its spec.accessFrom admits the GitTarget's -// namespace. A GitProvider in the GitTarget's own namespace needs no policy. -func TestGitTargetAdmitted_GitProviderAccessFrom(t *testing.T) { +// TestGitTargetAdmitted_GitBranchsProviderAccessFrom: a GitTarget writes with its GitBranch's +// GitProvider credential, so a GitBranch in another namespace is used only while that GitProvider's +// spec.accessFrom admits the GitTarget's namespace. A GitBranch in the GitTarget's own namespace +// needs no policy. +func TestGitTargetAdmitted_GitBranchsProviderAccessFrom(t *testing.T) { // The ClusterProvider gate passes in every case, so only the GitProvider's decides. source := providerNamed(configv1alpha3.DefaultClusterProviderName, &configv1alpha3.NamespaceMatcher{Names: []string{testNS}}) @@ -48,48 +59,63 @@ func TestGitTargetAdmitted_GitProviderAccessFrom(t *testing.T) { { name: "its own namespace, with no policy at all", objects: []client.Object{source}, - target: targetWriting("", "repo"), + target: targetWriting("", "main"), allowed: true, }, { name: "its own namespace, named explicitly", objects: []client.Object{source}, - target: targetWriting(testNS, "repo"), + target: targetWriting(testNS, "main"), allowed: true, }, { - name: "another namespace, with no policy", - objects: []client.Object{source, sharedGitProvider(nil), namespaceLabeled(nil)}, - target: targetWriting("platform", "monorepo"), + name: "another namespace, with no policy", + objects: []client.Object{ + source, + sharedGitProvider(nil), + sharedGitBranch("monorepo"), + namespaceLabeled(nil), + }, + target: targetWriting("platform", "main"), }, { name: "another namespace, admitted by name", - objects: []client.Object{source, namespaceLabeled(nil), + objects: []client.Object{source, namespaceLabeled(nil), sharedGitBranch("monorepo"), sharedGitProvider(&configv1alpha3.NamespaceMatcher{Names: []string{testNS}})}, - target: targetWriting("platform", "monorepo"), + target: targetWriting("platform", "main"), allowed: true, }, { name: "another namespace, admitted by its labels", - objects: []client.Object{source, namespaceLabeled(map[string]string{"tenant": "yes"}), + objects: []client.Object{ + source, + namespaceLabeled(map[string]string{"tenant": "yes"}), + sharedGitBranch("monorepo"), sharedGitProvider(&configv1alpha3.NamespaceMatcher{ Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"tenant": "yes"}}, - })}, - target: targetWriting("platform", "monorepo"), + }), + }, + target: targetWriting("platform", "main"), allowed: true, }, { name: "another namespace, not admitted", - objects: []client.Object{source, namespaceLabeled(nil), + objects: []client.Object{source, namespaceLabeled(nil), sharedGitBranch("monorepo"), sharedGitProvider(&configv1alpha3.NamespaceMatcher{Names: []string{"team-b"}})}, - target: targetWriting("platform", "monorepo"), + target: targetWriting("platform", "main"), }, { - name: "another namespace's GitProvider that does not exist wields nothing", + name: "another namespace's GitBranch that does not exist wields nothing", objects: []client.Object{source}, target: targetWriting("platform", "missing"), allowed: true, }, + { + name: "another namespace's GitBranch whose GitProvider does not exist wields nothing", + objects: []client.Object{source, sharedGitBranch("missing")}, + target: targetWriting("platform", "main"), + allowed: true, + }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { @@ -101,6 +127,7 @@ func TestGitTargetAdmitted_GitProviderAccessFrom(t *testing.T) { assert.Equal(t, tt.allowed, decision.Allowed, decision.Message) if !tt.allowed { assert.Equal(t, ReasonNamespaceNotAuthorized, decision.Reason) + assert.Contains(t, decision.Message, "GitBranch platform/main") assert.Contains(t, decision.Message, "GitProvider platform/monorepo") } }) diff --git a/internal/authz/source_namespace_test.go b/internal/authz/source_namespace_test.go index 48ff3a2b..135a70c2 100644 --- a/internal/authz/source_namespace_test.go +++ b/internal/authz/source_namespace_test.go @@ -44,9 +44,8 @@ func snTarget() *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: snTargetName, Namespace: snTenantNS}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "acme-git"}, + GitBranchRef: &meta.NamespacedObjectReference{Name: "acme-git-main"}, ClusterProviderRef: &meta.LocalObjectReference{Name: snProvider}, - Branch: "main", Path: "tenants/acme", }, } diff --git a/internal/controller/applytrigger_controller.go b/internal/controller/applytrigger_controller.go index aaf28ef9..abcddf41 100644 --- a/internal/controller/applytrigger_controller.go +++ b/internal/controller/applytrigger_controller.go @@ -497,10 +497,15 @@ func (r *ApplyTriggerReconciler) SetupWithManager(mgr ctrl.Manager) error { // What makes a GitTarget eligible re-runs the check of the trigger it names, and of the one it // named before: the GitTarget appearing, going, or changing what it names or writes. Watches(&configbutleraiv1alpha3.GitTarget{}, r.gitTargetToTriggers()). - // A GitProvider's URL is what its GitTargets write. + // A GitBranch and its GitProvider's URL are what its GitTargets write. + Watches( + &configbutleraiv1alpha3.GitBranch{}, + handler.EnqueueRequestsFromMapFunc(r.gitDestinationToTriggers), + builder.WithPredicates(predicate.GenerationChangedPredicate{}), + ). Watches( &configbutleraiv1alpha3.GitProvider{}, - handler.EnqueueRequestsFromMapFunc(r.gitProviderToTriggers), + handler.EnqueueRequestsFromMapFunc(r.gitDestinationToTriggers), builder.WithPredicates(predicate.GenerationChangedPredicate{}), ). // A ClusterProvider admits the trigger's namespace, and the namespaces of the GitTargets naming @@ -632,20 +637,23 @@ func (r *ApplyTriggerReconciler) gitTargetToTriggers() handler.EventHandler { } } -// gitProviderToTriggers maps a GitProvider event to the triggers of this kind named by the -// GitTargets that write through it. -func (r *ApplyTriggerReconciler) gitProviderToTriggers(ctx context.Context, obj client.Object) []ctrlreconcile.Request { - var targets configbutleraiv1alpha3.GitTargetList - if err := r.List(ctx, &targets); err != nil { +// gitDestinationToTriggers maps a GitBranch or GitProvider event to the triggers of this kind named +// by the GitTargets that write through it. +func (r *ApplyTriggerReconciler) gitDestinationToTriggers( + ctx context.Context, + obj client.Object, +) []ctrlreconcile.Request { + targets, err := gitTargetsOn(ctx, r.Client, obj) + if err != nil { logDependencyListError(ctx, err, "GitTargets", obj) return nil } seen := map[k8stypes.NamespacedName]bool{} var requests []ctrlreconcile.Request - for i := range targets.Items { - target := &targets.Items[i] + for i := range targets { + target := &targets[i] ref := target.Spec.ApplyTriggerRef - if ref == nil || ref.Kind != r.Kind || !namesGitProvider(target, obj) { + if ref == nil || ref.Kind != r.Kind { continue } key := k8stypes.NamespacedName{Namespace: target.Namespace, Name: ref.Name} diff --git a/internal/controller/applytrigger_controller_test.go b/internal/controller/applytrigger_controller_test.go index bdfc5702..db0f5bd5 100644 --- a/internal/controller/applytrigger_controller_test.go +++ b/internal/controller/applytrigger_controller_test.go @@ -137,8 +137,7 @@ func referencingTarget(name, kind string) *configbutleraiv1alpha3.GitTarget { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "team-a"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "platform"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("platform", "main")}, Path: name, ApplyTriggerRef: &configbutleraiv1alpha3.ApplyTriggerReference{Kind: kind, Name: "checkout"}, }, @@ -186,6 +185,8 @@ func triggerTestClient(t *testing.T, objects ...ctrlclient.Object) ctrlclient.Cl hasTarget = true case *configbutleraiv1alpha3.GitProvider: hasProvider = true + // The GitBranch its GitTargets write through: "-main". + kept = append(kept, gitBranch(obj.GetName(), obj.GetNamespace())) } if _, marker := obj.(unreferenced); !marker { kept = append(kept, obj) @@ -198,7 +199,7 @@ func triggerTestClient(t *testing.T, objects ...ctrlclient.Object) ctrlclient.Cl } } if !hasProvider { - objects = append(objects, platformProvider()) + objects = append(objects, platformProvider(), gitBranch("platform", "team-a")) } if !hasTarget { objects = append(objects, diff --git a/internal/controller/applytrigger_holder_test.go b/internal/controller/applytrigger_holder_test.go index ca47f795..8aefd950 100644 --- a/internal/controller/applytrigger_holder_test.go +++ b/internal/controller/applytrigger_holder_test.go @@ -123,7 +123,7 @@ func TestFluxTrigger_OneTriggerHoldsAKustomization(t *testing.T) { target.Namespace = inLineKey.Namespace gt := &GitTargetReconciler{Client: f.r.Client} verdict, _ := gt.observeApplyTrigger(context.Background(), beginStatus(nil, nil, target), target, - triggerProvider()) + triggerDestination()) assert.Equal(t, metav1.ConditionFalse, verdict.value.Status) assert.Equal(t, applier.ReasonHeldByAnotherTrigger, verdict.value.Reason) assert.Contains(t, verdict.value.Message, "held by FluxTrigger team-a/checkout") diff --git a/internal/controller/applytrigger_rule_test.go b/internal/controller/applytrigger_rule_test.go index a2778f22..92e479b1 100644 --- a/internal/controller/applytrigger_rule_test.go +++ b/internal/controller/applytrigger_rule_test.go @@ -176,7 +176,7 @@ func TestApplyTriggerRule_ARecreatedGitTargetReleasesAndReadopts(t *testing.T) { func TestApplyTriggerRule_TheBindingAggregatesItsGitTargets(t *testing.T) { onBranch := func(name, branch string) *configbutleraiv1alpha3.GitTarget { target := referencingTarget(name, configbutleraiv1alpha3.ApplyTriggerKindFlux) - target.Spec.Branch = branch + target.Spec.GitBranchRef.Name = gitBranchNameForTest("platform", branch) return target } elsewhere := onBranch("elsewhere", "main") @@ -195,7 +195,7 @@ func TestApplyTriggerRule_TheBindingAggregatesItsGitTargets(t *testing.T) { wantReason: applier.ReasonNotBound, wantNames: []string{"team-a/unprovided"}, objects: []ctrlclient.Object{func() ctrlclient.Object { target := onBranch("unprovided", "main") - target.Spec.GitProviderRef.Name = "absent" + target.Spec.GitBranchRef.Name = "absent" return target }(), onBranch("release", "release")}}, {name: "every one mismatched is stalled, with the shared reason", wantStatus: metav1.ConditionFalse, @@ -215,7 +215,8 @@ func TestApplyTriggerRule_TheBindingAggregatesItsGitTargets(t *testing.T) { } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - objects := append([]ctrlclient.Object{testFluxTrigger(configbutleraiv1alpha3.DefaultClusterProviderName)}, + objects := append([]ctrlclient.Object{testFluxTrigger(configbutleraiv1alpha3.DefaultClusterProviderName), + gitBranchOn("platform", "team-a", "release"), gitBranchOn("platform", "team-a", "staging")}, tc.objects...) f := ruleFixture(t, objects...) @@ -242,8 +243,9 @@ func TestApplyTriggerRule_TheBindingAggregatesItsGitTargets(t *testing.T) { // holding the Kustomization. func TestApplyTriggerRule_AnOlderTriggerNotInLineKeepsNoneOut(t *testing.T) { for name, older := range map[string]func(*configbutleraiv1alpha3.GitTarget, *[]ctrlclient.Object){ - "bound to another branch": func(target *configbutleraiv1alpha3.GitTarget, _ *[]ctrlclient.Object) { - target.Spec.Branch = "release" + "bound to another branch": func(target *configbutleraiv1alpha3.GitTarget, objects *[]ctrlclient.Object) { + target.Spec.GitBranchRef.Name = gitBranchNameForTest("platform", "release") + *objects = append(*objects, gitBranchOn("platform", target.Namespace, "release")) }, "not admitted": func(target *configbutleraiv1alpha3.GitTarget, objects *[]ctrlclient.Object) { target.Spec.ClusterProviderRef = &meta.LocalObjectReference{Name: "closed"} @@ -431,12 +433,12 @@ func TestApplyTriggerRule_AGitTargetEventRerunsTheTriggersItNames(t *testing.T) c := triggerTestClient(t, fluxTarget(), platformProvider()) r := &ApplyTriggerReconciler{Client: c, Kind: configbutleraiv1alpha3.ApplyTriggerKindFlux} - requests := r.gitProviderToTriggers(context.Background(), platformProvider()) + requests := r.gitDestinationToTriggers(context.Background(), platformProvider()) require.Len(t, requests, 1) assert.Equal(t, triggerKey, requests[0].NamespacedName) argo := &ApplyTriggerReconciler{Client: c, Kind: configbutleraiv1alpha3.ApplyTriggerKindArgoCD} - assert.Empty(t, argo.gitProviderToTriggers(context.Background(), platformProvider()), "another kind's") + assert.Empty(t, argo.gitDestinationToTriggers(context.Background(), platformProvider()), "another kind's") } // TestApplyTriggerRule_AnApplierInAnotherNamespaceNeedsTheOptIn is B1: a namespace a ClusterProvider diff --git a/internal/controller/applytrigger_schema_test.go b/internal/controller/applytrigger_schema_test.go index f03f1599..fdf06633 100644 --- a/internal/controller/applytrigger_schema_test.go +++ b/internal/controller/applytrigger_schema_test.go @@ -117,8 +117,7 @@ var _ = Describe("Apply trigger schema", func() { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "prov"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov", "main")}, Path: "apps/" + name, ApplyTriggerRef: ref, }, @@ -172,7 +171,10 @@ var _ = Describe("Apply trigger reconcile", func() { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "names-reconciled", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "prov"}, Branch: "main", Path: "apps/reconciled", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("prov", "main"), + }, + Path: "apps/reconciled", ApplyTriggerRef: &configbutleraiv1alpha3.ApplyTriggerReference{ Kind: configbutleraiv1alpha3.ApplyTriggerKindArgoCD, Name: "reconciled", }, diff --git a/internal/controller/clusterwatchrule_admission_test.go b/internal/controller/clusterwatchrule_admission_test.go index a6044522..cdf5794f 100644 --- a/internal/controller/clusterwatchrule_admission_test.go +++ b/internal/controller/clusterwatchrule_admission_test.go @@ -112,9 +112,8 @@ func cwaGitTarget() *configbutleraiv1alpha3.GitTarget { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: cwaTargetName, Namespace: cwaTargetNS}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "git"}, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: cwaProviderName}, - Branch: "main", Path: "clusters/prod", }, } @@ -311,7 +310,7 @@ func TestReconcile_RefusalStopsDataPlaneBeforePublishingStatus(t *testing.T) { // Pre-seed the store as an earlier, admitted reconcile would have left it, so "no compiled // rule at the end" cannot pass vacuously. f.store.AddOrUpdateClusterWatchRule( - *cwaClusterWatchRule(), cwaTargetName, cwaTargetNS, "git", cwaTargetNS, "main", "clusters/prod") + *cwaClusterWatchRule(), cwaTargetName, cwaTargetNS, "git", cwaTargetNS, "clusters/prod") require.Len(t, f.compiledNames(), 1, "precondition: the rule starts out compiled") f.wm.onReconcile = func() { @@ -428,7 +427,7 @@ func TestReconcile_AdmissionReadErrorRequeuesAndKeepsCompiledRule(t *testing.T) }) f.store.AddOrUpdateClusterWatchRule( - *cwaClusterWatchRule(), cwaTargetName, cwaTargetNS, "git", cwaTargetNS, "main", "clusters/prod") + *cwaClusterWatchRule(), cwaTargetName, cwaTargetNS, "git", cwaTargetNS, "clusters/prod") _, err := f.reconcile(ctx) diff --git a/internal/controller/clusterwatchrule_controller.go b/internal/controller/clusterwatchrule_controller.go index 3a89266a..7e3b4f45 100644 --- a/internal/controller/clusterwatchrule_controller.go +++ b/internal/controller/clusterwatchrule_controller.go @@ -32,7 +32,6 @@ import ( // ClusterWatchRule status condition reasons. const ( ClusterWatchRuleReasonValidating = "Validating" - ClusterWatchRuleReasonGitProviderNotFound = "GitProviderNotFound" ClusterWatchRuleReasonAccessDenied = "AccessDenied" ClusterWatchRuleReasonGitTargetNotFound = "GitTargetNotFound" ClusterWatchRuleReasonGitDestinationInvalid = "GitDestinationInvalid" @@ -186,33 +185,11 @@ func (r *ClusterWatchRuleReconciler) reconcileClusterWatchRuleViaTarget( } r.setGitTargetReadyCondition(st, target) - // Resolve GitProvider from target - providerName := target.Spec.GitProviderRef.Name - providerNS := target.GitProviderKey().Namespace - - var provider configbutleraiv1alpha3.GitProvider - providerKey := types.NamespacedName{Name: providerName, Namespace: providerNS} - if err := r.Get(ctx, providerKey, &provider); err != nil { - log.Error(err, "Failed to resolve GitProvider from GitTarget", - "gitProviderName", providerName, "gitProviderNamespace", providerNS) - st.set( - ConditionTypeGitTargetReady, - metav1.ConditionFalse, - ClusterWatchRuleReasonGitProviderNotFound, - fmt.Sprintf("GitProvider '%s/%s' (from GitTarget) not found: %v", - providerNS, providerName, err), - ) - return r.stallRule(ctx, st, ClusterWatchRuleReasonGitProviderNotFound, "Referenced GitProvider not found") - } - - // Ready check - // TODO: Check GitProvider readiness - // Admission AND compilation, in that order and in one call — see gateClusterWatchRule. There is // deliberately no AddOrUpdateClusterWatchRule here: routing every compilation through // watch.CompileClusterWatchRule is what stops the startup bootstrap from being a second, // ungated path into the store. - if handled, result, err := r.gateClusterWatchRule(ctx, st, clusterRule, target, provider, log); handled { + if handled, result, err := r.gateClusterWatchRule(ctx, st, clusterRule, target, log); handled { return result, err } @@ -256,11 +233,10 @@ func (r *ClusterWatchRuleReconciler) gateClusterWatchRule( st *reconcileStatus, clusterRule *configbutleraiv1alpha3.ClusterWatchRule, target configbutleraiv1alpha3.GitTarget, - provider configbutleraiv1alpha3.GitProvider, log logr.Logger, ) (bool, ctrl.Result, error) { decision, err := watch.CompileClusterWatchRule( - ctx, r.Client, r.RuleStore, *clusterRule, target, provider) + ctx, r.Client, r.RuleStore, *clusterRule, target) var selectorErr *watch.ObjectSelectorError if errors.As(err, &selectorErr) { // A refusal of the rule's own resources, not of its GitTarget, so it lands on @@ -403,15 +379,6 @@ func (r *ClusterWatchRuleReconciler) SetupWithManager(mgr ctrl.Manager) error { handler.EnqueueRequestsFromMapFunc(r.gitTargetToClusterWatchRules), builder.WithPredicates(gitTargetReadyProjectionChanged()), ). - // Nothing of the GitProvider's STATUS is mirrored here, so GenerationChangedPredicate is - // right on this edge: it reacts to a freshly applied or spec-changed provider while ignoring - // the status-only updates its controller writes itself — without it every GitProvider - // heartbeat would re-list and re-enqueue all ClusterWatchRules. - Watches( - &configbutleraiv1alpha3.GitProvider{}, - handler.EnqueueRequestsFromMapFunc(r.gitProviderToClusterWatchRules), - builder.WithPredicates(predicate.GenerationChangedPredicate{}), - ). // React to a ClusterProvider's accessFrom changing. Without this, REVOKING a // namespace stops the GitTarget (which does watch ClusterProvider) but leaves this rule's // compiled entry resident until the next periodic reconcile, so the admission gate would @@ -557,52 +524,3 @@ func (r *ClusterWatchRuleReconciler) gitTargetToClusterWatchRules( } return requests } - -// gitProviderToClusterWatchRules maps a GitProvider event to every -// ClusterWatchRule whose referenced GitTarget points at this provider. This -// closes the gap where a freshly-arrived GitProvider needs to nudge rules whose -// own GitTarget event would otherwise not fire (e.g. the GitTarget already -// existed but its provider lookup was failing). -func (r *ClusterWatchRuleReconciler) gitProviderToClusterWatchRules( - ctx context.Context, - obj client.Object, -) []ctrlreconcile.Request { - var targets configbutleraiv1alpha3.GitTargetList - if err := r.List(ctx, &targets); err != nil { - logDependencyListError(ctx, err, "GitTargets", obj) - return nil - } - - matchingTargets := make(map[types.NamespacedName]struct{}) - for i := range targets.Items { - t := &targets.Items[i] - if namesGitProvider(t, obj) { - matchingTargets[types.NamespacedName{Name: t.Name, Namespace: t.Namespace}] = struct{}{} - } - } - if len(matchingTargets) == 0 { - return nil - } - - var rules configbutleraiv1alpha3.ClusterWatchRuleList - if err := r.List(ctx, &rules); err != nil { - logDependencyListError(ctx, err, "ClusterWatchRules", obj) - return nil - } - - var requests []ctrlreconcile.Request - for i := range rules.Items { - rule := &rules.Items[i] - key := types.NamespacedName{ - Name: rule.Spec.GitTargetRef.Name, - Namespace: rule.Spec.GitTargetRef.Namespace, - } - if _, ok := matchingTargets[key]; !ok { - continue - } - requests = append(requests, ctrlreconcile.Request{ - NamespacedName: types.NamespacedName{Name: rule.Name}, - }) - } - return requests -} diff --git a/internal/controller/commitrequest_outcome_metric_test.go b/internal/controller/commitrequest_outcome_metric_test.go index c6453c80..52e7c666 100644 --- a/internal/controller/commitrequest_outcome_metric_test.go +++ b/internal/controller/commitrequest_outcome_metric_test.go @@ -356,8 +356,7 @@ func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } fresh := newCommitRequest("save-fresh") @@ -367,7 +366,7 @@ func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { stale.Spec.GitTargetRef = meta.LocalObjectReference{Name: target.Name} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. stale.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(stale.Spec).resolveTimeout())) - c := newCommitRequestClient(t, nil, fresh, stale, target) + c := newCommitRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) workers := git.NewWorkerManager(c, logr.Discard(), git.BranchWorkerLimits{}, itypes.SensitiveResourcePolicy{}) router := watch.NewEventRouter(workers, nil, c, logr.Discard()) r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: router, AuthorLookup: attributedAlice()} diff --git a/internal/controller/condition_helper_test.go b/internal/controller/condition_helper_test.go index e6800eae..36e27e78 100644 --- a/internal/controller/condition_helper_test.go +++ b/internal/controller/condition_helper_test.go @@ -61,8 +61,8 @@ func TestUpsertCondition_ChangesTransitionTimeWhenStatusChanges(t *testing.T) { { Type: GitTargetConditionValidated, Status: metav1.ConditionFalse, - Reason: GitTargetReasonProviderNotFound, - Message: "missing provider", + Reason: GitTargetReasonGitBranchNotFound, + Message: "missing GitBranch", ObservedGeneration: 2, LastTransitionTime: oldTransition, }, diff --git a/internal/controller/dependency_watches_test.go b/internal/controller/dependency_watches_test.go index b481994a..283242a8 100644 --- a/internal/controller/dependency_watches_test.go +++ b/internal/controller/dependency_watches_test.go @@ -66,8 +66,7 @@ func gitTarget(name, namespace, providerName string) *configbutleraiv1alpha3.Git return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: providerName}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest(providerName, "main")}, }, } } @@ -93,8 +92,18 @@ func clusterWatchRule(name, targetName, targetNamespace string) *configbutleraiv } } -// TestGitProviderToGitTargets verifies that a GitProvider event enqueues only -// the GitTargets in its namespace that reference it. +// gitBranch is the GitBranch "-main" in namespace, written through that GitProvider. +func gitBranch(providerName, namespace string) *configbutleraiv1alpha3.GitBranch { + return &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Name: gitBranchNameForTest(providerName, "main"), Namespace: namespace}, + Spec: configbutleraiv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: providerName}, Branch: "main", + }, + } +} + +// TestGitProviderToGitTargets verifies that a GitProvider event enqueues only the GitTargets on the +// GitBranches in its namespace that name it. func TestGitProviderToGitTargets(t *testing.T) { tests := []struct { name string @@ -148,10 +157,13 @@ func TestGitProviderToGitTargets(t *testing.T) { for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - client := newDependencyWatchesTestClient(t, tc.objects...) + objects := append([]ctrlclient.Object{ + gitBranch("provider-a", "ns-a"), gitBranch("provider-b", "ns-a"), gitBranch("provider-a", "ns-b"), + }, tc.objects...) + client := newDependencyWatchesTestClient(t, objects...) r := &GitTargetReconciler{Client: client} - got := r.gitProviderToGitTargets(context.Background(), tc.provider) + got := r.gitDestinationToGitTargets(context.Background(), tc.provider) assert.Equal(t, tc.want, requestsToKeys(got)) }) @@ -198,49 +210,6 @@ func TestGitTargetToClusterWatchRules(t *testing.T) { } } -// TestGitProviderToClusterWatchRules verifies that a GitProvider event -// transitively enqueues ClusterWatchRules via their GitTarget. -func TestGitProviderToClusterWatchRules(t *testing.T) { - tests := []struct { - name string - objects []ctrlclient.Object - provider ctrlclient.Object - want []string - }{ - { - name: "enqueues rules whose target points at the provider", - objects: []ctrlclient.Object{ - gitTarget("target-a", "ns-a", "provider-a"), - gitTarget("target-other", "ns-a", "provider-b"), - clusterWatchRule("rule-matched", "target-a", "ns-a"), - clusterWatchRule("rule-unrelated", "target-other", "ns-a"), - }, - provider: gitProvider("provider-a", "ns-a"), - want: []string{"/rule-matched"}, - }, - { - name: "no matching targets returns empty", - objects: []ctrlclient.Object{ - gitTarget("target-a", "ns-a", "provider-b"), - clusterWatchRule("rule-matched", "target-a", "ns-a"), - }, - provider: gitProvider("provider-a", "ns-a"), - want: nil, - }, - } - - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - client := newDependencyWatchesTestClient(t, tc.objects...) - r := &ClusterWatchRuleReconciler{Client: client} - - got := r.gitProviderToClusterWatchRules(context.Background(), tc.provider) - - assert.Equal(t, tc.want, requestsToKeys(got)) - }) - } -} - // TestGitTargetToWatchRules verifies that a GitTarget event enqueues every // WatchRule in the same namespace that names it. func TestGitTargetToWatchRules(t *testing.T) { @@ -281,49 +250,6 @@ func TestGitTargetToWatchRules(t *testing.T) { } } -// TestGitProviderToWatchRules verifies that a GitProvider event transitively -// enqueues WatchRules in the provider's namespace via their GitTarget. -func TestGitProviderToWatchRules(t *testing.T) { - tests := []struct { - name string - objects []ctrlclient.Object - provider ctrlclient.Object - want []string - }{ - { - name: "enqueues rules whose target points at the provider", - objects: []ctrlclient.Object{ - gitTarget("target-a", "ns-a", "provider-a"), - gitTarget("target-other", "ns-a", "provider-b"), - watchRule("rule-matched", "ns-a", "target-a"), - watchRule("rule-unrelated", "ns-a", "target-other"), - }, - provider: gitProvider("provider-a", "ns-a"), - want: []string{"ns-a/rule-matched"}, - }, - { - name: "no matching targets returns empty", - objects: []ctrlclient.Object{ - gitTarget("target-a", "ns-a", "provider-b"), - watchRule("rule-matched", "ns-a", "target-a"), - }, - provider: gitProvider("provider-a", "ns-a"), - want: nil, - }, - } - - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - client := newDependencyWatchesTestClient(t, tc.objects...) - r := &WatchRuleReconciler{Client: client} - - got := r.gitProviderToWatchRules(context.Background(), tc.provider) - - assert.Equal(t, tc.want, requestsToKeys(got)) - }) - } -} - // TestGenerationChangedPredicateFiltersStatusUpdates verifies PR #149 review // issue 2: the cross-kind watches use GenerationChangedPredicate, so they react // to a freshly applied or spec-changed dependency but ignore the status-only @@ -381,11 +307,9 @@ func TestDependencyMapFunctionsTolerateListErrors(t *testing.T) { provider := gitProvider("provider-a", "ns-a") target := gitTarget("target-a", "ns-a", "provider-a") - assert.Nil(t, (&GitTargetReconciler{Client: client}).gitProviderToGitTargets(ctx, provider)) + assert.Nil(t, (&GitTargetReconciler{Client: client}).gitDestinationToGitTargets(ctx, provider)) assert.Nil(t, (&WatchRuleReconciler{Client: client}).gitTargetToWatchRules(ctx, target)) - assert.Nil(t, (&WatchRuleReconciler{Client: client}).gitProviderToWatchRules(ctx, provider)) assert.Nil(t, (&ClusterWatchRuleReconciler{Client: client}).gitTargetToClusterWatchRules(ctx, target)) - assert.Nil(t, (&ClusterWatchRuleReconciler{Client: client}).gitProviderToClusterWatchRules(ctx, provider)) } // TestRequeueSteadyIntervalIsFiveMinutes locks the unified control-plane periodic reconcile diff --git a/internal/controller/git_branch_helpers_test.go b/internal/controller/git_branch_helpers_test.go new file mode 100644 index 00000000..4121454c --- /dev/null +++ b/internal/controller/git_branch_helpers_test.go @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "time" + + meta "github.com/fluxcd/pkg/apis/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + k8stypes "k8s.io/apimachinery/pkg/types" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" +) + +// gitBranchNameForTest is the GitBranch a test GitTarget references: one per provider and branch. +func gitBranchNameForTest(provider, branch string) string { return provider + "-" + branch } + +// shopRepo1Main is the GitBranch shop/repo1-main, which the worker tests' GitTargets reference. +var shopRepo1Main = k8stypes.NamespacedName{Namespace: "shop", Name: gitBranchNameForTest("repo1", "main")} + +// claimWorkerForTest makes shopRepo1Main hold its repository branch, as its reconcile does once its +// credential is proved, and returns the request its GitTargets make for repo. +func claimWorkerForTest(m *git.WorkerManager, repo git.RepoIdentity) git.WorkerRequest { + m.Claims().Observe(git.BranchKeyFor(repo.URL, "main"), git.BranchClaimant{ + GitBranch: shopRepo1Main, + UID: k8stypes.UID(shopRepo1Main.String()), + Created: time.Unix(0, 0), + }, true) + return git.WorkerRequest{GitBranch: shopRepo1Main, Provider: "repo1", Branch: "main", Repo: repo} +} + +// gitBranchOn is the GitBranch "-" in namespace, writing branch through provider. +func gitBranchOn(provider, namespace, branch string) *configbutleraiv1alpha3.GitBranch { + return &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Name: gitBranchNameForTest(provider, branch), Namespace: namespace}, + Spec: configbutleraiv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: provider}, Branch: branch, + }, + } +} diff --git a/internal/controller/gitbranch_controller.go b/internal/controller/gitbranch_controller.go new file mode 100644 index 00000000..5162e956 --- /dev/null +++ b/internal/controller/gitbranch_controller.go @@ -0,0 +1,441 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "fmt" + "path/filepath" + "sync" + "time" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + k8stypes "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/builder" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/predicate" + ctrlreconcile "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/repourl" + "github.com/ConfigButler/gitops-reverser/internal/telemetry" +) + +// GitBranchFinalizer holds a GitBranch until its branch worker has given every pending write and +// CommitRequest an outcome. It never deletes the branch on the remote. +const GitBranchFinalizer = "configbutler.ai/git-branch" + +const ( + // GitBranchReasonGitProviderNotFound is the stall for a GitBranch whose GitProvider does not + // exist in its namespace. + GitBranchReasonGitProviderNotFound = "GitProviderNotFound" + // GitBranchReasonGitProviderNotReady is the progress reason while the GitProvider is not Ready. + GitBranchReasonGitProviderNotReady = "GitProviderNotReady" + // GitBranchReasonBranchNotAllowed is the stall for a branch outside the GitProvider's + // allowedBranches. + GitBranchReasonBranchNotAllowed = "BranchNotAllowed" + // GitBranchReasonPushSessionFailed is the progress reason while the credential cannot open a + // push (receive-pack) session on the remote. A GitBranch that holds its branch keeps it. + GitBranchReasonPushSessionFailed = "PushSessionFailed" + // GitBranchReasonBranchConflict is the stall for a GitBranch whose repository branch another, + // older GitBranch holds. + GitBranchReasonBranchConflict = "BranchConflict" + // GitBranchReasonParentBranchNotFound is the stall for a branch that cannot be created: the + // remote carries neither it nor the parent it would be created from. + GitBranchReasonParentBranchNotFound = "ParentBranchNotFound" + // GitBranchReasonRecoveringParentBranch is progress while the parent branch has been found again + // but the branch worker has not yet published the writes it kept while the parent was missing. + GitBranchReasonRecoveringParentBranch = "RecoveringParentBranch" +) + +// proofReuse is how long a successful push-session proof stands before the next reconcile proves +// the credential again. The branch worker wakes its GitBranch on every push, and proving on each of +// those would cost a round trip per push; the steady interval is how stale any other reconcile-time +// fact here is allowed to be. +const proofReuse = RequeueSteadyInterval + +// GitBranchReconciler reconciles a GitBranch: it proves the credential, keeps the claim on the +// repository branch (git.BranchClaims), and publishes where the branch is on the remote. +type GitBranchReconciler struct { + client.Client + + Scheme *runtime.Scheme + WorkerManager *git.WorkerManager + // Recorder emits a Kubernetes Event on every persisted Ready transition. It may be nil in tests. + Recorder record.EventRecorder + + // SSHHostKeys and CredentialPolicy resolve the GitProvider's credential exactly as its branch + // worker does, so the credential proved is the one that will push. + SSHHostKeys git.SSHHostKeyConfig + CredentialPolicy git.CredentialTransportPolicy + + // proveProvider stands in for opening a push session with a GitProvider's credential. Nil + // means the real one; a test sets it. + proveProvider func(provider *configbutleraiv1alpha3.GitProvider) error + + // proofs is the last proof per GitBranch UID, so a reconcile woken by a push does not prove the + // credential again. Guarded by proofsMu. + proofsMu sync.Mutex + proofs map[k8stypes.UID]proofRecord + + // remotePublications is what each GitBranch last wrote to status.remote. See + // remotePublicationLedger. + remotePublications remotePublicationLedger + + // clock is the source of "now", so a test can hold time still. Nil means time.Now. + clock func() time.Time +} + +// proofRecord is one proof's outcome, and the GitProvider object it was made with. +type proofRecord struct { + at time.Time + provider k8stypes.UID + err error +} + +func (r *GitBranchReconciler) clockNow() time.Time { + if r.clock != nil { + return r.clock() + } + return time.Now() +} + +// +kubebuilder:rbac:groups=configbutler.ai,resources=gitbranches,verbs=get;list;watch;update;patch +// +kubebuilder:rbac:groups=configbutler.ai,resources=gitbranches/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=configbutler.ai,resources=gitbranches/finalizers,verbs=update + +// newGitBranchReadiness starts the accumulator that owns this kind's kstatus trio. +func newGitBranchReadiness(key git.BranchKey) *readiness { + return newReadiness(fmt.Sprintf("GitBranch holds %s", key), "GitBranch is not stalled") +} + +// Reconcile proves the GitBranch's credential, keeps its claim, and publishes its status. +func (r *GitBranchReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + var branch configbutleraiv1alpha3.GitBranch + if err := r.Get(ctx, req.NamespacedName, &branch); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + if !branch.DeletionTimestamp.IsZero() { + return ctrl.Result{}, r.finalize(ctx, &branch) + } + + st := beginStatus(r.Client, r.Recorder, &branch) + if err := r.ensureFinalizer(ctx, st, &branch); err != nil { + return ctrl.Result{}, err + } + if err := r.countGitTargets(ctx, &branch); err != nil { + return ctrl.Result{}, err + } + + rd, heldFor, err := r.evaluate(ctx, st, &branch) + if err != nil { + return ctrl.Result{}, err + } + st.applyReadiness(rd) + if err := st.commit(ctx); err != nil { + return ctrl.Result{}, err + } + cadence := RequeueStreamSettleInterval + if rd.converged() { + cadence = RequeueSteadyInterval + } + if heldFor > 0 && heldFor < cadence { + cadence = heldFor + } + return ctrl.Result{RequeueAfter: st.requeueAfter(cadence)}, nil +} + +// evaluate runs the gates in order and returns the verdict, and how long the publication floor +// holds back an answer about the remote. +func (r *GitBranchReconciler) evaluate( + ctx context.Context, + st *reconcileStatus, + branch *configbutleraiv1alpha3.GitBranch, +) (*readiness, time.Duration, error) { + var provider configbutleraiv1alpha3.GitProvider + if err := r.Get(ctx, branch.GitProviderKey(), &provider); err != nil { + if !apierrors.IsNotFound(err) { + return nil, 0, err + } + // The holder keeps its claim (rule 3): the claim is left as it is, so its worker keeps the + // writes it holds until the GitProvider is back or the GitBranch is deleted. + rd := newReadiness("", "GitBranch is not stalled") + rd.stalled(GitBranchReasonGitProviderNotFound, + fmt.Sprintf("Referenced GitProvider '%s' not found", branch.GitProviderKey())) + return rd, 0, nil + } + + key := git.BranchKeyFor(provider.Spec.URL, branch.Spec.Branch) + branch.Status.Repository = repourl.Normalize(provider.Spec.URL) + rd := newGitBranchReadiness(key) + claimant := git.BranchClaimant{ + GitBranch: client.ObjectKeyFromObject(branch), + UID: branch.UID, + Created: branch.CreationTimestamp.Time, + } + + if !branchAllowed(provider.Spec.AllowedBranches, branch.Spec.Branch) { + r.observeClaim(key, claimant, false) + rd.stalled(GitBranchReasonBranchNotAllowed, fmt.Sprintf( + "Branch '%s' does not match any pattern in allowedBranches list %v of GitProvider '%s'", + branch.Spec.Branch, provider.Spec.AllowedBranches, branch.GitProviderKey())) + return rd, 0, nil + } + + proofErr := r.proveCredential(ctx, branch, &provider) + holder, held := r.observeClaim(key, claimant, proofErr == nil) + if held && holder.UID != branch.UID { + rd.stalled(GitBranchReasonBranchConflict, fmt.Sprintf( + "GitBranch '%s' holds repository %s, branch '%s'. Only one GitBranch writes a repository "+ + "branch; reference that one from your GitTargets, or delete it first", + holder.GitBranch, key.Repository, key.Branch)) + return rd, 0, nil + } + if proofErr != nil { + message := fmt.Sprintf("The credential of GitProvider '%s' cannot open a push session on the remote: %v", + branch.GitProviderKey(), proofErr) + if held { + message += "; this GitBranch keeps the branch and its pending writes until it can" + } + rd.progressing(metav1.ConditionFalse, GitBranchReasonPushSessionFailed, message) + } + rd.progressingIf(!conditionIsTrue(provider.Status.Conditions, ConditionTypeReady), metav1.ConditionFalse, + GitBranchReasonGitProviderNotReady, fmt.Sprintf("Referenced GitProvider '%s' is not Ready", + branch.GitProviderKey())) + + if !held { + return rd, 0, nil + } + heldFor := r.observeWorker(st, rd, branch, key, &provider) + return rd, heldFor, nil +} + +// observeWorker publishes what the holder's branch worker last proved about the remote, and the +// gates that come from it: a parent branch that is missing, and work held back while it was. +func (r *GitBranchReconciler) observeWorker( + st *reconcileStatus, + rd *readiness, + branch *configbutleraiv1alpha3.GitBranch, + key git.BranchKey, + provider *configbutleraiv1alpha3.GitProvider, +) time.Duration { + repo := repoIdentityOf(provider) + observed, seen := r.observeRemote(key) + heldFor := r.publishRemote(st, branch, observed, seen, repo, r.clockNow()) + + if seen && observed.ParentState == git.ParentMissing && + observed.ParentRequested == branch.Spec.ParentBranch && !differentRepository(observed.Repo, repo) { + rd.stalled(GitBranchReasonParentBranchNotFound, parentBranchNotFoundMessage(branch, observed.ParentBranch)) + } + if r.WorkerManager == nil { + return heldFor + } + worker, live := r.WorkerManager.WorkerForGitBranch(client.ObjectKeyFromObject(branch)) + if !live { + return heldFor + } + open, found := worker.ParentRecovery() + rd.progressingIf(open && found, metav1.ConditionFalse, GitBranchReasonRecoveringParentBranch, + "The parent branch is on the remote again; the branch worker is publishing the work it held back") + return heldFor +} + +// observeClaim records the proof with the claim registry and returns the holder. +func (r *GitBranchReconciler) observeClaim( + key git.BranchKey, claimant git.BranchClaimant, proven bool, +) (git.BranchClaimant, bool) { + if r.WorkerManager == nil { + return git.BranchClaimant{}, false + } + return r.WorkerManager.Claims().Observe(key, claimant, proven) +} + +// proveCredential opens and closes a receive-pack session with the GitProvider's credential. A +// success stands for proofReuse, for the same GitProvider object; a failure is retried on every +// reconcile. +func (r *GitBranchReconciler) proveCredential( + ctx context.Context, + branch *configbutleraiv1alpha3.GitBranch, + provider *configbutleraiv1alpha3.GitProvider, +) error { + now := r.clockNow() + r.proofsMu.Lock() + last, proved := r.proofs[branch.UID] + r.proofsMu.Unlock() + if proved && last.err == nil && last.provider == provider.UID && now.Sub(last.at) < proofReuse { + return nil + } + + err := r.openPushSession(ctx, provider) + r.proofsMu.Lock() + if r.proofs == nil { + r.proofs = map[k8stypes.UID]proofRecord{} + } + r.proofs[branch.UID] = proofRecord{at: now, provider: provider.UID, err: err} + r.proofsMu.Unlock() + return err +} + +func (r *GitBranchReconciler) openPushSession( + ctx context.Context, + provider *configbutleraiv1alpha3.GitProvider, +) error { + if r.proveProvider != nil { + return r.proveProvider(provider) + } + auth, err := git.AuthForProvider(ctx, r.Client, provider, r.SSHHostKeys, r.CredentialPolicy) + if err != nil { + return err + } + return git.ProvePushSession(ctx, provider.Spec.URL, auth) +} + +// countGitTargets publishes status.gitTargetCount: the live GitTargets, from any namespace, that +// reference this GitBranch. It is read from the cache. +func (r *GitBranchReconciler) countGitTargets(ctx context.Context, branch *configbutleraiv1alpha3.GitBranch) error { + var targets configbutleraiv1alpha3.GitTargetList + if err := r.List(ctx, &targets); err != nil { + return err + } + self := client.ObjectKeyFromObject(branch) + var count int32 + for i := range targets.Items { + if targets.Items[i].DeletionTimestamp.IsZero() && targets.Items[i].GitBranchKey() == self { + count++ + } + } + branch.Status.GitTargetCount = count + return nil +} + +// ensureFinalizer puts the finalizer on a GitBranch before it can hold a branch, and carries the +// metadata that write left into the status session. +func (r *GitBranchReconciler) ensureFinalizer( + ctx context.Context, st *reconcileStatus, branch *configbutleraiv1alpha3.GitBranch, +) error { + update := branch.DeepCopy() + if !controllerutil.AddFinalizer(update, GitBranchFinalizer) { + return nil + } + if err := r.Update(ctx, update); err != nil { + return err + } + branch.SetFinalizers(update.GetFinalizers()) + branch.SetResourceVersion(update.GetResourceVersion()) + st.metadataWritten() + return nil +} + +// finalize gives up the branch. The claim goes first, so no GitTarget reconcile can start this +// GitBranch's worker again; then the worker is stopped, which closes its intake, pushes or refuses +// every pending write and settles every CommitRequest it holds (BranchWorker.Stop). Only then is +// the finalizer removed. Nothing deletes the branch on the remote. +func (r *GitBranchReconciler) finalize(ctx context.Context, branch *configbutleraiv1alpha3.GitBranch) error { + if !controllerutil.ContainsFinalizer(branch, GitBranchFinalizer) { + return nil + } + if r.WorkerManager != nil { + r.WorkerManager.Claims().Release(branch.UID) + r.WorkerManager.RetireGitBranch(client.ObjectKeyFromObject(branch)) + } + r.proofsMu.Lock() + delete(r.proofs, branch.UID) + r.proofsMu.Unlock() + r.remotePublications.forget(gitBranchRef(branch)) + telemetry.ForgetResourceConditions(conditionKindGitBranch, branch.Namespace, branch.Name) + + update := branch.DeepCopy() + controllerutil.RemoveFinalizer(update, GitBranchFinalizer) + return client.IgnoreNotFound(r.Update(ctx, update)) +} + +// branchAllowed reports whether branch matches one of the GitProvider's allowedBranches patterns. +func branchAllowed(patterns []string, branch string) bool { + for _, pattern := range patterns { + if match, err := filepath.Match(pattern, branch); err == nil && match { + return true + } + } + return false +} + +// parentBranchNotFoundMessage says which parent is missing and what fixes it. +func parentBranchNotFoundMessage(branch *configbutleraiv1alpha3.GitBranch, missing string) string { + switch { + case branch.Spec.ParentBranch != "": + return fmt.Sprintf( + "Branch '%s' does not exist on the remote, and neither does its parent branch '%s' to create "+ + "it from; nothing is written until one of them exists", + branch.Spec.Branch, branch.Spec.ParentBranch) + case missing != "": + return fmt.Sprintf( + "Branch '%s' does not exist on the remote, and the remote's default branch (HEAD -> '%s') does "+ + "not exist either; fix the remote's HEAD or set spec.parentBranch. Nothing is written until then", + branch.Spec.Branch, missing) + default: + return fmt.Sprintf( + "Branch '%s' does not exist on the remote, and the remote has no default branch to create it "+ + "from; set the remote's HEAD or spec.parentBranch. Nothing is written until then", + branch.Spec.Branch) + } +} + +// SetupWithManager sets up the controller with the Manager. +func (r *GitBranchReconciler) SetupWithManager(mgr ctrl.Manager) error { + b := ctrl.NewControllerManagedBy(mgr). + For(&configbutleraiv1alpha3.GitBranch{}, builder.WithPredicates(predicate.GenerationChangedPredicate{})). + // Every GitProvider event, status included: its Ready flipping is what un-blocks a GitBranch, + // and providers are few. + Watches(&configbutleraiv1alpha3.GitProvider{}, handler.EnqueueRequestsFromMapFunc(r.gitProviderToGitBranches)). + // gitBranchRef is immutable, so only a GitTarget appearing or going changes the count. + Watches(&configbutleraiv1alpha3.GitTarget{}, handler.EnqueueRequestsFromMapFunc(gitTargetToGitBranch), + builder.WithPredicates(predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + return e.ObjectOld.GetDeletionTimestamp().IsZero() != e.ObjectNew.GetDeletionTimestamp().IsZero() + }, + })). + Named("gitbranch") + if r.WorkerManager != nil && r.WorkerManager.GitBranchEvents() != nil { + b = b.WatchesRawSource(source.Channel(r.WorkerManager.GitBranchEvents(), &handler.EnqueueRequestForObject{})) + } + return b.Complete(r) +} + +// gitProviderToGitBranches enqueues the GitBranches in the provider's namespace that name it. +func (r *GitBranchReconciler) gitProviderToGitBranches(ctx context.Context, obj client.Object) []ctrlreconcile.Request { + var branches configbutleraiv1alpha3.GitBranchList + if err := r.List(ctx, &branches, client.InNamespace(obj.GetNamespace())); err != nil { + return nil + } + var requests []ctrlreconcile.Request + for i := range branches.Items { + if branches.Items[i].Spec.GitProviderRef.Name == obj.GetName() { + requests = append(requests, ctrlreconcile.Request{ + NamespacedName: client.ObjectKeyFromObject(&branches.Items[i]), + }) + } + } + return requests +} + +// gitTargetToGitBranch enqueues the GitBranch a GitTarget references. +func gitTargetToGitBranch(_ context.Context, obj client.Object) []ctrlreconcile.Request { + target, ok := obj.(*configbutleraiv1alpha3.GitTarget) + if !ok { + return nil + } + key := target.GitBranchKey() + if key.Name == "" { + return nil + } + return []ctrlreconcile.Request{{NamespacedName: key}} +} diff --git a/internal/controller/gitbranch_controller_test.go b/internal/controller/gitbranch_controller_test.go new file mode 100644 index 00000000..efd9bd26 --- /dev/null +++ b/internal/controller/gitbranch_controller_test.go @@ -0,0 +1,356 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "errors" + "testing" + "time" + + meta "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + k8stypes "k8s.io/apimachinery/pkg/types" + clientgoscheme "k8s.io/client-go/kubernetes/scheme" + ctrl "sigs.k8s.io/controller-runtime" + ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +const gitBranchTestURL = "https://github.com/acme/platform.git" + +var gitBranchEpoch = time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC) + +// branchProvider is a GitProvider in namespace for the test repository, allowing branches, Ready +// unless told otherwise. It has no secretRef, so its credential resolves to anonymous and the +// fixture's prover decides what that credential may do. +func branchProvider(namespace, name string, allowed ...string) *configbutleraiv1alpha3.GitProvider { + if len(allowed) == 0 { + allowed = []string{"*"} + } + return &configbutleraiv1alpha3.GitProvider{ + ObjectMeta: metav1.ObjectMeta{Namespace: namespace, Name: name, UID: k8stypes.UID("uid-" + name)}, + Spec: configbutleraiv1alpha3.GitProviderSpec{URL: gitBranchTestURL, AllowedBranches: allowed}, + Status: configbutleraiv1alpha3.GitProviderStatus{Conditions: []metav1.Condition{{ + Type: ConditionTypeReady, Status: metav1.ConditionTrue, Reason: ReasonSucceeded, + LastTransitionTime: metav1.Now(), + }}}, + } +} + +// branchObject is a GitBranch namespace/name writing branch through provider, created age ago. +func branchObject(namespace, name, provider, branch string, age time.Duration) *configbutleraiv1alpha3.GitBranch { + return &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: namespace, Name: name, UID: k8stypes.UID("uid-" + namespace + "-" + name), + CreationTimestamp: metav1.NewTime(gitBranchEpoch.Add(-age)), + }, + Spec: configbutleraiv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: provider}, Branch: branch, + }, + } +} + +// gitBranchFixture runs the GitBranch reconciler against a fake API and a started worker manager. +// Every credential proves itself unless a test sets r.proveProvider. +type gitBranchFixture struct { + r *GitBranchReconciler + client ctrlclient.Client + workers *git.WorkerManager +} + +func newGitBranchFixture(t *testing.T, objects ...ctrlclient.Object) *gitBranchFixture { + t.Helper() + scheme := runtime.NewScheme() + require.NoError(t, clientgoscheme.AddToScheme(scheme)) + require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(objects...). + WithStatusSubresource(&configbutleraiv1alpha3.GitBranch{}, &configbutleraiv1alpha3.GitProvider{}).Build() + workers := git.NewWorkerManager(c, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + go func() { _ = workers.Start(ctx) }() + time.Sleep(50 * time.Millisecond) // Start installs the context workers are started under + + f := &gitBranchFixture{client: c, workers: workers} + f.r = &GitBranchReconciler{Client: c, WorkerManager: workers} + f.r.proveProvider = func(*configbutleraiv1alpha3.GitProvider) error { return nil } + return f +} + +// reconcile runs one reconcile of namespace/name and returns the GitBranch as stored. +func (f *gitBranchFixture) reconcile(t *testing.T, namespace, name string) *configbutleraiv1alpha3.GitBranch { + t.Helper() + key := k8stypes.NamespacedName{Namespace: namespace, Name: name} + _, err := f.r.Reconcile(context.Background(), ctrl.Request{NamespacedName: key}) + require.NoError(t, err) + got := &configbutleraiv1alpha3.GitBranch{} + require.NoError(t, f.client.Get(context.Background(), key, got)) + return got +} + +func readyOf(b *configbutleraiv1alpha3.GitBranch) *metav1.Condition { + return findCondition(b.Status.Conditions, ConditionTypeReady) +} + +// TestGitBranch_AProvenGitBranchHoldsItsBranch: a GitBranch whose credential opens a push session +// holds the repository branch, is Ready, carries its finalizer, publishes the normalized repository +// and counts the GitTargets that reference it from any namespace. +func TestGitBranch_AProvenGitBranchHoldsItsBranch(t *testing.T) { + target := func(namespace, name string) *configbutleraiv1alpha3.GitTarget { + return &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: namespace, Name: name}, + Spec: configbutleraiv1alpha3.GitTargetSpec{ + GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, Path: name, + }, + } + } + f := newGitBranchFixture(t, + branchProvider("platform", "acme"), branchObject("platform", "main", "acme", "main", time.Hour), + target("platform", "infra"), target("team-a", "apps"), + &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "other"}, + Spec: configbutleraiv1alpha3.GitTargetSpec{ + GitBranchRef: &meta.NamespacedObjectReference{Name: "main"}, Path: "other", + }, + }) + + got := f.reconcile(t, "platform", "main") + + ready := readyOf(got) + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionTrue, ready.Status, ready.Message) + assert.Contains(t, got.Finalizers, GitBranchFinalizer) + assert.Equal(t, "github.com/acme/platform", got.Status.Repository) + assert.Equal(t, int32(2), got.Status.GitTargetCount, "team-a/other references team-a/main, another GitBranch") + key, _, held := f.workers.Claims().HeldBy(ctrlclient.ObjectKeyFromObject(got)) + require.True(t, held) + assert.Equal(t, git.BranchKeyFor(gitBranchTestURL, "main"), key) +} + +// TestGitBranch_ABadCredentialCreatedFirstBlocksNobody is the plan's first "done when" about the +// claim: a GitBranch with a bad credential, created first, does not stop a valid one created later +// from claiming the branch, and it reports which GitBranch holds it. +func TestGitBranch_ABadCredentialCreatedFirstBlocksNobody(t *testing.T) { + f := newGitBranchFixture(t, + branchProvider("squat", "acme"), branchObject("squat", "main", "acme", "main", 2*time.Hour), + branchProvider("platform", "acme"), branchObject("platform", "main", "acme", "main", time.Hour)) + f.r.proveProvider = func(provider *configbutleraiv1alpha3.GitProvider) error { + if provider.Namespace == "squat" { + return errors.New("authentication required") + } + return nil + } + + squat := f.reconcile(t, "squat", "main") + assert.Equal(t, GitBranchReasonPushSessionFailed, readyOf(squat).Reason) + assert.Contains(t, readyOf(squat).Message, "authentication required") + + holder := f.reconcile(t, "platform", "main") + assert.Equal(t, metav1.ConditionTrue, readyOf(holder).Status, readyOf(holder).Message) + + squat = f.reconcile(t, "squat", "main") + assert.Equal(t, GitBranchReasonBranchConflict, readyOf(squat).Reason) + assert.Contains(t, readyOf(squat).Message, "platform/main") + assert.True(t, conditionIsTrue(squat.Status.Conditions, ConditionTypeStalled)) +} + +// TestGitBranch_TwoValidGitBranchesForOneBranchRunOneWorker is the create race: both pass admission +// (nothing refuses there), both prove themselves, and the younger reports BranchConflict, also when +// it names another GitProvider for the same repository. +func TestGitBranch_TwoValidGitBranchesForOneBranchRunOneWorker(t *testing.T) { + f := newGitBranchFixture(t, + branchProvider("platform", "acme"), branchObject("platform", "main", "acme", "main", 2*time.Hour), + branchProvider("team-b", "mirror"), branchObject("team-b", "main", "mirror", "main", time.Hour)) + + younger := f.reconcile(t, "team-b", "main") + assert.Equal(t, metav1.ConditionTrue, readyOf(younger).Status, "alone, it holds the branch") + older := f.reconcile(t, "platform", "main") + assert.Equal(t, metav1.ConditionTrue, readyOf(older).Status, readyOf(older).Message) + + younger = f.reconcile(t, "team-b", "main") + assert.Equal(t, GitBranchReasonBranchConflict, readyOf(younger).Reason) + assert.Contains(t, readyOf(younger).Message, "platform/main") + holder, held := f.workers.Claims().Holder(git.BranchKeyFor(gitBranchTestURL, "main")) + require.True(t, held) + assert.Equal(t, "platform", holder.GitBranch.Namespace) +} + +// TestGitBranch_TheHolderKeepsItsBranchWhenItsCredentialFails is rule 3: Ready goes False with the +// failure, the claim stays, and a younger valid GitBranch does not take it over. +func TestGitBranch_TheHolderKeepsItsBranchWhenItsCredentialFails(t *testing.T) { + f := newGitBranchFixture(t, + branchProvider("platform", "acme"), branchObject("platform", "main", "acme", "main", 2*time.Hour), + branchProvider("team-b", "mirror"), branchObject("team-b", "main", "mirror", "main", time.Hour)) + f.reconcile(t, "platform", "main") + f.r.proveProvider = func(provider *configbutleraiv1alpha3.GitProvider) error { + if provider.Namespace == "platform" { + return errors.New("authentication required") + } + return nil + } + f.r.proofs = nil // the next reconcile proves again + + holder := f.reconcile(t, "platform", "main") + assert.Equal(t, GitBranchReasonPushSessionFailed, readyOf(holder).Reason) + assert.Contains(t, readyOf(holder).Message, "keeps the branch") + assert.False(t, conditionIsTrue(holder.Status.Conditions, ConditionTypeStalled), "it recovers on its own") + + younger := f.reconcile(t, "team-b", "main") + assert.Equal(t, GitBranchReasonBranchConflict, readyOf(younger).Reason) +} + +// TestGitBranch_TheBranchMustBeAllowed: allowedBranches globs moved here from the GitTarget. +func TestGitBranch_TheBranchMustBeAllowed(t *testing.T) { + for branch, allowed := range map[string]bool{ + "main": true, "develop": true, "feature/login": true, "release/v1.0": true, + "hotfix/urgent": false, "staging": false, + } { + t.Run(branch, func(t *testing.T) { + f := newGitBranchFixture(t, + branchProvider("platform", "acme", "main", "develop", "feature/*", "release/v*"), + branchObject("platform", "b", "acme", branch, time.Hour)) + + got := f.reconcile(t, "platform", "b") + + if allowed { + assert.Equal(t, metav1.ConditionTrue, readyOf(got).Status, readyOf(got).Message) + return + } + assert.Equal(t, GitBranchReasonBranchNotAllowed, readyOf(got).Reason) + assert.True(t, conditionIsTrue(got.Status.Conditions, ConditionTypeStalled)) + _, _, held := f.workers.Claims().HeldBy(ctrlclient.ObjectKeyFromObject(got)) + assert.False(t, held, "a branch outside allowedBranches is never claimed") + }) + } +} + +// TestGitBranch_AMissingOrUnreadyGitProvider: a missing GitProvider stalls the GitBranch; one that +// is not Ready keeps it progressing. +func TestGitBranch_AMissingOrUnreadyGitProvider(t *testing.T) { + f := newGitBranchFixture(t, branchObject("platform", "main", "acme", "main", time.Hour)) + got := f.reconcile(t, "platform", "main") + assert.Equal(t, GitBranchReasonGitProviderNotFound, readyOf(got).Reason) + assert.Contains(t, readyOf(got).Message, "platform/acme") + + unready := branchProvider("platform", "acme") + unready.Status.Conditions[0].Status = metav1.ConditionFalse + f = newGitBranchFixture(t, unready, branchObject("platform", "main", "acme", "main", time.Hour)) + got = f.reconcile(t, "platform", "main") + assert.Equal(t, GitBranchReasonGitProviderNotReady, readyOf(got).Reason) + assert.False(t, conditionIsTrue(got.Status.Conditions, ConditionTypeStalled)) +} + +// TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch: the finalizer stops the branch worker +// (which publishes or refuses its pending writes and settles its CommitRequests), releases the +// claim, and goes. The next candidate then takes the branch. +func TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch(t *testing.T) { + f := newGitBranchFixture(t, + branchProvider("platform", "acme"), branchObject("platform", "main", "acme", "main", 2*time.Hour), + branchProvider("team-b", "mirror"), branchObject("team-b", "main", "mirror", "main", time.Hour)) + f.reconcile(t, "platform", "main") + f.reconcile(t, "team-b", "main") + holder := k8stypes.NamespacedName{Namespace: "platform", Name: "main"} + require.NoError(t, f.workers.EnsureWorker(context.Background(), git.WorkerRequest{ + GitBranch: holder, Provider: "acme", Branch: "main", + Repo: git.RepoIdentity{ProviderUID: "uid-acme", URL: gitBranchTestURL}, + })) + + stored := &configbutleraiv1alpha3.GitBranch{} + require.NoError(t, f.client.Get(context.Background(), holder, stored)) + require.NoError(t, f.client.Delete(context.Background(), stored)) + _, err := f.r.Reconcile(context.Background(), ctrl.Request{NamespacedName: holder}) + require.NoError(t, err) + + _, live := f.workers.WorkerForGitBranch(holder) + assert.False(t, live, "the worker stopped before the finalizer went") + require.Error(t, f.client.Get(context.Background(), holder, stored), "the finalizer is gone, and so is the object") + + next := f.reconcile(t, "team-b", "main") + assert.Equal(t, metav1.ConditionTrue, readyOf(next).Status, readyOf(next).Message) +} + +// TestGitBranch_AMissingParentStallsTheBranch: ParentBranchNotFound moved from the GitTarget with its +// rules unchanged. A fetch that found neither the branch nor its configured parent stalls the +// GitBranch, for that parent only. +func TestGitBranch_AMissingParentStallsTheBranch(t *testing.T) { + branch := branchObject("platform", "edits", "acme", "edits", time.Hour) + branch.Spec.ParentBranch = "release" + f := newGitBranchFixture(t, branchProvider("platform", "acme"), branch) + got := f.reconcile(t, "platform", "edits") + require.Equal(t, metav1.ConditionTrue, readyOf(got).Status, "nothing has looked yet") + + key := ctrlclient.ObjectKeyFromObject(branch) + require.NoError(t, f.workers.EnsureWorker(context.Background(), git.WorkerRequest{ + GitBranch: key, Provider: "acme", Branch: "edits", ParentBranch: "release", + Repo: git.RepoIdentity{ProviderUID: "uid-acme", URL: gitBranchTestURL}, + })) + worker, _ := f.workers.WorkerForGitBranch(key) + worker.RecordMissingParentForTest("release") + + got = f.reconcile(t, "platform", "edits") + assert.Equal(t, GitBranchReasonParentBranchNotFound, readyOf(got).Reason) + assert.Contains(t, readyOf(got).Message, "'release'") + assert.True(t, conditionIsTrue(got.Status.Conditions, ConditionTypeStalled), "somebody has to create a branch") + require.NotNil(t, got.Status.Remote) + require.NotNil(t, got.Status.Remote.Parent) + assert.Equal(t, configbutleraiv1alpha3.GitBranchParentMissing, got.Status.Remote.Parent.State) + + worker.RecordMissingParentForTest("other") + got = f.reconcile(t, "platform", "edits") + assert.NotEqual(t, GitBranchReasonParentBranchNotFound, readyOf(got).Reason, + "an observation made under another parent says nothing about this one") +} + +// TestGitBranch_AnUnresolvedDefaultBranchStallsToo: with spec.parentBranch omitted, a remote whose +// HEAD names a branch it does not carry stalls the GitBranch, with a message that says what to fix. +func TestGitBranch_AnUnresolvedDefaultBranchStallsToo(t *testing.T) { + f := newGitBranchFixture(t, branchProvider("platform", "acme"), + branchObject("platform", "edits", "acme", "edits", time.Hour)) + f.reconcile(t, "platform", "edits") + key := k8stypes.NamespacedName{Namespace: "platform", Name: "edits"} + require.NoError(t, f.workers.EnsureWorker(context.Background(), git.WorkerRequest{ + GitBranch: key, Provider: "acme", Branch: "edits", + Repo: git.RepoIdentity{ProviderUID: "uid-acme", URL: gitBranchTestURL}, + })) + worker, _ := f.workers.WorkerForGitBranch(key) + + worker.RecordUnresolvedDefaultBranchForTest("master") + got := f.reconcile(t, "platform", "edits") + assert.Equal(t, GitBranchReasonParentBranchNotFound, readyOf(got).Reason) + assert.Contains(t, readyOf(got).Message, "HEAD -> 'master'") + assert.Contains(t, readyOf(got).Message, "spec.parentBranch") +} + +// TestGitBranch_ARepointedProviderMovesTheClaim: a GitProvider recreated against another repository +// is another key, so the GitBranch's claim moves with it. +func TestGitBranch_ARepointedProviderMovesTheClaim(t *testing.T) { + provider := branchProvider("platform", "acme") + f := newGitBranchFixture(t, provider, branchObject("platform", "main", "acme", "main", time.Hour)) + f.reconcile(t, "platform", "main") + + stored := &configbutleraiv1alpha3.GitProvider{} + require.NoError(t, f.client.Get(context.Background(), ctrlclient.ObjectKeyFromObject(provider), stored)) + require.NoError(t, f.client.Delete(context.Background(), stored)) + recreated := branchProvider("platform", "acme") + recreated.UID = "uid-acme-2" + recreated.ResourceVersion = "" + recreated.Spec.URL = "https://github.com/acme/other.git" + require.NoError(t, f.client.Create(context.Background(), recreated)) + + got := f.reconcile(t, "platform", "main") + assert.Equal(t, "github.com/acme/other", got.Status.Repository) + _, held := f.workers.Claims().Holder(git.BranchKeyFor(gitBranchTestURL, "main")) + assert.False(t, held, "the old repository's branch is free again") + key, _, held := f.workers.Claims().HeldBy(ctrlclient.ObjectKeyFromObject(got)) + require.True(t, held) + assert.Equal(t, "github.com/acme/other", key.Repository) +} diff --git a/internal/controller/gittarget_remote.go b/internal/controller/gitbranch_remote.go similarity index 70% rename from internal/controller/gittarget_remote.go rename to internal/controller/gitbranch_remote.go index ebb2fb8c..0739287d 100644 --- a/internal/controller/gittarget_remote.go +++ b/internal/controller/gitbranch_remote.go @@ -14,17 +14,17 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/types" ) -// remotePublicationLedger remembers, per GitTarget, what its last status.remote WRITE was: when it +// remotePublicationLedger remembers, per GitBranch, what its last status.remote WRITE was: when it // happened, and which repository the revision it published was proved against. // // Both halves are deliberately internal. The write TIME is what a rate limit has to measure // against — the observation's own timestamp is not, because an observation made just before a // publication and a fresh one made just after are a second apart while the two writes are not. -// The REPOSITORY is what makes invalidation follow this target's published state rather than the -// latest shared observation: one branch worker serves every GitTarget on a branch, so a sibling -// can already have proved the replacement repository while this target still publishes a revision -// from the one it has left, and comparing the shared observation against the provider would find -// them in agreement and leave the wrong revision standing. +// The REPOSITORY is what makes invalidation follow this GitBranch's published state rather than the +// latest observation: the observation can already have been proved against the replacement +// repository while this GitBranch still publishes a revision from the one it has left, and comparing +// the observation against the provider would find them in agreement and leave the wrong revision +// standing. // // Losing it costs nothing. An empty ledger publishes the next observation immediately and then // resumes limiting, which is the right behaviour after a restart: what is published is older than @@ -35,7 +35,7 @@ type remotePublicationLedger struct { } type remotePublication struct { - // uid is the GitTarget object the entry is about. A ledger keyed by namespace/name alone + // uid is the GitBranch object the entry is about. A ledger keyed by namespace/name alone // outlives the object that earned it: cleanup needs a reconcile that observes NotFound, and a // delete followed quickly by a recreate can be reconciled from the successor directly. The // successor would then inherit its predecessor's cooldown and have its FIRST observation @@ -45,7 +45,7 @@ type remotePublication struct { repo git.RepoIdentity } -// last returns what this GitTarget published, and whether the ledger knows. An entry belonging to +// last returns what this GitBranch published, and whether the ledger knows. An entry belonging to // a predecessor under the same name is not this object's, so it answers "nothing". func (l *remotePublicationLedger) last(ref types.ResourceReference, uid k8stypes.UID) (remotePublication, bool) { l.mu.Lock() @@ -77,64 +77,55 @@ func (l *remotePublicationLedger) forget(ref types.ResourceReference) { delete(l.entries, ref.Key()) } -// observeRemote reads what the data plane last proved about this GitTarget's BRANCH. -// -// The observation is shared: one branch worker serves every GitTarget on its (provider, branch), -// and what it proved — by a push, or by a fetch — is one fact about the branch, available to all -// of them at the moment it was proved, carrying the time it was proved. Sharing it costs nothing -// and re-proving it per target would cost a connection each. +// gitBranchRef is the ledger key of a GitBranch. +func gitBranchRef(branch *configbutleraiv1alpha3.GitBranch) types.ResourceReference { + return types.NewResourceReference(branch.Name, branch.Namespace) +} + +// observeRemote reads what the data plane last proved about this GitBranch's repository branch. // // Absent means nothing has looked yet, which is genuinely different from a look that found no // branch: the first leaves status.remote unset, the second publishes an observation with no // revision. -func (r *GitTargetReconciler) observeRemote( - target *configbutleraiv1alpha3.GitTarget, - repo git.RepoIdentity, -) (git.RemoteObservation, bool) { +func (r *GitBranchReconciler) observeRemote(key git.BranchKey) (git.RemoteObservation, bool) { if r.WorkerManager == nil { return git.RemoteObservation{}, false } - return r.WorkerManager.RemoteForBranch(git.BranchKeyFor(repo.URL, target.Spec.Branch)) + return r.WorkerManager.RemoteForBranch(key) } // publishRemote writes status.remote, sampled onto this reconcile and rate-limited. // -// DELIVERY IS NOT PUBLICATION. An observation reaches every GitTarget on the branch the instant it -// is proved, in memory, for nothing. A status write is the opposite kind of thing: fanning one -// push out to ten targets sharing a branch turns it into ten etcd writes, each invalidating the -// cached copy every watcher of the type holds. So this publishes what the target holds AT THE -// MOMENT IT RECONCILES, and nothing in the data plane wakes a target because a branch moved. -// -// The cadence is therefore the target's own: a converged one samples every RequeueSteadyInterval, -// a busy one much sooner, and nobody is scheduled a wake-up for a status field. On top of that -// sits one floor, RemotePublicationInterval, so that a target reconciling in a tight loop still -// writes this stanza at most once a minute. The floor is measured from the last WRITE, kept in the -// ledger above, not from the observation's timestamp. +// The branch worker wakes the GitBranch whenever it proves something (GitBranchEvents), and a busy +// branch proves something on every push. A status write per push would invalidate the cached copy +// every watcher of the type holds, so one floor, RemotePublicationInterval, keeps this stanza to at +// most one write a minute. The floor is measured from the last WRITE, kept in the ledger above, not +// from the observation's timestamp. // // Two things ignore the floor, because they are correctness rather than throughput: the FIRST // observation, which is the difference between "nothing has looked" and an answer, and any -// correction that involves the repository changing under the target. +// correction that involves the repository changing under the GitBranch. // // It returns how long the floor is holding news back, zero when nothing is held. A held answer // must not wait for whatever reconcile happens to come next: see requeueForRemoteAnswer. -func (r *GitTargetReconciler) publishRemote( +func (r *GitBranchReconciler) publishRemote( st *reconcileStatus, - target *configbutleraiv1alpha3.GitTarget, + branch *configbutleraiv1alpha3.GitBranch, observed git.RemoteObservation, seen bool, repo git.RepoIdentity, now time.Time, ) time.Duration { - ref := types.NewResourceReference(target.Name, target.Namespace) - published, everPublished := r.remotePublications.last(ref, target.UID) - // What this target PUBLISHED was proved against a repository it no longer points at: its + ref := gitBranchRef(branch) + published, everPublished := r.remotePublications.last(ref, branch.UID) + // What this GitBranch PUBLISHED was proved against a repository it no longer points at: its // GitProvider was recreated against a different URL. staleRepository := everPublished && differentRepository(published.repo, repo) if withdrawRemote(staleRepository, seen, observed, repo) { // Nothing proved about the repository in use now, so there is nothing to replace what is // published with, and it has to go rather than go stale. - target.Status.Remote = nil + branch.Status.Remote = nil st.afterPersist(func() { r.remotePublications.forget(ref) }) return 0 } @@ -149,14 +140,14 @@ func (r *GitTargetReconciler) publishRemote( VerifiedBy: string(observed.By), Parent: parentStatusOf(observed), } - if !remoteStatusIsNews(target.Status.Remote, next) { + if !remoteStatusIsNews(branch.Status.Remote, next) { return 0 } // The floor, skipped for the two corrections above and for the first publication. if everPublished && !staleRepository && now.Sub(published.at) < RemotePublicationInterval { return published.at.Add(RemotePublicationInterval).Sub(now) } - target.Status.Remote = next + branch.Status.Remote = next // Only once the write has landed, and dated THEN rather than now. A patch refused by the // optimistic lock leaves the object holding the winner's older answer, and a ledger that // recorded this publication anyway would hold the retry off behind a cooldown for a revision @@ -164,18 +155,17 @@ func (r *GitTargetReconciler) publishRemote( // callback exists: `now` was taken before the gates, the worker wiring and the patch itself, // so on a slow reconcile a cooldown dated from it can be half spent before the write lands, // and a reconcile queued behind it would publish again straight away. - st.afterPersist(func() { r.remotePublications.record(ref, target.UID, r.clockNow(), observed.Repo) }) + st.afterPersist(func() { r.remotePublications.record(ref, branch.UID, r.clockNow(), observed.Repo) }) return 0 } // withdrawRemote decides whether what is published has to be taken back rather than replaced. // // Two questions, and they are asked of different evidence. The ledger knows which repository THIS -// target published, which is what a sibling's observation cannot answer: one worker serves every -// GitTarget on a branch, so a sibling can already have proved the replacement while this target -// still publishes a revision from the repository it has left. The observation answers for the case -// the ledger cannot — a restart, or a target that has not published under this process — because -// it names the repository it was proved against. +// GitBranch published, which the latest observation cannot answer: it can already have proved the +// replacement while this GitBranch still publishes a revision from the repository it has left. The +// observation answers for the case the ledger cannot — a restart, or a GitBranch that has not +// published under this process — because it names the repository it was proved against. // // Both comparisons need two known halves: an unreadable GitProvider names no repository, and a // worker with no identity of its own records none. Neither is evidence of a mismatch. diff --git a/internal/controller/gittarget_remote_test.go b/internal/controller/gitbranch_remote_test.go similarity index 88% rename from internal/controller/gittarget_remote_test.go rename to internal/controller/gitbranch_remote_test.go index 2b3d2cea..301d3933 100644 --- a/internal/controller/gittarget_remote_test.go +++ b/internal/controller/gitbranch_remote_test.go @@ -36,8 +36,8 @@ var ( // the ledger as commit() does once the write has landed. Tests that care about a write which did // NOT land call publishRemote directly and never settle. func publishPersisted( - r *GitTargetReconciler, - target *configbutleraiv1alpha3.GitTarget, + r *GitBranchReconciler, + target *configbutleraiv1alpha3.GitBranch, observed git.RemoteObservation, seen bool, repo git.RepoIdentity, @@ -54,8 +54,8 @@ func publishPersisted( st.runPersisted() } -func remoteTestTarget() *configbutleraiv1alpha3.GitTarget { - return &configbutleraiv1alpha3.GitTarget{ +func remoteTestTarget() *configbutleraiv1alpha3.GitBranch { + return &configbutleraiv1alpha3.GitBranch{ ObjectMeta: metav1.ObjectMeta{Name: "checkout", Namespace: "shop", Generation: 4}, } } @@ -67,7 +67,7 @@ func observation(revision string, at time.Time, by git.ObservationSource, repo g // TestPublishRemote_AbsentUntilSomethingLooks keeps the two "no revision" cases apart. Nothing has // looked, so there is no stanza at all — which is not the same as a look that found no branch. func TestPublishRemote_AbsentUntilSomethingLooks(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() publishPersisted(r, target, git.RemoteObservation{}, false, git.RepoIdentity{}, time.Now()) @@ -78,7 +78,7 @@ func TestPublishRemote_AbsentUntilSomethingLooks(t *testing.T) { // TestPublishRemote_AnObservationWithNoRevisionIsStillPublished is the other half: the branch is // not on the remote, and saying so is the answer rather than the absence of one. func TestPublishRemote_AnObservationWithNoRevisionIsStillPublished(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() @@ -104,7 +104,7 @@ func TestPublishRemote_AnObservationWithNoRevisionIsStillPublished(t *testing.T) // made at 12:00:02 can be written at 12:01:00, and a revision proved at 12:01:02 then looks a full // minute newer than what is published and is written two seconds after it. func TestPublishRemote_TheFloorIsMeasuredFromTheLastWrite(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() start := time.Now() @@ -130,7 +130,7 @@ func TestPublishRemote_TheFloorIsMeasuredFromTheLastWrite(t *testing.T) { // first one has no rate: holding it back would leave a target that has just published showing no // branch at all, which reads as a broken data plane rather than as a sampled field. func TestPublishRemote_TheFirstObservationIsImmediate(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() @@ -150,7 +150,7 @@ func TestPublishRemote_TheFirstObservationIsImmediate(t *testing.T) { // target whose branch has not moved must not write status once per tick merely to advance a clock, // however long it has been since the last one. func TestPublishRemote_AConvergedTargetWritesNothing(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() publishPersisted(r, @@ -172,7 +172,7 @@ func TestPublishRemote_AConvergedTargetWritesNothing(t *testing.T) { // news — that is what makes lastVerifiedAt a freshness field — and like every other kind of news // it waits for the floor. func TestPublishRemote_AReProvedRevisionRefreshesTheClock(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() publishPersisted(r, @@ -227,7 +227,7 @@ func TestRemoteStatusIsNews_AMissingTimestampIsAlwaysNews(t *testing.T) { // until a look at the new one succeeds. If that repository is unreachable there never will be, so // the stanza goes rather than going stale. func TestPublishRemote_ARevisionFromAnotherRepositoryIsRemoved(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() publishPersisted(r, target, observation("aaaa", at, git.ObservedByPush, firstRepo), true, firstRepo, at) @@ -247,7 +247,7 @@ func TestPublishRemote_ARevisionFromAnotherRepositoryIsRemoved(t *testing.T) { // target is still publishing a revision from the repository it has left, and the floor would hold // the correction back on top of that. Invalidation follows what THIS target published. func TestPublishRemote_ASiblingsObservationCannotHideAStaleRepository(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() publishPersisted(r, target, observation("aaaa", at, git.ObservedByPush, firstRepo), true, firstRepo, at) @@ -277,7 +277,7 @@ func TestPublishRemote_AnUnknownIdentityProvesNothing(t *testing.T) { {"the worker has no identity", git.RepoIdentity{}, firstRepo}, } { t.Run(tc.name, func(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() @@ -293,7 +293,7 @@ func TestPublishRemote_AnUnknownIdentityProvesNothing(t *testing.T) { // first look at the repository the target points at NOW publishes again, with no latch to clear // and no floor to wait for, because the ledger was dropped with the revision it described. func TestPublishRemote_TheNewRepositoryRepopulatesTheStanza(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() publishPersisted(r, target, observation("aaaa", at, git.ObservedByPush, firstRepo), true, secondRepo, at) @@ -315,7 +315,7 @@ func TestPublishRemote_ARestartPublishesAgainstTheObservationAlone(t *testing.T) target.Status.Remote = &configbutleraiv1alpha3.GitBranchRemoteStatus{ Commit: "aaaa", LastVerifiedAt: &at, VerifiedBy: "Push", } - fresh := &GitTargetReconciler{} + fresh := &GitBranchReconciler{} publishPersisted( fresh, @@ -329,24 +329,16 @@ func TestPublishRemote_ARestartPublishesAgainstTheObservationAlone(t *testing.T) assert.Nil(t, target.Status.Remote, "the observation is about a repository this target has left") } -// TestObserveRemote_ReadsTheBranchItsGitTargetWrites. The observation is shared by every GitTarget -// on the branch, so it is looked up by (repository, branch) and not by target. -func TestObserveRemote_ReadsTheBranchItsGitTargetWrites(t *testing.T) { - target := remoteTestTarget() - target.Spec.GitProviderRef.Name = "repo1" - target.Spec.Branch = "main" +// TestObserveRemote_ReadsTheBranchItHolds. The observation is the branch worker's, looked up by +// (repository, branch). +func TestObserveRemote_ReadsTheBranchItHolds(t *testing.T) { + key := git.BranchKeyFor("https://example.invalid/repo1.git", "main") - _, seen := (&GitTargetReconciler{}).observeRemote( - target, - git.RepoIdentity{URL: "https://example.invalid/repo1.git"}, - ) + _, seen := (&GitBranchReconciler{}).observeRemote(key) assert.False(t, seen, "a reconcile with no worker manager has nothing to read") workers := git.NewWorkerManager(nil, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) - _, seen = (&GitTargetReconciler{WorkerManager: workers}).observeRemote( - target, - git.RepoIdentity{URL: "https://example.invalid/repo1.git"}, - ) + _, seen = (&GitBranchReconciler{WorkerManager: workers}).observeRemote(key) assert.False(t, seen, "and neither has one whose branch nothing has looked at") } @@ -354,21 +346,19 @@ func TestObserveRemote_ReadsTheBranchItsGitTargetWrites(t *testing.T) { // not armed: the refresher turned off, and a reconcile running without a worker manager. Neither // may reach for a worker, because the reconcile runs on every tick of every target. func TestRequestRemoteRefresh_DoesNothingWithoutADataPlane(t *testing.T) { - target := remoteTestTarget() + target := &configbutleraiv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{Name: "checkout", Namespace: "shop"}} assert.NotPanics(t, func() { - (&GitTargetReconciler{GitRefreshInterval: 0}).requestRemoteRefresh(target, "default") + (&GitTargetReconciler{GitRefreshInterval: 0}).requestRemoteRefresh(target) }, "a zero interval turns the refresher off") assert.NotPanics(t, func() { - (&GitTargetReconciler{GitRefreshInterval: time.Minute}).requestRemoteRefresh(target, "default") + (&GitTargetReconciler{GitRefreshInterval: time.Minute}).requestRemoteRefresh(target) }, "and a reconcile with no worker manager has nothing to ask") workers := git.NewWorkerManager(nil, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) assert.NotPanics(t, func() { - (&GitTargetReconciler{ - GitRefreshInterval: time.Minute, WorkerManager: workers, - }).requestRemoteRefresh(target, "default") + (&GitTargetReconciler{GitRefreshInterval: time.Minute, WorkerManager: workers}).requestRemoteRefresh(target) }, "and a target whose branch has no worker yet has nothing to refresh either") } @@ -376,7 +366,7 @@ func TestRequestRemoteRefresh_DoesNothingWithoutADataPlane(t *testing.T) { // removing a field: the status write is a JSON merge patch computed against the object as read, so // the fix works only if a field that was there and is now nil is emitted as an explicit null. func TestPublishRemote_TheRemovalReachesTheAPIAsADelete(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() target.Status.Conditions = []metav1.Condition{{ Type: "Ready", Status: metav1.ConditionTrue, Reason: "Ready", @@ -397,7 +387,7 @@ func TestPublishRemote_TheRemovalReachesTheAPIAsADelete(t *testing.T) { // memory, keyed by namespace/name, so an entry left behind would hold a rate limit against a name // that may be recreated tomorrow with nothing published at all. func TestPublishRemote_ADeletedTargetReleasesItsLedgerEntry(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() publishPersisted(r, target, observation("aaaa", at, git.ObservedByPush, firstRepo), true, firstRepo, at) @@ -418,36 +408,11 @@ func TestPublishRemote_ADeletedTargetReleasesItsLedgerEntry(t *testing.T) { assert.Equal(t, "bbbb", successor.Status.Remote.Commit) } -// TestRequeueForRemoteAnswer_ShortensOnlyWhenSomethingWasAsked. The refresh is enqueued during the -// reconcile, so its answer lands after the status write: a target that asked has to come back for -// it, or an idle one spends a connection every interval and publishes what it learned a whole tick -// late. A target that asked nothing keeps its own cadence. -func TestRequeueForRemoteAnswer_ShortensOnlyWhenSomethingWasAsked(t *testing.T) { - assert.Equal(t, RemotePublicationInterval, - requeueForRemoteAnswer(RequeueSteadyInterval, remoteWake{asked: true})) - assert.Equal(t, RequeueSteadyInterval, requeueForRemoteAnswer(RequeueSteadyInterval, remoteWake{})) - assert.Equal(t, time.Second, requeueForRemoteAnswer(time.Second, remoteWake{asked: true}), - "a target already on a faster loop is not slowed down to the publication interval") -} - -// TestRequeueForRemoteAnswer_ComesBackWhenTheFloorLifts is the reconcile after a lost requeue. The -// asking reconcile's one-minute requeue is folded away when the refresh's answer re-enqueues the -// target while it runs; the reconcile that follows holds that answer fresh, asks nothing, and is -// stopped by the floor. It must come back when the floor lifts, not a whole steady tick later. -func TestRequeueForRemoteAnswer_ComesBackWhenTheFloorLifts(t *testing.T) { - held := 59 * time.Second - assert.Equal(t, held, requeueForRemoteAnswer(RequeueSteadyInterval, remoteWake{heldFor: held})) - assert.Equal(t, held, requeueForRemoteAnswer(RequeueSteadyInterval, remoteWake{asked: true, heldFor: held}), - "the earlier of the two wakes wins") - assert.Equal(t, time.Second, requeueForRemoteAnswer(time.Second, remoteWake{heldFor: held}), - "a target already on a faster loop is not slowed down to the floor") -} - // TestPublishRemote_ReportsHowLongTheFloorHoldsAnAnswer: the floor delays news, and says for how // long, so the reconcile can come back for it. What is not news, and what is published, holds // nothing back. func TestPublishRemote_ReportsHowLongTheFloorHoldsAnAnswer(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() start := time.Now() previous := r.clock @@ -475,13 +440,13 @@ func TestPublishRemote_ReportsHowLongTheFloorHoldsAnAnswer(t *testing.T) { // remoteLedgerFixture drives publishRemote through the REAL status session, so what advances the // ledger is a patch the API server accepted rather than a callback a test chose to run. type remoteLedgerFixture struct { - r *GitTargetReconciler + r *GitBranchReconciler client client.Client conflict bool now time.Time } -func newRemoteLedgerFixture(t *testing.T, target *configbutleraiv1alpha3.GitTarget) *remoteLedgerFixture { +func newRemoteLedgerFixture(t *testing.T, target *configbutleraiv1alpha3.GitBranch) *remoteLedgerFixture { t.Helper() scheme := runtime.NewScheme() require.NoError(t, clientgoscheme.AddToScheme(scheme)) @@ -500,13 +465,13 @@ func newRemoteLedgerFixture(t *testing.T, target *configbutleraiv1alpha3.GitTarg // What losing the optimistic lock looks like: the object moved under this // reconcile, so the patch is refused. return apierrors.NewConflict( - schema.GroupResource{Group: "configbutler.ai", Resource: "gittargets"}, + schema.GroupResource{Group: "configbutler.ai", Resource: "gitbranches"}, obj.GetName(), errors.New("the object has been modified")) } return c.SubResource(subResource).Patch(ctx, obj, patch, opts...) }, }) - f.r = &GitTargetReconciler{Client: f.client, clock: func() time.Time { return f.now }} + f.r = &GitBranchReconciler{Client: f.client, clock: func() time.Time { return f.now }} return f } @@ -514,7 +479,7 @@ func newRemoteLedgerFixture(t *testing.T, target *configbutleraiv1alpha3.GitTarg // whether the write landed. func (f *remoteLedgerFixture) publish( t *testing.T, - target *configbutleraiv1alpha3.GitTarget, + target *configbutleraiv1alpha3.GitBranch, observed git.RemoteObservation, repo git.RepoIdentity, ) bool { @@ -544,7 +509,7 @@ func TestPublishRemote_ARejectedPatchDoesNotAdvanceTheLedger(t *testing.T) { // The retry, two seconds later, is not held off by a cooldown it never earned. f.conflict = false f.now = f.now.Add(2 * time.Second) - fresh := &configbutleraiv1alpha3.GitTarget{} + fresh := &configbutleraiv1alpha3.GitBranch{} require.NoError(t, f.client.Get(context.Background(), client.ObjectKeyFromObject(target), fresh)) require.Nil(t, fresh.Status.Remote, "nothing was published, so there is nothing to read back") @@ -581,7 +546,7 @@ func TestPublishRemote_ARejectedWithdrawalKeepsTheLedgerEntry(t *testing.T) { // And the next one does, against the object as it actually stands. f.conflict = false f.now = f.now.Add(time.Second) - stale := &configbutleraiv1alpha3.GitTarget{} + stale := &configbutleraiv1alpha3.GitBranch{} require.NoError(t, f.client.Get(context.Background(), client.ObjectKeyFromObject(target), stale)) require.NotNil(t, stale.Status.Remote, "the old repository's revision is still published") @@ -595,7 +560,7 @@ func TestPublishRemote_ARejectedWithdrawalKeepsTheLedgerEntry(t *testing.T) { // worker wiring and the patch, so on a slow reconcile a cooldown dated from it is already part // spent when the write lands — and a reconcile queued behind it publishes again straight away. func TestPublishRemote_TheCooldownStartsWhenTheWriteLands(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() evaluated := time.Now() landed := evaluated.Add(90 * time.Second) // a slow pass: gates, wiring, then the patch @@ -627,7 +592,7 @@ func TestPublishRemote_TheCooldownStartsWhenTheWriteLands(t *testing.T) { // has the same shape: forgetting the entry before the removal persists leaves the old repository's // revision standing with nothing to take it back. func TestPublishRemote_AWriteThatDidNotLandDoesNotAdvanceTheLedger(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} target := remoteTestTarget() at := time.Now() @@ -655,7 +620,7 @@ func TestPublishRemote_AWriteThatDidNotLandDoesNotAdvanceTheLedger(t *testing.T) // observation suppressed by its predecessor's cooldown. The entry carries the UID it was earned // under, which is the one thing that tells the two objects apart. func TestPublishRemote_ARecreatedTargetDoesNotInheritTheCooldown(t *testing.T) { - r := &GitTargetReconciler{} + r := &GitBranchReconciler{} at := time.Now() predecessor := remoteTestTarget() diff --git a/internal/controller/gitprovider_branches.go b/internal/controller/gitprovider_branches.go deleted file mode 100644 index 9d5f7fb5..00000000 --- a/internal/controller/gitprovider_branches.go +++ /dev/null @@ -1,76 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package controller - -import ( - "context" - "sort" - - "sigs.k8s.io/controller-runtime/pkg/client" - - configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" -) - -// publishBranchInventory writes status.branches: the branches this repository's GitTargets are -// configured to write, and how many reference each. -// -// It is a question about CONFIGURATION, so it is answered from the API and not from the data -// plane. A blocked or suspended GitTarget still says what the repository is for; a branch worker -// that happens to be running says only what has been reached so far, and a repository nothing has -// published to yet has no worker at all. That is the whole point of the field: "Ready=True with no -// branches" is configured-and-unused, which looks nothing like broken, and telling the two apart -// used to take a GitTarget listing. -// -// A GitProvider can be referenced from any namespace its accessFrom admits, so the list is -// cluster-wide, and it is read from the cache on the provider's own steady tick. There is deliberately no watch on -// GitTarget: this reconcile also proves the credential against the remote, so an edge per -// GitTarget edit would spend a network round trip and advance status.lastVerifiedAt to answer a -// question about configuration. The inventory lags a GitTarget edit by up to one steady interval, -// which is the same freshness every other configuration projection here has. -// -// A List that fails leaves what is published standing rather than emptying it: an inventory that -// collapses to nothing whenever the cache hiccups would report "configured and unused" about a -// repository with ten GitTargets on it. -func (r *GitProviderReconciler) publishBranchInventory( - ctx context.Context, - gitProvider *configbutleraiv1alpha3.GitProvider, -) error { - var targets configbutleraiv1alpha3.GitTargetList - if err := r.List(ctx, &targets); err != nil { - return err - } - gitProvider.Status.Branches = branchInventory(client.ObjectKeyFromObject(gitProvider), targets.Items) - return nil -} - -// branchInventory counts the live GitTargets that name provider, from any namespace, by branch, -// sorted by name. -// -// Sorting is not cosmetic. The status patch is computed against what is published, so an order -// that followed the list's arrival order would rewrite the field — and wake every watcher of the -// type — on a change that moved nothing. -func branchInventory( - provider client.ObjectKey, - targets []configbutleraiv1alpha3.GitTarget, -) []configbutleraiv1alpha3.GitProviderBranchStatus { - counts := map[string]int32{} - for i := range targets { - target := &targets[i] - // A target being deleted is not what the repository is for any more: its worker is - // retired by the sweep the delete drives, and counting it would leave a branch in the - // inventory that nothing writes. - if !target.DeletionTimestamp.IsZero() || target.GitProviderKey() != provider { - continue - } - counts[target.Spec.Branch]++ - } - // EMPTY rather than nil: "this repository is configured and unused" is a measurement, and a - // nil slice would serialize the same way as "nothing has read the GitTargets yet", which is - // what the field is absent for. - branches := make([]configbutleraiv1alpha3.GitProviderBranchStatus, 0, len(counts)) - for name, count := range counts { - branches = append(branches, configbutleraiv1alpha3.GitProviderBranchStatus{Name: name, GitTargetCount: count}) - } - sort.Slice(branches, func(i, j int) bool { return branches[i].Name < branches[j].Name }) - return branches -} diff --git a/internal/controller/gitprovider_branches_test.go b/internal/controller/gitprovider_branches_test.go deleted file mode 100644 index 600c05c7..00000000 --- a/internal/controller/gitprovider_branches_test.go +++ /dev/null @@ -1,133 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package controller - -import ( - "context" - "errors" - "testing" - "time" - - "github.com/fluxcd/pkg/apis/meta" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime" - clientgoscheme "k8s.io/client-go/kubernetes/scheme" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/client/fake" - "sigs.k8s.io/controller-runtime/pkg/client/interceptor" - - configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" -) - -// shopRepo1 is the GitProvider the inventory tests count for. -var shopRepo1 = client.ObjectKey{Namespace: "shop", Name: "repo1"} - -func inventoryTarget(name, provider, branch string) *configbutleraiv1alpha3.GitTarget { - return &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "shop"}, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: provider}, - Branch: branch, - Path: name, - }, - } -} - -// TestBranchInventory_CountsTheGitTargetsThatConfigureEachBranch is the whole field: what is this -// repository being used for, answered from the configuration and sorted so a status patch is sent -// only when the answer moves. -func TestBranchInventory_CountsTheGitTargetsThatConfigureEachBranch(t *testing.T) { - suspended := inventoryTarget("archive", "repo1", "main") - suspended.Spec.Suspend = true - deleting := inventoryTarget("leaving", "repo1", "retired") - deleting.DeletionTimestamp = &metav1.Time{Time: time.Now()} - - shared := inventoryTarget("team-b", "repo1", "main") - shared.Namespace, shared.Spec.GitProviderRef.Namespace = "team-b", "shop" - sameName := inventoryTarget("team-b-own", "repo1", "main") - sameName.Namespace = "team-b" // its own namespace's repo1, another GitProvider - - got := branchInventory(shopRepo1, []configbutleraiv1alpha3.GitTarget{ - *shared, - *sameName, - *inventoryTarget("release", "repo1", "rel"), - *inventoryTarget("apps", "repo1", "main"), - *inventoryTarget("infra", "repo1", "main"), - *suspended, - *deleting, - *inventoryTarget("other", "repo2", "main"), - }) - - assert.Equal(t, []configbutleraiv1alpha3.GitProviderBranchStatus{ - {Name: "main", GitTargetCount: 4}, - {Name: "rel", GitTargetCount: 1}, - }, got, - "every branch a live GitTarget of THIS provider configures, from any namespace, suspended ones "+ - "included, sorted by name") -} - -// TestBranchInventory_AConfiguredAndUnusedRepositoryReportsAnEmptyList is the state the field -// exists to distinguish. It is not an error, and it must not look like one — and it must not look -// like the third state either: an ABSENT list means nothing has read the GitTargets yet, so the -// measured-and-empty answer is an empty slice rather than nil. -func TestBranchInventory_AConfiguredAndUnusedRepositoryReportsAnEmptyList(t *testing.T) { - measured := branchInventory(shopRepo1, nil) - assert.NotNil(t, measured, "configured and unused is an answer, not the absence of one") - assert.Empty(t, measured) - - measured = branchInventory(shopRepo1, []configbutleraiv1alpha3.GitTarget{ - *inventoryTarget("other", "repo2", "main"), - }) - assert.NotNil(t, measured) - assert.Empty(t, measured, "a GitTarget on another provider says nothing about this one") -} - -// TestPublishBranchInventory_KeepsTheLastAnswerWhenTheTargetsCannotBeRead. An inventory that -// collapsed to nothing on a cache hiccup would report "configured and unused" about a repository -// with GitTargets all over it, which is exactly the misreading the field is here to prevent. -func TestPublishBranchInventory_KeepsTheLastAnswerWhenTheTargetsCannotBeRead(t *testing.T) { - scheme := runtime.NewScheme() - require.NoError(t, clientgoscheme.AddToScheme(scheme)) - require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) - k8sClient := interceptor.NewClient( - fake.NewClientBuilder().WithScheme(scheme).Build(), - interceptor.Funcs{ - List: func(context.Context, client.WithWatch, client.ObjectList, ...client.ListOption) error { - return errors.New("the cache is not ready") - }, - }) - r := &GitProviderReconciler{Client: k8sClient} - provider := &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{Name: "repo1", Namespace: "shop"}, - Status: configbutleraiv1alpha3.GitProviderStatus{ - Branches: []configbutleraiv1alpha3.GitProviderBranchStatus{{Name: "main", GitTargetCount: 2}}, - }, - } - - require.Error(t, r.publishBranchInventory(context.Background(), provider)) - assert.Equal(t, []configbutleraiv1alpha3.GitProviderBranchStatus{{Name: "main", GitTargetCount: 2}}, - provider.Status.Branches) -} - -// TestPublishBranchInventory_ReadsOnlyTheProvidersOwnNamespace. A GitProvider is referenced from -// its own namespace, and counting a same-named provider's targets in another tenant's namespace -// would report somebody else's configuration. -func TestPublishBranchInventory_ReadsOnlyTheProvidersOwnNamespace(t *testing.T) { - scheme := runtime.NewScheme() - require.NoError(t, clientgoscheme.AddToScheme(scheme)) - require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) - elsewhere := inventoryTarget("apps", "repo1", "other-tenant-branch") - elsewhere.Namespace = "warehouse" - k8sClient := fake.NewClientBuilder().WithScheme(scheme). - WithObjects(inventoryTarget("apps", "repo1", "main"), elsewhere).Build() - r := &GitProviderReconciler{Client: k8sClient} - provider := &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{Name: "repo1", Namespace: "shop"}, - } - - require.NoError(t, r.publishBranchInventory(context.Background(), provider)) - assert.Equal(t, []configbutleraiv1alpha3.GitProviderBranchStatus{{Name: "main", GitTargetCount: 1}}, - provider.Status.Branches) -} diff --git a/internal/controller/gitprovider_controller.go b/internal/controller/gitprovider_controller.go index 9e7f9e91..c2aaf243 100644 --- a/internal/controller/gitprovider_controller.go +++ b/internal/controller/gitprovider_controller.go @@ -113,17 +113,8 @@ func (r *GitProviderReconciler) reconcileGitProvider( "GitProvider is not stalled", ) - // Ahead of every gate, like the GitTarget's own projections and for the same reason: the - // inventory explains what a refused or unused repository is FOR, so publishing it only on the - // happy path would drop it exactly when it is wanted. It must also come AFTER beginStatus, - // which snapshots the status the patch is computed against. A failure to read the GitTargets - // leaves the last inventory standing and is not a reason to fail the reconcile. - if err := r.publishBranchInventory(ctx, gitProvider); err != nil { - log.Error(err, "Could not list the GitTargets for the branch inventory; the last one stands", - "name", gitProvider.Name, "namespace", gitProvider.Namespace) - } - // Ahead of the gates for the same reason: an operator wiring a Git host's webhook needs the - // path while the provider is still red. + // Ahead of every gate, and after beginStatus, which snapshots the status the patch is computed + // against: an operator wiring a Git host's webhook needs the path while the provider is still red. r.publishWebhookPath(ctx, log, gitProvider) if err := r.validateCommitConfiguration(gitProvider); err != nil { diff --git a/internal/controller/gitprovider_controller_test.go b/internal/controller/gitprovider_controller_test.go index bd7159e5..8a341af9 100644 --- a/internal/controller/gitprovider_controller_test.go +++ b/internal/controller/gitprovider_controller_test.go @@ -308,65 +308,9 @@ var _ = Describe("GitProvider Controller", func() { } }) - It("publishes the branches its GitTargets configure, even when the remote is unreachable", func() { - // The inventory is a question about configuration, so it is published ahead of every - // gate: an operator looking at a red GitProvider still needs to see what it is FOR. - // - // The inventory is re-derived on the provider's own reconcile, deliberately with no - // watch on GitTarget: this reconcile also proves the credential against the remote, so - // an edge per GitTarget edit would spend a round trip to answer a question about - // configuration. Each step below therefore drives one reconcile, inside Eventually - // because the suite's manager reconciles the same object and either write can lose the - // optimistic lock. - gitProvider = &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{Name: "test-provider-inventory", Namespace: "default"}, - Spec: configbutleraiv1alpha3.GitProviderSpec{ - URL: "https://example.invalid/unreachable.git", - AllowedBranches: []string{"main", "release"}, - }, - } - Expect(k8sClient.Create(ctx, gitProvider)).To(Succeed()) - - key := types.NamespacedName{Name: gitProvider.Name, Namespace: gitProvider.Namespace} - branches := func(g Gomega) []configbutleraiv1alpha3.GitProviderBranchStatus { - _, err := reconciler.Reconcile(ctx, reconcile.Request{NamespacedName: key}) - g.Expect(err).NotTo(HaveOccurred()) - published := &configbutleraiv1alpha3.GitProvider{} - g.Expect(k8sClient.Get(ctx, key, published)).To(Succeed()) - return published.Status.Branches - } - - Consistently(branches, "2s", "500ms").Should(And(Not(BeNil()), BeEmpty()), - "configured and unused is a state of its own, and it is not an error") - - targets := []*configbutleraiv1alpha3.GitTarget{ - newInventoryGitTarget("inventory-apps", gitProvider.Name, "main"), - newInventoryGitTarget("inventory-infra", gitProvider.Name, "main"), - newInventoryGitTarget("inventory-release", gitProvider.Name, "release"), - } - for _, target := range targets { - Expect(k8sClient.Create(ctx, target)).To(Succeed()) - } - DeferCleanup(func() { - for _, target := range targets { - _ = k8sClient.Delete(ctx, target) - } - }) - - Eventually(branches, "30s", "500ms").Should(Equal([]configbutleraiv1alpha3.GitProviderBranchStatus{ - {Name: "main", GitTargetCount: 2}, - {Name: "release", GitTargetCount: 1}, - }), "two folders share the main branch, and they share one branch worker with it") - - Expect(k8sClient.Delete(ctx, targets[2])).To(Succeed()) - Eventually(branches, "30s", "500ms").Should(Equal([]configbutleraiv1alpha3.GitProviderBranchStatus{ - {Name: "main", GitTargetCount: 2}, - }), "a branch nothing references any more is not what this repository is for") - }) - It("publishes status.webhookPath while spec.receiver is set, and clears it when removed", func() { - // Published ahead of every gate like the inventory: the remote is unreachable here, and - // the path is still what an operator needs to wire the Git host's webhook. + // Published ahead of every gate: the remote is unreachable here, and the path is still + // what an operator needs to wire the Git host's webhook. gitProvider = &configbutleraiv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "test-provider-receiver", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitProviderSpec{ @@ -600,16 +544,3 @@ func generateTestSSHKey() ([]byte, error) { return privateKeyPEM, nil } - -// newInventoryGitTarget is a minimal, valid GitTarget for the branch-inventory spec: the inventory -// reads the spec alone, so nothing here has to reconcile. -func newInventoryGitTarget(name, provider, branch string) *configbutleraiv1alpha3.GitTarget { - return &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: provider}, - Branch: branch, - Path: "apps/" + name, - }, - } -} diff --git a/internal/controller/gittarget_apply_trigger.go b/internal/controller/gittarget_apply_trigger.go index 01ef8362..56ae3d09 100644 --- a/internal/controller/gittarget_apply_trigger.go +++ b/internal/controller/gittarget_apply_trigger.go @@ -37,14 +37,14 @@ func (r *GitTargetReconciler) observeApplyTrigger( ctx context.Context, st *reconcileStatus, target *configbutleraiv1alpha3.GitTarget, - gitProvider *configbutleraiv1alpha3.GitProvider, + dest gitDestination, ) (applyTriggerVerdict, bool) { ref := target.Spec.ApplyTriggerRef if ref == nil { st.remove(GitTargetConditionApplyTriggerReady) return applyTriggerVerdict{}, false } - verdict := r.applyTriggerVerdict(ctx, target, gitProvider) + verdict := r.applyTriggerVerdict(ctx, target, dest) st.setValue(GitTargetConditionApplyTriggerReady, verdict.value) return verdict, true } @@ -52,7 +52,7 @@ func (r *GitTargetReconciler) observeApplyTrigger( func (r *GitTargetReconciler) applyTriggerVerdict( ctx context.Context, target *configbutleraiv1alpha3.GitTarget, - gitProvider *configbutleraiv1alpha3.GitProvider, + dest gitDestination, ) applyTriggerVerdict { ref := target.Spec.ApplyTriggerRef name := fmt.Sprintf("%s %s/%s", ref.Kind, target.Namespace, ref.Name) @@ -94,8 +94,8 @@ func (r *GitTargetReconciler) applyTriggerVerdict( } } - if gitProvider != nil { - want := applier.Binding{RepositoryURL: gitProvider.Spec.URL, Branch: target.Spec.Branch} + if dest.provider != nil && dest.branch != nil { + want := applier.Binding{RepositoryURL: dest.provider.Spec.URL, Branch: dest.branch.Spec.Branch} if mismatch := applier.BindingMismatch(trigger, want); mismatch != "" { return applyTriggerVerdict{ value: conditionValue{ diff --git a/internal/controller/gittarget_apply_trigger_test.go b/internal/controller/gittarget_apply_trigger_test.go index c3c99cf1..087fb548 100644 --- a/internal/controller/gittarget_apply_trigger_test.go +++ b/internal/controller/gittarget_apply_trigger_test.go @@ -25,7 +25,7 @@ func triggerTarget(named bool) *configbutleraiv1alpha3.GitTarget { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "checkout", Namespace: "team-a"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "platform"}, Branch: "main", Path: "apps", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("platform", "main")}, Path: "apps", }, } if named { @@ -36,8 +36,17 @@ func triggerTarget(named bool) *configbutleraiv1alpha3.GitTarget { return target } -func triggerProvider() *configbutleraiv1alpha3.GitProvider { - return &configbutleraiv1alpha3.GitProvider{Spec: configbutleraiv1alpha3.GitProviderSpec{URL: applierTestRepo}} +// triggerDestination is what triggerTarget's spec.gitBranchRef resolves to: branch main of the test +// repository. +func triggerDestination() gitDestination { + branch := gitBranch("platform", "team-a") + return gitDestination{ + key: ctrlclient.ObjectKeyFromObject(branch), + branch: branch, + provider: &configbutleraiv1alpha3.GitProvider{ + Spec: configbutleraiv1alpha3.GitProviderSpec{URL: applierTestRepo}, + }, + } } // readyTrigger is the FluxTrigger checkout, as its reconciler published it: Ready with status, @@ -67,7 +76,7 @@ func TestObserveApplyTrigger_NoReferenceRemovesTheCondition(t *testing.T) { st := beginStatus(nil, nil, target) r := &GitTargetReconciler{Client: triggerTestClient(t)} - _, present := r.observeApplyTrigger(context.Background(), st, target, triggerProvider()) + _, present := r.observeApplyTrigger(context.Background(), st, target, triggerDestination()) assert.False(t, present) assert.Nil(t, findCondition(target.Status.Conditions, GitTargetConditionApplyTriggerReady)) @@ -119,7 +128,7 @@ func TestObserveApplyTrigger_Verdicts(t *testing.T) { target := triggerTarget(true) st := beginStatus(nil, nil, target) - verdict, present := r.observeApplyTrigger(context.Background(), st, target, triggerProvider()) + verdict, present := r.observeApplyTrigger(context.Background(), st, target, triggerDestination()) require.True(t, present) assert.Equal(t, tc.wantStatus, verdict.value.Status, verdict.value.Message) @@ -145,7 +154,12 @@ func TestObserveApplyTrigger_AReadyOfAnOlderGenerationIsNotReady(t *testing.T) { r := &GitTargetReconciler{Client: triggerTestClient(t, trigger)} target := triggerTarget(true) - verdict, _ := r.observeApplyTrigger(context.Background(), beginStatus(nil, nil, target), target, triggerProvider()) + verdict, _ := r.observeApplyTrigger( + context.Background(), + beginStatus(nil, nil, target), + target, + triggerDestination(), + ) assert.Equal(t, metav1.ConditionUnknown, verdict.value.Status) } @@ -199,13 +213,23 @@ func TestObserveApplyTrigger_AFailedReadKeepsThePublishedVerdict(t *testing.T) { r := &GitTargetReconciler{Client: failing} target := triggerTarget(true) - verdict, _ := r.observeApplyTrigger(context.Background(), beginStatus(nil, nil, target), target, triggerProvider()) + verdict, _ := r.observeApplyTrigger( + context.Background(), + beginStatus(nil, nil, target), + target, + triggerDestination(), + ) assert.Equal(t, metav1.ConditionUnknown, verdict.value.Status, "nothing published yet") target = triggerTarget(true) target.Status.Conditions = []metav1.Condition{{ Type: GitTargetConditionApplyTriggerReady, Status: metav1.ConditionTrue, Reason: ReasonSucceeded, }} - verdict, _ = r.observeApplyTrigger(context.Background(), beginStatus(nil, nil, target), target, triggerProvider()) + verdict, _ = r.observeApplyTrigger( + context.Background(), + beginStatus(nil, nil, target), + target, + triggerDestination(), + ) assert.Equal(t, metav1.ConditionTrue, verdict.value.Status, "the published verdict stands") } diff --git a/internal/controller/gittarget_controller.go b/internal/controller/gittarget_controller.go index db6380ae..36e42689 100644 --- a/internal/controller/gittarget_controller.go +++ b/internal/controller/gittarget_controller.go @@ -6,7 +6,6 @@ import ( "context" "errors" "fmt" - "path/filepath" "strconv" "strings" "time" @@ -31,7 +30,6 @@ import ( "sigs.k8s.io/controller-runtime/pkg/source" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" - "github.com/ConfigButler/gitops-reverser/internal/authz" "github.com/ConfigButler/gitops-reverser/internal/git" "github.com/ConfigButler/gitops-reverser/internal/reconcile" "github.com/ConfigButler/gitops-reverser/internal/telemetry" @@ -55,12 +53,10 @@ const ( const ( // GitTargetReasonOK is the healthy reason. It is the shared Succeeded vocabulary rather than // a per-kind spelling; the name is kept for call-site stability. - GitTargetReasonOK = ReasonSucceeded - GitTargetReasonProviderNotFound = "ProviderNotFound" - GitTargetReasonBranchNotAllowed = "BranchNotAllowed" - // GitTargetReasonParentBranchNotFound is the Ready reason for a write branch that cannot be - // created: the remote carries neither it nor the spec.parentBranch it would start from. - GitTargetReasonParentBranchNotFound = "ParentBranchNotFound" + GitTargetReasonOK = ReasonSucceeded + // GitTargetReasonGitBranchNotFound is the Validated=False reason for a GitTarget whose + // spec.gitBranchRef names no GitBranch, or is not set. + GitTargetReasonGitBranchNotFound = "GitBranchNotFound" GitTargetReasonTargetConflict = "TargetConflict" GitTargetReasonNotChecked = "NotChecked" GitTargetReasonBlocked = "Blocked" @@ -145,27 +141,12 @@ type GitTargetReconciler struct { // so a standing annotation forces one re-read rather than one per reconcile forever. Its zero // value is usable. reconcileRequests reconcileRequestTracker - - // remotePublications is what each GitTarget last wrote to status.remote: when, and about which - // repository. It is internal by design; see remotePublicationLedger. - remotePublications remotePublicationLedger - - // clock is the source of "now" for the publication ledger, so a test can hold time still - // across a reconcile that is measured in wall clock. Nil means time.Now. - clock func() time.Time -} - -// clockNow is the reconciler's wall clock. -func (r *GitTargetReconciler) clockNow() time.Time { - if r.clock != nil { - return r.clock() - } - return time.Now() } // +kubebuilder:rbac:groups=configbutler.ai,resources=gittargets,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=configbutler.ai,resources=gittargets/status,verbs=get;update;patch // +kubebuilder:rbac:groups=configbutler.ai,resources=gitproviders,verbs=get;list;watch +// +kubebuilder:rbac:groups=configbutler.ai,resources=gitbranches,verbs=get;list;watch // +kubebuilder:rbac:groups=configbutler.ai,resources=fluxtriggers;argocdtriggers,verbs=get;list;watch // +kubebuilder:rbac:groups="",resources=secrets,verbs=get;create;update // +kubebuilder:rbac:groups="",resources=events,verbs=create;patch @@ -192,33 +173,17 @@ func (r *GitTargetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( st := beginStatus(r.Client, r.Recorder, &target) gitPathWasRefused := conditionIsFalse(target.Status.Conditions, GitTargetConditionGitPathAccepted) - // The branch worker keeps the writes decided while the parent is missing and publishes them when - // it returns, so nothing has to be re-derived; this only reports the progress. - providerNS := target.GitProviderKey().Namespace - recoveringParent := r.parentRecovering(&target, providerNS) - publication := r.branchPublication(&target, providerNS) - - // One read of the GitProvider for everything below it; see getGitProvider. - gitProvider, providerErr := r.getGitProvider(ctx, providerNS, target.Spec.GitProviderRef.Name) - if providerErr != nil { - return ctrl.Result{}, providerErr - } - - // remoteWake: whether this reconcile is about to ask the remote where the branch is, so the - // answer lands after the status write below, and how long the floor holds back an answer it - // already has. See requeueForRemoteAnswer. - remoteWake := r.publishGitObservations(st, &target, repoIdentityOf(gitProvider)) - - // Ahead of every gate, for the reason the layout stanza above is: the join series has to carry - // the targets that are NOT working. git_pushes_total names a branch that stopped advancing and - // cannot name the GitTargets behind it, and a mapping published only on the happy path would - // go missing exactly when that question is being asked. It is a statement about configuration, - // so it holds whether or not a worker ever started. See telemetry/branch_targets.go. - telemetry.RecordBranchTarget( - target.Namespace, target.Name, providerNS, target.Spec.GitProviderRef.Name, target.Spec.Branch, - target.SourceCluster()) + publication := r.branchPublication(&target) + + // One read of the GitBranch and its GitProvider for everything below it; see resolveDestination. + dest, destErr := r.resolveDestination(ctx, &target) + if destErr != nil { + return ctrl.Result{}, destErr + } - validated, validationMsg, validationErr := r.evaluateValidatedGate(ctx, st, &target, providerNS, gitProvider) + r.publishAheadOfGates(st, &target, dest) + + validated, validationMsg, validationErr := r.evaluateValidatedGate(ctx, st, &target, dest) if validationErr != nil { return ctrl.Result{}, validationErr } @@ -253,10 +218,17 @@ func (r *GitTargetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( }) } + // The branch is written only by the GitBranch holding its claim, so a target whose GitBranch does + // not hold it has no worker to be wired to. That clears on its own once the GitBranch proves + // itself, so it is progress, not a stall. + if blocked, result, err := r.blockOnGitBranch(ctx, st, dest); blocked { + return result, err + } + // Ensure the branch worker exists and register the GitTarget's event stream before declaring // streams. The watch data plane can route live events as soon as it reaches Streaming, so the // destination worker must already be wired. - wired, wiringMessage := r.evaluateWorkerWiringGate(ctx, &target, providerNS, repoIdentityOf(gitProvider), log) + wired, wiringMessage := r.evaluateWorkerWiringGate(ctx, &target, dest, log) if !wired { return r.stall(ctx, st, blockedGate{ reason: GitTargetReadyReasonWorkerUnavailable, @@ -270,7 +242,7 @@ func (r *GitTargetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( // status without touching Git. Nothing new schedules anything, and the request is ENQUEUED // rather than run here — the reconcile must not block on a network round trip, and the work // belongs on the one goroutine allowed to touch the repository. - r.requestRemoteRefresh(&target, providerNS) + r.requestRemoteRefresh(&target) // One read of the source ClusterProvider serves everything below it: the audit route captured on // Declare and the ClusterProviderReady projection. Reading it twice invited the two to disagree. @@ -289,15 +261,14 @@ func (r *GitTargetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( st.setValue(GitTargetConditionGitPathAccepted, observed.axes.GitPath) st.setValue(GitTargetConditionRenderMatchesLive, observed.axes.Render) - refs := r.publishReferenceReadiness(st, &target, providerNS, gitProvider, sourceProvider) - triggerVerdict, hasTrigger := r.observeApplyTrigger(ctx, st, &target, gitProvider) + refs := r.publishReferenceReadiness(st, &target, gitBranchReadiness(dest, true), sourceProvider) + triggerVerdict, hasTrigger := r.observeApplyTrigger(ctx, st, &target, dest) rd := newGitTargetReadiness() convergeAsSuspended(rd, &target) - gitTargetReadinessGates(rd, observed, r.parentBranchReadiness(&target, repoIdentityOf(gitProvider)), - refs.gitProvider, refs.clusterProvider, refs.sourceCluster, publicationCondition(publication)) + gitTargetReadinessGates(rd, observed, + refs.gitBranch, refs.clusterProvider, refs.sourceCluster, publicationCondition(publication)) applyTriggerReadiness(rd, triggerVerdict, hasTrigger) - parentRecoveryReadiness(rd, recoveringParent) st.applyReadiness(rd) if err := st.commit(ctx); err != nil { @@ -306,7 +277,7 @@ func (r *GitTargetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( // requeueAfter shortens the cadence when the status write lost a race: the object then holds // the WINNER's older answer and nothing re-enqueues it, because the For() predicate filters // status-only updates by design. - requeue := st.requeueAfter(requeueForRemoteAnswer(gitTargetRequeue(rd), remoteWake)) + requeue := st.requeueAfter(gitTargetRequeue(rd)) // TEMPORARY at Info, while Failure A is open. The data plane can converge and this status not // follow it: a run has shown the render gate reaching True and both this GitTarget and every // WatchRule on it still publishing "Rechecking" two minutes later, with no dropped reconcile @@ -323,6 +294,25 @@ func (r *GitTargetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ( return ctrl.Result{RequeueAfter: requeue}, nil } +// publishAheadOfGates writes what holds whether or not any gate passes. The folder's layout is +// published ahead of every gate, so a target held unready still shows what the last scan resolved: +// it exists to explain a refused or surprising write. +// +// The branch-target join series is published here for the same reason: it has to carry the targets +// that are NOT working. git_pushes_total names a branch that stopped advancing and cannot name the +// GitTargets behind it, and a mapping published only on the happy path would go missing exactly +// when that question is being asked. It is a statement about configuration, so it holds whether or +// not a worker ever started. See telemetry/branch_targets.go. +func (r *GitTargetReconciler) publishAheadOfGates( + st *reconcileStatus, target *configbutleraiv1alpha3.GitTarget, dest gitDestination, +) { + layout, scanned := r.observeLayout(target) + publishLayout(st, target, layout, scanned) + telemetry.RecordBranchTarget( + target.Namespace, target.Name, dest.key.Namespace, dest.providerName(), dest.branchName(), + target.SourceCluster()) +} + // convergeAsSuspended gives a suspended GitTarget the converged outcome, under its own reason. // // A suspended target is healthy, not faulty: not writing is the configured outcome, and no @@ -364,6 +354,9 @@ type blockedGate struct { blocked string // requeueAfter overrides the steady cadence this gate would otherwise get. Zero means take it. requeueAfter time.Duration + // progressing publishes the outcome as progress (Ready=False, Reconciling=True) rather than a + // stall: the gate clears on its own, and a watch re-runs this reconcile when it does. + progressing bool } // stall publishes a terminal control-plane outcome and ends the reconcile. @@ -384,7 +377,11 @@ func (r *GitTargetReconciler) stall( GitTargetReasonNotChecked, gate.blocked) rd := newGitTargetReadiness() - rd.stalled(gate.reason, gate.message) + if gate.progressing { + rd.progressing(metav1.ConditionFalse, gate.reason, gate.message) + } else { + rd.stalled(gate.reason, gate.message) + } st.applyReadiness(rd) if err := st.commit(ctx); err != nil { @@ -395,7 +392,13 @@ func (r *GitTargetReconciler) stall( // strands the object on the previous answer for five minutes. This path returned gate.result // directly and was the one place that skipped the check. cadence := gate.requeueAfter - if cadence == 0 { + switch { + case cadence != 0: + case gate.progressing: + // Not converged, so the not-Ready cadence: a gate that clears on its own is looked at again + // soon, as gitTargetRequeue does for every other progressing target. + cadence = RequeueStreamSettleInterval + default: cadence = RequeueSteadyInterval } return ctrl.Result{RequeueAfter: st.requeueAfter(cadence)}, nil @@ -427,16 +430,18 @@ func (r *GitTargetReconciler) evaluateValidatedGate( ctx context.Context, st *reconcileStatus, target *configbutleraiv1alpha3.GitTarget, - providerNS string, - provider *configbutleraiv1alpha3.GitProvider, + dest gitDestination, ) (bool, string, error) { - validated, message, reason := validateProviderAndBranch(target, providerNS, provider) - if !validated { - st.set(GitTargetConditionValidated, metav1.ConditionFalse, reason, message) - return false, fmt.Sprintf("Validated gate failed: %s", reason), nil + if dest.branch == nil { + message := fmt.Sprintf("Referenced GitBranch '%s' not found", dest.key) + if dest.key.Name == "" { + message = "spec.gitBranchRef is not set" + } + st.set(GitTargetConditionValidated, metav1.ConditionFalse, GitTargetReasonGitBranchNotFound, message) + return false, fmt.Sprintf("Validated gate failed: %s", GitTargetReasonGitBranchNotFound), nil } - conflict, conflictMsg, conflictReason, conflictErr := r.checkForConflicts(ctx, target, providerNS) + conflict, conflictMsg, conflictReason, conflictErr := r.checkForConflicts(ctx, target) if conflictErr != nil { return false, "", conflictErr } @@ -483,7 +488,7 @@ func (r *GitTargetReconciler) evaluateValidatedGate( st.set(GitTargetConditionValidated, metav1.ConditionTrue, GitTargetReasonOK, - "Provider, branch, placement and commit validation passed", + "GitBranch, placement and commit validation passed", ) return true, "", nil } @@ -531,6 +536,26 @@ func (r *GitTargetReconciler) evaluateEncryptionGate( return true, "", 0 } +// blockOnGitBranch publishes GitBranchReady, and ends the reconcile as progress while the target +// cannot be wired: its GitBranch does not hold the branch, or names a GitProvider that does not exist. +func (r *GitTargetReconciler) blockOnGitBranch( + ctx context.Context, st *reconcileStatus, dest gitDestination, +) (bool, ctrl.Result, error) { + held := r.holdsBranch(dest) + branchReady := gitBranchReadiness(dest, held) + st.setValue(GitTargetConditionGitBranchReady, branchReady) + if held && dest.provider != nil { + return false, ctrl.Result{}, nil + } + result, err := r.stall(ctx, st, blockedGate{ + reason: GitTargetReasonGitBranchNotReady, + message: branchReady.Message, + blocked: "Blocked by GitBranchReady=False", + progressing: true, + }) + return true, result, err +} + // evaluateWorkerWiringGate ensures the GitTarget's branch worker exists and registers its // GitTargetEventStream, the route live watch events use to reach the branch worker. This is // internal plumbing rather than a status condition of its own: rare failures fold into Ready @@ -538,15 +563,14 @@ func (r *GitTargetReconciler) evaluateEncryptionGate( func (r *GitTargetReconciler) evaluateWorkerWiringGate( ctx context.Context, target *configbutleraiv1alpha3.GitTarget, - providerNS string, - repo git.RepoIdentity, + dest gitDestination, log logr.Logger, ) (bool, string) { if r.EventRouter == nil { return true, "" } - if _, err := r.ensureEventStream(ctx, target, providerNS, repo, log); err != nil { + if _, err := r.ensureEventStream(ctx, target, dest.workerRequest(), log); err != nil { return false, fmt.Sprintf("Failed to wire branch worker/event stream for %s/%s: %v", target.Namespace, target.Name, err) } @@ -743,10 +767,10 @@ func renderAxis(renderFidelity watch.RenderFidelityStatus) conditionValue { } // referenceReadiness is what the three objects a GitTarget REFERENCES say about themselves: -// destination-side (the GitProvider), source-config side (the ClusterProvider), and the runtime +// destination-side (the GitBranch), source-config side (the ClusterProvider), and the runtime // reachability of the source cluster. type referenceReadiness struct { - gitProvider conditionValue + gitBranch conditionValue clusterProvider conditionValue sourceCluster conditionValue } @@ -757,17 +781,16 @@ type referenceReadiness struct { func (r *GitTargetReconciler) publishReferenceReadiness( st *reconcileStatus, target *configbutleraiv1alpha3.GitTarget, - providerNS string, - gitProvider *configbutleraiv1alpha3.GitProvider, + gitBranch conditionValue, sourceProvider *configbutleraiv1alpha3.ClusterProvider, ) referenceReadiness { refs := referenceReadiness{ - gitProvider: gitProviderReadiness(target, providerNS, gitProvider), + gitBranch: gitBranch, clusterProvider: clusterProviderReadiness(sourceProvider), sourceCluster: r.observeSourceReachable(target), } st.setValue(GitTargetConditionSourceClusterReachable, refs.sourceCluster) - st.setValue(GitTargetConditionGitProviderReady, refs.gitProvider) + st.setValue(GitTargetConditionGitBranchReady, refs.gitBranch) st.setValue(GitTargetConditionClusterProviderReady, refs.clusterProvider) return refs } @@ -782,24 +805,22 @@ func (r *GitTargetReconciler) publishReferenceReadiness( func gitTargetReadinessGates( rd *readiness, observed dataPlaneObservation, - parent, provider, clusterProvider, sourceReach, publication conditionValue, + gitBranch, clusterProvider, sourceReach, publication conditionValue, ) { // Terminal, most specific first. Each of these needs a human: the folder holds content the - // operator will not manage, the write branch has no parent to be created from, a watch is - // refused, or what was written no longer matches live. + // operator will not manage, a watch is refused, or what was written no longer matches live. rd.stalledIf(observed.axes.GitPath.Status == metav1.ConditionFalse, observed.axes.GitPath.Reason, observed.axes.GitPath.Message) - rd.stalledIf(parent.Status == metav1.ConditionFalse, parent.Reason, parent.Message) rd.stalledIf(observed.streams.Blocked > 0, observed.streams.Reason, observed.streams.Message) rd.stalledIf(observed.axes.Render.Status == metav1.ConditionFalse, observed.axes.Render.Reason, observed.axes.Render.Message) // Transient. Each of these clears on its own and each has a Watches() edge that re-runs this // reconcile when it does, so they are progress, never a stall. Dependencies before this - // object's own data plane: "your GitProvider is down" is a more actionable answer than + // object's own data plane: "your GitBranch is down" is a more actionable answer than // "streams are still replaying", and it is usually the cause of the latter. - rd.progressingIf(provider.Status == metav1.ConditionFalse, metav1.ConditionFalse, - provider.Reason, provider.Message) + rd.progressingIf(gitBranch.Status == metav1.ConditionFalse, metav1.ConditionFalse, + gitBranch.Reason, gitBranch.Message) rd.progressingIf(clusterProvider.Status == metav1.ConditionFalse, metav1.ConditionFalse, clusterProvider.Reason, clusterProvider.Message) rd.progressingIf(sourceReach.Status == metav1.ConditionFalse, metav1.ConditionFalse, @@ -829,74 +850,21 @@ func gitTargetReadinessGates( "Stream declaration has not landed yet; the data-plane surface is not observable") } -// parentBranchReadiness reports whether the write branch can be created: False when the last -// look at the remote found neither the write branch nor the parent it would be created from. That -// parent is the configured spec.parentBranch, or, when it is omitted, the remote's default branch: -// a repository that is not empty but whose HEAD names no branch it carries refuses as well, because -// starting there would make an orphan. -// -// It is read from the branch worker's remote observation, so it costs no round trip and holds -// with periodic refresh disabled: every fetch on the write path records the same observation. The -// observation counts only when it was made under the parent configured now, and only for the -// repository the GitProvider names now (a repoint proves nothing about the new one). -func (r *GitTargetReconciler) parentBranchReadiness( - target *configbutleraiv1alpha3.GitTarget, - repo git.RepoIdentity, -) conditionValue { - ok := conditionValue{Status: metav1.ConditionTrue, Reason: GitTargetReasonOK} - observed, seen := r.observeRemote(target, repo) - if !seen || observed.ParentState != git.ParentMissing || - observed.ParentRequested != target.Spec.ParentBranch || differentRepository(observed.Repo, repo) { - return ok - } - return conditionValue{ - Status: metav1.ConditionFalse, - Reason: GitTargetReasonParentBranchNotFound, - Message: parentBranchNotFoundMessage(target, observed.ParentBranch), - } -} - -// parentBranchNotFoundMessage says which parent is missing and what fixes it. -func parentBranchNotFoundMessage(target *configbutleraiv1alpha3.GitTarget, missing string) string { - switch { - case target.Spec.ParentBranch != "": - return fmt.Sprintf( - "Branch '%s' does not exist on the remote, and neither does its parent branch '%s' to create "+ - "it from; nothing is written until one of them exists", - target.Spec.Branch, target.Spec.ParentBranch) - case missing != "": - return fmt.Sprintf( - "Branch '%s' does not exist on the remote, and the remote's default branch (HEAD -> '%s') does "+ - "not exist either; fix the remote's HEAD or set spec.parentBranch. Nothing is written until then", - target.Spec.Branch, missing) - default: - return fmt.Sprintf( - "Branch '%s' does not exist on the remote, and the remote has no default branch to create it "+ - "from; set the remote's HEAD or spec.parentBranch. Nothing is written until then", - target.Spec.Branch) - } -} - // requestRemoteRefresh asks this target's branch worker to re-prove where the branch is, if // nothing has proved it within the interval. // // The worker decides whether to spend a connection: the age test is its own, because only it // knows when it last pushed. A target sharing a branch with a busy one therefore costs nothing -// here and still gets its status renewed, because the worker reports what it knows before it -// skips. +// here, and what the worker learns reaches the GitBranch's status.remote. // // A suspended target refreshes too, for the reason its layout is still scanned: a stopped valve // that also stopped looking would freeze what an operator reads at whatever Git looked like when // someone panicked. Refreshing writes nothing, so there is no valve to respect. -func (r *GitTargetReconciler) requestRemoteRefresh( - target *configbutleraiv1alpha3.GitTarget, - providerNS string, -) { +func (r *GitTargetReconciler) requestRemoteRefresh(target *configbutleraiv1alpha3.GitTarget) { if r.GitRefreshInterval <= 0 || r.WorkerManager == nil { return } - worker, exists := r.WorkerManager.GetWorkerForTarget( - target.Spec.GitProviderRef.Name, providerNS, target.Spec.Branch) + worker, exists := r.WorkerManager.WorkerForTarget(target) if !exists || worker == nil { // Nothing has been published for this target yet, so there is no checkout to refresh and // the first publication reads the remote anyway. @@ -909,71 +877,6 @@ func (r *GitTargetReconciler) requestRemoteRefresh( }) } -// publishGitObservations writes what the data plane last read about this target's folder and -// about its branch on the remote: status.placement with the LayoutResolved condition, and -// status.remote. -// -// It runs ahead of every gate, so a target held unready still shows both. That ordering is the -// point rather than a convenience: these stanzas exist to explain a refused or surprising write, -// and a projection that ran only on the happy path would be missing exactly when it is wanted. - -func (r *GitTargetReconciler) publishGitObservations( - st *reconcileStatus, - target *configbutleraiv1alpha3.GitTarget, - repo git.RepoIdentity, -) remoteWake { - layout, scanned := r.observeLayout(target) - publishLayout(st, target, layout, scanned) - - now := r.clockNow() - remote, remoteSeen := r.observeRemote(target, repo) - heldFor := r.publishRemote(st, target, remote, remoteSeen, repo, now) - return remoteWake{ - // Whether the refresh this reconcile goes on to enqueue will actually reach the remote: the - // worker spends a connection only when what it holds is older than the interval configured. - asked: r.GitRefreshInterval > 0 && (!remoteSeen || remote.Age(now) >= r.GitRefreshInterval), - heldFor: heldFor, - } -} - -// remoteWake is what status.remote needs from the next reconcile's timing. -type remoteWake struct { - // asked: this reconcile asked the remote where the branch is, and the answer lands after it. - asked bool - // heldFor: the floor holds back an answer this target already has, for this long. - heldFor time.Duration -} - -// requeueForRemoteAnswer shortens the cadence for a target that has just ASKED the remote where -// its branch is, or that holds an answer the floor has not let out yet. -// -// It is the one piece of scheduling this status field gets, and it exists because the refresh is -// enqueued during the reconcile: the worker answers moments later, after the status write, so the -// answer would otherwise sit undelivered until the next steady tick. An idle target would then -// spend a connection every interval and publish what it learned a whole tick late — visible to an -// operator as a target that follows a foreign push ten minutes after noticing it. -// -// Asking is not enough on its own, because the requeue it returns can be lost. A refresh that -// finds the branch moved re-enqueues the target while the asking reconcile is still running, and -// the queue folds a delayed requeue into an item that is already waiting (controller-runtime's -// priority queue keeps the earlier time). The reconcile that then runs holds a fresh answer, so it -// asks nothing, and the floor stops it publishing: without heldFor it returned the steady cadence, -// and the answer waited the whole tick after all. So whichever reconcile meets the held answer -// comes back when the floor lifts. -// -// Nothing else is threaded anywhere. A reconcile that asked nothing and holds nothing keeps its -// own cadence, and no early return has to carry a deadline. -func requeueForRemoteAnswer(cadence time.Duration, wake remoteWake) time.Duration { - next := cadence - if wake.asked && RemotePublicationInterval < next { - next = RemotePublicationInterval - } - if wake.heldFor > 0 && wake.heldFor < next { - next = wake.heldFor - } - return next -} - // ensureEventStream wires this GitTarget to the branch worker for the repository its GitProvider // names NOW, and wires the route live events take to reach it. // @@ -984,52 +887,28 @@ func requeueForRemoteAnswer(cadence time.Duration, wake remoteWake) time.Duratio func (r *GitTargetReconciler) ensureEventStream( ctx context.Context, target *configbutleraiv1alpha3.GitTarget, - providerNS string, - repo git.RepoIdentity, + req git.WorkerRequest, log logr.Logger, ) (*reconcile.GitTargetEventStream, error) { if r.WorkerManager == nil { return nil, errors.New("worker manager is not configured") } - err := r.WorkerManager.EnsureWorker( - context.Background(), - target.Spec.GitProviderRef.Name, - providerNS, - target.Spec.Branch, - repo, - target.Spec.ParentBranch, - ) - if err != nil { - return nil, fmt.Errorf( - "failed to ensure branch worker for provider=%s/%s branch=%s: %w", - providerNS, - target.Spec.GitProviderRef.Name, - target.Spec.Branch, - err, - ) + if err := r.WorkerManager.EnsureWorker(context.Background(), req); err != nil { + return nil, fmt.Errorf("failed to ensure branch worker for GitBranch %s: %w", req.GitBranch, err) } // Asked on EVERY reconcile, not only on the one that did the replacing. The recovery is held // by the worker manager until it has been delivered, so a delivery that failed is retried by // whichever GitTarget on the branch reconciles next. - branch := git.BranchKeyFor(repo.URL, target.Spec.Branch) + branch := git.BranchKeyFor(req.Repo.URL, req.Branch) if r.WorkerManager.ReplacementPending(branch) && - r.recoverBranchAfterReplacement(ctx, branch, target.GitProviderKey(), log) { + r.recoverBranchAfterReplacement(ctx, branch, req.GitBranch, log) { r.WorkerManager.AcknowledgeReplacement(branch) } - worker, ensured := r.WorkerManager.GetWorkerForTarget( - target.Spec.GitProviderRef.Name, - providerNS, - target.Spec.Branch, - ) + worker, ensured := r.WorkerManager.WorkerForGitBranch(req.GitBranch) if !ensured { - return nil, fmt.Errorf( - "branch worker not found for provider=%s/%s branch=%s", - providerNS, - target.Spec.GitProviderRef.Name, - target.Spec.Branch, - ) + return nil, fmt.Errorf("branch worker not found for GitBranch %s", req.GitBranch) } gitDest := types.NewResourceReference(target.Name, target.Namespace) @@ -1047,45 +926,6 @@ func (r *GitTargetReconciler) ensureEventStream( return stream, nil } -// validateProviderAndBranch answers the two questions the GitProvider decides: is there one, and -// does it admit this branch. It takes the provider the reconcile already read rather than reading -// its own, so the branch worker cannot be wired for a different object than the one validated -// here; a nil provider is the not-found case. -func validateProviderAndBranch( - target *configbutleraiv1alpha3.GitTarget, - providerNS string, - gp *configbutleraiv1alpha3.GitProvider, -) (bool, string, string) { - if gp == nil { - msg := fmt.Sprintf("Referenced GitProvider '%s/%s' not found", providerNS, target.Spec.GitProviderRef.Name) - return false, msg, GitTargetReasonProviderNotFound - } - - branchAllowed := false - for _, pattern := range gp.Spec.AllowedBranches { - match, matchErr := filepath.Match(pattern, target.Spec.Branch) - if matchErr != nil { - continue - } - if match { - branchAllowed = true - break - } - } - if !branchAllowed { - msg := fmt.Sprintf( - "Branch '%s' does not match any pattern in allowedBranches list %v of GitProvider '%s/%s'", - target.Spec.Branch, - gp.Spec.AllowedBranches, - providerNS, - target.Spec.GitProviderRef.Name, - ) - return false, msg, GitTargetReasonBranchNotAllowed - } - - return true, "", "" -} - // recoverBranchAfterReplacement re-establishes every GitTarget on a branch whose worker was just // replaced, because the replacement leaves that branch's state describing a repository that is no // longer the destination. @@ -1112,7 +952,7 @@ func validateProviderAndBranch( func (r *GitTargetReconciler) recoverBranchAfterReplacement( ctx context.Context, branch git.BranchKey, - provider k8stypes.NamespacedName, + gitBranch k8stypes.NamespacedName, log logr.Logger, ) bool { if r.EventRouter == nil || r.EventRouter.WatchManager == nil { @@ -1121,14 +961,14 @@ func (r *GitTargetReconciler) recoverBranchAfterReplacement( var targets configbutleraiv1alpha3.GitTargetList if err := r.List(ctx, &targets); err != nil { log.Error(err, "Could not re-establish the branch after its worker was replaced; it stays pending", - "branch", branch.String(), "provider", provider.String()) + "branch", branch.String(), "gitBranch", gitBranch.String()) return false } gate := r.WorkerManager.RenderFidelityGate() recovered := 0 for i := range targets.Items { affected := &targets.Items[i] - if !git.WritesThrough(affected, provider, branch.Branch) { + if !git.WritesThrough(affected, gitBranch) { continue } if !affected.DeletionTimestamp.IsZero() { @@ -1140,32 +980,72 @@ func (r *GitTargetReconciler) recoverBranchAfterReplacement( recovered++ } log.Info("Branch worker replaced; re-establishing every GitTarget on the branch", - "branch", branch.String(), "provider", provider.String(), "gitTargets", recovered) + "branch", branch.String(), "gitBranch", gitBranch.String(), "gitTargets", recovered) return true } -// getGitProvider reads the GitTarget's GitProvider ONCE for the whole reconcile. -// -// That one object answers the Validated gate's questions, projects GitProviderReady, and names the -// repository the branch worker has to be about. Reading it three times let those three disagree -// about which repository this target points at, which is the disagreement the worker's repository +// gitDestination is what a GitTarget's spec.gitBranchRef resolves to, read ONCE for the whole +// reconcile: the GitBranch, and that GitBranch's GitProvider. That one pair answers the Validated +// gate, projects GitBranchReady, names the repository the branch worker has to be about, and the +// binding the apply trigger is checked against. Reading them several times let those disagree about +// which repository this target points at, which is the disagreement the worker's repository // identity exists to prevent. -// -// A provider that is not there comes back (nil, nil): its absence is a configuration state the -// Validated gate reports as ProviderNotFound, not a reconcile failure to retry with a backoff. -func (r *GitTargetReconciler) getGitProvider( +type gitDestination struct { + // key is the referenced GitBranch; zero when spec.gitBranchRef is not set. + key k8stypes.NamespacedName + // branch is nil when the GitBranch does not exist. + branch *configbutleraiv1alpha3.GitBranch + // provider is nil when the GitBranch, or its GitProvider, does not exist. + provider *configbutleraiv1alpha3.GitProvider +} + +// resolveDestination reads the GitBranch and its GitProvider. Objects that are not there come back +// nil: their absence is a configuration state reported in status, not a reconcile failure. +func (r *GitTargetReconciler) resolveDestination( ctx context.Context, - providerNS, providerName string, -) (*configbutleraiv1alpha3.GitProvider, error) { - var gp configbutleraiv1alpha3.GitProvider - key := k8stypes.NamespacedName{Name: providerName, Namespace: providerNS} - if err := r.Get(ctx, key, &gp); err != nil { - if apierrors.IsNotFound(err) { - return nil, nil //nolint:nilnil // a GitProvider that is not there is a state, not an error - } - return nil, err + target *configbutleraiv1alpha3.GitTarget, +) (gitDestination, error) { + branch, provider, err := git.ResolveGitBranch(ctx, r.Client, target) + if err != nil { + return gitDestination{}, err + } + return gitDestination{key: target.GitBranchKey(), branch: branch, provider: provider}, nil +} + +func (d gitDestination) providerName() string { + if d.branch == nil { + return "" + } + return d.branch.Spec.GitProviderRef.Name +} + +func (d gitDestination) branchName() string { + if d.branch == nil { + return "" + } + return d.branch.Spec.Branch +} + +// workerRequest is the branch worker this destination needs. +func (d gitDestination) workerRequest() git.WorkerRequest { + return git.WorkerRequest{ + GitBranch: d.key, + Provider: d.providerName(), + Branch: d.branchName(), + Repo: repoIdentityOf(d.provider), + ParentBranch: d.branch.Spec.ParentBranch, } - return &gp, nil +} + +// holdsBranch reports whether the referenced GitBranch holds its repository branch now. It reads the +// claim registry, which is what EnsureWorker checks, rather than the GitBranch's status, which can +// lag it. +func (r *GitTargetReconciler) holdsBranch(dest gitDestination) bool { + if dest.branch == nil || r.WorkerManager == nil { + return false + } + _, claimant, held := r.WorkerManager.Claims().HeldBy(dest.key) + return held && claimant.UID == dest.branch.UID } // repoIdentityOf is which repository a GitProvider names, in the form the data plane keys a branch @@ -1181,38 +1061,21 @@ func repoIdentityOf(provider *configbutleraiv1alpha3.GitProvider) git.RepoIdenti func (r *GitTargetReconciler) checkForConflicts( ctx context.Context, target *configbutleraiv1alpha3.GitTarget, - providerNS string, ) (bool, string, string, error) { var allTargets configbutleraiv1alpha3.GitTargetList if err := r.List(ctx, &allTargets); err != nil { return false, "", "", fmt.Errorf("list GitTargets for conflict validation: %w", err) } - lost, msg, err := r.losesRepositoryBranch(ctx, target, allTargets.Items) - if err != nil || lost { - return lost, msg, GitTargetReasonTargetConflict, err - } - for i := range allTargets.Items { existing := &allTargets.Items[i] if existing.Namespace == target.Namespace && existing.Name == target.Name { continue } - if existing.GitProviderKey() != target.GitProviderKey() { - continue - } - if existing.Spec.Branch != target.Spec.Branch { + if existing.GitBranchKey() != target.GitBranchKey() { continue } - // Checked whatever either path looks like: a target with a path its writer will reject - // still wires the shared worker, and would set its parent. - if lost, msg := losesParentBranchConflict(target, existing, providerNS); lost { - return true, msg, GitTargetReasonTargetConflict, nil - } - if existing.Spec.ParentBranch != target.Spec.ParentBranch { - continue // the other target loses, on its own reconcile - } - if lost, msg := losesPathConflict(target, existing, providerNS); lost { + if lost, msg := losesPathConflict(target, existing); lost { return true, msg, GitTargetReasonTargetConflict, nil } } @@ -1220,51 +1083,6 @@ func (r *GitTargetReconciler) checkForConflicts( return false, "", "", nil } -// losesRepositoryBranch reports whether another GitProvider owns the repository branch target writes: -// a GitTarget older than target writes the same repository and branch through a GitProvider other -// than target's. One worker writes a repository branch, with one GitProvider's credential, so the -// later GitTarget is refused (git/branch_owner.go). The GitProviders are read from the cache. -func (r *GitTargetReconciler) losesRepositoryBranch( - ctx context.Context, - target *configbutleraiv1alpha3.GitTarget, - targets []configbutleraiv1alpha3.GitTarget, -) (bool, string, error) { - var providers configbutleraiv1alpha3.GitProviderList - if err := r.List(ctx, &providers); err != nil { - return false, "", fmt.Errorf("list GitProviders for conflict validation: %w", err) - } - urls := make(map[k8stypes.NamespacedName]string, len(providers.Items)) - for i := range providers.Items { - urls[client.ObjectKeyFromObject(&providers.Items[i])] = providers.Items[i].Spec.URL - } - url, known := urls[target.GitProviderKey()] - if !known { - return false, "", nil - } - branch := git.BranchKeyFor(url, target.Spec.Branch) - owners, err := git.BranchOwners(targets, func(provider k8stypes.NamespacedName) (string, bool) { - u, ok := urls[provider] - return u, ok - }, func(candidate *configbutleraiv1alpha3.GitTarget) (bool, error) { - decision, err := authz.GitTargetAdmitted(ctx, r.Client, candidate) - return decision.Allowed, err - }) - if err != nil { - return false, "", fmt.Errorf("decide the owner of the repository branch: %w", err) - } - owner, owned := owners[branch] - if !owned || owner.Provider == target.GitProviderKey() { - return false, "", nil - } - return true, fmt.Sprintf( - "Conflict detected. GitTarget '%s' (older) writes repository %s, branch '%s' through GitProvider '%s', "+ - "and this GitTarget writes it through GitProvider '%s'. Every GitTarget writing one repository branch "+ - "must name the same GitProvider: to share it from another namespace, set spec.gitProviderRef.namespace "+ - "and admit this namespace in that GitProvider's spec.accessFrom. This GitTarget will not be processed.", - owner.Target, branch.Repository, branch.Branch, owner.Provider, target.GitProviderKey(), - ), nil -} - // losesPathConflict reports whether target loses to existing, on the same write branch, because // their paths are equal or nested. Two such targets fight over which documents each one owns. The // later-created target loses (ties broken deterministically by identity) so every materialized @@ -1273,7 +1091,7 @@ func (r *GitTargetReconciler) losesRepositoryBranch( // A path the writer would reject (absolute, backslashes, ".." traversal) owns nothing, so it must // neither block others nor be blocked here — its own write path fails it. Only well-formed paths // participate in overlap detection. -func losesPathConflict(target, existing *configbutleraiv1alpha3.GitTarget, providerNS string) (bool, string) { +func losesPathConflict(target, existing *configbutleraiv1alpha3.GitTarget) (bool, string) { if !git.IsValidTargetPath(target.Spec.Path) || !git.IsValidTargetPath(existing.Spec.Path) || !gitTargetPathsOverlap(target.Spec.Path, existing.Spec.Path) || @@ -1282,60 +1100,25 @@ func losesPathConflict(target, existing *configbutleraiv1alpha3.GitTarget, provi } if normalizeGitTargetPath(target.Spec.Path) == normalizeGitTargetPath(existing.Spec.Path) { return true, fmt.Sprintf( - "Conflict detected. Another GitTarget '%s/%s' (created at %s) is already using GitProvider '%s/%s', branch '%s', path '%s'. This GitTarget was created later and will not be processed.", + "Conflict detected. Another GitTarget '%s/%s' (created at %s) is already using GitBranch '%s', path '%s'. This GitTarget was created later and will not be processed.", existing.Namespace, existing.Name, existing.CreationTimestamp.Format(time.RFC3339), - providerNS, - target.Spec.GitProviderRef.Name, - target.Spec.Branch, + target.GitBranchKey(), target.Spec.Path, ) } return true, fmt.Sprintf( - "Conflict detected. This GitTarget's path '%s' overlaps the path '%s' of GitTarget '%s/%s' (created at %s) on GitProvider '%s/%s', branch '%s' — one path nests inside the other (sibling paths are allowed). This GitTarget was created later and will not be processed.", + "Conflict detected. This GitTarget's path '%s' overlaps the path '%s' of GitTarget '%s/%s' (created at %s) on GitBranch '%s' — one path nests inside the other (sibling paths are allowed). This GitTarget was created later and will not be processed.", target.Spec.Path, existing.Spec.Path, existing.Namespace, existing.Name, existing.CreationTimestamp.Format(time.RFC3339), - providerNS, - target.Spec.GitProviderRef.Name, - target.Spec.Branch, + target.GitBranchKey(), ) } -// losesParentBranchConflict reports whether target loses to existing, on the same write branch, -// because the two name different parent branches. One branch worker owns one checkout per write -// branch, so every target on it has to agree on where the branch starts. Compared as written, with -// omitted as its own value: the answer needs no remote round trip and cannot flip when somebody -// changes the remote's default branch. -func losesParentBranchConflict(target, existing *configbutleraiv1alpha3.GitTarget, providerNS string) (bool, string) { - if existing.Spec.ParentBranch == target.Spec.ParentBranch || !gitTargetLosesConflict(target, existing) { - return false, "" - } - return true, parentBranchConflictMessage(target, existing, providerNS) -} - -// parentBranchConflictMessage explains a TargetConflict over spec.parentBranch. -func parentBranchConflictMessage(target, existing *configbutleraiv1alpha3.GitTarget, providerNS string) string { - return fmt.Sprintf( - "Conflict detected. GitTarget '%s/%s' (created at %s) writes GitProvider '%s/%s', branch '%s' with parent "+ - "branch %s, and this GitTarget names parent branch %s. All GitTargets on one branch must name the same "+ - "parent branch. This GitTarget was created later and will not be processed.", - existing.Namespace, existing.Name, existing.CreationTimestamp.Format(time.RFC3339), - providerNS, target.Spec.GitProviderRef.Name, target.Spec.Branch, - describeParentBranch(existing.Spec.ParentBranch), describeParentBranch(target.Spec.ParentBranch), - ) -} - -func describeParentBranch(name string) string { - if name == "" { - return "(omitted: the remote's default branch)" - } - return "'" + name + "'" -} - func (r *GitTargetReconciler) ensureEncryptionSecret( ctx context.Context, target *configbutleraiv1alpha3.GitTarget, @@ -1544,10 +1327,6 @@ func (r *GitTargetReconciler) cleanupDeletedGitTarget( // condition gauge is released on the same terms and for a sharper reason: a condition series // that outlives its object reports Ready=False forever and the alert on it never clears. r.reconcileRequests.forget(gitDest) - // Same terms: the publication ledger is this reconciler's memory of what the object's status - // said, and an entry that outlives the object is a rate limit held against a name that may be - // recreated tomorrow with nothing published. - r.remotePublications.forget(gitDest) telemetry.ForgetResourceConditions(conditionKindGitTarget, namespacedName.Namespace, namespacedName.Name) // Same terms, same reason: a join series that outlives its GitTarget keeps attributing a live // branch's push failures to an object that no longer exists. @@ -1631,13 +1410,20 @@ func (r *GitTargetReconciler) SetupWithManager(mgr ctrl.Manager) error { // cluster. Generated-age-Secret recovery and out-of-band age-key updates are // picked up by the periodic reconcile (RequeueSteadyInterval) instead. // See docs/rbac.md. - // GenerationChangedPredicate keeps this watch reacting to a freshly - // applied or spec-changed GitProvider while ignoring the status-only - // updates the controllers write themselves — without it every provider - // heartbeat would re-list and re-enqueue all dependent GitTargets. + // React to the referenced GitBranch appearing, going, or its Ready changing (it proved its + // credential, took or lost its branch), so GitBranchReady and the wiring gate follow it within + // one reconcile. Its status.remote moves on every push and is filtered out. + Watches( + &configbutleraiv1alpha3.GitBranch{}, + handler.EnqueueRequestsFromMapFunc(r.gitDestinationToGitTargets), + builder.WithPredicates(gitBranchReadyOrSpecChanged()), + ). + // A GitProvider's accessFrom decides which namespaces may reference its GitBranches, so a + // changed or freshly applied provider re-runs the GitTargets on those GitBranches. + // GenerationChangedPredicate ignores the status-only updates its controller writes itself. Watches( &configbutleraiv1alpha3.GitProvider{}, - handler.EnqueueRequestsFromMapFunc(r.gitProviderToGitTargets), + handler.EnqueueRequestsFromMapFunc(r.gitDestinationToGitTargets), builder.WithPredicates(predicate.GenerationChangedPredicate{}), ). // React to the referenced (cluster-scoped) ClusterProvider becoming Ready/NotReady so the @@ -1750,35 +1536,59 @@ func (r *GitTargetReconciler) clusterWatchRuleToGitTarget( }}} } -// namesGitProvider reports whether a GitTarget writes through the GitProvider obj, from whichever -// namespace. -func namesGitProvider(target *configbutleraiv1alpha3.GitTarget, obj client.Object) bool { - return target.GitProviderKey() == client.ObjectKeyFromObject(obj) +// gitTargetsOn lists the GitTargets, from any namespace, that write through obj: a GitBranch they +// reference, or a GitProvider that one of their GitBranches names. Both reads are from the cache. +func gitTargetsOn( + ctx context.Context, + reader client.Reader, + obj client.Object, +) ([]configbutleraiv1alpha3.GitTarget, error) { + branches := map[k8stypes.NamespacedName]bool{} + switch obj.(type) { + case *configbutleraiv1alpha3.GitBranch: + branches[client.ObjectKeyFromObject(obj)] = true + case *configbutleraiv1alpha3.GitProvider: + var list configbutleraiv1alpha3.GitBranchList + if err := reader.List(ctx, &list, client.InNamespace(obj.GetNamespace())); err != nil { + return nil, err + } + for i := range list.Items { + if list.Items[i].Spec.GitProviderRef.Name == obj.GetName() { + branches[client.ObjectKeyFromObject(&list.Items[i])] = true + } + } + } + if len(branches) == 0 { + return nil, nil + } + var targets configbutleraiv1alpha3.GitTargetList + if err := reader.List(ctx, &targets); err != nil { + return nil, err + } + var on []configbutleraiv1alpha3.GitTarget + for i := range targets.Items { + if branches[targets.Items[i].GitBranchKey()] { + on = append(on, targets.Items[i]) + } + } + return on, nil } -// gitProviderToGitTargets maps a GitProvider event to every GitTarget that references it, in any -// namespace, so a freshly-arrived provider re-enqueues any dependents currently stuck on -// ProviderNotFound, and a changed accessFrom grants or revokes at once, instead of waiting for the -// periodic RequeueSteadyInterval. -func (r *GitTargetReconciler) gitProviderToGitTargets( +// gitDestinationToGitTargets maps a GitBranch or GitProvider event to every GitTarget writing +// through it (gitTargetsOn): a GitBranch taking or losing its branch changes whether they can be +// wired, and a GitProvider's accessFrom decides which of their namespaces may reference it. +func (r *GitTargetReconciler) gitDestinationToGitTargets( ctx context.Context, obj client.Object, ) []ctrlreconcile.Request { - var targets configbutleraiv1alpha3.GitTargetList - if err := r.List(ctx, &targets); err != nil { + targets, err := gitTargetsOn(ctx, r.Client, obj) + if err != nil { logDependencyListError(ctx, err, "GitTargets", obj) return nil } - - var requests []ctrlreconcile.Request - for i := range targets.Items { - t := &targets.Items[i] - if !namesGitProvider(t, obj) { - continue - } - requests = append(requests, ctrlreconcile.Request{ - NamespacedName: k8stypes.NamespacedName{Name: t.Name, Namespace: t.Namespace}, - }) + requests := make([]ctrlreconcile.Request, 0, len(targets)) + for i := range targets { + requests = append(requests, ctrlreconcile.Request{NamespacedName: client.ObjectKeyFromObject(&targets[i])}) } return requests } @@ -1919,3 +1729,28 @@ func clusterProviderReadyOrSpecChanged() predicate.Predicate { }, } } + +// gitBranchReadyOrSpecChanged is the GitBranch watch predicate for the GitTarget controller: fire on +// create/delete, and on an update when the spec changed or the Ready condition's status or reason +// did. A status.remote write alone is filtered. +func gitBranchReadyOrSpecChanged() predicate.Predicate { + readyOf := func(b *configbutleraiv1alpha3.GitBranch) string { + if c := findCondition(b.Status.Conditions, ConditionTypeReady); c != nil { + return string(c.Status) + "/" + c.Reason + } + return "" + } + return predicate.Funcs{ + CreateFunc: func(event.CreateEvent) bool { return true }, + DeleteFunc: func(event.DeleteEvent) bool { return true }, + GenericFunc: func(event.GenericEvent) bool { return false }, + UpdateFunc: func(e event.UpdateEvent) bool { + oldB, ok1 := e.ObjectOld.(*configbutleraiv1alpha3.GitBranch) + newB, ok2 := e.ObjectNew.(*configbutleraiv1alpha3.GitBranch) + if !ok1 || !ok2 { + return true + } + return oldB.Generation != newB.Generation || readyOf(oldB) != readyOf(newB) + }, + } +} diff --git a/internal/controller/gittarget_controller_test.go b/internal/controller/gittarget_controller_test.go index 88e89b58..f654098d 100644 --- a/internal/controller/gittarget_controller_test.go +++ b/internal/controller/gittarget_controller_test.go @@ -16,6 +16,7 @@ import ( "k8s.io/apimachinery/pkg/types" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" ) var _ = Describe("GitTarget Controller Security", func() { @@ -24,245 +25,42 @@ var _ = Describe("GitTarget Controller Security", func() { interval = time.Millisecond * 250 ) - Context("When a branch is not allowed by GitProvider", func() { - It("Should report missing provider through status instead of admission rejection", func() { + Context("When the referenced GitBranch is missing", func() { + It("Should report a missing GitBranch through status instead of admission rejection", func() { ctx := context.Background() gitTarget := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{ - Name: "test-target-missing-provider", + Name: "test-target-missing-branch", Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "missing-provider", - }, - Branch: "main", - Path: "test-folder", - }, - } - Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) - - gitTargetLookupKey := types.NamespacedName{ - Name: "test-target-missing-provider", - Namespace: "default", - } - - createdGitTarget := &configbutleraiv1alpha3.GitTarget{} - Eventually(func() bool { - err := k8sClient.Get(ctx, gitTargetLookupKey, createdGitTarget) - if err != nil { - return false - } - for _, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionValidated && - condition.Reason == GitTargetReasonProviderNotFound { - return true - } - } - return false - }, timeout, interval).Should(BeTrue()) - - var readyCondition *metav1.Condition - var validatedCondition *metav1.Condition - for i, condition := range createdGitTarget.Status.Conditions { - switch condition.Type { - case GitTargetConditionReady: - readyCondition = &createdGitTarget.Status.Conditions[i] - case GitTargetConditionValidated: - validatedCondition = &createdGitTarget.Status.Conditions[i] - } - } - - Expect(readyCondition).NotTo(BeNil(), "Ready condition should exist") - Expect(readyCondition.Status).To(Equal(metav1.ConditionFalse)) - Expect(readyCondition.Reason).To(Equal(GitTargetReadyReasonValidationFailed)) - - Expect(validatedCondition).NotTo(BeNil(), "Validated condition should exist") - Expect(validatedCondition.Status).To(Equal(metav1.ConditionFalse)) - Expect(validatedCondition.Reason).To(Equal(GitTargetReasonProviderNotFound)) - Expect(validatedCondition.Message).To( - ContainSubstring("Referenced GitProvider 'default/missing-provider' not found"), - ) - - Expect(k8sClient.Delete(ctx, gitTarget)).Should(Succeed()) - }) - - It("Should clear branch push status when the branch is not allowed", func() { - ctx := context.Background() - - // Create a GitProvider that only allows 'main' and 'develop' branches - gitProvider := &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-provider-security", - Namespace: "default", - }, - Spec: configbutleraiv1alpha3.GitProviderSpec{ - URL: "https://github.com/test-org/test-repo.git", - AllowedBranches: []string{"main", "develop"}, - SecretRef: &meta.LocalObjectReference{ - Name: "test-secret", // Dummy secret - }, - }, - } - Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) - - // Create a GitTarget referencing an unauthorized branch - unauthorizedBranch := "feature/unauthorized" - gitTarget := &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-target-security", - Namespace: "default", - }, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-security", - }, - Branch: unauthorizedBranch, - Path: "test-folder", - }, - } - Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) - - // Wait for reconciliation and verify status - gitTargetLookupKey := types.NamespacedName{ - Name: "test-target-security", - Namespace: "default", - } - - createdGitTarget := &configbutleraiv1alpha3.GitTarget{} - - // Wait for the controller to reach the final BranchNotAllowed state. - // The reconciler may pass through transient reasons (e.g. ProviderNotFound) before the - // GitProvider cache is warm, so wait for the specific reason this test asserts on. - Eventually(func() bool { - err := k8sClient.Get(ctx, gitTargetLookupKey, createdGitTarget) - if err != nil { - return false - } - for _, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionValidated && - condition.Reason == GitTargetReasonBranchNotAllowed { - return true - } - } - return false - }, timeout, interval).Should(BeTrue()) - - // Verify the Ready condition is False with BranchNotAllowed reason - Expect(createdGitTarget.Status.Conditions).NotTo(BeEmpty()) - var readyCondition *metav1.Condition - for i, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionReady { - readyCondition = &createdGitTarget.Status.Conditions[i] - break - } - } - Expect(readyCondition).NotTo(BeNil(), "Ready condition should exist") - Expect(readyCondition.Status).To(Equal(metav1.ConditionFalse), "Ready should be False") - Expect(readyCondition.Reason).To(Equal(GitTargetReadyReasonValidationFailed)) - var validatedCondition *metav1.Condition - for i, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionValidated { - validatedCondition = &createdGitTarget.Status.Conditions[i] - break - } - } - Expect(validatedCondition).NotTo(BeNil(), "Validated condition should exist") - Expect(validatedCondition.Reason).To(Equal(GitTargetReasonBranchNotAllowed)) - - // Cleanup - Expect(k8sClient.Delete(ctx, gitTarget)).Should(Succeed()) - Expect(k8sClient.Delete(ctx, gitProvider)).Should(Succeed()) - }) - - It("Should populate status fields when branch IS allowed", func() { - ctx := context.Background() - - // Create a GitProvider with wildcard pattern - gitProvider := &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-provider-allowed", - Namespace: "default", - }, - Spec: configbutleraiv1alpha3.GitProviderSpec{ - URL: "https://github.com/test-org/test-repo.git", - AllowedBranches: []string{"main", "feature/*"}, - SecretRef: &meta.LocalObjectReference{ - Name: "test-secret", - }, - }, - } - Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) - - // Create a GitTarget with an ALLOWED branch - gitTarget := &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-target-allowed", - Namespace: "default", - }, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-allowed", - }, - Branch: "feature/allowed", - Path: "allowed-folder", + GitBranchRef: &meta.NamespacedObjectReference{Name: "missing-branch"}, + Path: "test-folder", }, } Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) - // Wait for reconciliation - gitTargetLookupKey := types.NamespacedName{ - Name: "test-target-allowed", - Namespace: "default", - } - - createdGitTarget := &configbutleraiv1alpha3.GitTarget{} - - // Wait for the controller to reconcile - Eventually(func() bool { - err := k8sClient.Get(ctx, gitTargetLookupKey, createdGitTarget) - if err != nil { - return false - } - // Check if Ready condition exists - for _, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionReady { - return true - } - } - return false - }, timeout, interval).Should(BeTrue()) - - // When branch IS allowed, the Ready condition should eventually be True - // (may be False initially if repo is not accessible, but that's expected) - // The key point is that sensitive fields are NOT cleared - var readyCondition *metav1.Condition - for i, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionReady { - readyCondition = &createdGitTarget.Status.Conditions[i] - break - } - } - Expect(readyCondition).NotTo(BeNil(), "Ready condition should exist") - - // If branch is allowed but repo is not accessible, reason should be RepositoryUnavailable - // NOT BranchNotAllowed - if readyCondition.Status == metav1.ConditionFalse { - Expect(readyCondition.Reason).NotTo(Equal(GitTargetReasonBranchNotAllowed), - "When branch is allowed, reason should not be BranchNotAllowed") - } - - // The key security verification: when branch IS allowed (even if repo unavailable), - // the controller attempts to populate status fields and does NOT clear them - // (they may be empty due to repo inaccessibility, but won't be explicitly cleared) + key := types.NamespacedName{Name: "test-target-missing-branch", Namespace: "default"} + Eventually(func(g Gomega) { + var got configbutleraiv1alpha3.GitTarget + g.Expect(k8sClient.Get(ctx, key, &got)).To(Succeed()) + validated := findCondition(got.Status.Conditions, GitTargetConditionValidated) + g.Expect(validated).NotTo(BeNil()) + g.Expect(validated.Status).To(Equal(metav1.ConditionFalse)) + g.Expect(validated.Reason).To(Equal(GitTargetReasonGitBranchNotFound)) + g.Expect(validated.Message). + To(ContainSubstring("Referenced GitBranch 'default/missing-branch' not found")) + ready := findCondition(got.Status.Conditions, GitTargetConditionReady) + g.Expect(ready).NotTo(BeNil()) + g.Expect(ready.Status).To(Equal(metav1.ConditionFalse)) + g.Expect(ready.Reason).To(Equal(GitTargetReadyReasonValidationFailed)) + }, timeout, interval).Should(Succeed()) - // Cleanup Expect(k8sClient.Delete(ctx, gitTarget)).Should(Succeed()) - Expect(k8sClient.Delete(ctx, gitProvider)).Should(Succeed()) }) - It("Should surface kstatus and data-plane status without EventStreamLive", func() { + It("Should wait for a GitBranch that does not hold its branch, without stalling", func() { ctx := context.Background() gitProvider := &configbutleraiv1alpha3.GitProvider{ @@ -279,6 +77,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") gitTarget := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{ @@ -286,19 +85,18 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-two-axis", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-two-axis", "main"), }, - Branch: "main", - Path: "two-axis-folder", + Path: "two-axis-folder", }, } Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) key := types.NamespacedName{Name: "test-target-two-axis", Namespace: "default"} - // With no EventRouter wired in the suite, a validated target has no resolved streams. - // Ready stays false while Reconciling carries progress, and the retired EventStreamLive - // condition / snapshot stay absent. + // No GitBranch reconciler runs in this suite, so the GitBranch never proves itself and holds + // nothing. That clears on its own, so the target is progressing, not stalled, and the + // retired EventStreamLive condition stays absent. Eventually(func(g Gomega) { var got configbutleraiv1alpha3.GitTarget g.Expect(k8sClient.Get(ctx, key, &got)).To(Succeed()) @@ -306,6 +104,9 @@ var _ = Describe("GitTarget Controller Security", func() { g.Expect(conditionIsTrue(got.Status.Conditions, GitTargetConditionReconciling)).To(BeTrue()) g.Expect(conditionIsTrue(got.Status.Conditions, GitTargetConditionStalled)).To(BeFalse()) g.Expect(conditionIsTrue(got.Status.Conditions, GitTargetConditionStreamsRunning)).To(BeFalse()) + branchReady := findCondition(got.Status.Conditions, GitTargetConditionGitBranchReady) + g.Expect(branchReady).NotTo(BeNil()) + g.Expect(branchReady.Reason).To(Equal(GitTargetReasonGitBranchNotReady)) for _, c := range got.Status.Conditions { g.Expect(c.Type).NotTo(Equal("EventStreamLive"), "EventStreamLive condition was removed") } @@ -314,120 +115,6 @@ var _ = Describe("GitTarget Controller Security", func() { Expect(k8sClient.Delete(ctx, gitTarget)).Should(Succeed()) Expect(k8sClient.Delete(ctx, gitProvider)).Should(Succeed()) }) - - It("Should support glob patterns in allowedBranches", func() { - ctx := context.Background() - - // Create a GitProvider with various glob patterns - gitProvider := &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-provider-glob", - Namespace: "default", - }, - Spec: configbutleraiv1alpha3.GitProviderSpec{ - URL: "https://github.com/test-org/test-repo.git", - AllowedBranches: []string{ - "main", - "develop", - "feature/*", - "release/v*", - }, - SecretRef: &meta.LocalObjectReference{ - Name: "test-secret", - }, - }, - } - Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) - - // Test cases for different branches - testCases := []struct { - branch string - shouldBeAllowed bool - }{ - {"main", true}, - {"develop", true}, - {"feature/login", true}, - {"feature/payment", true}, - {"release/v1.0", true}, - {"release/v2.5", true}, - {"hotfix/urgent", false}, - {"staging", false}, - } - - for i, tc := range testCases { - // Generate a valid K8s name (no slashes or special chars) - targetName := "test-target-glob-" + string(rune('a'+i)) - - gitTarget := &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{ - Name: targetName, - Namespace: "default", - }, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-glob", - }, - Branch: tc.branch, - Path: "glob-test", - }, - } - - Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) - - // Wait for reconciliation - gitTargetLookupKey := types.NamespacedName{ - Name: targetName, - Namespace: "default", - } - - createdGitTarget := &configbutleraiv1alpha3.GitTarget{} - - Eventually(func() bool { - err := k8sClient.Get(ctx, gitTargetLookupKey, createdGitTarget) - if err != nil { - return false - } - for _, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionReady { - return true - } - } - return false - }, timeout, interval).Should(BeTrue()) - - // Verify the condition based on whether branch should be allowed - var readyCondition *metav1.Condition - for i, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionReady { - readyCondition = &createdGitTarget.Status.Conditions[i] - break - } - } - - if !tc.shouldBeAllowed { - Expect(readyCondition.Status).To(Equal(metav1.ConditionFalse)) - Expect(readyCondition.Reason).To(Equal(GitTargetReadyReasonValidationFailed)) - var validatedCondition *metav1.Condition - for i, condition := range createdGitTarget.Status.Conditions { - if condition.Type == GitTargetConditionValidated { - validatedCondition = &createdGitTarget.Status.Conditions[i] - break - } - } - Expect(validatedCondition).NotTo(BeNil()) - Expect(validatedCondition.Reason).To(Equal(GitTargetReasonBranchNotAllowed)) - } else { - // If allowed, reason should not be BranchNotAllowed - Expect(readyCondition.Reason).NotTo(Equal(GitTargetReasonBranchNotAllowed)) - } - - // Cleanup - Expect(k8sClient.Delete(ctx, gitTarget)).Should(Succeed()) - } - - // Cleanup - Expect(k8sClient.Delete(ctx, gitProvider)).Should(Succeed()) - }) }) Context("When checking for conflicts during reconciliation loop", func() { @@ -449,6 +136,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") // Create first GitTarget (winner - created first) firstTarget := &configbutleraiv1alpha3.GitTarget{ @@ -457,11 +145,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-conflict", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-conflict", "main"), }, - Branch: "main", - Path: "conflict-folder", + Path: "conflict-folder", }, } Expect(k8sClient.Create(ctx, firstTarget)).Should(Succeed()) @@ -492,11 +179,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-conflict", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-conflict", "main"), }, - Branch: "main", - Path: "conflict-folder", + Path: "conflict-folder", }, } Expect(k8sClient.Create(ctx, secondTarget)).Should(Succeed()) @@ -568,6 +254,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") // Create first GitTarget firstTarget := &configbutleraiv1alpha3.GitTarget{ @@ -576,11 +263,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-no-conflict", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-no-conflict", "main"), }, - Branch: "main", - Path: "folder-a", + Path: "folder-a", }, } Expect(k8sClient.Create(ctx, firstTarget)).Should(Succeed()) @@ -592,11 +278,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-no-conflict", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-no-conflict", "main"), }, - Branch: "main", - Path: "folder-b", // Different! + Path: "folder-b", // Different! }, } Expect(k8sClient.Create(ctx, secondTarget)).Should(Succeed()) @@ -656,6 +341,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") // First GitTarget owns the parent folder "team" (winner - created first) firstTarget := &configbutleraiv1alpha3.GitTarget{ @@ -664,11 +350,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-nested", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-nested", "main"), }, - Branch: "main", - Path: "team", + Path: "team", }, } Expect(k8sClient.Create(ctx, firstTarget)).Should(Succeed()) @@ -698,11 +383,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-nested", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-nested", "main"), }, - Branch: "main", - Path: "team/app", + Path: "team/app", }, } Expect(k8sClient.Create(ctx, secondTarget)).Should(Succeed()) @@ -771,6 +455,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{ @@ -778,11 +463,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-generate-enc-secret", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-generate-enc-secret", "main"), }, - Branch: "main", - Path: "test-path", + Path: "test-path", Encryption: &configbutleraiv1alpha3.EncryptionSpec{ Provider: "sops", SecretRef: meta.LocalObjectReference{ @@ -836,6 +520,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{ @@ -843,11 +528,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-no-generate-enc-secret", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-no-generate-enc-secret", "main"), }, - Branch: "main", - Path: "test-path", + Path: "test-path", Encryption: &configbutleraiv1alpha3.EncryptionSpec{ Provider: "sops", SecretRef: meta.LocalObjectReference{ @@ -913,6 +597,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{ @@ -920,11 +605,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-no-enc-recipients", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-no-enc-recipients", "main"), }, - Branch: "main", - Path: "test-path", + Path: "test-path", Encryption: &configbutleraiv1alpha3.EncryptionSpec{ Provider: "sops", SecretRef: meta.LocalObjectReference{ @@ -994,6 +678,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") seedSecret := &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ @@ -1013,11 +698,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-update-enc-secret", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-update-enc-secret", "main"), }, - Branch: "main", - Path: "test-path", + Path: "test-path", Encryption: &configbutleraiv1alpha3.EncryptionSpec{ Provider: "sops", SecretRef: meta.LocalObjectReference{ @@ -1066,6 +750,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{ @@ -1073,11 +758,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-recreate-enc-secret", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-recreate-enc-secret", "main"), }, - Branch: "main", - Path: "test-path", + Path: "test-path", Encryption: &configbutleraiv1alpha3.EncryptionSpec{ Provider: "sops", SecretRef: meta.LocalObjectReference{ @@ -1145,6 +829,7 @@ var _ = Describe("GitTarget Controller Security", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) + createGitBranchFor(ctx, gitProvider, "main") existingIdentity, err := age.GenerateX25519Identity() Expect(err).NotTo(HaveOccurred()) @@ -1168,11 +853,10 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider-existing-agekey", + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("test-provider-existing-agekey", "main"), }, - Branch: "main", - Path: "test-path", + Path: "test-path", Encryption: &configbutleraiv1alpha3.EncryptionSpec{ Provider: "sops", SecretRef: meta.LocalObjectReference{ @@ -1208,6 +892,35 @@ var _ = Describe("GitTarget Controller Security", func() { }) }) +// createGitBranchFor creates the GitBranch "-" writing branch through provider, in +// the provider's namespace, and deletes it when the spec ends. +func createGitBranchFor(ctx context.Context, provider *configbutleraiv1alpha3.GitProvider, branch string) { + gitBranch := &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{ + Name: gitBranchNameForTest(provider.Name, branch), Namespace: provider.Namespace, + }, + Spec: configbutleraiv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: provider.Name}, Branch: branch, + }, + } + Expect(k8sClient.Create(ctx, gitBranch)).Should(Succeed()) + DeferCleanup(func() { _ = k8sClient.Delete(context.Background(), gitBranch) }) +} + +// holdGitBranchForTest creates the GitBranch "-" and makes it hold its repository +// branch, as its reconcile would once its credential is proved. No GitBranch reconciler runs in +// this suite. +func holdGitBranchForTest(ctx context.Context, provider *configbutleraiv1alpha3.GitProvider, branch string) { + createGitBranchFor(ctx, provider, branch) + stored := &configbutleraiv1alpha3.GitBranch{} + key := types.NamespacedName{Namespace: provider.Namespace, Name: gitBranchNameForTest(provider.Name, branch)} + Expect(k8sClient.Get(ctx, key, stored)).To(Succeed()) + testWorkerManager.Claims().Observe(git.BranchKeyFor(provider.Spec.URL, branch), git.BranchClaimant{ + GitBranch: key, UID: stored.UID, Created: stored.CreationTimestamp.Time, + }, true) + DeferCleanup(func() { testWorkerManager.Claims().Release(stored.UID) }) +} + func findFirstAgeKeyEntry(data map[string][]byte) (string, []byte) { for key, value := range data { if strings.HasSuffix(key, ".agekey") { diff --git a/internal/controller/gittarget_controller_unit_test.go b/internal/controller/gittarget_controller_unit_test.go index f97852ea..4f00b2b7 100644 --- a/internal/controller/gittarget_controller_unit_test.go +++ b/internal/controller/gittarget_controller_unit_test.go @@ -94,13 +94,12 @@ func TestCheckForConflicts_ListErrorFailsClosed(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "target-a", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "provider-a"}, - Branch: "main", - Path: "apps", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("provider-a", "main")}, + Path: "apps", }, } - conflict, _, _, err := reconciler.checkForConflicts(context.Background(), target, target.Namespace) + conflict, _, _, err := reconciler.checkForConflicts(context.Background(), target) require.Error(t, err) assert.Contains(t, err.Error(), "list GitTargets for conflict validation") diff --git a/internal/controller/gittarget_gitbranch_test.go b/internal/controller/gittarget_gitbranch_test.go new file mode 100644 index 00000000..421bcc99 --- /dev/null +++ b/internal/controller/gittarget_gitbranch_test.go @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "testing" + "time" + + meta "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + k8stypes "k8s.io/apimachinery/pkg/types" + clientgoscheme "k8s.io/client-go/kubernetes/scheme" + ctrl "sigs.k8s.io/controller-runtime" + ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + "sigs.k8s.io/controller-runtime/pkg/event" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/types" + "github.com/ConfigButler/gitops-reverser/internal/watch" +) + +// destinationFixture is a GitTarget team-a/apps on the GitBranch platform/main, which team-a may +// reference through the GitProvider platform/acme's accessFrom, with a started worker manager and a +// router with no watch plane. +type destinationFixture struct { + r *GitTargetReconciler + client ctrlclient.Client + workers *git.WorkerManager + branch *configbutleraiv1alpha3.GitBranch +} + +func newDestinationFixture(t *testing.T) *destinationFixture { + t.Helper() + scheme := runtime.NewScheme() + require.NoError(t, clientgoscheme.AddToScheme(scheme)) + require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) + provider := branchProvider("platform", "acme") + provider.Spec.AccessFrom = &configbutleraiv1alpha3.NamespaceMatcher{Names: []string{"team-a"}} + branch := branchObject("platform", "main", "acme", "main", time.Hour) + target := &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}, + Spec: configbutleraiv1alpha3.GitTargetSpec{ + GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, + Path: "apps", + }, + } + c := fake.NewClientBuilder().WithScheme(scheme). + WithObjects(provider, branch, target, admitting(configbutleraiv1alpha3.DefaultClusterProviderName, "team-a")). + WithStatusSubresource(&configbutleraiv1alpha3.GitTarget{}).Build() + workers := git.NewWorkerManager(c, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + go func() { _ = workers.Start(ctx) }() + time.Sleep(50 * time.Millisecond) + return &destinationFixture{ + r: &GitTargetReconciler{ + Client: c, WorkerManager: workers, EventRouter: watch.NewEventRouter(workers, nil, c, logr.Discard()), + }, + client: c, workers: workers, branch: branch, + } +} + +func (f *destinationFixture) reconcile(t *testing.T) (ctrl.Result, *configbutleraiv1alpha3.GitTarget) { + t.Helper() + key := k8stypes.NamespacedName{Namespace: "team-a", Name: "apps"} + result, err := f.r.Reconcile(context.Background(), ctrl.Request{NamespacedName: key}) + require.NoError(t, err) + got := &configbutleraiv1alpha3.GitTarget{} + require.NoError(t, f.client.Get(context.Background(), key, got)) + return result, got +} + +// TestReconcile_AGitBranchThatHoldsNothingKeepsTheTargetWaiting: until its GitBranch holds the +// branch, a GitTarget gets no worker and reports GitBranchNotReady as progress, coming back on the +// not-Ready cadence. +func TestReconcile_AGitBranchThatHoldsNothingKeepsTheTargetWaiting(t *testing.T) { + f := newDestinationFixture(t) + + result, got := f.reconcile(t) + + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + assert.Equal(t, GitTargetReasonGitBranchNotReady, ready.Reason) + assert.False(t, conditionIsTrue(got.Status.Conditions, ConditionTypeStalled)) + assert.Equal(t, RequeueStreamSettleInterval, result.RequeueAfter) + _, wired := f.workers.WorkerForTarget(got) + assert.False(t, wired) +} + +// TestReconcile_ATargetOnAHeldGitBranchInAnotherNamespaceIsWired: the GitBranch holds the branch and +// its GitProvider admits the target's namespace, so the target is wired to the holder's worker with +// that provider's identity. +func TestReconcile_ATargetOnAHeldGitBranchInAnotherNamespaceIsWired(t *testing.T) { + f := newDestinationFixture(t) + f.workers.Claims().Observe(git.BranchKeyFor(gitBranchTestURL, "main"), git.BranchClaimant{ + GitBranch: ctrlclient.ObjectKeyFromObject( + f.branch, + ), + UID: f.branch.UID, + Created: f.branch.CreationTimestamp.Time, + }, true) + + _, got := f.reconcile(t) + + branchReady := apimeta.FindStatusCondition(got.Status.Conditions, GitTargetConditionGitBranchReady) + require.NotNil(t, branchReady) + assert.Equal(t, metav1.ConditionTrue, branchReady.Status, branchReady.Message) + worker, wired := f.workers.WorkerForTarget(got) + require.True(t, wired) + assert.Equal(t, "platform", worker.GitProviderNamespace) + assert.Equal(t, "acme", worker.GitProviderRef) + assert.Equal(t, "main", worker.Branch) +} + +// TestReconcile_AGitBranchWhoseProviderDoesNotAdmitTheNamespace: the cross-namespace reference is +// decided by the branch's GitProvider, against the TARGET's namespace. +func TestReconcile_AGitBranchWhoseProviderDoesNotAdmitTheNamespace(t *testing.T) { + f := newDestinationFixture(t) + provider := &configbutleraiv1alpha3.GitProvider{} + require.NoError(t, f.client.Get(context.Background(), k8stypes.NamespacedName{Namespace: "platform", Name: "acme"}, + provider)) + provider.Spec.AccessFrom = nil + require.NoError(t, f.client.Update(context.Background(), provider)) + + _, got := f.reconcile(t) + + validated := apimeta.FindStatusCondition(got.Status.Conditions, GitTargetConditionValidated) + require.NotNil(t, validated) + assert.Equal(t, GitTargetReasonNamespaceNotAuthorized, validated.Reason) + assert.Contains(t, validated.Message, "GitBranch platform/main") +} + +// TestGitBranchWatches_MapToWhatTheyChange: a GitProvider event reaches the GitBranches in its +// namespace that name it; a GitTarget event reaches the GitBranch it references, from any namespace. +func TestGitBranchWatches_MapToWhatTheyChange(t *testing.T) { + scheme := runtime.NewScheme() + require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects( + branchObject("platform", "main", "acme", "main", time.Hour), + branchObject("platform", "release", "acme", "release", time.Hour), + branchObject("platform", "other", "mirror", "main", time.Hour), + branchObject("team-b", "main", "acme", "main", time.Hour), + ).Build() + r := &GitBranchReconciler{Client: c} + + assert.Equal(t, []string{"platform/main", "platform/release"}, + requestsToKeys(r.gitProviderToGitBranches(context.Background(), branchProvider("platform", "acme")))) + + target := &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}, + Spec: configbutleraiv1alpha3.GitTargetSpec{ + GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, + }, + } + assert.Equal(t, []string{"platform/main"}, requestsToKeys(gitTargetToGitBranch(context.Background(), target))) + target.Spec.GitBranchRef = nil + assert.Empty(t, gitTargetToGitBranch(context.Background(), target)) + assert.Empty(t, gitTargetToGitBranch(context.Background(), branchProvider("platform", "acme"))) +} + +// TestGitBranchReadyOrSpecChanged: a GitTarget is woken when its GitBranch appears, goes, or its +// Ready changes; a status.remote write alone, which moves on every push, is filtered. +func TestGitBranchReadyOrSpecChanged(t *testing.T) { + p := gitBranchReadyOrSpecChanged() + ready := func(status metav1.ConditionStatus, reason string) *configbutleraiv1alpha3.GitBranch { + b := branchObject("platform", "main", "acme", "main", time.Hour) + b.Status.Conditions = []metav1.Condition{{Type: ConditionTypeReady, Status: status, Reason: reason}} + return b + } + held := ready(metav1.ConditionTrue, ReasonSucceeded) + pushed := held.DeepCopy() + pushed.Status.Remote = &configbutleraiv1alpha3.GitBranchRemoteStatus{Commit: "abc"} + + assert.True(t, p.Create(event.CreateEvent{Object: held})) + assert.True(t, p.Delete(event.DeleteEvent{Object: held})) + assert.False(t, p.Generic(event.GenericEvent{Object: held})) + assert.False(t, p.Update(event.UpdateEvent{ObjectOld: held, ObjectNew: pushed}), "status.remote alone") + assert.True(t, p.Update(event.UpdateEvent{ObjectOld: held, + ObjectNew: ready(metav1.ConditionFalse, GitBranchReasonPushSessionFailed)}), "Ready went False") + assert.True(t, p.Update(event.UpdateEvent{ObjectOld: ready(metav1.ConditionFalse, GitBranchReasonPushSessionFailed), + ObjectNew: ready(metav1.ConditionFalse, GitBranchReasonBranchConflict)}), "the reason changed") + assert.True(t, p.Update(event.UpdateEvent{ObjectOld: held, ObjectNew: branchProvider("platform", "acme")}), + "an object it cannot read is let through") +} + +// TestGitTargetsOn_FindsTheTargetsOfABranchOrItsProvider is the shared mapping behind the GitTarget +// and apply-trigger watches on GitBranch and GitProvider. +func TestGitTargetsOn_FindsTheTargetsOfABranchOrItsProvider(t *testing.T) { + scheme := runtime.NewScheme() + require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) + target := func(namespace, name, branchNS, branch string) *configbutleraiv1alpha3.GitTarget { + return &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: namespace, Name: name}, + Spec: configbutleraiv1alpha3.GitTargetSpec{ + GitBranchRef: &meta.NamespacedObjectReference{Name: branch, Namespace: branchNS}, + }, + } + } + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects( + branchObject("platform", "main", "acme", "main", time.Hour), + target("platform", "infra", "", "main"), target("team-a", "apps", "platform", "main"), + target("team-a", "own", "", "main"), + ).Build() + + byBranch, err := gitTargetsOn(context.Background(), c, branchObject("platform", "main", "acme", "main", 0)) + require.NoError(t, err) + assert.Len(t, byBranch, 2) + byProvider, err := gitTargetsOn(context.Background(), c, branchProvider("platform", "acme")) + require.NoError(t, err) + assert.Len(t, byProvider, 2) + none, err := gitTargetsOn(context.Background(), c, branchProvider("platform", "unused")) + require.NoError(t, err) + assert.Empty(t, none) +} diff --git a/internal/controller/gittarget_immutability_test.go b/internal/controller/gittarget_immutability_test.go index bdb0a66c..49ca194e 100644 --- a/internal/controller/gittarget_immutability_test.go +++ b/internal/controller/gittarget_immutability_test.go @@ -12,32 +12,44 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// A GitTarget's destination — gitProviderRef, branch, path — is immutable: changing where -// it materializes would orphan the old materialization, so the API server rejects the -// change (CEL transition rules) and a relocation is a delete + recreate. This replaces -// the alternative of reconciling a destination move (which would need a generation-aware -// snapshot gate and worker rebinding); making it immutable removes that whole class of -// bug instead of handling it. +// A GitTarget's destination — gitBranchRef and path — is immutable: changing where it materializes +// would orphan the old materialization, so the API server rejects the change (CEL transition rules) +// and a relocation is a delete + recreate. A GitBranch's spec is the repository branch it claims, so +// all of it is immutable the same way. var _ = Describe("GitTarget Destination Immutability", func() { const ( timeout = time.Second * 10 interval = time.Millisecond * 250 ) - It("rejects changes to gitProviderRef, branch, and path but allows a no-op update", func() { + // expectImmutable loops past any optimistic-lock conflict from a concurrent status write, so + // the assertion lands on the real immutability rejection, not a transient 409. + expectImmutable := func( + ctx context.Context, key types.NamespacedName, obj client.Object, mutate func(), wantMsg string, + ) { + Eventually(func(g Gomega) { + g.Expect(k8sClient.Get(ctx, key, obj)).To(Succeed()) + mutate() + err := k8sClient.Update(ctx, obj) + g.Expect(err).To(HaveOccurred()) + g.Expect(err.Error()).To(ContainSubstring(wantMsg)) + }, timeout, interval).Should(Succeed()) + } + + It("rejects changes to gitBranchRef and path but allows a no-op update", func() { ctx := context.Background() key := types.NamespacedName{Name: "immutable-target", Namespace: "default"} gitTarget := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: key.Name, Namespace: key.Namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "prov-a"}, - Branch: "main", - Path: "apps", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov-a", "main")}, + Path: "apps", }, } Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) @@ -51,82 +63,57 @@ var _ = Describe("GitTarget Destination Immutability", func() { g.Expect(k8sClient.Update(ctx, current)).To(Succeed()) }, timeout, interval).Should(Succeed()) - // Each destination field is immutable. Eventually loops past any optimistic-lock - // conflict from a concurrent status write so the assertion lands on the real - // immutability rejection, not a transient 409. - expectImmutable := func(mutate func(*configbutleraiv1alpha3.GitTarget), wantMsg string) { - Eventually(func(g Gomega) { - current := &configbutleraiv1alpha3.GitTarget{} - g.Expect(k8sClient.Get(ctx, key, current)).To(Succeed()) - mutate(current) - err := k8sClient.Update(ctx, current) - g.Expect(err).To(HaveOccurred()) - g.Expect(err.Error()).To(ContainSubstring(wantMsg)) - }, timeout, interval).Should(Succeed()) - } - - expectImmutable(func(gt *configbutleraiv1alpha3.GitTarget) { - gt.Spec.Path = "moved" - }, "spec.path is immutable") - expectImmutable(func(gt *configbutleraiv1alpha3.GitTarget) { - gt.Spec.Branch = "develop" - }, "spec.branch is immutable") - expectImmutable(func(gt *configbutleraiv1alpha3.GitTarget) { - gt.Spec.GitProviderRef.Name = "prov-b" - }, "spec.gitProviderRef is immutable") + current := &configbutleraiv1alpha3.GitTarget{} + expectImmutable(ctx, key, current, func() { current.Spec.Path = "moved" }, "spec.path is immutable") + expectImmutable(ctx, key, current, func() { current.Spec.GitBranchRef.Name = "other" }, + "spec.gitBranchRef is immutable") + expectImmutable(ctx, key, current, func() { current.Spec.GitBranchRef.Namespace = "platform" }, + "spec.gitBranchRef is immutable") }) - // spec.parentBranch decides what history a new write branch gets, so it is fixed at creation - // like the destination, including adding or removing it. It can be omitted, but never empty. - It("rejects changing, adding or removing spec.parentBranch, and an empty one", func() { + It("rejects any change to a GitBranch's spec, including adding or removing parentBranch", func() { ctx := context.Background() - withParent := types.NamespacedName{Name: "parent-branch-target", Namespace: "default"} - withoutParent := types.NamespacedName{Name: "no-parent-branch-target", Namespace: "default"} - + withParent := types.NamespacedName{Name: "immutable-branch-parent", Namespace: "default"} + withoutParent := types.NamespacedName{Name: "immutable-branch", Namespace: "default"} for _, tc := range []struct { key types.NamespacedName parent string }{{withParent, "main"}, {withoutParent, ""}} { - gitTarget := &configbutleraiv1alpha3.GitTarget{ + gitBranch := &configbutleraiv1alpha3.GitBranch{ ObjectMeta: metav1.ObjectMeta{Name: tc.key.Name, Namespace: tc.key.Namespace}, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "prov-a"}, + Spec: configbutleraiv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: "prov-a"}, Branch: "edits", - Path: "apps-" + tc.key.Name, ParentBranch: tc.parent, }, } - Expect(k8sClient.Create(ctx, gitTarget)).Should(Succeed()) - DeferCleanup(func() { _ = k8sClient.Delete(ctx, gitTarget) }) + Expect(k8sClient.Create(ctx, gitBranch)).Should(Succeed()) + DeferCleanup(func() { _ = k8sClient.Delete(ctx, gitBranch) }) } - expectRejected := func(key types.NamespacedName, parent string) { - Eventually(func(g Gomega) { - current := &configbutleraiv1alpha3.GitTarget{} - g.Expect(k8sClient.Get(ctx, key, current)).To(Succeed()) - current.Spec.ParentBranch = parent - err := k8sClient.Update(ctx, current) - g.Expect(err).To(HaveOccurred()) - g.Expect(err.Error()).To(ContainSubstring("spec.parentBranch is immutable")) - }, timeout, interval).Should(Succeed()) - } - expectRejected(withParent, "release") // changed - expectRejected(withParent, "") // removed - expectRejected(withoutParent, "release") // added + current := &configbutleraiv1alpha3.GitBranch{} + expectImmutable(ctx, withParent, current, func() { current.Spec.GitProviderRef.Name = "prov-b" }, + "spec.gitProviderRef is immutable") + expectImmutable(ctx, withParent, current, func() { current.Spec.Branch = "other" }, "spec.branch is immutable") + expectImmutable(ctx, withParent, current, func() { current.Spec.ParentBranch = "release" }, + "spec.parentBranch is immutable") + expectImmutable(ctx, withParent, current, func() { current.Spec.ParentBranch = "" }, + "spec.parentBranch is immutable") + expectImmutable(ctx, withoutParent, current, func() { current.Spec.ParentBranch = "release" }, + "spec.parentBranch is immutable") empty := &unstructured.Unstructured{Object: map[string]interface{}{ "apiVersion": "configbutler.ai/v1alpha3", - "kind": "GitTarget", - "metadata": map[string]interface{}{"name": "empty-parent-target", "namespace": "default"}, + "kind": "GitBranch", + "metadata": map[string]interface{}{"name": "empty-parent-branch", "namespace": "default"}, "spec": map[string]interface{}{ "gitProviderRef": map[string]interface{}{"name": "prov-a"}, "branch": "edits", - "path": "apps", "parentBranch": "", }, }} err := k8sClient.Create(ctx, empty) - Expect(err).To(HaveOccurred()) + Expect(err).To(HaveOccurred(), "parentBranch can be omitted, never empty") Expect(err.Error()).To(ContainSubstring("spec.parentBranch")) }) @@ -137,8 +124,7 @@ var _ = Describe("GitTarget Destination Immutability", func() { base := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: key.Name, Namespace: key.Namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "prov-a"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov-a", "main")}, }, } diff --git a/internal/controller/gittarget_layout_test.go b/internal/controller/gittarget_layout_test.go index 8fd9a0e2..3a96d5ed 100644 --- a/internal/controller/gittarget_layout_test.go +++ b/internal/controller/gittarget_layout_test.go @@ -206,7 +206,7 @@ func TestGitTargetReadiness_StalledFollowsGitPathAccepted(t *testing.T) { }, Render: conditionValue{Status: metav1.ConditionTrue}, }, - }, healthyDependency(), healthyDependency(), healthyDependency(), healthyDependency(), + }, healthyDependency(), healthyDependency(), healthyDependency(), healthyDependency()) trio := rd.trio() diff --git a/internal/controller/gittarget_parent_branch_test.go b/internal/controller/gittarget_parent_branch_test.go deleted file mode 100644 index 974b8139..00000000 --- a/internal/controller/gittarget_parent_branch_test.go +++ /dev/null @@ -1,302 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package controller - -import ( - "context" - "testing" - "time" - - meta "github.com/fluxcd/pkg/apis/meta" - "github.com/go-logr/logr" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime" - clientgoscheme "k8s.io/client-go/kubernetes/scheme" - "sigs.k8s.io/controller-runtime/pkg/client/fake" - - configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" - "github.com/ConfigButler/gitops-reverser/internal/git" - "github.com/ConfigButler/gitops-reverser/internal/types" - "github.com/ConfigButler/gitops-reverser/internal/watch" -) - -func parentBranchTarget(name, branch, path, parent string, created time.Time) *configbutleraiv1alpha3.GitTarget { - return &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{ - Name: name, Namespace: "shop", CreationTimestamp: metav1.NewTime(created), - }, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "repo1"}, - Branch: branch, - Path: path, - ParentBranch: parent, - }, - } -} - -func parentBranchScheme(t *testing.T) *runtime.Scheme { - t.Helper() - scheme := runtime.NewScheme() - require.NoError(t, clientgoscheme.AddToScheme(scheme)) - require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) - return scheme -} - -// TestCheckForConflicts_ParentBranch: one write branch, one checkout, one parent. Two targets on the -// same branch that name different parents conflict even on sibling paths, and the later one loses. -// Omitted is its own value, so it conflicts with an explicit "main". -func TestCheckForConflicts_ParentBranch(t *testing.T) { - earlier := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) - later := earlier.Add(time.Hour) - - cases := []struct { - name string - existing *configbutleraiv1alpha3.GitTarget - target *configbutleraiv1alpha3.GitTarget - wantConflict bool - }{ - { - name: "omitted versus main conflicts", - existing: parentBranchTarget("first", "edits", "apps/a", "", earlier), - target: parentBranchTarget("second", "edits", "apps/b", "main", later), - wantConflict: true, - }, - { - name: "the earlier target wins", - existing: parentBranchTarget("second", "edits", "apps/b", "main", later), - target: parentBranchTarget("first", "edits", "apps/a", "", earlier), - }, - { - name: "the same parent on sibling paths is fine", - existing: parentBranchTarget("first", "edits", "apps/a", "release", earlier), - target: parentBranchTarget("second", "edits", "apps/b", "release", later), - }, - { - name: "another write branch is another checkout", - existing: parentBranchTarget("first", "edits", "apps/a", "", earlier), - target: parentBranchTarget("second", "other", "apps/b", "release", later), - }, - } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - k8sClient := fake.NewClientBuilder().WithScheme(parentBranchScheme(t)).WithObjects(tc.existing).Build() - r := &GitTargetReconciler{Client: k8sClient} - - conflict, msg, reason, err := r.checkForConflicts(context.Background(), tc.target, "shop") - - require.NoError(t, err) - assert.Equal(t, tc.wantConflict, conflict, msg) - if tc.wantConflict { - assert.Equal(t, GitTargetReasonTargetConflict, reason) - assert.Contains(t, msg, "(omitted: the remote's default branch)") - assert.Contains(t, msg, "'main'") - } - }) - } -} - -func startParentBranchWorkers(t *testing.T) (*GitTargetReconciler, *git.WorkerManager) { - t.Helper() - k8sClient := fake.NewClientBuilder().WithScheme(parentBranchScheme(t)).Build() - workers := git.NewWorkerManager(k8sClient, logr.Discard(), git.BranchWorkerLimits{}, - types.SensitiveResourcePolicy{}) - ctx, cancel := context.WithCancel(context.Background()) - t.Cleanup(cancel) - go func() { _ = workers.Start(ctx) }() - time.Sleep(100 * time.Millisecond) // Start installs the context EnsureWorker starts workers under - router := watch.NewEventRouter(workers, nil, k8sClient, logr.Discard()) - return &GitTargetReconciler{Client: k8sClient, WorkerManager: workers, EventRouter: router}, workers -} - -// TestEnsureEventStream_AParentBranchChangeKeepsTheWorker: changing spec.parentBranch abandons -// nothing written, so it is not a repoint. The worker stays and re-reads the remote. -func TestEnsureEventStream_AParentBranchChangeKeepsTheWorker(t *testing.T) { - r, workers := startParentBranchWorkers(t) - repo := git.RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/repo.git"} - target := parentBranchTarget("apps", "edits", "apps", "", time.Now()) - - _, err := r.ensureEventStream(context.Background(), target, "shop", repo, logr.Discard()) - require.NoError(t, err) - before, ok := workers.GetWorkerForTarget("repo1", "shop", "edits") - require.True(t, ok) - before.SeedRemoteObservationForTest("", time.Now(), "Fetch") - - target.Spec.ParentBranch = "release" - _, err = r.ensureEventStream(context.Background(), target, "shop", repo, logr.Discard()) - require.NoError(t, err) - - after, ok := workers.GetWorkerForTarget("repo1", "shop", "edits") - require.True(t, ok) - assert.Same(t, before, after, "the worker and its clone are kept") - assert.Equal(t, "release", after.ParentBranch()) - assert.False(t, after.BaseTrustedForTest(), "and its checkout of the old parent is no longer trusted") -} - -// TestParentBranchReadiness: a fetch that found neither the write branch nor its configured parent -// stalls the GitTarget as ParentBranchNotFound, for that parent only. -func TestParentBranchReadiness(t *testing.T) { - r, workers := startParentBranchWorkers(t) - repo := git.RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/repo.git"} - target := parentBranchTarget("apps", "edits", "apps", "release", time.Now()) - - assert.Equal(t, metav1.ConditionTrue, r.parentBranchReadiness(target, repo).Status, - "nothing has looked yet") - - _, err := r.ensureEventStream(context.Background(), target, "shop", repo, logr.Discard()) - require.NoError(t, err) - worker, _ := workers.GetWorkerForTarget("repo1", "shop", "edits") - worker.RecordMissingParentForTest("release") - - got := r.parentBranchReadiness(target, repo) - assert.Equal(t, metav1.ConditionFalse, got.Status) - assert.Equal(t, GitTargetReasonParentBranchNotFound, got.Reason) - assert.Contains(t, got.Message, "'release'") - - rd := newGitTargetReadiness() - gitTargetReadinessGates(rd, dataPlaneObservation{axes: gitTargetAxes{ - Streams: conditionValue{Status: metav1.ConditionTrue}, - GitPath: conditionValue{Status: metav1.ConditionTrue}, - Render: conditionValue{Status: metav1.ConditionTrue}, - }}, got, healthyDependency(), healthyDependency(), healthyDependency(), conditionValue{}) - trio := rd.trio() - assert.Equal(t, metav1.ConditionTrue, trio.Stalled.Status, "it needs somebody to create the branch") - assert.Equal(t, GitTargetReasonParentBranchNotFound, trio.Ready.Reason) - - target.Spec.ParentBranch = "other" - assert.Equal(t, metav1.ConditionTrue, r.parentBranchReadiness(target, repo).Status, - "the observation is about another parent") - target.Spec.ParentBranch = "" - assert.Equal(t, metav1.ConditionTrue, r.parentBranchReadiness(target, repo).Status) -} - -// TestParentBranchReadiness_AnOmittedParentCanStall: with spec.parentBranch omitted, a remote that -// is not empty but whose default branch does not resolve stalls the target too, with a message -// that names what to fix. An observation made under a configured parent says nothing about it. -func TestParentBranchReadiness_AnOmittedParentCanStall(t *testing.T) { - r, workers := startParentBranchWorkers(t) - repo := git.RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/repo.git"} - target := parentBranchTarget("apps", "edits", "apps", "", time.Now()) - _, err := r.ensureEventStream(context.Background(), target, "shop", repo, logr.Discard()) - require.NoError(t, err) - worker, _ := workers.GetWorkerForTarget("repo1", "shop", "edits") - - worker.RecordUnresolvedDefaultBranchForTest("master") - got := r.parentBranchReadiness(target, repo) - assert.Equal(t, metav1.ConditionFalse, got.Status) - assert.Equal(t, GitTargetReasonParentBranchNotFound, got.Reason) - assert.Contains(t, got.Message, "HEAD -> 'master'") - assert.Contains(t, got.Message, "spec.parentBranch") - - worker.RecordUnresolvedDefaultBranchForTest("") - got = r.parentBranchReadiness(target, repo) - assert.Equal(t, GitTargetReasonParentBranchNotFound, got.Reason) - assert.Contains(t, got.Message, "no default branch") - - worker.RecordMissingParentForTest("release") - assert.Equal(t, metav1.ConditionTrue, r.parentBranchReadiness(target, repo).Status, - "an observation made under a configured parent is about another configuration") -} - -// TestParentRecovering_ReadsTheWorkerOnly: the reconciler reports recovery from the branch worker's -// memory and nothing else. No worker manager, no worker, or a worker with no pending writes is not -// recovering. -func TestParentRecovering_ReadsTheWorkerOnly(t *testing.T) { - target := parentBranchTarget("apps", "edits", "apps", "release", time.Now()) - assert.False(t, (&GitTargetReconciler{}).parentRecovering(target, "shop"), "no worker manager") - - r, _ := startParentBranchWorkers(t) - assert.False(t, r.parentRecovering(target, "shop"), "no worker for the target yet") - - repo := git.RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/repo.git"} - _, err := r.ensureEventStream(context.Background(), target, "shop", repo, logr.Discard()) - require.NoError(t, err) - assert.False(t, r.parentRecovering(target, "shop"), "a worker with no pending writes") -} - -func TestParentRecoveryReadiness_IsProgressNotAStall(t *testing.T) { - rd := newGitTargetReadiness() - parentRecoveryReadiness(rd, true) - trio := rd.trio() - assert.Equal(t, metav1.ConditionFalse, trio.Ready.Status) - assert.Equal(t, GitTargetReasonRecoveringParentBranch, trio.Ready.Reason) - assert.Equal(t, metav1.ConditionTrue, trio.Reconciling.Status) - assert.NotEqual(t, metav1.ConditionTrue, trio.Stalled.Status) - assert.Equal(t, RequeueStreamSettleInterval, gitTargetRequeue(rd), "it keeps the not-Ready requeue") - - rd = newGitTargetReadiness() - parentRecoveryReadiness(rd, false) - assert.True(t, rd.converged()) -} - -// TestCheckForConflicts_ParentBranchAppliesToInvalidPaths: a target whose path its writer will -// reject still wires the shared worker, so it must not escape the parent check in either role and -// repoint the healthy sibling's parent. -func TestCheckForConflicts_ParentBranchAppliesToInvalidPaths(t *testing.T) { - earlier := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) - healthy := parentBranchTarget("first", "edits", "apps/a", "main", earlier) - invalid := parentBranchTarget("second", "edits", "../bad", "release", earlier.Add(time.Hour)) - - r, workers := startParentBranchWorkers(t) - require.NoError(t, r.Client.Create(context.Background(), healthy)) - _, err := r.ensureEventStream(context.Background(), healthy, "shop", firstRepo, logr.Discard()) - require.NoError(t, err) - - conflict, _, reason, err := r.checkForConflicts(context.Background(), invalid, "shop") - require.NoError(t, err) - assert.True(t, conflict, "an invalid path must not bypass the parent conflict") - assert.Equal(t, GitTargetReasonTargetConflict, reason) - worker, _ := workers.GetWorkerForTarget("repo1", "shop", "edits") - assert.Equal(t, "main", worker.ParentBranch()) - - // And as the earlier object, the invalid target still wins the parent. - k8sClient := fake.NewClientBuilder().WithScheme(parentBranchScheme(t)). - WithObjects(parentBranchTarget("older", "edits", "../bad", "release", earlier)).Build() - conflict, _, _, err = (&GitTargetReconciler{Client: k8sClient}).checkForConflicts( - context.Background(), parentBranchTarget("newer", "edits", "apps/a", "main", earlier.Add(time.Hour)), "shop") - require.NoError(t, err) - assert.True(t, conflict) -} - -// TestParentBranchReadiness_IgnoresAnotherRepository: the manager keeps a branch's observation across -// a repoint, and a missing parent in the old repository says nothing about the new one. -func TestParentBranchReadiness_IgnoresAnotherRepository(t *testing.T) { - r, workers := startParentBranchWorkers(t) - target := parentBranchTarget("apps", "edits", "apps", "release", time.Now()) - _, err := r.ensureEventStream(context.Background(), target, "shop", firstRepo, logr.Discard()) - require.NoError(t, err) - worker, _ := workers.GetWorkerForTarget("repo1", "shop", "edits") - worker.RecordMissingParentForTest("release") - require.Equal(t, metav1.ConditionFalse, r.parentBranchReadiness(target, firstRepo).Status) - - _, err = r.ensureEventStream(context.Background(), target, "shop", secondRepo, logr.Discard()) - require.NoError(t, err) - assert.Equal(t, metav1.ConditionTrue, r.parentBranchReadiness(target, secondRepo).Status, - "the old repository's missing parent is not evidence about the replacement") -} - -// TestParentStatusOf: status.remote.parent is present only while the write branch is absent and -// a parent was named, and a moved parent is news even when the write branch is still absent. -func TestParentStatusOf(t *testing.T) { - absent := git.RemoteObservation{ParentState: git.ParentFound, ParentBranch: "main", ParentCommit: "aaaa"} - assert.Equal(t, &configbutleraiv1alpha3.GitBranchParentStatus{ - State: configbutleraiv1alpha3.GitBranchParentFound, Branch: "main", Commit: "aaaa", - }, parentStatusOf(absent)) - assert.Nil(t, parentStatusOf(git.RemoteObservation{Commit: "bbbb"}), - "an existing write branch does not follow its parent") - assert.Nil(t, parentStatusOf(git.RemoteObservation{}), "an observation that names no parent state") - assert.Equal(t, &configbutleraiv1alpha3.GitBranchParentStatus{State: configbutleraiv1alpha3.GitBranchParentUnborn}, - parentStatusOf(git.RemoteObservation{ParentState: git.ParentUnborn}), "an empty repository") - assert.Equal(t, &configbutleraiv1alpha3.GitBranchParentStatus{ - State: configbutleraiv1alpha3.GitBranchParentMissing, Branch: "release", - }, parentStatusOf(git.RemoteObservation{ParentState: git.ParentMissing, ParentBranch: "release"}), - "a missing parent is named with no commit") - - at := metav1.NewTime(time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)) - published := &configbutleraiv1alpha3.GitBranchRemoteStatus{LastVerifiedAt: &at, Parent: parentStatusOf(absent)} - moved := published.DeepCopy() - moved.Parent.Commit = "cccc" - assert.True(t, remoteStatusIsNews(published, moved), "the parent moved") - assert.False(t, remoteStatusIsNews(published, published.DeepCopy())) -} diff --git a/internal/controller/gittarget_parent_recovery.go b/internal/controller/gittarget_parent_recovery.go deleted file mode 100644 index 7931ae7c..00000000 --- a/internal/controller/gittarget_parent_recovery.go +++ /dev/null @@ -1,39 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package controller - -import ( - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - - configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" -) - -// GitTargetReasonRecoveringParentBranch is the Ready reason while the parent branch has been found -// again but the branch worker has not yet published the writes it kept while the parent was -// missing. It is progress, not a stall. -const GitTargetReasonRecoveringParentBranch = "RecoveringParentBranch" - -// parentRecovering reports whether the branch worker has found the parent branch again and still -// has to publish the writes it kept while the parent was missing. It reads memory only, and costs no -// connection. -func (r *GitTargetReconciler) parentRecovering( - target *configbutleraiv1alpha3.GitTarget, - providerNS string, -) bool { - if r.WorkerManager == nil { - return false - } - worker, ok := r.WorkerManager.GetWorkerForTarget(target.Spec.GitProviderRef.Name, providerNS, target.Spec.Branch) - if !ok { - return false - } - open, found := worker.ParentRecovery() - return open && found -} - -// parentRecoveryReadiness is the progress gate for a target whose parent is back but whose -// held-back work is not published yet. -func parentRecoveryReadiness(rd *readiness, recovering bool) { - rd.progressingIf(recovering, metav1.ConditionFalse, GitTargetReasonRecoveringParentBranch, - "The parent branch is on the remote again; the branch worker is publishing the work it held back") -} diff --git a/internal/controller/gittarget_placement_validation_test.go b/internal/controller/gittarget_placement_validation_test.go index f27f411b..0dd986a4 100644 --- a/internal/controller/gittarget_placement_validation_test.go +++ b/internal/controller/gittarget_placement_validation_test.go @@ -268,9 +268,8 @@ func TestEvaluateValidatedGate_InvalidPlacementPolicy(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "target-a", Namespace: ns}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "provider-a"}, - Branch: "main", - Path: "apps", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("provider-a", "main")}, + Path: "apps", Placement: &configbutleraiv1alpha3.GitTargetPlacementSpec{ Default: "{bogus}/all.yaml", }, @@ -281,7 +280,11 @@ func TestEvaluateValidatedGate_InvalidPlacementPolicy(t *testing.T) { reconciler := &GitTargetReconciler{Client: client} st := beginStatus(client, nil, target) - validated, msg, err := reconciler.evaluateValidatedGate(context.Background(), st, target, ns, provider) + branch := &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Name: gitBranchNameForTest("provider-a", "main"), Namespace: ns}, + } + dest := gitDestination{key: target.GitBranchKey(), branch: branch, provider: provider} + validated, msg, err := reconciler.evaluateValidatedGate(context.Background(), st, target, dest) require.NoError(t, err) assert.False(t, validated) diff --git a/internal/controller/gittarget_publication.go b/internal/controller/gittarget_publication.go index fe0fc7bf..e4820831 100644 --- a/internal/controller/gittarget_publication.go +++ b/internal/controller/gittarget_publication.go @@ -13,14 +13,11 @@ import ( // remote, and if not, since when and why. Every GitTarget on the branch reads the same report, // because they share the worker: a sibling's success cannot clear it, and only a landed push does. // It reads memory only, and costs no connection. -func (r *GitTargetReconciler) branchPublication( - target *configbutleraiv1alpha3.GitTarget, - providerNS string, -) git.PublicationStatus { +func (r *GitTargetReconciler) branchPublication(target *configbutleraiv1alpha3.GitTarget) git.PublicationStatus { if r.WorkerManager == nil { return git.PublicationStatus{} } - worker, ok := r.WorkerManager.GetWorkerForTarget(target.Spec.GitProviderRef.Name, providerNS, target.Spec.Branch) + worker, ok := r.WorkerManager.WorkerForTarget(target) if !ok { return git.PublicationStatus{} } diff --git a/internal/controller/gittarget_publication_test.go b/internal/controller/gittarget_publication_test.go index 081b91a6..2b7ab3fc 100644 --- a/internal/controller/gittarget_publication_test.go +++ b/internal/controller/gittarget_publication_test.go @@ -33,7 +33,7 @@ func convergedObservation() dataPlaneObservation { func publicationGates(observed dataPlaneObservation, publication git.PublicationStatus) kstatusTrio { rd := newGitTargetReadiness() - gitTargetReadinessGates(rd, observed, healthyDependency(), healthyDependency(), healthyDependency(), + gitTargetReadinessGates(rd, observed, healthyDependency(), healthyDependency(), healthyDependency(), publicationCondition(publication)) return rd.trio() } diff --git a/internal/controller/gittarget_shared_provider_test.go b/internal/controller/gittarget_shared_branch_test.go similarity index 51% rename from internal/controller/gittarget_shared_provider_test.go rename to internal/controller/gittarget_shared_branch_test.go index 09bb5b3f..708b0772 100644 --- a/internal/controller/gittarget_shared_provider_test.go +++ b/internal/controller/gittarget_shared_branch_test.go @@ -19,8 +19,8 @@ import ( configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// sharedProviderTarget is a GitTarget named name in namespace, writing path on main through the -// GitProvider providerNS/monorepo (empty providerNS: its own namespace), created age ago. +// sharedProviderTarget is a GitTarget named name in namespace, writing path through the GitBranch +// providerNS/monorepo-main (empty providerNS: its own namespace), created age ago. func sharedProviderTarget( namespace, name, providerNS, path string, age time.Duration, @@ -31,9 +31,11 @@ func sharedProviderTarget( CreationTimestamp: metav1.NewTime(time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC).Add(-age)), }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Namespace: providerNS, Name: "monorepo"}, - Branch: "main", - Path: path, + GitBranchRef: &meta.NamespacedObjectReference{ + Name: gitBranchNameForTest("monorepo", "main"), + Namespace: providerNS, + }, + Path: path, }, } } @@ -43,7 +45,7 @@ func sharedProviderReconciler(t *testing.T, objects ...ctrlclient.Object) *GitTa scheme := runtime.NewScheme() require.NoError(t, clientgoscheme.AddToScheme(scheme)) require.NoError(t, configbutleraiv1alpha3.AddToScheme(scheme)) - // The chart's default ClusterProvider admits every namespace; only the GitProviders decide here. + // The chart's default ClusterProvider admits every namespace. objects = append(objects, &configbutleraiv1alpha3.ClusterProvider{ ObjectMeta: metav1.ObjectMeta{Name: configbutleraiv1alpha3.DefaultClusterProviderName}, Spec: configbutleraiv1alpha3.ClusterProviderSpec{ @@ -59,55 +61,22 @@ func sharedProviderReconciler(t *testing.T, objects ...ctrlclient.Object) *GitTa return &GitTargetReconciler{Client: fake.NewClientBuilder().WithScheme(scheme).WithObjects(objects...).Build()} } -// TestCheckForConflicts_ARepositoryBranchHasOneGitProvider: one worker writes a repository branch, -// with one GitProvider's credential. A GitTarget writing it through another GitProvider than the -// oldest GitTarget's is refused, and the message says how to share the owner's instead. -func TestCheckForConflicts_ARepositoryBranchHasOneGitProvider(t *testing.T) { - owner := sharedProviderTarget("platform", "apps", "", "apps", 2*time.Hour) - late := sharedProviderTarget("team-b", "infra", "", "infra", time.Hour) - r := sharedProviderReconciler(t, owner, late) - - conflict, msg, reason, err := r.checkForConflicts(context.Background(), late, "team-b") - require.NoError(t, err) - assert.True(t, conflict) - assert.Equal(t, GitTargetReasonTargetConflict, reason) - assert.Contains(t, msg, "GitProvider 'platform/monorepo'") - assert.Contains(t, msg, "spec.gitProviderRef.namespace") - - conflict, _, _, err = r.checkForConflicts(context.Background(), owner, "platform") - require.NoError(t, err) - assert.False(t, conflict, "the oldest GitTarget's GitProvider owns the branch") -} - -// TestCheckForConflicts_ASharedGitProviderIsOneBranchAcrossNamespaces: a GitTarget that shares the -// owner's GitProvider from another namespace is on the owner's branch: no ownership conflict, and -// the path rules hold between it and the owner's GitTargets as within one namespace. -func TestCheckForConflicts_ASharedGitProviderIsOneBranchAcrossNamespaces(t *testing.T) { +// TestCheckForConflicts_ASharedGitBranchIsOneBranchAcrossNamespaces: GitTargets in several +// namespaces referencing one GitBranch write one branch, and the path rules hold between them as +// within one namespace. Who may write the branch at all is the GitBranch's claim, not a conflict. +func TestCheckForConflicts_ASharedGitBranchIsOneBranchAcrossNamespaces(t *testing.T) { owner := sharedProviderTarget("platform", "apps", "", "apps", 2*time.Hour) shared := sharedProviderTarget("team-b", "infra", "platform", "infra", time.Hour) nested := sharedProviderTarget("team-c", "inside", "platform", "apps/inside", time.Minute) r := sharedProviderReconciler(t, owner, shared, nested) - conflict, msg, _, err := r.checkForConflicts(context.Background(), shared, "platform") + conflict, msg, _, err := r.checkForConflicts(context.Background(), shared) require.NoError(t, err) assert.False(t, conflict, msg) - conflict, msg, reason, err := r.checkForConflicts(context.Background(), nested, "platform") + conflict, msg, reason, err := r.checkForConflicts(context.Background(), nested) require.NoError(t, err) assert.True(t, conflict, "a path nested in another namespace's folder on the same branch still overlaps") assert.Equal(t, GitTargetReasonTargetConflict, reason) assert.Contains(t, msg, "platform/apps") } - -// TestCheckForConflicts_AGitTargetThatMayNotWriteKeepsNoBranch: an older GitTarget naming a -// GitProvider whose accessFrom does not admit its namespace may not write the branch, so it must not -// get the GitTarget that may refused. -func TestCheckForConflicts_AGitTargetThatMayNotWriteKeepsNoBranch(t *testing.T) { - refused := sharedProviderTarget("team-x", "early", "platform", "early", 2*time.Hour) - own := sharedProviderTarget("team-b", "infra", "", "infra", time.Hour) - r := sharedProviderReconciler(t, refused, own) - - conflict, msg, _, err := r.checkForConflicts(context.Background(), own, "team-b") - require.NoError(t, err) - assert.False(t, conflict, msg) -} diff --git a/internal/controller/gittarget_source_cluster.go b/internal/controller/gittarget_source_cluster.go index 8320cb44..f89bf4c5 100644 --- a/internal/controller/gittarget_source_cluster.go +++ b/internal/controller/gittarget_source_cluster.go @@ -56,18 +56,19 @@ const ( // controller reads without a network dial); an otherwise-valid kubeconfig whose API server // cannot be contacted is SourceClusterReachable=False. GitTargetConditionSourceClusterReachable = "SourceClusterReachable" - // GitTargetConditionGitProviderReady projects the referenced GitProvider's Ready onto the - // GitTarget, so one `kubectl get gittarget` separates source-side (SourceClusterReachable) - // from destination-side (GitProviderReady) failure. - GitTargetConditionGitProviderReady = "GitProviderReady" + // GitTargetConditionGitBranchReady projects the referenced GitBranch's Ready onto the GitTarget, + // and whether it holds its repository branch, so one `kubectl get gittarget` separates + // source-side (SourceClusterReachable) from destination-side (GitBranchReady) failure. + GitTargetConditionGitBranchReady = "GitBranchReady" - // GitTargetReasonGitProviderNotReady is the GitProviderReady=False reason. - GitTargetReasonGitProviderNotReady = "GitProviderNotReady" + // GitTargetReasonGitBranchNotReady is the GitBranchReady=False reason: the GitBranch exists but + // does not hold its branch, or reports itself not Ready. + GitTargetReasonGitBranchNotReady = "GitBranchNotReady" // GitTargetConditionClusterProviderReady projects the referenced ClusterProvider's Ready onto // the GitTarget, so one `kubectl get gittarget` shows whether the SOURCE cluster's provider is // healthy — distinct from SourceClusterReachable (the data plane's runtime reach) and - // GitProviderReady (the destination). It follows the GitProviderReady contract: only an + // GitBranchReady (the destination). It follows the GitBranchReady contract: only an // EXPLICIT Ready=False downgrades the GitTarget; a provider that has not reported readiness yet // is Unknown and does not. A MISSING provider never reaches this projection at all — that is the // Validated gate's hard denial (GitTargetReasonClusterProviderNotFound), not a readiness axis. @@ -148,49 +149,37 @@ func describeKubeConfigKey(specKey string) string { return specKey } -// gitProviderReadiness projects the referenced GitProvider's Ready condition. It takes the -// provider the reconcile already read — the same object the Validated gate and the branch worker's -// identity came from, so the three cannot describe different objects — and runs only after that -// gate has confirmed it exists. It returns Unknown — which does NOT downgrade Ready — when -// readiness cannot be observed (no provider in hand, or one that has not reported a Ready -// condition yet), so a not-yet-reconciled provider never blocks its GitTarget; only an EXPLICIT -// Ready=False downgrades. -func gitProviderReadiness( - target *configbutleraiv1alpha3.GitTarget, - providerNS string, - gp *configbutleraiv1alpha3.GitProvider, -) conditionValue { - key := k8stypes.NamespacedName{Name: target.Spec.GitProviderRef.Name, Namespace: providerNS} - notReady := func(status metav1.ConditionStatus, format string, args ...any) conditionValue { +// gitBranchReadiness projects the referenced GitBranch onto the GitTarget. held is whether it holds +// its repository branch now (the claim registry), which decides whether this target can be wired; +// the GitBranch's own Ready carries the reason when it cannot, and its message is repeated here. +// A GitBranch that holds its branch and reports Ready=False (its credential failing, a parent +// branch missing) is not Ready either: the target cannot publish through it. +func gitBranchReadiness(dest gitDestination, held bool) conditionValue { + notReady := func(format string, args ...any) conditionValue { return conditionValue{ - Status: status, - Reason: GitTargetReasonGitProviderNotReady, + Status: metav1.ConditionFalse, + Reason: GitTargetReasonGitBranchNotReady, Message: fmt.Sprintf(format, args...), } } - - if gp == nil { - return notReady(metav1.ConditionUnknown, "referenced GitProvider %s readiness not observed", key) + if dest.branch == nil { + return notReady("referenced GitBranch %s not found", dest.key) } - c := findCondition(gp.Status.Conditions, ConditionTypeReady) + c := findCondition(dest.branch.Status.Conditions, ConditionTypeReady) switch { - case c == nil: - return notReady(metav1.ConditionUnknown, "referenced GitProvider %s has not reported readiness yet", key) - case c.Status == metav1.ConditionTrue: - return conditionValue{ - Status: metav1.ConditionTrue, - Reason: ReasonSucceeded, - Message: fmt.Sprintf("referenced GitProvider %s is Ready", key), - } - case c.Status == metav1.ConditionFalse: - if c.Message != "" { - return notReady(metav1.ConditionFalse, "referenced GitProvider %s is not Ready: %s", key, c.Message) - } - return notReady(metav1.ConditionFalse, "referenced GitProvider %s is not Ready", key) - default: - // Ready=Unknown is the provider mid-reconcile saying it does not know yet, which is the - // case the contract above refuses to downgrade on — same rule as clusterProviderReadiness. - return notReady(metav1.ConditionUnknown, "referenced GitProvider %s readiness is unknown", key) + case !held && c != nil && c.Status == metav1.ConditionFalse && c.Message != "": + return notReady("referenced GitBranch %s does not hold its branch: %s", dest.key, c.Message) + case !held: + return notReady("referenced GitBranch %s does not hold its branch yet", dest.key) + case dest.provider == nil: + return notReady("referenced GitBranch %s names a GitProvider that does not exist", dest.key) + case c != nil && c.Status == metav1.ConditionFalse: + return notReady("referenced GitBranch %s is not Ready: %s", dest.key, c.Message) + } + return conditionValue{ + Status: metav1.ConditionTrue, + Reason: ReasonSucceeded, + Message: fmt.Sprintf("referenced GitBranch %s holds its branch", dest.key), } } diff --git a/internal/controller/gittarget_source_cluster_test.go b/internal/controller/gittarget_source_cluster_test.go index 0512bb1d..980ec939 100644 --- a/internal/controller/gittarget_source_cluster_test.go +++ b/internal/controller/gittarget_source_cluster_test.go @@ -335,15 +335,14 @@ func TestReconcile_UnauthorizedNamespaceStartsNoWatch(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "gt", Namespace: ns, UID: "gt-uid"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "gp"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("gp", "main")}, Path: "apps", ClusterProviderRef: &meta.LocalObjectReference{Name: providerName}, }, } cl := fake.NewClientBuilder().WithScheme(scScheme(t)). - WithObjects(provider, gitProvider, target, + WithObjects(provider, gitProvider, gitBranchOn("gp", ns, "main"), target, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: ns}}). WithStatusSubresource(&configbutleraiv1alpha3.GitTarget{}). Build() @@ -441,51 +440,50 @@ func TestClusterProviderReadiness_AllScenarios(t *testing.T) { } } -func TestGitProviderReadiness_AllScenarios(t *testing.T) { - provider := func(conds []metav1.Condition) *configbutleraiv1alpha3.GitProvider { - return &configbutleraiv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{Name: "prov", Namespace: "team-a"}, - Status: configbutleraiv1alpha3.GitProviderStatus{Conditions: conds}, +// TestGitBranchReadiness_AllScenarios: the projection follows the claim first (only a GitBranch that +// holds its branch can be wired) and the GitBranch's own Ready after that. +func TestGitBranchReadiness_AllScenarios(t *testing.T) { + branch := func(conds ...metav1.Condition) *configbutleraiv1alpha3.GitBranch { + return &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Name: "main", Namespace: "platform"}, + Status: configbutleraiv1alpha3.GitBranchStatus{Conditions: conds}, } } + provider := &configbutleraiv1alpha3.GitProvider{} + key := k8stypes.NamespacedName{Namespace: "platform", Name: "main"} ready := metav1.Condition{Type: ConditionTypeReady, Status: metav1.ConditionTrue, Reason: ReasonSucceeded} - notReady := metav1.Condition{ - Type: ConditionTypeReady, - Status: metav1.ConditionFalse, - Reason: "BadRepo", - Message: "no repo", - } + conflict := metav1.Condition{Type: ConditionTypeReady, Status: metav1.ConditionFalse, + Reason: GitBranchReasonBranchConflict, Message: "held by team-b/main"} + failing := metav1.Condition{Type: ConditionTypeReady, Status: metav1.ConditionFalse, + Reason: GitBranchReasonPushSessionFailed, Message: "authentication required"} tests := []struct { - name string - gp *configbutleraiv1alpha3.GitProvider - want metav1.ConditionStatus + name string + dest gitDestination + held bool + want metav1.ConditionStatus + message string }{ - {"ready", provider([]metav1.Condition{ready}), metav1.ConditionTrue}, - {"not ready -> False (downgrades)", provider([]metav1.Condition{notReady}), metav1.ConditionFalse}, - {"no condition -> Unknown (does not downgrade)", provider(nil), metav1.ConditionUnknown}, - { - "Ready=Unknown -> Unknown (provider mid-reconcile does not downgrade)", - provider([]metav1.Condition{{ - Type: ConditionTypeReady, - Status: metav1.ConditionUnknown, - Reason: ReasonProgressing, - }}), - metav1.ConditionUnknown, - }, - {"absent provider -> Unknown", nil, metav1.ConditionUnknown}, + {"holds its branch", gitDestination{key: key, branch: branch(ready), provider: provider}, true, + metav1.ConditionTrue, "holds its branch"}, + {"not found", gitDestination{key: key}, false, metav1.ConditionFalse, "not found"}, + {"holds nothing yet", gitDestination{key: key, branch: branch(), provider: provider}, false, + metav1.ConditionFalse, "does not hold its branch yet"}, + {"another holds it", gitDestination{key: key, branch: branch(conflict), provider: provider}, false, + metav1.ConditionFalse, "held by team-b/main"}, + {"holds it with a failing credential", gitDestination{key: key, branch: branch(failing), provider: provider}, + true, metav1.ConditionFalse, "authentication required"}, + {"its GitProvider is gone", gitDestination{key: key, branch: branch(ready)}, true, + metav1.ConditionFalse, "GitProvider that does not exist"}, } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - target := &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "gt", Namespace: "team-a"}, - Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "prov"}, - }, - } - got := gitProviderReadiness(target, "team-a", tc.gp) + got := gitBranchReadiness(tc.dest, tc.held) assert.Equal(t, tc.want, got.Status) - assert.NotEmpty(t, got.Message) + assert.Contains(t, got.Message, tc.message) + if tc.want == metav1.ConditionFalse { + assert.Equal(t, GitTargetReasonGitBranchNotReady, got.Reason) + } }) } } diff --git a/internal/controller/gittarget_status_test.go b/internal/controller/gittarget_status_test.go index e32e40cd..dd24e8e5 100644 --- a/internal/controller/gittarget_status_test.go +++ b/internal/controller/gittarget_status_test.go @@ -116,13 +116,13 @@ func TestGitTargetReadinessGates(t *testing.T) { want: kstatus.InProgressStatus, wantReadyReason: GitTargetReasonRenderRechecking, }, { - name: "unready GitProvider holds the target below Ready", streams: running, + name: "unready GitBranch holds the target below Ready", streams: running, gitPath: accepted, render: matches, provider: conditionValue{ - Status: metav1.ConditionFalse, Reason: GitTargetReasonGitProviderNotReady, Message: "no repo", + Status: metav1.ConditionFalse, Reason: GitTargetReasonGitBranchNotReady, Message: "no repo", }, clusterProvider: healthy, sourceReach: healthy, - want: kstatus.InProgressStatus, wantReadyReason: GitTargetReasonGitProviderNotReady, + want: kstatus.InProgressStatus, wantReadyReason: GitTargetReasonGitBranchNotReady, }, { // THE F1 REGRESSION. A refused Git path is terminal, and an unready provider is @@ -130,14 +130,14 @@ func TestGitTargetReadinessGates(t *testing.T) { // Stalled=False over the refusal, flipping the object from kstatus Failed to // InProgress — so `kubectl wait` and every CI gate built on kstatus waited out its // timeout on a target that was never going to converge. - name: "refused path AND unready provider stays Failed", streams: running, render: matches, + name: "refused path AND unready GitBranch stays Failed", streams: running, render: matches, gitPath: watch.GitPathAcceptanceStatus{ Accepted: false, Reason: GitTargetReasonUnsupportedContent, Message: "Git path refused at kustomization.yaml: uses patches", }, provider: conditionValue{ - Status: metav1.ConditionFalse, Reason: GitTargetReasonGitProviderNotReady, Message: "no repo", + Status: metav1.ConditionFalse, Reason: GitTargetReasonGitBranchNotReady, Message: "no repo", }, clusterProvider: healthy, sourceReach: conditionValue{ @@ -164,7 +164,7 @@ func TestGitTargetReadinessGates(t *testing.T) { st.setValue(GitTargetConditionRenderMatchesLive, observed.axes.Render) rd := newGitTargetReadiness() - gitTargetReadinessGates(rd, observed, conditionValue{}, tt.provider, tt.clusterProvider, tt.sourceReach, + gitTargetReadinessGates(rd, observed, tt.provider, tt.clusterProvider, tt.sourceReach, conditionValue{}) st.applyReadiness(rd) diff --git a/internal/controller/gittarget_worker_cleanup_test.go b/internal/controller/gittarget_worker_cleanup_test.go index a79fd7ec..48c44dba 100644 --- a/internal/controller/gittarget_worker_cleanup_test.go +++ b/internal/controller/gittarget_worker_cleanup_test.go @@ -8,6 +8,8 @@ import ( "testing" "time" + meta "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -45,23 +47,26 @@ func TestCleanupDeletedGitTarget_StopsAWorkerNothingNeedsAnyMore(t *testing.T) { time.Sleep(100 * time.Millisecond) func() { - err := workers.EnsureWorker(ctx, "repo1", "shop", "main", git.RepoIdentity{URL: "file:///repo1.git"}, "") + err := workers.EnsureWorker( + ctx, + claimWorkerForTest(workers, git.RepoIdentity{URL: "file:///repo1.git"}), + ) require.NoError(t, err) }() - _, exists := workers.GetWorkerForTarget("repo1", "shop", "main") + _, exists := workers.WorkerForGitBranch(shopRepo1Main) require.True(t, exists) r := &GitTargetReconciler{Client: k8sClient, WorkerManager: workers} _ = r.cleanupDeletedGitTarget(ctx, k8stypes.NamespacedName{Name: "checkout", Namespace: "shop"}, logr.Discard()) - _, exists = workers.GetWorkerForTarget("repo1", "shop", "main") + _, exists = workers.WorkerForGitBranch(shopRepo1Main) assert.False(t, exists, "the GitTarget is gone and nothing else lists this branch") } // TestCleanupDeletedGitTarget_LeavesAWorkerAnotherGitTargetStillUses. The sweep reads the API // rather than counting registrations, and this is why: one worker serves every GitTarget on its -// (provider, branch), so one deletion says nothing about the rest. +// GitBranch, so one deletion says nothing about the rest. func TestCleanupDeletedGitTarget_LeavesAWorkerAnotherGitTargetStillUses(t *testing.T) { scheme := runtime.NewScheme() require.NoError(t, clientgoscheme.AddToScheme(scheme)) @@ -70,8 +75,7 @@ func TestCleanupDeletedGitTarget_LeavesAWorkerAnotherGitTargetStillUses(t *testi sibling := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "invoices", Namespace: "shop"}, } - sibling.Spec.GitProviderRef.Name = "repo1" - sibling.Spec.Branch = "main" + sibling.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")} k8sClient := fake.NewClientBuilder().WithScheme(scheme).WithObjects(sibling).Build() workers := git.NewWorkerManager(k8sClient, logr.Discard(), git.BranchWorkerLimits{}, @@ -81,7 +85,10 @@ func TestCleanupDeletedGitTarget_LeavesAWorkerAnotherGitTargetStillUses(t *testi go func() { _ = workers.Start(ctx) }() time.Sleep(100 * time.Millisecond) func() { - err := workers.EnsureWorker(ctx, "repo1", "shop", "main", git.RepoIdentity{URL: "file:///repo1.git"}, "") + err := workers.EnsureWorker( + ctx, + claimWorkerForTest(workers, git.RepoIdentity{URL: "file:///repo1.git"}), + ) require.NoError(t, err) }() @@ -89,7 +96,7 @@ func TestCleanupDeletedGitTarget_LeavesAWorkerAnotherGitTargetStillUses(t *testi _ = r.cleanupDeletedGitTarget(ctx, k8stypes.NamespacedName{Name: "checkout", Namespace: "shop"}, logr.Discard()) - _, exists := workers.GetWorkerForTarget("repo1", "shop", "main") + _, exists := workers.WorkerForGitBranch(shopRepo1Main) assert.True(t, exists, "invoices still writes to this branch") } @@ -125,7 +132,10 @@ func TestCleanupDeletedGitTarget_RetriesWhenTheSweepCannotRead(t *testing.T) { go func() { _ = workers.Start(ctx) }() time.Sleep(100 * time.Millisecond) func() { - err := workers.EnsureWorker(ctx, "repo1", "shop", "main", git.RepoIdentity{URL: "file:///repo1.git"}, "") + err := workers.EnsureWorker( + ctx, + claimWorkerForTest(workers, git.RepoIdentity{URL: "file:///repo1.git"}), + ) require.NoError(t, err) }() @@ -134,7 +144,7 @@ func TestCleanupDeletedGitTarget_RetriesWhenTheSweepCannotRead(t *testing.T) { k8stypes.NamespacedName{Name: "checkout", Namespace: "shop"}, logr.Discard()) require.Error(t, err, "the caller has to come back, or nothing ever retires this worker") - _, exists := workers.GetWorkerForTarget("repo1", "shop", "main") + _, exists := workers.WorkerForGitBranch(shopRepo1Main) assert.True(t, exists, "and the worker is still there, which is why the retry matters") } @@ -158,7 +168,10 @@ func TestHandleFetchError_RequeuesAFailedCleanup(t *testing.T) { go func() { _ = workers.Start(ctx) }() time.Sleep(100 * time.Millisecond) func() { - err := workers.EnsureWorker(ctx, "repo1", "shop", "main", git.RepoIdentity{URL: "file:///repo1.git"}, "") + err := workers.EnsureWorker( + ctx, + claimWorkerForTest(workers, git.RepoIdentity{URL: "file:///repo1.git"}), + ) require.NoError(t, err) }() diff --git a/internal/controller/gittarget_worker_repoint_test.go b/internal/controller/gittarget_worker_repoint_test.go index affa6612..52840632 100644 --- a/internal/controller/gittarget_worker_repoint_test.go +++ b/internal/controller/gittarget_worker_repoint_test.go @@ -52,26 +52,40 @@ func TestEnsureEventStream_RepointsTheStreamAtTheReplacementWorker(t *testing.T) target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "apps", Namespace: "shop"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "repo1"}, - Branch: "main", - Path: "apps", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")}, + Path: "apps", }, } before := git.RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/first.git"} after := git.RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"} - first, err := r.ensureEventStream(ctx, target, "shop", before, logr.Discard()) + first, err := r.ensureEventStream( + ctx, + target, + claimWorkerForTest(r.WorkerManager, before), + logr.Discard(), + ) require.NoError(t, err) - steady, err := r.ensureEventStream(ctx, target, "shop", before, logr.Discard()) + steady, err := r.ensureEventStream( + ctx, + target, + claimWorkerForTest(r.WorkerManager, before), + logr.Discard(), + ) require.NoError(t, err) assert.Same(t, first, steady, "a steady tick must not re-register anything") - repointed, err := r.ensureEventStream(ctx, target, "shop", after, logr.Discard()) + repointed, err := r.ensureEventStream( + ctx, + target, + claimWorkerForTest(r.WorkerManager, after), + logr.Discard(), + ) require.NoError(t, err) assert.NotSame(t, first, repointed, "the route has to reach the worker for the repository named NOW") - worker, ok := workers.GetWorkerForTarget("repo1", "shop", "main") + worker, ok := workers.WorkerForGitBranch(shopRepo1Main) require.True(t, ok) assert.True(t, repointed.ServesWorker(worker)) assert.Same(t, repointed, router.GetGitTargetEventStream(types.NewResourceReference("apps", "shop")), @@ -85,10 +99,10 @@ func TestEnsureEventStream_RepointsTheStreamAtTheReplacementWorker(t *testing.T) // declaration is level-triggered, its inputs did not change, so the next pass is a no-op and an // idle target would leave the new repository empty. // -// One worker serves every GitTarget on its (provider, branch), and the siblings have no reason of -// their own to reconcile, so recovering only the target whose reconcile noticed leaves the rest -// pointing at a repository nothing will repopulate. Targets on another branch, and on another -// provider, must be left alone: forcing a replay is a full cluster snapshot per target. +// One worker serves every GitTarget on its GitBranch, and the siblings have no reason of their own +// to reconcile, so recovering only the target whose reconcile noticed leaves the rest pointing at a +// repository nothing will repopulate. Targets on another GitBranch must be left alone: forcing a +// replay is a full cluster snapshot per target. func TestRecoverBranchAfterReplacement_ReachesEverySiblingOnTheBranch(t *testing.T) { scheme := runtime.NewScheme() require.NoError(t, clientgoscheme.AddToScheme(scheme)) @@ -98,9 +112,8 @@ func TestRecoverBranchAfterReplacement_ReachesEverySiblingOnTheBranch(t *testing return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "shop", UID: k8stypes.UID("uid-" + name)}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: provider}, - Branch: branch, - Path: name, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest(provider, branch)}, + Path: name, }, } } @@ -131,8 +144,8 @@ func TestRecoverBranchAfterReplacement_ReachesEverySiblingOnTheBranch(t *testing require.False(t, workers.RenderFidelityGate().AllowsWrites(appsRef)) branch := git.BranchKeyFor("https://example.invalid/second.git", "main") - provider := k8stypes.NamespacedName{Namespace: "shop", Name: "repo1"} - require.True(t, r.recoverBranchAfterReplacement(context.Background(), branch, provider, logr.Discard()), + require.True(t, r.recoverBranchAfterReplacement(context.Background(), branch, + shopRepo1Main, logr.Discard()), "the targets were listed and told, so the recovery is delivered") assert.ElementsMatch(t, []string{"shop/apps", "shop/infra"}, manager.ForcedRecheckTargetsForTest(), @@ -161,9 +174,8 @@ func TestEnsureEventStream_ARecoveryThatCouldNotBeDeliveredIsRetried(t *testing. target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "apps", Namespace: "shop", UID: k8stypes.UID("uid-apps")}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "repo1"}, - Branch: "main", - Path: "apps", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")}, + Path: "apps", }, } listFails := true @@ -197,11 +209,21 @@ func TestEnsureEventStream_ARecoveryThatCouldNotBeDeliveredIsRetried(t *testing. after := git.RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"} branch := git.BranchKeyFor(after.URL, "main") - _, err := r.ensureEventStream(ctx, target, "shop", before, logr.Discard()) + _, err := r.ensureEventStream( + ctx, + target, + claimWorkerForTest(r.WorkerManager, before), + logr.Discard(), + ) require.NoError(t, err) // The repoint. Its recovery cannot be delivered, and the reconcile does not fail for it. - _, err = r.ensureEventStream(ctx, target, "shop", after, logr.Discard()) + _, err = r.ensureEventStream( + ctx, + target, + claimWorkerForTest(r.WorkerManager, after), + logr.Discard(), + ) require.NoError(t, err) require.Empty(t, manager.ForcedRecheckTargetsForTest(), "precondition: nothing was told") assert.True(t, workers.ReplacementPending(branch), @@ -209,7 +231,12 @@ func TestEnsureEventStream_ARecoveryThatCouldNotBeDeliveredIsRetried(t *testing. // The next reconcile — this target's own, or a sibling's — delivers it. listFails = false - _, err = r.ensureEventStream(ctx, target, "shop", after, logr.Discard()) + _, err = r.ensureEventStream( + ctx, + target, + claimWorkerForTest(r.WorkerManager, after), + logr.Discard(), + ) require.NoError(t, err) assert.Equal(t, []string{"shop/apps"}, manager.ForcedRecheckTargetsForTest(), diff --git a/internal/controller/resource_condition_metrics.go b/internal/controller/resource_condition_metrics.go index 2d71b54b..fe3e896d 100644 --- a/internal/controller/resource_condition_metrics.go +++ b/internal/controller/resource_condition_metrics.go @@ -31,6 +31,7 @@ const ( conditionKindWatchRule = "WatchRule" conditionKindClusterWatchRule = "ClusterWatchRule" conditionKindGitProvider = "GitProvider" + conditionKindGitBranch = "GitBranch" conditionKindClusterProvider = "ClusterProvider" conditionKindFluxTrigger = configbutleraiv1alpha3.ApplyTriggerKindFlux conditionKindArgoCDTrigger = configbutleraiv1alpha3.ApplyTriggerKindArgoCD @@ -55,6 +56,8 @@ func conditionMetricKind(obj client.Object) string { return conditionKindClusterWatchRule case *configbutleraiv1alpha3.GitProvider: return conditionKindGitProvider + case *configbutleraiv1alpha3.GitBranch: + return conditionKindGitBranch case *configbutleraiv1alpha3.ClusterProvider: return conditionKindClusterProvider case *configbutleraiv1alpha3.FluxTrigger: diff --git a/internal/controller/status_event_severity_test.go b/internal/controller/status_event_severity_test.go index 992a2045..e07539af 100644 --- a/internal/controller/status_event_severity_test.go +++ b/internal/controller/status_event_severity_test.go @@ -46,7 +46,7 @@ func TestRecordReadyTransition_SeverityFollowsStalled(t *testing.T) { // kstatus Failed. Waiting changes nothing, which is exactly what a Warning should mean. "a stalled gate is a Warning": { contribute: func(rd *readiness) { - rd.stalled(GitTargetReasonBranchNotAllowed, "branch is not allowed") + rd.stalled(GitTargetReasonGitBranchNotFound, "GitBranch not found") }, want: corev1.EventTypeWarning, }, diff --git a/internal/controller/stream_status_test.go b/internal/controller/stream_status_test.go index 07af241b..88692b9d 100644 --- a/internal/controller/stream_status_test.go +++ b/internal/controller/stream_status_test.go @@ -83,8 +83,8 @@ func TestRuleReadiness_StallsOnTheGitTargetsVerdictWhateverItsReason(t *testing. }{ {"refused folder", stalledTarget(GitTargetReasonAmbiguousLayout), metav1.ConditionTrue, GitTargetReasonAmbiguousLayout}, - {"no parent branch", stalledTarget(GitTargetReasonParentBranchNotFound), - metav1.ConditionTrue, GitTargetReasonParentBranchNotFound}, + {"a conflicting path", stalledTarget(GitTargetReasonTargetConflict), + metav1.ConditionTrue, GitTargetReasonTargetConflict}, {"apply trigger on another source", stalledTarget(applier.ReasonSourceMismatch), metav1.ConditionTrue, applier.ReasonSourceMismatch}, {"a reason this package has never heard of", stalledTarget("SomeFutureReason"), diff --git a/internal/controller/suite_test.go b/internal/controller/suite_test.go index 0732ac58..9e7604cc 100644 --- a/internal/controller/suite_test.go +++ b/internal/controller/suite_test.go @@ -39,6 +39,9 @@ var ( cancel context.CancelFunc mgr manager.Manager testRuleStore *rulestore.RuleStore + // testWorkerManager is the suite's worker manager. No GitBranch reconciler runs here, so a spec + // that needs a GitBranch to hold its branch claims it directly (holdGitBranchForTest). + testWorkerManager *git.WorkerManager ) func TestControllers(t *testing.T) { @@ -98,9 +101,10 @@ var _ = BeforeSuite(func() { testRuleStore = rulestore.NewStore() // Initialize WorkerManager for new architecture - workerManager := git.NewWorkerManager( + testWorkerManager = git.NewWorkerManager( mgr.GetClient(), logf.Log.WithName("worker-manager"), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}, ) + workerManager := testWorkerManager err = mgr.Add(workerManager) Expect(err).NotTo(HaveOccurred()) diff --git a/internal/controller/superseded_fields_admission_test.go b/internal/controller/superseded_fields_admission_test.go index 9230ab70..669657e8 100644 --- a/internal/controller/superseded_fields_admission_test.go +++ b/internal/controller/superseded_fields_admission_test.go @@ -87,9 +87,8 @@ var _ = Describe("Superseded source-scope fields", func() { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "renamed-commit", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "any-provider"}, - Branch: "main", - Path: "clusters/prod", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("any-provider", "main")}, + Path: "clusters/prod", Commit: &configbutleraiv1alpha3.GitTargetCommitSpec{ Window: &configbutleraiv1alpha3.CommitWindow{ IdleTimeout: &metav1.Duration{Duration: 30 * time.Second}, @@ -197,18 +196,26 @@ var _ = Describe("Reference names must not be empty", func() { Expect(err.Error()).To(ContainSubstring("must not be empty")) }) - It("refuses an empty name on the provider and target references", func() { + It("refuses an empty name on the branch, provider and target references", func() { ctx := context.Background() target := &configbutleraiv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "empty-provider-name", Namespace: "default"}, + ObjectMeta: metav1.ObjectMeta{Name: "empty-branch-name", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: ""}, - Branch: "main", - Path: "clusters/prod", + GitBranchRef: &meta.NamespacedObjectReference{Name: ""}, + Path: "clusters/prod", }, } Expect(k8sClient.Create(ctx, target)).NotTo(Succeed(), + "an empty gitBranchRef.name names nothing and must be refused at admission") + + branch := &configbutleraiv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Name: "empty-provider-name", Namespace: "default"}, + Spec: configbutleraiv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: ""}, Branch: "main", + }, + } + Expect(k8sClient.Create(ctx, branch)).NotTo(Succeed(), "an empty gitProviderRef.name names nothing and must be refused at admission") rule := &configbutleraiv1alpha3.WatchRule{ diff --git a/internal/controller/watchrule_controller.go b/internal/controller/watchrule_controller.go index 4af4c6c6..6053b060 100644 --- a/internal/controller/watchrule_controller.go +++ b/internal/controller/watchrule_controller.go @@ -31,7 +31,6 @@ import ( // WatchRule status condition reasons. const ( WatchRuleReasonValidating = "Validating" - WatchRuleReasonGitProviderNotFound = "GitProviderNotFound" WatchRuleReasonAccessDenied = "AccessDenied" WatchRuleReasonGitTargetNotFound = "GitTargetNotFound" WatchRuleReasonGitDestinationInvalid = "GitDestinationInvalid" @@ -175,30 +174,11 @@ func (r *WatchRuleReconciler) reconcileWatchRuleViaTarget( ready := gitTargetReadyCondition(target) st.set(ConditionTypeGitTargetReady, ready.Status, ready.Reason, ready.Message) - // Resolve the GitProvider named by the target, in its own namespace unless the reference names - // another one. - providerName := target.Spec.GitProviderRef.Name - providerNS := target.GitProviderKey().Namespace - - var provider configbutleraiv1alpha3.GitProvider - providerKey := types.NamespacedName{Name: providerName, Namespace: providerNS} - if err := r.Get(ctx, providerKey, &provider); err != nil { - log.Error(err, "Failed to resolve GitProvider from GitTarget", - "gitProviderName", providerName, "gitProviderNamespace", providerNS) - st.set( - ConditionTypeGitTargetReady, - metav1.ConditionFalse, - WatchRuleReasonGitProviderNotFound, - fmt.Sprintf("GitProvider '%s/%s' (from GitTarget) not found: %v", providerNS, providerName, err), - ) - return r.stallRule(ctx, st, WatchRuleReasonGitProviderNotFound, "Referenced GitProvider not found") - } - // Source-namespace gate AND compilation, in that order and in one call — see // gateSourceNamespace. There is deliberately no AddOrUpdateWatchRule here: routing every // compilation through watch.CompileWatchRule is what stops the startup bootstrap from being a // second, ungated path into the store. - if handled, result, err := r.gateSourceNamespace(ctx, st, watchRule, target, provider, log); handled { + if handled, result, err := r.gateSourceNamespace(ctx, st, watchRule, target, log); handled { return result, err } @@ -300,15 +280,6 @@ func (r *WatchRuleReconciler) SetupWithManager(mgr ctrl.Manager) error { handler.EnqueueRequestsFromMapFunc(r.gitTargetToWatchRules), builder.WithPredicates(gitTargetReadyProjectionChanged()), ). - // Nothing of the GitProvider's STATUS is mirrored here, so GenerationChangedPredicate is - // right on this edge: it reacts to a freshly applied or spec-changed provider while ignoring - // the status-only updates its controller writes itself — without it every GitProvider - // heartbeat would re-list and re-enqueue all WatchRules. - Watches( - &configbutleraiv1alpha3.GitProvider{}, - handler.EnqueueRequestsFromMapFunc(r.gitProviderToWatchRules), - builder.WithPredicates(predicate.GenerationChangedPredicate{}), - ). // React to a ClusterProvider's allowCrossNamespace.sources (or accessFrom) // changing. The GitTarget->WatchRules edge above still cannot be relied on to carry this: a // ClusterProvider change reaches the GitTarget as a status update that need not move the @@ -410,49 +381,3 @@ func (r *WatchRuleReconciler) gitTargetToWatchRules( } return requests } - -// gitProviderToWatchRules maps a GitProvider event to every WatchRule (in the -// GitProvider's namespace) whose referenced GitTarget points at this provider. -// Mirrors the equivalent helper on ClusterWatchRuleReconciler so that an -// arriving provider doesn't have to wait for a separate GitTarget event to -// reach the rule. -func (r *WatchRuleReconciler) gitProviderToWatchRules( - ctx context.Context, - obj client.Object, -) []ctrlreconcile.Request { - var targets configbutleraiv1alpha3.GitTargetList - if err := r.List(ctx, &targets); err != nil { - logDependencyListError(ctx, err, "GitTargets", obj) - return nil - } - - matchingTargets := make(map[types.NamespacedName]struct{}) - for i := range targets.Items { - t := &targets.Items[i] - if namesGitProvider(t, obj) { - matchingTargets[types.NamespacedName{Name: t.Name, Namespace: t.Namespace}] = struct{}{} - } - } - if len(matchingTargets) == 0 { - return nil - } - - var rules configbutleraiv1alpha3.WatchRuleList - if err := r.List(ctx, &rules); err != nil { - logDependencyListError(ctx, err, "WatchRules", obj) - return nil - } - - var requests []ctrlreconcile.Request - for i := range rules.Items { - rule := &rules.Items[i] - target := types.NamespacedName{Name: rule.Spec.GitTargetRef.Name, Namespace: rule.Namespace} - if _, ok := matchingTargets[target]; !ok { - continue - } - requests = append(requests, ctrlreconcile.Request{ - NamespacedName: types.NamespacedName{Name: rule.Name, Namespace: rule.Namespace}, - }) - } - return requests -} diff --git a/internal/controller/watchrule_controller_test.go b/internal/controller/watchrule_controller_test.go index 801430e9..2be6b478 100644 --- a/internal/controller/watchrule_controller_test.go +++ b/internal/controller/watchrule_controller_test.go @@ -79,11 +79,8 @@ var _ = Describe("WatchRule Controller", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "test-provider", - }, - Branch: "main", - Path: "default/test", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("test-provider", "main")}, + Path: "default/test", }, } Expect(k8sClient.Create(ctx, target)).To(Succeed()) @@ -184,9 +181,9 @@ var _ = Describe("WatchRule Controller", func() { }, } Expect(k8sClient.Create(ctx, gitProvider)).Should(Succeed()) - // No wait on GitProvider readiness: reconcileWatchRuleViaTarget only requires the - // GitProvider to exist (it is resolved by name), not to be Ready. The GitTarget - // readiness wait below is the dependency gate this spec actually needs. + // The GitTarget readiness wait below is the dependency gate this spec needs: its + // GitBranch has to hold the branch for the target to get past the claim. + holdGitBranchForTest(ctx, gitProvider, "main") By("Creating GitTarget in same namespace referencing the GitProvider") target := &configbutleraiv1alpha3.GitTarget{ @@ -195,11 +192,8 @@ var _ = Describe("WatchRule Controller", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: "local-provider", - }, - Branch: "main", - Path: "ns/default", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("local-provider", "main")}, + Path: "ns/default", }, } Expect(k8sClient.Create(ctx, target)).Should(Succeed()) diff --git a/internal/controller/watchrule_source_namespace.go b/internal/controller/watchrule_source_namespace.go index cab23e4f..180e8a59 100644 --- a/internal/controller/watchrule_source_namespace.go +++ b/internal/controller/watchrule_source_namespace.go @@ -55,10 +55,9 @@ func (r *WatchRuleReconciler) gateSourceNamespace( st *reconcileStatus, watchRule *configbutleraiv1alpha3.WatchRule, target configbutleraiv1alpha3.GitTarget, - provider configbutleraiv1alpha3.GitProvider, log logr.Logger, ) (bool, ctrl.Result, error) { - resolved, err := watch.CompileWatchRule(ctx, r.Client, r.RuleStore, *watchRule, target, provider) + resolved, err := watch.CompileWatchRule(ctx, r.Client, r.RuleStore, *watchRule, target) var selectorErr *watch.ObjectSelectorError if errors.As(err, &selectorErr) { result, refuseErr := r.refuseRuleResources(ctx, st, watchRule, watch.ReasonInvalidObjectSelector, diff --git a/internal/controller/watchrule_source_namespace_test.go b/internal/controller/watchrule_source_namespace_test.go index 3aaedf27..7863e0f7 100644 --- a/internal/controller/watchrule_source_namespace_test.go +++ b/internal/controller/watchrule_source_namespace_test.go @@ -36,9 +36,8 @@ func wrsnGitTarget() *configbutleraiv1alpha3.GitTarget { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: wrsnTarget, Namespace: wrsnTenantNS}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "git"}, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: wrsnProvider}, - Branch: "main", Path: "tenants/acme", }, } diff --git a/internal/controller/wiring/readiness_wiring_test.go b/internal/controller/wiring/readiness_wiring_test.go index d4a360fb..6230be3f 100644 --- a/internal/controller/wiring/readiness_wiring_test.go +++ b/internal/controller/wiring/readiness_wiring_test.go @@ -130,9 +130,8 @@ func seedTarget(ctx context.Context, t *testing.T, name, clusterProvider string) target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "target", Namespace: namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "provider"}, - Branch: "main", - Path: "mirror", + GitBranchRef: &meta.NamespacedObjectReference{Name: "provider-main"}, + Path: "mirror", }, } if clusterProvider != "" { diff --git a/internal/git/apply_recovery.go b/internal/git/apply_recovery.go index dc27e72a..acafe59b 100644 --- a/internal/git/apply_recovery.go +++ b/internal/git/apply_recovery.go @@ -107,6 +107,7 @@ func (l *branchWorkerEventLoop) followRecorded(state *applierState, recorded *v1 ID: attempt.id, Branch: w.branchKey(), Provider: w.provider(), + GitBranch: w.GitBranch, Applier: state.ref, Reason: recorded.Reason, Commit: recorded.Commit, diff --git a/internal/git/apply_request.go b/internal/git/apply_request.go index b1e92a6a..aa4e2185 100644 --- a/internal/git/apply_request.go +++ b/internal/git/apply_request.go @@ -87,9 +87,11 @@ type ApplyRequest struct { ID uint64 // Branch is the branch the request is about. Branch BranchKey - // Provider is the GitProvider the branch's worker writes through; its GitTargets are the ones - // that write the branch through it. + // Provider is the GitProvider the branch's worker writes through. Provider k8stypes.NamespacedName + // GitBranch is the GitBranch the worker writes for; its GitTargets are the ones that reference + // it. + GitBranch k8stypes.NamespacedName // Applier is what is asked to apply. Applier ApplierRef // Reason is what the applier is asked: the strongest of the targets' reasons. diff --git a/internal/git/apply_trigger.go b/internal/git/apply_trigger.go index b9e2230f..db5a27e7 100644 --- a/internal/git/apply_trigger.go +++ b/internal/git/apply_trigger.go @@ -189,7 +189,7 @@ func (w *BranchWorker) appliersOnBranch(ctx context.Context) ([]branchTarget, er for i := range targets.Items { target := &targets.Items[i] if !target.DeletionTimestamp.IsZero() || target.Spec.ApplyTriggerRef == nil || - !WritesThrough(target, w.provider(), w.Branch) { + !WritesThrough(target, w.GitBranch) { continue } trigger, err := ResolveApplyTrigger(ctx, w.Client, target) @@ -728,6 +728,7 @@ func (l *branchWorkerEventLoop) sendAttempt(state *applierState, tip string, ref ID: attempt.id, Branch: w.branchKey(), Provider: w.provider(), + GitBranch: w.GitBranch, Applier: state.ref, Reason: reason, Commit: tip, diff --git a/internal/git/apply_trigger_test.go b/internal/git/apply_trigger_test.go index c56e6abe..802c3d1f 100644 --- a/internal/git/apply_trigger_test.go +++ b/internal/git/apply_trigger_test.go @@ -109,9 +109,8 @@ func applierTestTarget(name string, applier bool, mutate ...func(*configv1alpha3 target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "team-a", UID: k8stypes.UID("uid-" + name)}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "test-provider"}, - Branch: "main", - Path: "apps/" + name, + GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")}, + Path: "apps/" + name, }, } if applier { @@ -134,6 +133,7 @@ func applyTriggerLoop(t *testing.T, targets ...client.Object) (*branchWorkerEven w := newMetricsTestWorker() w.GitProviderNamespace = "team-a" + w.GitBranch = testGitBranch(w.GitProviderRef, w.GitProviderNamespace, w.Branch) w.repo = RepoIdentity{URL: "https://github.com/acme/platform"} w.Client = fake.NewClientBuilder().WithScheme(scheme).WithObjects(testTriggersFor(targets)...). WithStatusSubresource(&configv1alpha3.FluxTrigger{}, &configv1alpha3.ArgoCDTrigger{}).Build() @@ -166,9 +166,11 @@ func TestServiceAppliers_ForeignCommitAsksEveryApplierOnTheBranch(t *testing.T) applierTestTarget("alpha", true), applierTestTarget("bravo", true, func(g *configv1alpha3.GitTarget) { g.Spec.Suspend = true }), applierTestTarget("no-applier", false), - applierTestTarget("other-branch", true, func(g *configv1alpha3.GitTarget) { g.Spec.Branch = "release" }), + applierTestTarget("other-branch", true, func(g *configv1alpha3.GitTarget) { + g.Spec.GitBranchRef.Name = testGitBranchName("test-provider", "release") + }), applierTestTarget("other-provider", true, func(g *configv1alpha3.GitTarget) { - g.Spec.GitProviderRef.Name = "elsewhere" + g.Spec.GitBranchRef.Name = testGitBranchName("elsewhere", "main") }), ) observeForeignCommit(loop.w) diff --git a/internal/git/branch_owner.go b/internal/git/branch_owner.go index 5e2f90cb..3462d202 100644 --- a/internal/git/branch_owner.go +++ b/internal/git/branch_owner.go @@ -3,81 +3,156 @@ package git import ( - k8stypes "k8s.io/apimachinery/pkg/types" + "sync" + "time" - "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + k8stypes "k8s.io/apimachinery/pkg/types" ) -// One GitProvider per repository branch. +// One GitBranch per repository branch. +// +// A branch worker is keyed by its repository and branch (BranchKey) and pushes with one credential, +// so exactly one GitBranch may write a repository branch: the one holding its claim. The rules are +// the plan's (docs/design/gitbranch-and-kubeprovider-plan.md): // -// A branch worker is keyed by its repository and branch (BranchKey), and pushes with one -// credential. So every GitTarget that writes one repository branch must name the same -// GitProvider; a GitTarget in another namespace reaches a shared one through -// spec.gitProviderRef.namespace, which the provider's spec.accessFrom admits. When GitTargets on -// one repository branch name different GitProviders anyway, the oldest GitTarget's GitProvider -// owns the branch, and the others are refused (TargetConflict) and are written by no worker. The -// GitTarget reconciler, the worker sweep and EnsureWorker all decide it here, so they cannot -// disagree. - -// BranchOwner is the GitProvider that writes a repository branch, and the GitTarget that decided -// it: the oldest one writing that branch. -type BranchOwner struct { - Provider k8stypes.NamespacedName - Target k8stypes.NamespacedName +// 1. A GitBranch becomes a candidate once its credential opens a receive-pack session on the +// remote (ProvePushSession). An unproven GitBranch claims nothing, so creating one first +// cannot block anybody. +// 2. Among the candidates for one key, the oldest by creationTimestamp, then UID, holds the claim. +// 3. The holder stays a candidate through a later credential or network failure, so it keeps the +// claim and does not hand the branch to another GitBranch. Any other candidate whose proof +// fails stops being one. +// 4. The claim lives here, in the manager's memory. After a restart it is recomputed by 1 and 2. +// +// The GitBranch reconciler feeds it (Observe, Release); the GitTarget reconciler, the worker sweep +// and EnsureWorker read it, so they cannot disagree about who writes a branch. + +// BranchClaimant is a GitBranch as the claim sees it: which object, and how old. +type BranchClaimant struct { + GitBranch k8stypes.NamespacedName + UID k8stypes.UID + Created time.Time } -// BranchOwners decides the owner of every repository branch the targets write. urlOf resolves a -// GitProvider to its spec.url; a GitTarget whose GitProvider it cannot resolve writes nothing yet, -// and decides nothing. A GitTarget being deleted decides nothing either, so the next one takes the -// branch over at once. Nor does one admitted reports refused (authz.GitTargetAdmitted: its -// ClusterProvider or its GitProvider does not admit its namespace): it may not write, so it must not -// keep the branch from a GitTarget that may. An error from admitted is returned, never read as a -// refusal, so a read that fails moves no branch. -func BranchOwners( - targets []v1alpha3.GitTarget, - urlOf func(provider k8stypes.NamespacedName) (string, bool), - admitted func(target *v1alpha3.GitTarget) (bool, error), -) (map[BranchKey]BranchOwner, error) { - owners := map[BranchKey]BranchOwner{} - oldest := map[BranchKey]*v1alpha3.GitTarget{} - for i := range targets { - target := &targets[i] - if !target.DeletionTimestamp.IsZero() { - continue - } - url, ok := urlOf(target.GitProviderKey()) - if !ok { - continue - } - allowed, err := admitted(target) - if err != nil { - return nil, err - } - if !allowed { - continue - } - key := BranchKeyFor(url, target.Spec.Branch) - if key == (BranchKey{}) { - continue - } - if current, held := oldest[key]; held && !OlderGitTarget(target, current) { +// olderThan orders two claimants: creationTimestamp first, then UID, so every caller and every +// restart elects the same one. +func (c BranchClaimant) olderThan(other BranchClaimant) bool { + if !c.Created.Equal(other.Created) { + return c.Created.Before(other.Created) + } + return c.UID < other.UID +} + +type claimEntry struct { + claimant BranchClaimant + key BranchKey + candidate bool +} + +// BranchClaims records which GitBranch holds each repository branch. Its zero value is not usable; +// use NewBranchClaims. +type BranchClaims struct { + mu sync.RWMutex + entries map[k8stypes.UID]*claimEntry + holders map[BranchKey]k8stypes.UID +} + +// NewBranchClaims returns an empty claim registry. +func NewBranchClaims() *BranchClaims { + return &BranchClaims{ + entries: map[k8stypes.UID]*claimEntry{}, + holders: map[BranchKey]k8stypes.UID{}, + } +} + +// Observe records what a GitBranch's reconcile found: the key it names now and whether its proof +// succeeded. It returns who holds the key afterwards. A GitBranch that names another key than it +// did before (its GitProvider was recreated against another repository) gives up the old one first. +func (c *BranchClaims) Observe(key BranchKey, claimant BranchClaimant, proven bool) (BranchClaimant, bool) { + c.mu.Lock() + defer c.mu.Unlock() + entry, known := c.entries[claimant.UID] + if known && entry.key != key { + c.releaseLocked(claimant.UID) + known = false + } + if !known { + entry = &claimEntry{claimant: claimant, key: key} + c.entries[claimant.UID] = entry + } + // Rule 3: the holder's candidacy survives a failed proof. Everyone else's follows the proof. + if c.holders[key] != claimant.UID { + entry.candidate = proven + } + c.electLocked(key) + return c.holderLocked(key) +} + +// Release forgets a GitBranch: it is being deleted, and its worker has stopped. If it held its key, +// the oldest remaining candidate takes it over. +func (c *BranchClaims) Release(uid k8stypes.UID) { + c.mu.Lock() + defer c.mu.Unlock() + c.releaseLocked(uid) +} + +func (c *BranchClaims) releaseLocked(uid k8stypes.UID) { + entry, known := c.entries[uid] + if !known { + return + } + delete(c.entries, uid) + if c.holders[entry.key] == uid { + delete(c.holders, entry.key) + } + c.electLocked(entry.key) +} + +// electLocked applies rule 2: the oldest candidate for key holds it. A holder is always a candidate, +// so an older GitBranch that proves itself later takes the key over; that is the rule, and it is +// what makes the outcome the same after a restart. +func (c *BranchClaims) electLocked(key BranchKey) { + var oldest *claimEntry + for _, entry := range c.entries { + if entry.key != key || !entry.candidate { continue } - oldest[key] = target - owners[key] = BranchOwner{ - Provider: target.GitProviderKey(), - Target: k8stypes.NamespacedName{Namespace: target.Namespace, Name: target.Name}, + if oldest == nil || entry.claimant.olderThan(oldest.claimant) { + oldest = entry } } - return owners, nil + if oldest == nil { + delete(c.holders, key) + return + } + c.holders[key] = oldest.claimant.UID +} + +// Holder returns the GitBranch holding key. +func (c *BranchClaims) Holder(key BranchKey) (BranchClaimant, bool) { + c.mu.RLock() + defer c.mu.RUnlock() + return c.holderLocked(key) +} + +func (c *BranchClaims) holderLocked(key BranchKey) (BranchClaimant, bool) { + uid, held := c.holders[key] + if !held { + return BranchClaimant{}, false + } + return c.entries[uid].claimant, true } -// OlderGitTarget reports whether a was created before b, ties broken by namespace and name, so every -// caller orders two GitTargets the same way. -func OlderGitTarget(a, b *v1alpha3.GitTarget) bool { - at, bt := a.CreationTimestamp.Time, b.CreationTimestamp.Time - if !at.Equal(bt) { - return at.Before(bt) +// HeldBy returns the key the named GitBranch holds, if it holds one. GitTargets reference a +// GitBranch by name; a recreated object with that name is a new UID, and it holds nothing until it +// has proved itself (Observe), because Release forgot the old one before its finalizer went. +func (c *BranchClaims) HeldBy(gitBranch k8stypes.NamespacedName) (BranchKey, BranchClaimant, bool) { + c.mu.RLock() + defer c.mu.RUnlock() + for key, uid := range c.holders { + if claimant := c.entries[uid].claimant; claimant.GitBranch == gitBranch { + return key, claimant, true + } } - return a.Namespace+"/"+a.Name < b.Namespace+"/"+b.Name + return BranchKey{}, BranchClaimant{}, false } diff --git a/internal/git/branch_owner_test.go b/internal/git/branch_owner_test.go index 18edfb78..0c86bc22 100644 --- a/internal/git/branch_owner_test.go +++ b/internal/git/branch_owner_test.go @@ -3,126 +3,140 @@ package git import ( - "errors" - "strings" "testing" "time" - "github.com/fluxcd/pkg/apis/meta" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" k8stypes "k8s.io/apimachinery/pkg/types" - - "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// ownerTarget is a GitTarget in namespace writing main through the GitProvider provider -// ("namespace/name", or a bare name for its own namespace), created age ago. -func ownerTarget(namespace, name, provider string, age time.Duration) v1alpha3.GitTarget { - ref := meta.NamespacedObjectReference{Name: provider} - if ns, n, found := strings.Cut(provider, "/"); found { - ref = meta.NamespacedObjectReference{Namespace: ns, Name: n} - } - return v1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{ - Namespace: namespace, Name: name, - CreationTimestamp: metav1.NewTime(time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC).Add(-age)), - }, - Spec: v1alpha3.GitTargetSpec{GitProviderRef: ref, Branch: "main", Path: name}, - } -} +var claimEpoch = time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC) -// urlsOf resolves the GitProviders the ownership tests name. -func urlsOf(urls map[string]string) func(k8stypes.NamespacedName) (string, bool) { - return func(provider k8stypes.NamespacedName) (string, bool) { - url, ok := urls[provider.String()] - return url, ok +// claimant is a GitBranch namespace/name created age ago, with a UID of its name. +func claimant(namespace, name string, age time.Duration) BranchClaimant { + return BranchClaimant{ + GitBranch: k8stypes.NamespacedName{Namespace: namespace, Name: name}, + UID: k8stypes.UID(namespace + "/" + name), + Created: claimEpoch.Add(-age), } } -// TestBranchOwners_TheOldestGitTargetsGitProviderOwnsTheBranch: GitTargets on one repository branch -// through two GitProviders, spelled two ways. One worker writes the branch, with one credential, so -// the oldest GitTarget decides whose; a GitTarget sharing that GitProvider from another namespace is -// on the owner's side. -func TestBranchOwners_TheOldestGitTargetsGitProviderOwnsTheBranch(t *testing.T) { - urls := urlsOf(map[string]string{ - "team-a/monorepo": "https://github.com/acme/platform.git", - "team-b/monorepo": "git@github.com:acme/platform", - }) - targets := []v1alpha3.GitTarget{ - ownerTarget("team-b", "late", "monorepo", time.Hour), - ownerTarget("team-a", "first", "monorepo", 2*time.Hour), - ownerTarget("team-c", "shared", "team-a/monorepo", 3*time.Minute), - } +// TestBranchClaims_AnUnprovenGitBranchClaimsNothing is rule 1: a GitBranch with a bad credential, +// created first, does not stop a valid one created later from claiming the branch. +func TestBranchClaims_AnUnprovenGitBranchClaimsNothing(t *testing.T) { + claims := NewBranchClaims() + key := BranchKeyFor("https://github.com/acme/platform.git", "main") + squatter := claimant("team-x", "main", 2*time.Hour) + owner := claimant("platform", "main", time.Hour) - owners, err := BranchOwners(targets, urls, admitAll) - require.NoError(t, err) + _, held := claims.Observe(key, squatter, false) + assert.False(t, held, "an unproven GitBranch holds nothing") - key := BranchKeyFor("https://github.com/acme/platform", "main") - assert.Len(t, owners, 1, "two spellings of one repository are one branch") - assert.Equal(t, BranchOwner{ - Provider: k8stypes.NamespacedName{Namespace: "team-a", Name: "monorepo"}, - Target: k8stypes.NamespacedName{Namespace: "team-a", Name: "first"}, - }, owners[key]) + holder, held := claims.Observe(key, owner, true) + require.True(t, held) + assert.Equal(t, owner, holder) } -// TestBranchOwners_ADeletedOrUnresolvedGitTargetDecidesNothing: a GitTarget being deleted hands the -// branch to the next one at once, and one whose GitProvider cannot be read writes nothing yet. -func TestBranchOwners_ADeletedOrUnresolvedGitTargetDecidesNothing(t *testing.T) { - urls := urlsOf(map[string]string{ - "team-a/monorepo": "https://github.com/acme/platform", - "team-b/monorepo": "https://github.com/acme/platform", - }) - leaving := ownerTarget("team-a", "leaving", "monorepo", 2*time.Hour) - leaving.DeletionTimestamp = &metav1.Time{Time: time.Now()} - targets := []v1alpha3.GitTarget{ - leaving, - ownerTarget("team-x", "unread", "gone", 3*time.Hour), - ownerTarget("team-b", "next", "monorepo", time.Hour), - } - - owners, err := BranchOwners(targets, urls, admitAll) - require.NoError(t, err) - owner := owners[BranchKeyFor("https://github.com/acme/platform", "main")] +// TestBranchClaims_TheOldestCandidateHoldsTheBranch is rule 2, across two spellings of one +// repository: the younger GitBranch learns who holds it, whichever reconciles first. +func TestBranchClaims_TheOldestCandidateHoldsTheBranch(t *testing.T) { + claims := NewBranchClaims() + https := BranchKeyFor("https://github.com/acme/platform.git", "main") + ssh := BranchKeyFor("git@github.com:acme/platform", "main") + require.Equal(t, https, ssh, "one repository, spelled two ways, is one key") + older := claimant("team-a", "main", 2*time.Hour) + younger := claimant("team-b", "main", time.Hour) + + holder, _ := claims.Observe(ssh, younger, true) + assert.Equal(t, younger, holder, "alone, the younger GitBranch holds the branch") + holder, _ = claims.Observe(https, older, true) + assert.Equal(t, older, holder, "an older candidate takes the branch over") + holder, _ = claims.Observe(ssh, younger, true) + assert.Equal(t, older, holder) + + other := BranchKeyFor("https://github.com/acme/platform.git", "release") + holder, _ = claims.Observe(other, claimant("team-b", "release", time.Hour), true) + assert.Equal(t, "release", holder.GitBranch.Name, "another branch is another claim") +} - assert.Equal(t, k8stypes.NamespacedName{Namespace: "team-b", Name: "monorepo"}, owner.Provider) +// TestBranchClaims_TheHolderKeepsTheBranchThroughAFailedProof is rule 3: a holder whose credential +// starts failing keeps the branch, and does not hand it to a younger candidate. A candidate that is +// not the holder stops being one when its proof fails. +func TestBranchClaims_TheHolderKeepsTheBranchThroughAFailedProof(t *testing.T) { + claims := NewBranchClaims() + key := BranchKeyFor("https://github.com/acme/platform.git", "main") + holder := claimant("platform", "main", 2*time.Hour) + younger := claimant("team-b", "main", time.Hour) + claims.Observe(key, holder, true) + claims.Observe(key, younger, true) + + got, held := claims.Observe(key, holder, false) + require.True(t, held) + assert.Equal(t, holder, got, "the holder keeps the branch") + + claims.Observe(key, younger, false) + claims.Release(holder.UID) + _, held = claims.Holder(key) + assert.False(t, held, "a candidate whose proof failed does not inherit the branch") + + got, held = claims.Observe(key, younger, true) + require.True(t, held) + assert.Equal(t, younger, got) } -// TestBranchOwners_AGitTargetThatMayNotWriteDecidesNothing: an older GitTarget whose GitProvider -// does not admit its namespace may not write, so it must not keep the branch from a younger one that -// may. A read that fails is an error, never a refusal: it must not move the branch. -func TestBranchOwners_AGitTargetThatMayNotWriteDecidesNothing(t *testing.T) { - urls := urlsOf(map[string]string{ - "platform/monorepo": "https://github.com/acme/platform", - "team-b/monorepo": "https://github.com/acme/platform", - }) - targets := []v1alpha3.GitTarget{ - ownerTarget("team-x", "refused", "platform/monorepo", 2*time.Hour), - ownerTarget("team-b", "admitted", "monorepo", time.Hour), +// TestBranchClaims_ReleaseHandsTheBranchToTheNextCandidate: a deleted holder gives the branch to the +// oldest remaining candidate, and HeldBy follows it. +func TestBranchClaims_ReleaseHandsTheBranchToTheNextCandidate(t *testing.T) { + claims := NewBranchClaims() + key := BranchKeyFor("https://github.com/acme/platform.git", "main") + first := claimant("platform", "main", 3*time.Hour) + second := claimant("team-a", "main", 2*time.Hour) + third := claimant("team-b", "main", time.Hour) + for _, c := range []BranchClaimant{third, first, second} { + claims.Observe(key, c, true) } - refuseTeamX := func(target *v1alpha3.GitTarget) (bool, error) { return target.Namespace != "team-x", nil } - owners, err := BranchOwners(targets, urls, refuseTeamX) - require.NoError(t, err) - assert.Equal(t, k8stypes.NamespacedName{Namespace: "team-b", Name: "monorepo"}, - owners[BranchKeyFor("https://github.com/acme/platform", "main")].Provider) + claims.Release(first.UID) + gotKey, got, held := claims.HeldBy(second.GitBranch) + require.True(t, held) + assert.Equal(t, key, gotKey) + assert.Equal(t, second, got) + _, _, held = claims.HeldBy(first.GitBranch) + assert.False(t, held, "a released GitBranch holds nothing") +} - _, err = BranchOwners(targets, urls, func(*v1alpha3.GitTarget) (bool, error) { - return false, errors.New("the cache is not having it") - }) - require.Error(t, err) +// TestBranchClaims_AGitBranchThatNamesAnotherRepositoryGivesTheOldOneUp: its GitProvider was +// recreated against another URL, so its claim moves with it and the old branch is free again. +func TestBranchClaims_AGitBranchThatNamesAnotherRepositoryGivesTheOldOneUp(t *testing.T) { + claims := NewBranchClaims() + before := BranchKeyFor("https://github.com/acme/old.git", "main") + after := BranchKeyFor("https://github.com/acme/new.git", "main") + branch := claimant("platform", "main", time.Hour) + claims.Observe(before, branch, true) + + claims.Observe(after, branch, true) + _, held := claims.Holder(before) + assert.False(t, held) + got, held := claims.Holder(after) + require.True(t, held) + assert.Equal(t, branch, got) } -// admitAll admits every GitTarget. -func admitAll(*v1alpha3.GitTarget) (bool, error) { return true, nil } - -// TestOlderGitTarget_BreaksATieByIdentity: two GitTargets created in the same second are still -// ordered, the same way by every caller. -func TestOlderGitTarget_BreaksATieByIdentity(t *testing.T) { - a := ownerTarget("team-a", "x", "p", time.Hour) - b := ownerTarget("team-b", "x", "p", time.Hour) - assert.True(t, OlderGitTarget(&a, &b)) - assert.False(t, OlderGitTarget(&b, &a)) - assert.False(t, OlderGitTarget(&a, &a)) +// TestBranchClaims_ARecreatedGitBranchStartsWithoutTheClaim: the UID holds the claim, so an object +// recreated under the same name holds nothing until its own proof. +func TestBranchClaims_ARecreatedGitBranchStartsWithoutTheClaim(t *testing.T) { + claims := NewBranchClaims() + key := BranchKeyFor("https://github.com/acme/platform.git", "main") + old := claimant("platform", "main", time.Hour) + claims.Observe(key, old, true) + claims.Release(old.UID) + + recreated := old + recreated.UID = "platform/main-2" + _, _, held := claims.HeldBy(recreated.GitBranch) + assert.False(t, held) + got, held := claims.Observe(key, recreated, true) + require.True(t, held) + assert.Equal(t, recreated.UID, got.UID) } diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 68e24e63..8dd5b423 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -66,11 +66,14 @@ var ( advertiseRemoteBranchFn = advertiseRemoteBranch ) -// BranchWorker processes events for a single (GitProvider, Branch) combination. +// BranchWorker processes events for a single repository branch, for the GitBranch holding it. // It can serve multiple GitTargets that write to different paths in the same branch. // This design ensures serialized commits per branch, preventing merge conflicts. type BranchWorker struct { - // Identity (immutable after creation) + // Identity (immutable after creation). GitBranch is the claim holder this worker writes for; + // GitProviderRef and GitProviderNamespace name that GitBranch's GitProvider, whose credential + // and signing key it uses. + GitBranch types.NamespacedName GitProviderRef string GitProviderNamespace string Branch string diff --git a/internal/git/branch_worker_loop_test.go b/internal/git/branch_worker_loop_test.go index 5dffc7b9..50f19665 100644 --- a/internal/git/branch_worker_loop_test.go +++ b/internal/git/branch_worker_loop_test.go @@ -29,9 +29,8 @@ func TestCommitWindowFor_DefaultsAndParsing(t *testing.T) { d := func(v time.Duration) *metav1.Duration { return &metav1.Duration{Duration: v} } target := func(name string, window *configv1alpha3.CommitWindow) *configv1alpha3.GitTarget { spec := configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "p"}, - Branch: "main", - Path: "clusters/prod", + GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName("p", "main")}, + Path: "clusters/prod", } if window != nil { spec.Commit = &configv1alpha3.GitTargetCommitSpec{Window: window} diff --git a/internal/git/branch_worker_queue_depth_test.go b/internal/git/branch_worker_queue_depth_test.go index ad1e72d8..f76f03aa 100644 --- a/internal/git/branch_worker_queue_depth_test.go +++ b/internal/git/branch_worker_queue_depth_test.go @@ -134,11 +134,7 @@ func TestEnsureWorker_WaitsForTheWorkerItReplacesToStop(t *testing.T) { go func() { _ = manager.EnsureWorker( context.Background(), - "test-provider", - "test-ns", - "main", - RepoIdentity{URL: testRepoURL}, - "", + claimForTest(manager, "test-provider", "test-ns", "main", RepoIdentity{URL: testRepoURL}), ) close(ensured) }() @@ -179,6 +175,7 @@ func TestReconcileWorkers_DoesNotStopAWorkerEnsureWorkerJustHandedOut(t *testing } retiring := newMetricsTestWorker() retiring.repo = RepoIdentity{URL: testRepoURL} + retiring.GitBranch = testGitBranch(retiring.GitProviderRef, retiring.GitProviderNamespace, retiring.Branch) manager.workers[key] = retiring ensured := make(chan struct{}) @@ -188,7 +185,10 @@ func TestReconcileWorkers_DoesNotStopAWorkerEnsureWorkerJustHandedOut(t *testing // The SAME repository the retiring worker is about, deliberately: that is what makes // an unguarded EnsureWorker take its "already there" path, which is the regression // this test is here to catch. - _ = manager.EnsureWorker(context.Background(), "test-provider", "test-ns", "main", retiring.repo, "") + _ = manager.EnsureWorker( + context.Background(), + claimForTest(manager, "test-provider", "test-ns", "main", retiring.repo), + ) }() // Long enough for an unguarded EnsureWorker to finish inside the window; a guarded one is // blocked on lifecycleMu and finishes after the sweep instead. @@ -254,11 +254,13 @@ func TestReconcileWorkers_DoesNotStopAWorkerCreatedAfterItsSnapshot(t *testing.T defer close(ensured) _ = manager.EnsureWorker( context.Background(), - "newcomer-provider", - "test-ns", - "main", - RepoIdentity{URL: testRepoURL}, - "", + claimForTest( + manager, + "newcomer-provider", + "test-ns", + "main", + RepoIdentity{URL: testRepoURL}, + ), ) }() time.Sleep(200 * time.Millisecond) diff --git a/internal/git/branch_worker_split_test.go b/internal/git/branch_worker_split_test.go index 30b6909a..3f57e38a 100644 --- a/internal/git/branch_worker_split_test.go +++ b/internal/git/branch_worker_split_test.go @@ -97,12 +97,11 @@ func createGitTarget( Namespace: "default", }, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: worker.GitProviderRef, + GitBranchRef: &meta.NamespacedObjectReference{ + Name: testGitBranchName(worker.GitProviderRef, worker.Branch), }, - Branch: worker.Branch, - Path: path, - Prune: prune, + Path: path, + Prune: prune, }, })) } diff --git a/internal/git/branch_worker_test.go b/internal/git/branch_worker_test.go index db9b9c20..749dc1a8 100644 --- a/internal/git/branch_worker_test.go +++ b/internal/git/branch_worker_test.go @@ -513,10 +513,7 @@ func TestBranchWorker_EnsurePathBootstrapped_RendersAllResolvedRecipients(t *tes target := &configv1alpha3.GitTarget{} target.Name = "bootstrap-target" target.Namespace = "default" - target.Spec.GitProviderRef = meta.NamespacedObjectReference{ - Name: "test-repo", - } - target.Spec.Branch = "main" + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} target.Spec.Path = "clusters/dev" target.Spec.Encryption = &configv1alpha3.EncryptionSpec{ Provider: "sops", @@ -833,8 +830,7 @@ func TestBranchWorker_CommitAndPushRequest_UsesProviderCommitterAndTargetMessage target := &configv1alpha3.GitTarget{} target.Name = "audit-target" target.Namespace = "default" - target.Spec.GitProviderRef = meta.NamespacedObjectReference{Name: "test-repo"} - target.Spec.Branch = "main" + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} target.Spec.Path = "clusters/dev" target.Spec.Commit = &configv1alpha3.GitTargetCommitSpec{ Message: &configv1alpha3.CommitMessageSpec{ @@ -932,8 +928,7 @@ func TestBranchWorker_CommitAndPushRequest_UsesBatchTemplateForAtomicRequest(t * target := &configv1alpha3.GitTarget{} target.Name = "demo-target" target.Namespace = "default" - target.Spec.GitProviderRef = meta.NamespacedObjectReference{Name: "test-repo"} - target.Spec.Branch = "main" + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} target.Spec.Path = "clusters/dev" target.Spec.Commit = &configv1alpha3.GitTargetCommitSpec{ Message: &configv1alpha3.CommitMessageSpec{ @@ -1248,10 +1243,7 @@ func createTargetWithEncryption( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = namespace - target.Spec.GitProviderRef = meta.NamespacedObjectReference{ - Name: providerName, - } - target.Spec.Branch = branch + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path target.Spec.Encryption = &configv1alpha3.EncryptionSpec{ Provider: "sops", @@ -1278,10 +1270,7 @@ func createTargetWithoutEncryption( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = namespace - target.Spec.GitProviderRef = meta.NamespacedObjectReference{ - Name: providerName, - } - target.Spec.Branch = branch + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path require.NoError(t, k8sClient.Create(ctx, target)) } @@ -1304,10 +1293,7 @@ func createTargetWithEncryptionSecretData( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = namespace - target.Spec.GitProviderRef = meta.NamespacedObjectReference{ - Name: providerName, - } - target.Spec.Branch = branch + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path target.Spec.Encryption = &configv1alpha3.EncryptionSpec{ Provider: "sops", diff --git a/internal/git/claim_helpers_test.go b/internal/git/claim_helpers_test.go new file mode 100644 index 00000000..08f73dbc --- /dev/null +++ b/internal/git/claim_helpers_test.go @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "time" + + k8stypes "k8s.io/apimachinery/pkg/types" +) + +// testGitBranch is the GitBranch a test worker writes for: one per provider and branch, in the +// provider's namespace, as a GitBranch's gitProviderRef is always local. +func testGitBranch(provider, namespace, branch string) k8stypes.NamespacedName { + return k8stypes.NamespacedName{Namespace: namespace, Name: testGitBranchName(provider, branch)} +} + +// claimForTest makes testGitBranch(provider, namespace, branch) hold the repository branch, as its +// reconcile does once its credential is proved, and returns the request its GitTargets' reconcile +// would make. +func claimForTest(m *WorkerManager, provider, namespace, branch string, repo RepoIdentity) WorkerRequest { + gitBranch := testGitBranch(provider, namespace, branch) + m.Claims().Observe(BranchKeyFor(repo.URL, branch), BranchClaimant{ + GitBranch: gitBranch, + UID: k8stypes.UID(gitBranch.String()), + Created: time.Unix(0, 0), + }, true) + return WorkerRequest{ + GitBranch: gitBranch, + Provider: provider, + Branch: branch, + Repo: repo, + } +} + +// testGitBranchName is the name of testGitBranch(provider, namespace, branch). +func testGitBranchName(provider, branch string) string { return provider + "-" + branch } diff --git a/internal/git/credentials.go b/internal/git/credentials.go index 31b315c5..3f3e60a5 100644 --- a/internal/git/credentials.go +++ b/internal/git/credentials.go @@ -64,6 +64,18 @@ func getAuthFromSecret( return CredentialOptionsForProvider(provider, cred, transportPolicy) } +// AuthForProvider resolves the GitProvider's credential into the transport options its branch +// worker pushes with, so a caller proving the credential proves the one that will be used. +func AuthForProvider( + ctx context.Context, + k8sClient client.Client, + provider *v1alpha3.GitProvider, + hostKeys SSHHostKeyConfig, + transportPolicy CredentialTransportPolicy, +) ([]gitclient.Option, error) { + return getAuthFromSecret(ctx, k8sClient, provider, hostKeys, transportPolicy) +} + // CredentialOptionsForProvider applies transport policy to a resolved credential. func CredentialOptionsForProvider( provider *v1alpha3.GitProvider, diff --git a/internal/git/git_atomic_push.go b/internal/git/git_atomic_push.go index ff303375..f1a50438 100644 --- a/internal/git/git_atomic_push.go +++ b/internal/git/git_atomic_push.go @@ -44,7 +44,12 @@ func getPushSession( return nil, errors.New("remote origin has no URL configured") } - endpoint, err := transport.ParseURL(urls[0]) + return openReceivePack(ctx, urls[0], auth) +} + +// openReceivePack opens a receive-pack session on rawURL: the handshake a push starts with. +func openReceivePack(ctx context.Context, rawURL string, auth []gitclient.Option) (transport.Session, error) { + endpoint, err := transport.ParseURL(rawURL) if err != nil { return nil, fmt.Errorf("failed to parse remote URL: %w", err) } @@ -60,6 +65,23 @@ func getPushSession( return session, nil } +// ProvePushSession opens a receive-pack session on repoURL with auth and closes it again, sending +// nothing. It is the proof a GitBranch gives before it may claim a repository branch. +// +// What it proves depends on the host. GitHub and GitLab refuse the session without push permission. +// Gitea opens it for any authenticated user who can read the repository (its AGit flow pushes to +// refs/for/* with read access) and refuses a missing or wrong credential. A host that advertises +// receive-pack anonymously proves only that it is reachable. +func ProvePushSession(ctx context.Context, repoURL string, auth []gitclient.Option) error { + callCtx, cancel := boundGitCall(ctx) + defer cancel() + session, err := openReceivePack(callCtx, repoURL, auth) + if err = boundedCallError(callCtx, err); err != nil { + return err + } + return session.Close() +} + // advertisedHashes indexes a v6 advertisement by reference name. v5 handed back a // map[string]plumbing.Hash directly; v6's transport.RemoteRefs carries a []*plumbing.Reference so // that fields can be added without breaking the interface, so the lookup is built here. diff --git a/internal/git/git_branch_ref.go b/internal/git/git_branch_ref.go new file mode 100644 index 00000000..2dcfa0e5 --- /dev/null +++ b/internal/git/git_branch_ref.go @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "context" + "fmt" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/ConfigButler/gitops-reverser/api/v1alpha3" +) + +// ResolveGitBranch reads the GitBranch target references, and that GitBranch's GitProvider. An +// object that does not exist comes back nil with no error: a missing reference is a status, not a +// failure to retry. A target with no spec.gitBranchRef resolves to nothing. +func ResolveGitBranch( + ctx context.Context, + reader client.Reader, + target *v1alpha3.GitTarget, +) (*v1alpha3.GitBranch, *v1alpha3.GitProvider, error) { + key := target.GitBranchKey() + if key.Name == "" { + return nil, nil, nil + } + branch := &v1alpha3.GitBranch{} + if err := reader.Get(ctx, key, branch); err != nil { + if apierrors.IsNotFound(err) { + return nil, nil, nil + } + return nil, nil, fmt.Errorf("read GitBranch %s: %w", key, err) + } + provider := &v1alpha3.GitProvider{} + if err := reader.Get(ctx, branch.GitProviderKey(), provider); err != nil { + if apierrors.IsNotFound(err) { + return branch, nil, nil + } + return nil, nil, fmt.Errorf("read GitProvider %s: %w", branch.GitProviderKey(), err) + } + return branch, provider, nil +} diff --git a/internal/git/git_branch_ref_test.go b/internal/git/git_branch_ref_test.go new file mode 100644 index 00000000..fe228df8 --- /dev/null +++ b/internal/git/git_branch_ref_test.go @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "context" + "testing" + + "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +func refTarget(namespace, gitBranch string) *configv1alpha3.GitTarget { + target := &configv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}} + if gitBranch != "" { + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: gitBranch, Namespace: namespace} + } + return target +} + +// TestResolveGitBranch_FollowsTheTargetToItsBranchAndProvider: a GitTarget's reference resolves to +// its GitBranch and that GitBranch's GitProvider, in the GitBranch's namespace. Anything missing is +// nil without an error: it is a status, not a failure to retry. +func TestResolveGitBranch_FollowsTheTargetToItsBranchAndProvider(t *testing.T) { + branch := &configv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Namespace: "platform", Name: "main"}, + Spec: configv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: "acme"}, Branch: "main", + }, + } + orphan := &configv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Namespace: "platform", Name: "orphan"}, + Spec: configv1alpha3.GitBranchSpec{GitProviderRef: meta.LocalObjectReference{Name: "gone"}, Branch: "x"}, + } + provider := &configv1alpha3.GitProvider{ + ObjectMeta: metav1.ObjectMeta{Namespace: "platform", Name: "acme"}, + Spec: configv1alpha3.GitProviderSpec{URL: "https://github.com/acme/platform"}, + } + reader := fake.NewClientBuilder().WithScheme(setupScheme()).WithObjects(branch, orphan, provider).Build() + ctx := context.Background() + + gotBranch, gotProvider, err := ResolveGitBranch(ctx, reader, refTarget("platform", "main")) + require.NoError(t, err) + assert.Equal(t, "main", gotBranch.Spec.Branch) + assert.Equal(t, provider.Spec.URL, gotProvider.Spec.URL) + + gotBranch, gotProvider, err = ResolveGitBranch(ctx, reader, refTarget("platform", "orphan")) + require.NoError(t, err) + assert.NotNil(t, gotBranch) + assert.Nil(t, gotProvider, "a GitBranch whose GitProvider is gone") + + gotBranch, _, err = ResolveGitBranch(ctx, reader, refTarget("", "main")) + require.NoError(t, err) + assert.Nil(t, gotBranch, "no namespace means the target's own, where there is no such GitBranch") + + gotBranch, _, err = ResolveGitBranch(ctx, reader, refTarget("", "")) + require.NoError(t, err) + assert.Nil(t, gotBranch, "a target with no reference resolves to nothing") +} + +// TestAuthForProvider_AnAnonymousProviderSendsNothing: a GitProvider with no secretRef proves +// itself, and pushes, with no credential at all. +func TestAuthForProvider_AnAnonymousProviderSendsNothing(t *testing.T) { + provider := &configv1alpha3.GitProvider{ + ObjectMeta: metav1.ObjectMeta{Namespace: "platform", Name: "acme"}, + Spec: configv1alpha3.GitProviderSpec{URL: "https://github.com/acme/platform"}, + } + k8s := fake.NewClientBuilder().WithScheme(setupScheme()).Build() + + auth, err := AuthForProvider(context.Background(), k8s, provider, SSHHostKeyConfig{}, CredentialTransportPolicy{}) + require.NoError(t, err) + assert.Empty(t, auth) +} + +// TestWorkerForTarget_AndNotifyPushFindTheHoldersWorker: a GitTarget reaches its worker through its +// gitBranchRef, and a push notification, which names a GitProvider and a branch, reaches the worker +// of the GitBranch writing them. +func TestWorkerForTarget_AndNotifyPushFindTheHoldersWorker(t *testing.T) { + m, ctx := startedManager(t) + request := claimForTest(m, "acme", "team-a", "main", RepoIdentity{URL: ownerTestURL}) + require.NoError(t, m.EnsureWorker(ctx, request)) + + worker, found := m.WorkerForTarget(refTarget("", testGitBranchName("acme", "main"))) + require.True(t, found) + assert.Equal(t, request.GitBranch, worker.GitBranch) + _, found = m.WorkerForTarget(refTarget("", "other")) + assert.False(t, found) + + assert.True(t, m.NotifyPush("team-a", "acme", "main", "abc")) + assert.False(t, m.NotifyPush("team-a", "acme", "release", "abc"), "no worker writes that branch") + assert.False(t, m.NotifyPush("team-b", "acme", "main", "abc"), "nor through that GitProvider") +} + +// TestRecordRemoteObservation_WakesTheGitBranch: what the worker proves reaches the GitBranch's +// reconcile at once, so status.remote follows a push within one reconcile. +func TestRecordRemoteObservation_WakesTheGitBranch(t *testing.T) { + m := NewWorkerManager(nil, logr.Discard(), BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) + key := BranchKeyFor(ownerTestURL, "main") + gitBranch := testGitBranch("acme", "team-a", "main") + m.workers[key] = &BranchWorker{GitBranch: gitBranch, Branch: "main"} + + m.recordRemoteObservation(key, RemoteObservation{Commit: "abc"}) + + require.Len(t, m.GitBranchEvents(), 1) + woken := (<-m.GitBranchEvents()).Object + assert.Equal(t, gitBranch, client.ObjectKeyFromObject(woken)) +} diff --git a/internal/git/git_roundtrip_ledger_test.go b/internal/git/git_roundtrip_ledger_test.go index e067f64a..5fa46371 100644 --- a/internal/git/git_roundtrip_ledger_test.go +++ b/internal/git/git_roundtrip_ledger_test.go @@ -119,6 +119,7 @@ func newLedgerFixtureOnBranch(t *testing.T, slug string, seeded bool, branch str BranchWorkerLimits{}, ) worker.ctx = ctx + worker.GitBranch = testGitBranch(providerName, "default", branch) t.Cleanup(func() { _ = os.RemoveAll(worker.repoRootPath()) }) return &ledgerFixture{t: t, sim: sim, repoDir: repoDir, worker: worker} @@ -174,10 +175,11 @@ func (f *ledgerFixture) createLedgerTarget(path string, prune *configv1alpha3.Pr require.NoError(f.t, f.worker.Client.Create(f.worker.ctx, &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: ledgerTargetName, Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: f.worker.GitProviderRef}, - Branch: f.worker.Branch, - Path: path, - Prune: prune, + GitBranchRef: &meta.NamespacedObjectReference{ + Name: testGitBranchName(f.worker.GitProviderRef, f.worker.Branch), + }, + Path: path, + Prune: prune, }, })) } diff --git a/internal/git/prove_push_session_test.go b/internal/git/prove_push_session_test.go new file mode 100644 index 00000000..d7590880 --- /dev/null +++ b/internal/git/prove_push_session_test.go @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "context" + "net/http" + "os/exec" + "path/filepath" + "testing" + + gogithttp "github.com/go-git/go-git/v6/plumbing/transport/http" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// pushSessionServer serves a seeded bare repository over canonical git's HTTP backend, the way a +// Git host does: anyone may fetch, and a push session needs the user "bot" with the password +// "secret". receivePack opts the repository into pushes over HTTP at all. +func pushSessionServer(t *testing.T, receivePack bool) string { + t.Helper() + projectRoot := t.TempDir() + repoDir := filepath.Join(projectRoot, "repo.git") + createBareRepo(t, repoDir) + if receivePack { + out, err := exec.Command("git", "-C", repoDir, "config", "http.receivepack", "true").CombinedOutput() + require.NoError(t, err, "git config http.receivepack: %s", out) + } + simulateClientCommitOnDisk(t, repoDir, "main", "README.md", "seed\n") + + sim := startRealGitServer(t, projectRoot, repoDir) + around := func(w http.ResponseWriter, r *http.Request, backend http.Handler) { + if r.URL.Query().Get("service") == "git-receive-pack" || isReceivePack(r) { + if user, password, ok := r.BasicAuth(); !ok || user != "bot" || password != "secret" { + w.Header().Set("WWW-Authenticate", `Basic realm="git"`) + http.Error(w, "authentication required", http.StatusUnauthorized) + return + } + } + backend.ServeHTTP(w, r) + } + sim.around.Store(&around) + return sim.RepoURL +} + +// botAuth is the user "bot" with password. +func botAuth(password string) Credential { + return Credential{Basic: &gogithttp.BasicAuth{Username: "bot", Password: password}} +} + +// TestProvePushSession_NeedsTheCredentialThatMayPush: the proof a GitBranch gives before it claims +// a branch is a receive-pack session opened with its GitProvider's credential, over a real Git +// server (file:// is not one). Fetching anonymously proves nothing about pushing. +func TestProvePushSession_NeedsTheCredentialThatMayPush(t *testing.T) { + url := pushSessionServer(t, true) + ctx := context.Background() + + _, err := CheckRepo(ctx, url, nil) + require.NoError(t, err, "an anonymous fetch works, which is why it cannot be the proof") + + require.Error(t, ProvePushSession(ctx, url, nil), "no credential") + require.Error(t, ProvePushSession(ctx, url, botAuth("wrong").Options()), "a wrong credential") + assert.NoError(t, ProvePushSession(ctx, url, botAuth("secret").Options())) +} + +// TestProvePushSession_FailsWhereTheHostRefusesPushes: a host that does not offer receive-pack for +// the repository refuses the session whatever the credential. +func TestProvePushSession_FailsWhereTheHostRefusesPushes(t *testing.T) { + url := pushSessionServer(t, false) + + assert.Error(t, ProvePushSession(context.Background(), url, botAuth("secret").Options())) +} + +// TestProvePushSession_FailsForARepositoryThatIsNotThere: a URL that names nothing proves nothing. +func TestProvePushSession_FailsForARepositoryThatIsNotThere(t *testing.T) { + url := pushSessionServer(t, true) + + assert.Error(t, ProvePushSession(context.Background(), url+"-missing", botAuth("secret").Options())) +} diff --git a/internal/git/publication.go b/internal/git/publication.go index fdae6cb6..2c253075 100644 --- a/internal/git/publication.go +++ b/internal/git/publication.go @@ -180,19 +180,20 @@ func (m *WorkerManager) PublicationEvents() <-chan event.GenericEvent { return m // notifyPublication tells every GitTarget on the branch that its worker's report changed. It reads // the GitTargets from the cache, and never blocks: the worker's loop calls it, by way of a // goroutine, and a status that lags is better than a loop that waits for a controller. -func (m *WorkerManager) notifyPublication(ctx context.Context, provider k8stypes.NamespacedName, branch string) { +func (m *WorkerManager) notifyPublication(ctx context.Context, gitBranch k8stypes.NamespacedName) { + m.notifyGitBranchLocked(gitBranch) if m.Client == nil || m.publicationEvents == nil { return } var targets configv1alpha3.GitTargetList if err := m.Client.List(ctx, &targets); err != nil { m.Log.V(1).Info("Cannot list the GitTargets of a branch to report its publication", - "provider", provider.String(), "branch", branch, "error", err.Error()) + "gitBranch", gitBranch.String(), "error", err.Error()) return } for i := range targets.Items { target := &targets.Items[i] - if !WritesThrough(target, provider, branch) { + if !WritesThrough(target, gitBranch) { continue } select { diff --git a/internal/git/publication_test.go b/internal/git/publication_test.go index 7092b6ce..d8adb0ec 100644 --- a/internal/git/publication_test.go +++ b/internal/git/publication_test.go @@ -130,30 +130,32 @@ func TestPublication_MetricsDescribeTheBacklog(t *testing.T) { assert.False(t, ok, "nothing pending, no retry due") } -// A change of the report reaches every GitTarget on the branch, and none on another branch: they -// share the worker, so they share its outage, and a second branch's worker is its own. +// A change of the report reaches every GitTarget on the GitBranch, and none on another: they share +// the worker, so they share its outage, and a second GitBranch's worker is its own. The GitBranch +// itself is woken too, to republish its status. func TestPublication_ChangesReachEveryGitTargetOnTheBranch(t *testing.T) { - target := func(name, branch string) *configv1alpha3.GitTarget { + target := func(namespace, name, gitBranch string) *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "shop"}, + ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "repo"}, - Branch: branch, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranch, Namespace: "shop"}, }, } } - shared := target("team-b", "main") - shared.Namespace, shared.Spec.GitProviderRef.Namespace = "team-b", "shop" k8s := fake.NewClientBuilder().WithScheme(replayScheme(t)).WithObjects( - target("apps", "main"), target("infra", "main"), target("other", "release"), shared).Build() + target("shop", "apps", "main"), target("shop", "infra", "main"), target("shop", "other", "release"), + target("team-b", "team-b", "main")).Build() manager := NewWorkerManager(k8s, logr.Discard(), BranchWorkerLimits{}, itypes.SensitiveResourcePolicy{}) - manager.notifyPublication(context.Background(), k8stypes.NamespacedName{Namespace: "shop", Name: "repo"}, "main") + manager.notifyPublication(context.Background(), k8stypes.NamespacedName{Namespace: "shop", Name: "main"}) var told []string for len(manager.PublicationEvents()) > 0 { told = append(told, (<-manager.PublicationEvents()).Object.GetName()) } assert.ElementsMatch(t, []string{"apps", "infra", "team-b"}, told, - "a GitTarget in another namespace writing through the shared GitProvider is on the branch too") + "a GitTarget in another namespace referencing the GitBranch is on the branch too") + require.Len(t, manager.GitBranchEvents(), 1) + woken := (<-manager.GitBranchEvents()).Object + assert.Equal(t, "shop/main", woken.GetNamespace()+"/"+woken.GetName()) } diff --git a/internal/git/refusal_reapply_test.go b/internal/git/refusal_reapply_test.go index 9567bd4e..b5667974 100644 --- a/internal/git/refusal_reapply_test.go +++ b/internal/git/refusal_reapply_test.go @@ -44,8 +44,7 @@ func refusalTouchWorker(t *testing.T, spec configv1alpha3.GitTargetSpec) *Branch require.NoError(t, clientgoscheme.AddToScheme(scheme)) require.NoError(t, configv1alpha3.AddToScheme(scheme)) - spec.GitProviderRef.Name = "test-provider" - spec.Branch = "main" + spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")} if spec.Path == "" { spec.Path = "apps" } @@ -65,6 +64,7 @@ func refusalTouchWorker(t *testing.T, spec configv1alpha3.GitTargetSpec) *Branch // A GitTarget names its GitProvider in its own namespace, which is where a worker looks for // the appliers on its branch. w.GitProviderNamespace = target.Namespace + w.GitBranch = testGitBranch(w.GitProviderRef, w.GitProviderNamespace, w.Branch) w.Client = client w.Log = logr.Discard() w.ctx = context.Background() @@ -246,8 +246,7 @@ func TestRefusalReapply_TrailingRequestRechecksConsent(t *testing.T) { setSpec := func(spec configv1alpha3.GitTargetSpec) { target := &configv1alpha3.GitTarget{} require.NoError(t, w.Client.Get(w.ctx, client.ObjectKey{Name: "editing", Namespace: "team-a"}, target)) - spec.GitProviderRef = target.Spec.GitProviderRef - spec.Branch, spec.Path = target.Spec.Branch, target.Spec.Path + spec.GitBranchRef, spec.Path = target.Spec.GitBranchRef, target.Spec.Path target.Spec = spec require.NoError(t, w.Client.Update(w.ctx, target)) } @@ -407,13 +406,13 @@ func TestRefusalReapply_OneTargetDoesNotDropAnothersPendingRequest(t *testing.T) target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - Branch: "main", Path: "apps/" + name, + Path: "apps/" + name, OnRefusal: configv1alpha3.RefusalActionReapply, ApplyTriggerRef: testApplier(name), Suspend: suspend, }, } - target.Spec.GitProviderRef.Name = "test-provider" + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")} return target } // B is suspended, so its delayed request must be declined when the timer fires. A is not. diff --git a/internal/git/remote_report_test.go b/internal/git/remote_report_test.go index a0616b61..1ddc211e 100644 --- a/internal/git/remote_report_test.go +++ b/internal/git/remote_report_test.go @@ -56,12 +56,12 @@ func TestRemoteForBranch_SurvivesTheWorkerItWasProvedBy(t *testing.T) { after := RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"} key := BranchKeyFor(after.URL, "main") // where the GitTarget looks: its GitProvider names after - require.NoError(t, m.EnsureWorker(ctx, "repo1", "gitops-system", "main", before, "")) - worker, ok := m.GetWorkerForTarget("repo1", "gitops-system", "main") + require.NoError(t, m.EnsureWorker(ctx, claimForTest(m, "repo1", "gitops-system", "main", before))) + worker, ok := m.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) worker.recordRemoteObservation("aaaa", ObservedByPush) - require.NoError(t, m.EnsureWorker(ctx, "repo1", "gitops-system", "main", after, "")) + require.NoError(t, m.EnsureWorker(ctx, claimForTest(m, "repo1", "gitops-system", "main", after))) observed, known := m.RemoteForBranch(key) require.True(t, known, diff --git a/internal/git/secret_write_test.go b/internal/git/secret_write_test.go index 99ee6e7f..ac2fec2c 100644 --- a/internal/git/secret_write_test.go +++ b/internal/git/secret_write_test.go @@ -68,11 +68,8 @@ func secretTargetObjects(t *testing.T, providerName, branch, path string) []clie Namespace: "default", }, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{ - Name: providerName, - }, - Branch: branch, - Path: path, + GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)}, + Path: path, Encryption: &configv1alpha3.EncryptionSpec{ Provider: "sops", Age: &configv1alpha3.AgeEncryptionSpec{ diff --git a/internal/git/types.go b/internal/git/types.go index 156ad259..df10a33b 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -177,10 +177,20 @@ func (k BranchKey) String() string { return k.Repository + "@" + k.Branch } -// WritesThrough reports whether target writes branch through the GitProvider provider: the pair a -// branch worker serves. -func WritesThrough(target *v1alpha3.GitTarget, provider k8stypes.NamespacedName, branch string) bool { - return target.GitProviderKey() == provider && target.Spec.Branch == branch +// OlderGitTarget reports whether a was created before b, ties broken by namespace and name, so every +// caller orders two GitTargets the same way. +func OlderGitTarget(a, b *v1alpha3.GitTarget) bool { + at, bt := a.CreationTimestamp.Time, b.CreationTimestamp.Time + if !at.Equal(bt) { + return at.Before(bt) + } + return a.Namespace+"/"+a.Name < b.Namespace+"/"+b.Name +} + +// WritesThrough reports whether target writes through gitBranch: the GitBranch a branch worker +// serves. +func WritesThrough(target *v1alpha3.GitTarget, gitBranch k8stypes.NamespacedName) bool { + return gitBranch.Name != "" && target.GitBranchKey() == gitBranch } // RepoIdentity is WHICH REPOSITORY a branch worker's clone, base trust and observations are diff --git a/internal/git/types_test.go b/internal/git/types_test.go index 90b06057..b9244aa0 100644 --- a/internal/git/types_test.go +++ b/internal/git/types_test.go @@ -28,25 +28,25 @@ func TestBranchKeyFor_OneKeyPerRepositoryBranch(t *testing.T) { assert.Equal(t, "github.com/acme/platform@main", https.String()) } -// TestWritesThrough_IsTheGitProviderAndTheBranch: the GitTargets a worker serves are those that -// write its branch through its GitProvider, from whichever namespace. -func TestWritesThrough_IsTheGitProviderAndTheBranch(t *testing.T) { - shared := k8stypes.NamespacedName{Namespace: "platform", Name: "monorepo"} +// TestWritesThrough_IsTheGitBranch: the GitTargets a worker serves are those that reference its +// GitBranch, from whichever namespace. +func TestWritesThrough_IsTheGitBranch(t *testing.T) { + shared := k8stypes.NamespacedName{Namespace: "platform", Name: "main"} target := &v1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}, Spec: v1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "monorepo", Namespace: "platform"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, }, } - assert.True(t, WritesThrough(target, shared, "main")) - assert.False(t, WritesThrough(target, shared, "develop")) - assert.False(t, WritesThrough(target, k8stypes.NamespacedName{Namespace: "team-a", Name: "monorepo"}, "main"), - "a GitProvider of the same name in the GitTarget's own namespace is another one") + assert.True(t, WritesThrough(target, shared)) + assert.False(t, WritesThrough(target, k8stypes.NamespacedName{Namespace: "team-a", Name: "main"}), + "a GitBranch of the same name in the GitTarget's own namespace is another one") - target.Spec.GitProviderRef.Namespace = "" - assert.True(t, WritesThrough(target, k8stypes.NamespacedName{Namespace: "team-a", Name: "monorepo"}, "main"), + target.Spec.GitBranchRef.Namespace = "" + assert.True(t, WritesThrough(target, k8stypes.NamespacedName{Namespace: "team-a", Name: "main"}), "no namespace means the GitTarget's own") + target.Spec.GitBranchRef = nil + assert.False(t, WritesThrough(target, k8stypes.NamespacedName{}), "no reference writes through nothing") } // TestEventWithObject verifies event creation with objects. diff --git a/internal/git/worker_branch_owner_test.go b/internal/git/worker_branch_owner_test.go index 73110ef7..af13ebde 100644 --- a/internal/git/worker_branch_owner_test.go +++ b/internal/git/worker_branch_owner_test.go @@ -11,79 +11,94 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "sigs.k8s.io/controller-runtime/pkg/client" + k8stypes "k8s.io/apimachinery/pkg/types" configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) const ownerTestURL = "https://example.invalid/platform.git" -// TestEnsureWorker_RefusesASecondGitProviderOnOneRepositoryBranch: a worker is keyed by repository -// and branch, and pushes with one GitProvider's credential. A second GitProvider naming the same -// repository, however spelled, gets no worker of its own on that branch; on another branch it does. -func TestEnsureWorker_RefusesASecondGitProviderOnOneRepositoryBranch(t *testing.T) { +// TestEnsureWorker_OnlyTheHolderGetsAWorker: a worker is keyed by repository and branch, and pushes +// with the credential of the GitBranch holding it. A GitBranch that does not hold the branch gets no +// worker on it, however its GitProvider spells the repository; on another branch it can. +func TestEnsureWorker_OnlyTheHolderGetsAWorker(t *testing.T) { m, ctx := startedManager(t) teamA := RepoIdentity{ProviderUID: "uid-a", URL: ownerTestURL} teamB := RepoIdentity{ProviderUID: "uid-b", URL: "git@example.invalid:platform"} - require.NoError(t, m.EnsureWorker(ctx, "monorepo", "team-a", "main", teamA, "")) - err := m.EnsureWorker(ctx, "monorepo", "team-b", "main", teamB, "") - - require.ErrorIs(t, err, ErrBranchOwnedElsewhere) - _, found := m.GetWorkerForTarget("monorepo", "team-b", "main") - assert.False(t, found, "the refused GitProvider has no worker") - owner, found := m.GetWorkerForTarget("monorepo", "team-a", "main") + require.NoError(t, m.EnsureWorker(ctx, claimForTest(m, "monorepo", "team-a", "main", teamA))) + refused := WorkerRequest{ + GitBranch: testGitBranch("monorepo", "team-b", "main"), Provider: "monorepo", Branch: "main", Repo: teamB, + } + require.ErrorIs(t, m.EnsureWorker(ctx, refused), ErrBranchOwnedElsewhere) + _, found := m.WorkerForGitBranch(refused.GitBranch) + assert.False(t, found, "the GitBranch that does not hold the branch has no worker") + owner, found := m.WorkerForGitBranch(testGitBranch("monorepo", "team-a", "main")) require.True(t, found) assert.Equal(t, BranchKeyFor(ownerTestURL, "main"), owner.branchKey()) - require.NoError(t, m.EnsureWorker(ctx, "monorepo", "team-b", "release", teamB, ""), + require.NoError(t, m.EnsureWorker(ctx, claimForTest(m, "monorepo", "team-b", "release", teamB)), "another branch of the repository is another worker") } -// TestReconcileWorkers_StopsTheWorkerOfAGitProviderThatDoesNotOwnItsBranch: two GitTargets created -// too close together for either reconcile to see the other can both be wired. The sweep decides as -// the reconciler does, so the younger GitTarget's worker is stopped, and the owner's next reconcile -// gets the branch. -func TestReconcileWorkers_StopsTheWorkerOfAGitProviderThatDoesNotOwnItsBranch(t *testing.T) { +// TestReconcileWorkers_StopsTheWorkerOfAGitBranchThatLostItsBranch: an older GitBranch that proves +// itself later takes the branch over (rule 2). The sweep stops the previous holder's worker, the new +// holder's next reconcile starts its own, and the sweep keeps that one while a GitTarget references +// its GitBranch. +func TestReconcileWorkers_StopsTheWorkerOfAGitBranchThatLostItsBranch(t *testing.T) { m, ctx := startedManager(t) - created := time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC) - // A GitTarget its ClusterProvider does not admit may not write, and decides nothing. - require.NoError(t, m.Client.Create(ctx, &configv1alpha3.ClusterProvider{ - ObjectMeta: metav1.ObjectMeta{Name: configv1alpha3.DefaultClusterProviderName}, - Spec: configv1alpha3.ClusterProviderSpec{ - AccessFrom: &configv1alpha3.NamespaceMatcher{Selector: &metav1.LabelSelector{}}, - }, - })) - for _, ns := range []string{"team-a", "team-b"} { - require.NoError(t, m.Client.Create(ctx, &configv1alpha3.GitProvider{ - ObjectMeta: metav1.ObjectMeta{Namespace: ns, Name: "monorepo"}, - Spec: configv1alpha3.GitProviderSpec{URL: ownerTestURL, AllowedBranches: []string{"*"}}, - })) - } - target := func(ns string, age time.Duration) client.Object { - return &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{ - Namespace: ns, Name: "apps", CreationTimestamp: metav1.NewTime(created.Add(-age)), - }, + repo := RepoIdentity{URL: ownerTestURL} + key := BranchKeyFor(ownerTestURL, "main") + younger := testGitBranch("monorepo", "team-b", "main") + older := testGitBranch("monorepo", "team-a", "main") + for _, gitBranch := range []k8stypes.NamespacedName{younger, older} { + require.NoError(t, m.Client.Create(ctx, &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: gitBranch.Namespace, Name: "apps"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "monorepo"}, Branch: "main", Path: ns, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranch.Name}, Path: gitBranch.Namespace, }, - } + })) } - require.NoError(t, m.Client.Create(ctx, target("team-a", 2*time.Hour))) - require.NoError(t, m.Client.Create(ctx, target("team-b", time.Hour))) - // The younger GitTarget's reconcile won the race to the worker. - require.NoError(t, m.EnsureWorker(ctx, "monorepo", "team-b", "main", RepoIdentity{URL: ownerTestURL}, "")) - require.ErrorIs(t, m.EnsureWorker(ctx, "monorepo", "team-a", "main", RepoIdentity{URL: ownerTestURL}, ""), - ErrBranchOwnedElsewhere) + require.NoError(t, m.EnsureWorker(ctx, claimForTest(m, "monorepo", "team-b", "main", repo))) + m.Claims(). + Observe(key, BranchClaimant{GitBranch: older, UID: "older", Created: time.Unix(0, 0).Add(-time.Hour)}, true) + require.ErrorIs( + t, + m.EnsureWorker(ctx, WorkerRequest{GitBranch: older, Provider: "monorepo", Branch: "main", Repo: repo}), + ErrBranchOwnedElsewhere, + "the previous holder's worker is still live", + ) require.NoError(t, m.ReconcileWorkers(ctx)) - _, found := m.GetWorkerForTarget("monorepo", "team-b", "main") - assert.False(t, found, "the GitProvider of the younger GitTarget does not own the branch") + _, found := m.WorkerForGitBranch(younger) + assert.False(t, found, "the GitBranch that lost the branch loses its worker") - require.NoError(t, m.EnsureWorker(ctx, "monorepo", "team-a", "main", RepoIdentity{URL: ownerTestURL}, "")) + require.NoError( + t, + m.EnsureWorker(ctx, WorkerRequest{GitBranch: older, Provider: "monorepo", Branch: "main", Repo: repo}), + ) require.NoError(t, m.ReconcileWorkers(context.Background())) - _, found = m.GetWorkerForTarget("monorepo", "team-a", "main") - assert.True(t, found, "and the owner's worker is the one the sweep keeps") + _, found = m.WorkerForGitBranch(older) + assert.True(t, found, "and the holder's worker is the one the sweep keeps") +} + +// TestRetireGitBranch_StopsTheWorkerAndForgetsTheBranch: a GitBranch being deleted stops its worker +// before its finalizer goes, and nothing the manager kept about the branch outlives it. +func TestRetireGitBranch_StopsTheWorkerAndForgetsTheBranch(t *testing.T) { + m, ctx := startedManager(t) + repo := RepoIdentity{URL: ownerTestURL} + request := claimForTest(m, "monorepo", "platform", "main", repo) + require.NoError(t, m.EnsureWorker(ctx, request)) + worker, found := m.WorkerForGitBranch(request.GitBranch) + require.True(t, found) + m.recordRemoteObservation(worker.branchKey(), RemoteObservation{Commit: "abc", At: time.Now()}) + + m.RetireGitBranch(request.GitBranch) + + _, found = m.WorkerForGitBranch(request.GitBranch) + assert.False(t, found) + _, known := m.RemoteForBranch(worker.branchKey()) + assert.False(t, known, "the next holder starts from the remote") + m.RetireGitBranch(request.GitBranch) // a second call finds nothing to do } diff --git a/internal/git/worker_manager.go b/internal/git/worker_manager.go index 883bad14..90e7670a 100644 --- a/internal/git/worker_manager.go +++ b/internal/git/worker_manager.go @@ -167,6 +167,16 @@ type WorkerManager struct { // PublicationEvents. publicationEvents chan event.GenericEvent + // gitBranchEvents carries the GitBranch whose worker proved something new about the remote or + // whose publication report changed. See GitBranchEvents. + gitBranchEvents chan event.GenericEvent + + // claims is which GitBranch holds each repository branch (branch_owner.go). The GitBranch + // reconciler writes it; EnsureWorker and the sweep read it. Reach it through Claims, which makes + // one for a manager not built by NewWorkerManager. + claims *BranchClaims + claimsOnce sync.Once + // initialSnapshots and resyncRecheck are the watch side's half of the applier-echo rules every // worker follows (apply_epoch.go). Set once at startup (SetSnapshotHooks); nil in the CLI and in // tests. @@ -212,10 +222,22 @@ func NewWorkerManager( replacements: make(map[BranchKey]struct{}), renderFidelityGate: NewRenderFidelityGate(), publicationEvents: make(chan event.GenericEvent, publicationEventBuffer), + gitBranchEvents: make(chan event.GenericEvent, publicationEventBuffer), + claims: NewBranchClaims(), applierAdmitted: authz.ApplierAdmitted, } } +// Claims is the registry of which GitBranch holds each repository branch. +func (m *WorkerManager) Claims() *BranchClaims { + m.claimsOnce.Do(func() { + if m.claims == nil { + m.claims = NewBranchClaims() + } + }) + return m.claims +} + // RenderFidelityGate returns the manager-wide target gate used by branch workers. The gate is // safe for concurrent watch and worker access. func (m *WorkerManager) RenderFidelityGate() *RenderFidelityGate { @@ -322,6 +344,9 @@ func (m *WorkerManager) recordRemoteObservation(key BranchKey, observed RemoteOb m.remotes = map[BranchKey]RemoteObservation{} } m.remotes[key] = observed + if worker, live := m.workers[key]; live { + m.notifyGitBranchLocked(worker.GitBranch) + } } // RemoteForBranch is what this branch was last proved to be at, and whether anything has looked at @@ -338,16 +363,36 @@ func (m *WorkerManager) RemoteForBranch(key BranchKey) (RemoteObservation, bool) return observed, known } -// EnsureWorker ensures a worker exists for the given (provider, branch), keyed by the repository -// branch it writes (BranchKey), and that it is the worker for the repository the GitProvider names -// NOW. A second GitProvider on a repository branch another one's worker writes is refused -// (ErrBranchOwnedElsewhere). +// WorkerRequest names the branch worker a GitTarget needs: the GitBranch holding the branch, the +// GitProvider (in the GitBranch's namespace) it writes with, and the repository that provider names. +type WorkerRequest struct { + // GitBranch is the claim holder the worker writes for. + GitBranch k8stypes.NamespacedName + // Provider is the GitBranch's GitProvider, by name, in the GitBranch's namespace. + Provider string + // Branch is the GitBranch's spec.branch. + Branch string + // Repo is the repository the GitProvider names now. + Repo RepoIdentity + // ParentBranch is the GitBranch's spec.parentBranch. + ParentBranch string +} + +func (r WorkerRequest) provider() k8stypes.NamespacedName { + return k8stypes.NamespacedName{Namespace: r.GitBranch.Namespace, Name: r.Provider} +} + +// EnsureWorker ensures a worker exists for the repository branch req names, keyed by that branch +// (BranchKey), and that it is the worker for the repository the GitProvider names NOW. Only the +// GitBranch holding the branch's claim (branch_owner.go) gets one: any other is refused +// (ErrBranchOwnedElsewhere), and so is a request while another GitBranch's worker is still live on +// the branch, which the sweep retires. // -// repo is the caller's answer to "which repository is this"; the reconcile has already read the -// GitProvider one gate earlier, so it costs no API call. A slot holding a worker for a DIFFERENT -// repository is not corrected field by field — the clone, the base trust, the observation and the -// retained writes are all statements about the old one — so that worker is stopped, its checkout -// reclaimed, and a fresh one takes the slot. +// req.Repo is the caller's answer to "which repository is this"; the reconcile has already read the +// GitProvider, so it costs no API call. A slot holding a worker for a DIFFERENT repository is not +// corrected field by field — the clone, the base trust, the observation and the retained writes are +// all statements about the old one — so that worker is stopped, its checkout reclaimed, and a fresh +// one takes the slot. // // A replacement is not complete when it returns: the new worker has no clone and no queue, so // every GitTarget on that branch has to re-establish its folder against the new repository. That @@ -355,43 +400,39 @@ func (m *WorkerManager) RemoteForBranch(key BranchKey) (RemoteObservation, bool) // and the caller can fail. See ReplacementPending; the delivery is the GitTarget reconcile's // worker wiring gate. // -// parentBranch is the GitTarget's spec.parentBranch. It is immutable on a GitTarget, so a live -// worker sees a different one only when every target on its branch was replaced before the worker -// sweep ran. That is not a replacement: the worker keeps its clone, and SetParentBranch raises the -// parent generation, so nothing planned under the old parent is pushed and retained writes are -// rebuilt on the new one. +// req.ParentBranch is immutable on a GitBranch, so a live worker sees a different one only when its +// GitBranch was replaced before the worker sweep ran. That is not a replacement: the worker keeps +// its clone, and SetParentBranch raises the parent generation, so nothing planned under the old +// parent is pushed and retained writes are rebuilt on the new one. // // Worker creation/start is protected by the manager lock. -func (m *WorkerManager) EnsureWorker( - _ context.Context, - providerName, providerNamespace string, - branch string, - repo RepoIdentity, - parentBranch string, -) error { +func (m *WorkerManager) EnsureWorker(_ context.Context, req WorkerRequest) error { // Held across the whole check-and-create so a replacement cannot start while the worker it // replaces is still stopping; they would share a clone. m.lifecycleMu.Lock() defer m.lifecycleMu.Unlock() - key := BranchKeyFor(repo.URL, branch) + repo := req.Repo + key := BranchKeyFor(repo.URL, req.Branch) if key == (BranchKey{}) { - return fmt.Errorf("GitProvider %s/%s names no repository", providerNamespace, providerName) + return fmt.Errorf("GitProvider %s names no repository", req.provider()) + } + if err := m.refuseUnclaimed(key, req.GitBranch); err != nil { + return err } - provider := k8stypes.NamespacedName{Namespace: providerNamespace, Name: providerName} m.mu.RLock() existing, exists := m.workers[key] - previous, hadPrevious := m.workerServingLocked(provider, branch) + previous, hadPrevious := m.workerServingLocked(req.GitBranch) m.mu.RUnlock() - if exists && existing.provider() != provider { - return fmt.Errorf("%w: %s, through GitProvider %s", ErrBranchOwnedElsewhere, key, existing.provider()) + if exists && existing.GitBranch != req.GitBranch { + return fmt.Errorf("%w: %s, by GitBranch %s", ErrBranchOwnedElsewhere, key, existing.GitBranch) } - if !exists && hadPrevious { - // This GitProvider's worker on this branch writes another repository: the GitProvider was - // recreated under its name with another URL. The new repository is a new key, and the old - // worker's clone, base trust and retained writes are about the old repository. + if !exists && hadPrevious && previous != key { + // This GitBranch's worker writes another repository: its GitProvider was recreated under + // its name with another URL. The new repository is a new key, and the old worker's clone, + // base trust and retained writes are about the old repository. m.Log.Info("GitProvider now names a different repository; replacing the branch worker", "key", key.String(), "was", previous.String()) m.removeWorkersLocked([]BranchKey{previous}, "the GitProvider names a different repository") @@ -400,9 +441,9 @@ func (m *WorkerManager) EnsureWorker( } if exists { if existing.repo == repo { - if existing.SetParentBranch(parentBranch) { + if existing.SetParentBranch(req.ParentBranch) { m.Log.Info("Parent branch changed; the worker re-reads the remote on its next cycle", - "key", key.String(), "parentBranch", parentBranch) + "key", key.String(), "parentBranch", req.ParentBranch) } return nil } @@ -421,20 +462,28 @@ func (m *WorkerManager) EnsureWorker( defer m.mu.Unlock() if _, live := m.workers[key]; !live { - return m.startWorkerLocked(key, provider, repo, parentBranch) + return m.startWorkerLocked(key, req) } return nil } +// refuseUnclaimed is EnsureWorker's claim check: only the GitBranch holding key gets its worker. +func (m *WorkerManager) refuseUnclaimed(key BranchKey, gitBranch k8stypes.NamespacedName) error { + holder, held := m.Claims().Holder(key) + switch { + case !held: + return fmt.Errorf("%w: %s, and no GitBranch holds it yet", ErrBranchOwnedElsewhere, key) + case holder.GitBranch != gitBranch: + return fmt.Errorf("%w: %s, held by GitBranch %s", ErrBranchOwnedElsewhere, key, holder.GitBranch) + } + return nil +} + // startWorkerLocked creates and starts the worker for key, writing through provider, and records // it. Called with lifecycleMu and mu held. -func (m *WorkerManager) startWorkerLocked( - key BranchKey, - provider k8stypes.NamespacedName, - repo RepoIdentity, - parentBranch string, -) error { +func (m *WorkerManager) startWorkerLocked(key BranchKey, req WorkerRequest) error { + provider, repo, parentBranch := req.provider(), req.Repo, req.ParentBranch providerName, providerNamespace, branch := provider.Name, provider.Namespace, key.Branch m.Log.Info("Creating new branch worker", "key", key.String(), "repository", repo.String()) worker := NewBranchWorker( @@ -450,6 +499,7 @@ func (m *WorkerManager) startWorkerLocked( // Inject the resolver before Start: the field is read only by the event-loop // goroutine Start spawns, so setting it here (under m.mu, before that goroutine // exists) is race-free. + worker.GitBranch = req.GitBranch worker.SetParentBranch(parentBranch) worker.mapper = m.mapper worker.clusterMapper = m.clusterMapper @@ -459,7 +509,8 @@ func (m *WorkerManager) startWorkerLocked( worker.layoutReporter = m.layoutReporter worker.remoteReporter = func(observed RemoteObservation) { m.recordRemoteObservation(key, observed) } managerCtx := m.ctx // read under m.mu; the worker's loop calls the reporter later - worker.publicationReporter = func() { go m.notifyPublication(managerCtx, provider, branch) } + gitBranch := req.GitBranch + worker.publicationReporter = func() { go m.notifyPublication(managerCtx, gitBranch) } worker.scanAcceptance = m.scanAcceptance worker.applyRequester = m.applyRequester worker.triggerStatus = m.triggerStatus @@ -477,24 +528,43 @@ func (m *WorkerManager) startWorkerLocked( return nil } -// ErrBranchOwnedElsewhere is EnsureWorker's refusal to let a second GitProvider write a repository -// branch (branch_owner.go). The GitTarget reconciler refuses such a GitTarget first; this holds -// when two GitTargets are created too close together for either reconcile to see the other, and -// while a branch passes to another GitProvider. The worker sweep stops the worker of a GitProvider -// that does not own its branch, and the owner's next reconcile starts its own. -var ErrBranchOwnedElsewhere = errors.New("another GitProvider's branch worker writes this repository branch") +// ErrBranchOwnedElsewhere is EnsureWorker's refusal to start a worker for a GitBranch that does not +// hold its repository branch (branch_owner.go). The GitTarget reconciler checks the claim first; +// this holds while a claim passes to another GitBranch, until the sweep has retired the previous +// holder's worker and the new holder's next reconcile starts its own. +var ErrBranchOwnedElsewhere = errors.New("another GitBranch holds this repository branch") -// workerServingLocked is the key of the worker that writes branch through provider, if one is live. -// Called with m.mu held. There are few workers, so it looks at each. -func (m *WorkerManager) workerServingLocked(provider k8stypes.NamespacedName, branch string) (BranchKey, bool) { +// workerServingLocked is the key of the worker that writes for gitBranch, if one is live. Called +// with m.mu held. There are few workers, so it looks at each. +func (m *WorkerManager) workerServingLocked(gitBranch k8stypes.NamespacedName) (BranchKey, bool) { for key, worker := range m.workers { - if worker.provider() == provider && key.Branch == branch { + if worker.GitBranch == gitBranch { return key, true } } return BranchKey{}, false } +// notifyGitBranchLocked wakes the GitBranch reconciler for gitBranch. It never blocks: a full buffer +// drops the event, and the GitBranch's own requeue reads the worker later. Called with m.mu held or +// from the worker's reporter goroutine; it touches only the channel. +func (m *WorkerManager) notifyGitBranchLocked(gitBranch k8stypes.NamespacedName) { + if m.gitBranchEvents == nil || gitBranch.Name == "" { + return + } + object := &configv1alpha3.GitBranch{} + object.Name, object.Namespace = gitBranch.Name, gitBranch.Namespace + select { + case m.gitBranchEvents <- event.GenericEvent{Object: object}: + default: + } +} + +// GitBranchEvents carries a GitBranch whose branch worker proved something new about the remote or +// whose publication report changed, so the GitBranch controller publishes status.remote within one +// reconcile instead of at its next requeue. Nil for a manager not built by NewWorkerManager. +func (m *WorkerManager) GitBranchEvents() <-chan event.GenericEvent { return m.gitBranchEvents } + // carryRemote moves the last observation of a replaced worker's branch to the key of the worker // replacing it. It is stamped with the old repository, which is how the GitTarget's status sees // that what it published describes a repository it has left, and takes it back. Called with @@ -588,26 +658,45 @@ func (m *WorkerManager) removeWorkersLocked(keys []BranchKey, reason string) { } } -// GetWorkerForTarget finds the worker for a target's (provider, branch). -// Returns the worker and true if found, nil and false otherwise. -// This is used by EventRouter to dispatch events to the correct worker. -func (m *WorkerManager) GetWorkerForTarget( - providerName, providerNamespace string, - branch string, -) (*BranchWorker, bool) { +// RetireGitBranch stops the worker writing for gitBranch, if one is live, and returns once it has +// stopped. Stopping closes the worker's intake, pushes or refuses every pending write and settles +// every CommitRequest it holds, so a GitBranch being deleted calls this before its finalizer goes. +// What the manager kept about the branch goes with it: the next holder starts from the remote. +func (m *WorkerManager) RetireGitBranch(gitBranch k8stypes.NamespacedName) { + m.lifecycleMu.Lock() + defer m.lifecycleMu.Unlock() + m.mu.RLock() + key, live := m.workerServingLocked(gitBranch) + m.mu.RUnlock() + if !live { + return + } + m.removeWorkersLocked([]BranchKey{key}, "its GitBranch is being deleted") + m.mu.Lock() + delete(m.replacements, key) + delete(m.remotes, key) + m.mu.Unlock() +} + +// WorkerForGitBranch finds the worker writing for gitBranch: the worker of the repository branch +// that GitBranch holds. A GitTarget reaches its worker through its spec.gitBranchRef, so this is +// the lookup the event router and the GitTarget reconciler use, with no read on the event path. +func (m *WorkerManager) WorkerForGitBranch(gitBranch k8stypes.NamespacedName) (*BranchWorker, bool) { m.mu.RLock() defer m.mu.RUnlock() - key, exists := m.workerServingLocked( - k8stypes.NamespacedName{Namespace: providerNamespace, Name: providerName}, - branch, - ) + key, exists := m.workerServingLocked(gitBranch) if !exists { return nil, false } return m.workers[key], true } +// WorkerForTarget is WorkerForGitBranch for the GitBranch a GitTarget references. +func (m *WorkerManager) WorkerForTarget(target *configv1alpha3.GitTarget) (*BranchWorker, bool) { + return m.WorkerForGitBranch(target.GitBranchKey()) +} + // CommitRequestOwner returns the worker that holds a CommitRequest: the one that accepted its // attach and has not given it back. It is found whatever has happened to the request's GitTarget // since, including a worker retired and still finishing its last push, so a caller that finds no @@ -661,17 +750,12 @@ func (m *WorkerManager) ReconcileWorkers(ctx context.Context) error { return fmt.Errorf("failed to list GitTargets: %w", err) } - owners, urls, err := m.branchOwners(ctx, targetList.Items) - if err != nil { - return err - } - - needed := workersNeeded(targetList.Items, owners, urls) + needed := workersNeeded(targetList.Items, m.Claims()) m.mu.RLock() orphans := make([]BranchKey, 0, len(m.workers)) for key, worker := range m.workers { - if !needed.keeps(key, worker.provider()) { + if gitBranch, kept := needed[key]; !kept || gitBranch != worker.GitBranch { orphans = append(orphans, key) } } @@ -699,7 +783,7 @@ func (m *WorkerManager) ReconcileWorkers(ctx context.Context) error { m.Log.V(1).Info("Worker reconciliation complete", "activeWorkers", remaining, - "neededWorkers", len(needed.owned), + "neededWorkers", len(needed), "stopped", len(orphans)) return nil @@ -814,86 +898,40 @@ func (m *WorkerManager) NeedLeaderElection() bool { // reports whether such a worker exists. after is the commit the delivery named; it is logged and // never trusted: the worker reads where the branch is from the remote itself. func (m *WorkerManager) NotifyPush(providerNamespace, providerName, branch, after string) bool { - worker, found := m.GetWorkerForTarget(providerName, providerNamespace, branch) - if !found { + provider := k8stypes.NamespacedName{Namespace: providerNamespace, Name: providerName} + m.mu.RLock() + var found *BranchWorker + for key, worker := range m.workers { + if worker.provider() == provider && key.Branch == branch { + found = worker + break + } + } + m.mu.RUnlock() + if found == nil { return false } - worker.EnqueuePushNotification(after) + found.EnqueuePushNotification(after) return true } -// neededWorkers is which workers the live GitTargets need: the owner of every repository branch they -// write (branch_owner.go), and, for a GitTarget whose GitProvider cannot be read, any worker that -// serves that GitProvider and branch, so its retained writes wait for the GitProvider to be back, -// as they always did. -type neededWorkers struct { - owned map[BranchKey]k8stypes.NamespacedName - unresolved map[k8stypes.NamespacedName]map[string]bool -} - -func workersNeeded( - targets []configv1alpha3.GitTarget, - owners map[BranchKey]BranchOwner, - urls map[k8stypes.NamespacedName]string, -) neededWorkers { - needed := neededWorkers{ - owned: map[BranchKey]k8stypes.NamespacedName{}, - unresolved: map[k8stypes.NamespacedName]map[string]bool{}, - } +// workersNeeded is the worker each repository branch needs, by the GitBranch it writes for: the +// holder of a branch that at least one live GitTarget references. A held branch nothing references +// needs no worker, and one whose GitTargets are all gone has its worker retired. +func workersNeeded(targets []configv1alpha3.GitTarget, claims *BranchClaims) map[BranchKey]k8stypes.NamespacedName { + needed := map[BranchKey]k8stypes.NamespacedName{} for i := range targets { target := &targets[i] if !target.DeletionTimestamp.IsZero() { continue } - provider := target.GitProviderKey() - url, known := urls[provider] - if !known { - if needed.unresolved[provider] == nil { - needed.unresolved[provider] = map[string]bool{} - } - needed.unresolved[provider][target.Spec.Branch] = true + gitBranch := target.GitBranchKey() + if gitBranch.Name == "" { continue } - key := BranchKeyFor(url, target.Spec.Branch) - if owner, owned := owners[key]; owned && owner.Provider == provider { - needed.owned[key] = provider + if key, _, held := claims.HeldBy(gitBranch); held { + needed[key] = gitBranch } } return needed } - -// keeps reports whether the worker under key, writing through provider, is needed. -func (n neededWorkers) keeps(key BranchKey, provider k8stypes.NamespacedName) bool { - if owner, ok := n.owned[key]; ok && owner == provider { - return true - } - return n.unresolved[provider][key.Branch] -} - -// branchOwners decides which GitProvider owns each repository branch the targets write -// (branch_owner.go), and returns the URL of every GitProvider it read. The client is cached. -func (m *WorkerManager) branchOwners( - ctx context.Context, - targets []configv1alpha3.GitTarget, -) (map[BranchKey]BranchOwner, map[k8stypes.NamespacedName]string, error) { - var providers configv1alpha3.GitProviderList - if err := m.Client.List(ctx, &providers); err != nil { - return nil, nil, fmt.Errorf("failed to list GitProviders: %w", err) - } - urls := make(map[k8stypes.NamespacedName]string, len(providers.Items)) - for i := range providers.Items { - provider := &providers.Items[i] - urls[k8stypes.NamespacedName{Namespace: provider.Namespace, Name: provider.Name}] = provider.Spec.URL - } - owners, err := BranchOwners(targets, func(key k8stypes.NamespacedName) (string, bool) { - url, ok := urls[key] - return url, ok - }, func(target *configv1alpha3.GitTarget) (bool, error) { - decision, err := authz.GitTargetAdmitted(ctx, m.Client, target) - return decision.Allowed, err - }) - if err != nil { - return nil, nil, fmt.Errorf("decide branch owners: %w", err) - } - return owners, urls, nil -} diff --git a/internal/git/worker_manager_test.go b/internal/git/worker_manager_test.go index 27956c10..83b1e951 100644 --- a/internal/git/worker_manager_test.go +++ b/internal/git/worker_manager_test.go @@ -45,7 +45,7 @@ func TestWorkerManager_SetMapperInjectsIntoWorkers(t *testing.T) { repo := createProviderWithLocalRepo(ctx, t, client, "repo1") mustEnsureWorker(ctx, t, manager, "repo1", testProviderNamespace, "main", repo) - worker, exists := manager.GetWorkerForTarget("repo1", testProviderNamespace, "main") + worker, exists := manager.WorkerForGitBranch(testGitBranch("repo1", testProviderNamespace, "main")) require.True(t, exists) require.NotNil(t, worker) assert.NotNil(t, worker.mapper, "the created worker must carry the injected mapper") @@ -68,7 +68,7 @@ func mustEnsureWorker( repo RepoIdentity, ) bool { t.Helper() - require.NoError(t, m.EnsureWorker(ctx, providerName, providerNamespace, branch, repo, "")) + require.NoError(t, m.EnsureWorker(ctx, claimForTest(m, providerName, providerNamespace, branch, repo))) key := BranchKeyFor(repo.URL, branch) pending := m.ReplacementPending(key) m.AcknowledgeReplacement(key) @@ -117,10 +117,7 @@ func createTargetForRegister( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = testTargetNamespace - target.Spec.GitProviderRef = meta.NamespacedObjectReference{ - Name: providerName, - } - target.Spec.Branch = branch + target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path require.NoError(t, k8sClient.Create(ctx, target)) } @@ -144,13 +141,13 @@ func TestEnsureWorker_CreatesAWorkerWithTheIdentityItWasAskedFor(t *testing.T) { createTargetForRegister(ctx, t, client, "target1", "repo1", "main", "clusters/prod") // Register first target - err := manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", repo, "") + err := manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", repo)) if err != nil { t.Fatalf("Failed to register target: %v", err) } // Verify worker was created - worker, exists := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + worker, exists := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) if !exists { t.Fatal("Worker should exist after registration") } @@ -209,7 +206,7 @@ func TestWorkerManagerMultipleTargetsSameBranch(t *testing.T) { } // Verify worker exists for both targets - _, exists := manager.GetWorkerForTarget("shared-repo", "gitops-system", "main") + _, exists := manager.WorkerForGitBranch(testGitBranch("shared-repo", "gitops-system", "main")) if !exists { t.Fatal("Worker should exist") } @@ -237,12 +234,12 @@ func TestWorkerManagerDifferentBranches(t *testing.T) { createTargetForRegister(ctx, t, client, "target-dev", "repo1", "develop", "base/") // Register targets for same repo, different branches - err := manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", repo, "") + err := manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", repo)) if err != nil { t.Fatalf("Failed to register target-main: %v", err) } - err = manager.EnsureWorker(ctx, "repo1", "gitops-system", "develop", repo, "") + err = manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "develop", repo)) if err != nil { t.Fatalf("Failed to register target-dev: %v", err) } @@ -257,12 +254,12 @@ func TestWorkerManagerDifferentBranches(t *testing.T) { } // Verify each worker exists and has correct branch - workerMain, exists := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + workerMain, exists := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) if !exists || workerMain.Branch != "main" { t.Error("Main branch worker not found or has wrong branch") } - workerDev, exists := manager.GetWorkerForTarget("repo1", "gitops-system", "develop") + workerDev, exists := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "develop")) if !exists || workerDev.Branch != "develop" { t.Error("Develop branch worker not found or has wrong branch") } @@ -302,14 +299,14 @@ func TestReconcileWorkers_KeepsAWorkerASiblingStillNeeds(t *testing.T) { require.NoError(t, k8sClient.Delete(ctx, targetForRegister("target1"))) require.NoError(t, manager.ReconcileWorkers(ctx)) - _, exists := manager.GetWorkerForTarget("repo1", testTargetNamespace, "main") + _, exists := manager.WorkerForGitBranch(testGitBranch("repo1", testTargetNamespace, "main")) assert.True(t, exists, "target2 still writes to this branch, so its worker must survive") // And now the last one. require.NoError(t, k8sClient.Delete(ctx, targetForRegister("target2"))) require.NoError(t, manager.ReconcileWorkers(ctx)) - _, exists = manager.GetWorkerForTarget("repo1", testTargetNamespace, "main") + _, exists = manager.WorkerForGitBranch(testGitBranch("repo1", testTargetNamespace, "main")) assert.False(t, exists, "nothing needs this worker now, and it holds a goroutine, a queue and a clone until it stops") } @@ -337,7 +334,7 @@ func TestReconcileWorkers_StopsNothingWhenTheTargetsCannotBeRead(t *testing.T) { err := manager.ReconcileWorkers(ctx) require.Error(t, err) - _, exists := manager.GetWorkerForTarget("repo1", testTargetNamespace, "main") + _, exists := manager.WorkerForGitBranch(testGitBranch("repo1", testTargetNamespace, "main")) assert.True(t, exists, "a read that failed proves nothing about which workers are needed") } @@ -362,7 +359,7 @@ func TestWorkerManagerConcurrentRegistration(t *testing.T) { done := make(chan bool, 10) for i := range 10 { go func(index int) { - err := manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", repo, "") + err := manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", repo)) if err != nil { t.Errorf("Failed to ensure worker %d: %v", index, err) } @@ -396,7 +393,7 @@ func TestWorkerManagerGetNonexistentWorker(t *testing.T) { manager := NewWorkerManager(client, log, BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) - worker, exists := manager.GetWorkerForTarget("nonexistent", "default", "main") + worker, exists := manager.WorkerForGitBranch(testGitBranch("nonexistent", "default", "main")) if exists { t.Error("Should return exists=false for nonexistent worker") } @@ -437,7 +434,7 @@ func TestReconcileWorkers_RemovesTheRetiredWorkersCheckout(t *testing.T) { createTargetForRegister(ctx, t, k8sClient, "target1", "repo1", "main", "apps/") mustEnsureWorker(ctx, t, manager, "repo1", testTargetNamespace, "main", repo) - worker, exists := manager.GetWorkerForTarget("repo1", testTargetNamespace, "main") + worker, exists := manager.WorkerForGitBranch(testGitBranch("repo1", testTargetNamespace, "main")) require.True(t, exists) // Stand in for the clone a first publication would leave behind. checkout := worker.repoPath() @@ -527,8 +524,8 @@ func TestWorkerLifecycle_BuildsAndReclaimsItsOwnState(t *testing.T) { root := workerStateRoot repo := RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/first.git"} - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", repo, "")) - worker, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", repo))) + worker, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) require.True(t, strings.HasPrefix(worker.repoPath(), root+string(filepath.Separator)), "precondition: the worker keeps its state under the root this test owns") @@ -555,7 +552,7 @@ func TestWorkerLifecycle_BuildsAndReclaimsItsOwnState(t *testing.T) { require.NoError(t, err, "and neither is another provider's") _, err = os.Stat(root) require.NoError(t, err, "least of all the root every worker's state lives under") - _, stillThere := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + _, stillThere := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) assert.False(t, stillThere) } @@ -571,8 +568,8 @@ func TestWorkerLifecycle_AReplacementDoesNotReclaimItsPredecessorsCheckout(t *te before := RepoIdentity{ProviderUID: "uid-1", URL: sameURL} after := RepoIdentity{ProviderUID: "uid-2", URL: sameURL} - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", before, "")) - old, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", before))) + old, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) require.NoError(t, os.MkdirAll(old.repoPath(), 0o750)) @@ -583,8 +580,8 @@ func TestWorkerLifecycle_AReplacementDoesNotReclaimItsPredecessorsCheckout(t *te sentinel := filepath.Join(successor, "HEAD") require.NoError(t, os.WriteFile(sentinel, []byte("ref: refs/heads/main\n"), 0o600)) - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", after, "")) - replacement, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", after))) + replacement, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) require.NotSame(t, old, replacement) diff --git a/internal/git/worker_repo_identity_test.go b/internal/git/worker_repo_identity_test.go index 562abda8..11ab2fde 100644 --- a/internal/git/worker_repo_identity_test.go +++ b/internal/git/worker_repo_identity_test.go @@ -59,14 +59,14 @@ func TestEnsureWorker_KeepsTheWorkerWhenTheRepositoryIsUnchanged(t *testing.T) { repo := RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/repo.git"} mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", repo) - first, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + first, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) for range 3 { mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", repo) } - second, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + second, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) assert.Same(t, first, second, "a steady tick must leave the worker exactly where it was") assert.NoError(t, first.ctx.Err(), "and must not have stopped it") @@ -81,12 +81,12 @@ func TestEnsureWorker_ReplacesTheWorkerWhenTheProviderIsRecreated(t *testing.T) after := RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"} mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", before) - old, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + old, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", after) - replacement, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + replacement, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) assert.NotSame(t, old, replacement, "the slot holds the worker for the repository named NOW") assert.Equal(t, after, replacement.repo) @@ -103,13 +103,13 @@ func TestEnsureWorker_ReplacesOnARecreateThatKeepsTheURL(t *testing.T) { mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-1", URL: url}) - old, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + old, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-2", URL: url}) - replacement, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + replacement, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) assert.NotSame(t, old, replacement, "a recreated GitProvider is a new object, so it is a new subject") } @@ -123,13 +123,13 @@ func TestEnsureWorker_ReplacesWhenTheURLMovedUnderTheSameObject(t *testing.T) { mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/first.git"}) - old, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + old, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/second.git"}) - replacement, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + replacement, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) assert.NotSame(t, old, replacement, "a differing URL is a different repository, whatever the UID says") } @@ -143,7 +143,7 @@ func TestEnsureWorker_ReplacementTakesTheOldCheckoutWithIt(t *testing.T) { mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-1", URL: oldURL}) - old, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + old, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) // Stand in for the clone a first publication would have left behind. @@ -169,7 +169,7 @@ func TestEnsureWorker_ReplacementDropsWhatTheOldWorkerWasHolding(t *testing.T) { mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-1", URL: "https://example.invalid/first.git"}) - old, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + old, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) // Enqueued without a loop that can reach a remote, so they are still in hand at replacement. @@ -180,7 +180,7 @@ func TestEnsureWorker_ReplacementDropsWhatTheOldWorkerWasHolding(t *testing.T) { mustEnsureWorker(ctx, t, manager, "repo1", "gitops-system", "main", RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"}) - replacement, ok := manager.GetWorkerForTarget("repo1", "gitops-system", "main") + replacement, ok := manager.WorkerForGitBranch(testGitBranch("repo1", "gitops-system", "main")) require.True(t, ok) assert.Zero(t, replacement.queueDepth(), "the new repository starts empty; nothing computed against the old tree follows it") @@ -352,21 +352,21 @@ func TestEnsureWorker_AnUndeliveredRecoveryIsStillPendingOnTheNextPass(t *testin after := RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"} key := BranchKeyFor(after.URL, "main") - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", before, "")) + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", before))) require.False(t, manager.ReplacementPending(key)) - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", after, "")) + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", after))) require.True(t, manager.ReplacementPending(key)) // The delivery fails, so nothing acknowledges it. Two more reconciles of this target, and of // a sibling on the same branch, still find the recovery waiting. - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", after, "")) + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", after))) assert.True(t, manager.ReplacementPending(key), "a recovery nobody delivered must survive the calls that come after it") manager.AcknowledgeReplacement(key) assert.False(t, manager.ReplacementPending(key)) - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", after, "")) + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", after))) assert.False(t, manager.ReplacementPending(key), "and a delivered one must not be asked for again on every steady tick") } @@ -379,8 +379,8 @@ func TestReconcileWorkers_DropsARecoveryForABranchNothingNeeds(t *testing.T) { after := RepoIdentity{ProviderUID: "uid-2", URL: "https://example.invalid/second.git"} key := BranchKeyFor(after.URL, "main") - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", before, "")) - require.NoError(t, manager.EnsureWorker(ctx, "repo1", "gitops-system", "main", after, "")) + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", before))) + require.NoError(t, manager.EnsureWorker(ctx, claimForTest(manager, "repo1", "gitops-system", "main", after))) require.True(t, manager.ReplacementPending(key)) // No GitTarget names this branch, so the sweep retires its worker. @@ -410,8 +410,8 @@ func TestSweep_RetiresAWorkerNoDeleteWasEverObserved(t *testing.T) { require.NoError(t, manager.ReconcileWorkers(ctx)) - _, stranded := manager.GetWorkerForTarget("repo1", testTargetNamespace, "main") + _, stranded := manager.WorkerForGitBranch(testGitBranch("repo1", testTargetNamespace, "main")) assert.False(t, stranded, "no GitTarget names this branch any more, and no delete was ever seen") - _, live := manager.GetWorkerForTarget("repo1", testTargetNamespace, "release") + _, live := manager.WorkerForGitBranch(testGitBranch("repo1", testTargetNamespace, "release")) assert.True(t, live, "and the successor's worker is untouched") } diff --git a/internal/git/write_boundary_precondition_test.go b/internal/git/write_boundary_precondition_test.go index ede69e2d..bd01cc58 100644 --- a/internal/git/write_boundary_precondition_test.go +++ b/internal/git/write_boundary_precondition_test.go @@ -203,9 +203,8 @@ func diamondGitTarget(providerName, branch string) *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "podinfo-test", Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: providerName}, - Branch: branch, - Path: "", + GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)}, + Path: "", }, } } diff --git a/internal/rulestore/object_selector_test.go b/internal/rulestore/object_selector_test.go index 112e62c2..a5d0804a 100644 --- a/internal/rulestore/object_selector_test.go +++ b/internal/rulestore/object_selector_test.go @@ -37,7 +37,7 @@ func TestAddOrUpdateWatchRule_CompilesTheCanonicalObjectSelector(t *testing.T) { }, }, } - store.AddOrUpdateWatchRule(rule, ownNamespaceScope(rule), "target", "team-a", "p", "team-a", "main", "path") + store.AddOrUpdateWatchRule(rule, ownNamespaceScope(rule), "target", "team-a", "p", "team-a", "path") compiled, ok := store.GetWatchRule(types.NamespacedName{Name: "r", Namespace: "team-a"}) if !ok { @@ -68,7 +68,7 @@ func TestAddOrUpdateClusterWatchRule_CompilesTheCanonicalObjectSelector(t *testi }}, }, } - store.AddOrUpdateClusterWatchRule(rule, "target", "ns", "p", "ns", "main", "path") + store.AddOrUpdateClusterWatchRule(rule, "target", "ns", "p", "ns", "path") rules := store.SnapshotClusterWatchRules() if len(rules) != 1 || len(rules[0].Rules) != 1 { diff --git a/internal/rulestore/store.go b/internal/rulestore/store.go index 6b6549fd..a72ef8d2 100644 --- a/internal/rulestore/store.go +++ b/internal/rulestore/store.go @@ -28,10 +28,9 @@ type CompiledRule struct { GitTargetNamespace string // Resolved values (from GitTarget) - GitProviderRef string - GitProviderNamespace string - Branch string - Path string + GitBranchRef string + GitBranchNamespace string + Path string // CreatedAt is the WatchRule's creation timestamp. Where two rules on one GitTarget ask for // overlapping collections with different object selectors, the older rule (then the lower @@ -135,10 +134,9 @@ type CompiledClusterRule struct { GitTargetNamespace string // Resolved values (from GitTarget) - GitProviderRef string - GitProviderNamespace string - Branch string - Path string + GitBranchRef string + GitBranchNamespace string + Path string // CreatedAt is the ClusterWatchRule's creation timestamp; see CompiledRule.CreatedAt. CreatedAt metav1.Time @@ -190,18 +188,16 @@ func NewStore() *RuleStore { // authz.ResolveWatchRuleSourceScope). // - gitTargetName: the name of the GitTarget // - gitTargetNamespace: the namespace containing the GitTarget -// - gitProviderName: the name of the resolved GitProvider (from GitTarget.Spec.Provider) -// - gitProviderNamespace: the namespace containing the resolved GitProvider -// - branch: the Git branch to write to (from GitTarget.Spec.Branch) +// - gitBranchName: the GitBranch the GitTarget writes (from GitTarget.Spec.GitBranchRef) +// - gitBranchNamespace: the namespace containing that GitBranch // - path: POSIX-like relative path prefix for writes (from GitTarget.Spec.Path, sanitized upstream) func (s *RuleStore) AddOrUpdateWatchRule( rule configv1alpha3.WatchRule, sourceNamespaces [][]string, gitTargetName string, gitTargetNamespace string, - gitProviderName string, - gitProviderNamespace string, - branch string, + gitBranchName string, + gitBranchNamespace string, path string, ) { s.mu.Lock() @@ -213,16 +209,15 @@ func (s *RuleStore) AddOrUpdateWatchRule( } compiled := CompiledRule{ - Source: key, - GitTargetRef: gitTargetName, - GitTargetNamespace: gitTargetNamespace, - GitProviderRef: gitProviderName, - GitProviderNamespace: gitProviderNamespace, - Branch: branch, - Path: path, - CreatedAt: rule.CreationTimestamp, - IsClusterScoped: false, - ResourceRules: make([]CompiledResourceRule, 0, len(rule.Spec.Rules)), + Source: key, + GitTargetRef: gitTargetName, + GitTargetNamespace: gitTargetNamespace, + GitBranchRef: gitBranchName, + GitBranchNamespace: gitBranchNamespace, + Path: path, + CreatedAt: rule.CreationTimestamp, + IsClusterScoped: false, + ResourceRules: make([]CompiledResourceRule, 0, len(rule.Spec.Rules)), } for i, r := range rule.Spec.Rules { @@ -265,17 +260,15 @@ func (s *RuleStore) GetWatchRule(key types.NamespacedName) (CompiledRule, bool) // - rule: the ClusterWatchRule to add or update // - gitTargetName: the name of the GitTarget // - gitTargetNamespace: the namespace containing the GitTarget -// - gitProviderName: the name of the resolved GitProvider (from GitTarget.Spec.Provider) -// - gitProviderNamespace: the namespace containing the resolved GitProvider -// - branch: the Git branch to write to (from GitTarget.Spec.Branch) +// - gitBranchName: the GitBranch the GitTarget writes (from GitTarget.Spec.GitBranchRef) +// - gitBranchNamespace: the namespace containing that GitBranch // - path: POSIX-like relative path prefix for writes (from GitTarget.Spec.Path, sanitized upstream) func (s *RuleStore) AddOrUpdateClusterWatchRule( rule configv1alpha3.ClusterWatchRule, gitTargetName string, gitTargetNamespace string, - gitProviderName string, - gitProviderNamespace string, - branch string, + gitBranchName string, + gitBranchNamespace string, path string, ) { s.mu.Lock() @@ -287,15 +280,14 @@ func (s *RuleStore) AddOrUpdateClusterWatchRule( } compiled := CompiledClusterRule{ - Source: key, - GitTargetRef: gitTargetName, - GitTargetNamespace: gitTargetNamespace, - GitProviderRef: gitProviderName, - GitProviderNamespace: gitProviderNamespace, - Branch: branch, - Path: path, - CreatedAt: rule.CreationTimestamp, - Rules: make([]CompiledClusterResourceRule, 0, len(rule.Spec.Rules)), + Source: key, + GitTargetRef: gitTargetName, + GitTargetNamespace: gitTargetNamespace, + GitBranchRef: gitBranchName, + GitBranchNamespace: gitBranchNamespace, + Path: path, + CreatedAt: rule.CreationTimestamp, + Rules: make([]CompiledClusterResourceRule, 0, len(rule.Spec.Rules)), } for _, r := range rule.Spec.Rules { diff --git a/internal/rulestore/store_test.go b/internal/rulestore/store_test.go index b42420eb..3aea2ee4 100644 --- a/internal/rulestore/store_test.go +++ b/internal/rulestore/store_test.go @@ -65,7 +65,6 @@ func TestAddOrUpdateWatchRule(t *testing.T) { "default", "test-provider", "gitops-system", - "main", "clusters/prod", ) @@ -80,14 +79,11 @@ func TestAddOrUpdateWatchRule(t *testing.T) { if compiled.Source != key { t.Errorf("Source mismatch: got %v, want %v", compiled.Source, key) } - if compiled.GitProviderRef != "test-provider" { - t.Errorf("GitProviderRef mismatch: got %s, want test-provider", compiled.GitProviderRef) + if compiled.GitBranchRef != "test-provider" { + t.Errorf("GitBranchRef mismatch: got %s, want test-provider", compiled.GitBranchRef) } - if compiled.GitProviderNamespace != "gitops-system" { - t.Errorf("GitProviderNamespace mismatch: got %s, want gitops-system", compiled.GitProviderNamespace) - } - if compiled.Branch != "main" { - t.Errorf("Branch mismatch: got %s, want main", compiled.Branch) + if compiled.GitBranchNamespace != "gitops-system" { + t.Errorf("GitBranchNamespace mismatch: got %s, want gitops-system", compiled.GitBranchNamespace) } if compiled.Path != "clusters/prod" { t.Errorf("Path mismatch: got %s, want clusters/prod", compiled.Path) @@ -107,7 +103,6 @@ func TestAddOrUpdateWatchRule(t *testing.T) { "default", "updated-provider", "gitops-system", - "develop", "clusters/staging", ) @@ -115,11 +110,8 @@ func TestAddOrUpdateWatchRule(t *testing.T) { if !exists { t.Fatal("Rule was deleted instead of updated") } - if compiled.GitProviderRef != "updated-provider" { - t.Errorf("GitProviderRef not updated: got %s, want updated-provider", compiled.GitProviderRef) - } - if compiled.Branch != "develop" { - t.Errorf("Branch not updated: got %s, want develop", compiled.Branch) + if compiled.GitBranchRef != "updated-provider" { + t.Errorf("GitBranchRef not updated: got %s, want updated-provider", compiled.GitBranchRef) } if compiled.Path != "clusters/staging" { t.Errorf("Path not updated: got %s, want clusters/staging", compiled.Path) @@ -150,7 +142,6 @@ func TestAddOrUpdateClusterWatchRule(t *testing.T) { "gitops-system", "cluster-provider", "gitops-system", - "main", "cluster/audit", ) @@ -165,11 +156,8 @@ func TestAddOrUpdateClusterWatchRule(t *testing.T) { if compiled.Source != key { t.Errorf("Source mismatch: got %v, want %v", compiled.Source, key) } - if compiled.GitProviderRef != "cluster-provider" { - t.Errorf("GitProviderRef mismatch: got %s, want cluster-provider", compiled.GitProviderRef) - } - if compiled.Branch != "main" { - t.Errorf("Branch mismatch: got %s, want main", compiled.Branch) + if compiled.GitBranchRef != "cluster-provider" { + t.Errorf("GitBranchRef mismatch: got %s, want cluster-provider", compiled.GitBranchRef) } if compiled.Path != "cluster/audit" { t.Errorf("Path mismatch: got %s, want cluster/audit", compiled.Path) @@ -200,7 +188,7 @@ func TestDelete(t *testing.T) { // Add rule store.AddOrUpdateWatchRule( - rule, ownNamespaceScope(rule), "test-dest", "default", "test-repo", "gitops-system", "main", "test") + rule, ownNamespaceScope(rule), "test-dest", "default", "test-repo", "gitops-system", "test") // Verify it exists if _, exists := store.rules[key]; !exists { @@ -234,7 +222,7 @@ func TestDeleteClusterWatchRule(t *testing.T) { key := types.NamespacedName{Name: "delete-cluster-test", Namespace: ""} // Add rule - store.AddOrUpdateClusterWatchRule(rule, "test-dest", "gitops-system", "test-repo", "gitops-system", "main", "test") + store.AddOrUpdateClusterWatchRule(rule, "test-dest", "gitops-system", "test-repo", "gitops-system", "test") // Verify it exists if _, exists := store.clusterRules[key]; !exists { @@ -284,9 +272,9 @@ func TestGetMatchingRules(t *testing.T) { rule2.Namespace = "default" store.AddOrUpdateWatchRule( - rule1, ownNamespaceScope(rule1), "dest1", "default", "repo1", "gitops-system", "main", "test1") + rule1, ownNamespaceScope(rule1), "dest1", "default", "repo1", "gitops-system", "test1") store.AddOrUpdateWatchRule( - rule2, ownNamespaceScope(rule2), "dest2", "default", "repo2", "gitops-system", "main", "test2") + rule2, ownNamespaceScope(rule2), "dest2", "default", "repo2", "gitops-system", "test2") tests := []struct { name string @@ -393,7 +381,7 @@ func TestGetMatchingRules_OverlappingRulesAreAdditive(t *testing.T) { for _, name := range []string{"pods-a", "pods-b"} { rule := podsRule(name) store.AddOrUpdateWatchRule(rule, ownNamespaceScope(rule), "dest-a", "default", "repo", - "gitops-system", "main", "a") + "gitops-system", "a") } matches := store.GetMatchingRules(nil, "pods", "", "v1", false) @@ -448,7 +436,6 @@ func TestGetMatchingClusterRules(t *testing.T) { "gitops-system", "repo1", "gitops-system", - "main", "cluster", ) store.AddOrUpdateClusterWatchRule( @@ -457,7 +444,6 @@ func TestGetMatchingClusterRules(t *testing.T) { "gitops-system", "repo2", "gitops-system", - "main", "namespaced", ) @@ -640,9 +626,9 @@ func TestSnapshotWatchRules(t *testing.T) { rule2.Namespace = "default" store.AddOrUpdateWatchRule( - rule1, ownNamespaceScope(rule1), "dest1", "default", "repo1", "gitops-system", "main", "test1") + rule1, ownNamespaceScope(rule1), "dest1", "default", "repo1", "gitops-system", "test1") store.AddOrUpdateWatchRule( - rule2, ownNamespaceScope(rule2), "dest2", "default", "repo2", "gitops-system", "main", "test2") + rule2, ownNamespaceScope(rule2), "dest2", "default", "repo2", "gitops-system", "test2") // Get snapshot snapshot := store.SnapshotWatchRules() @@ -652,12 +638,12 @@ func TestSnapshotWatchRules(t *testing.T) { } // Verify snapshot is independent (deep copy) - snapshot[0].Branch = "modified" + snapshot[0].Path = "modified" // Get another snapshot to verify original wasn't modified snapshot2 := store.SnapshotWatchRules() for _, rule := range snapshot2 { - if rule.Branch == "modified" { + if rule.Path == "modified" { t.Error("Snapshot modification affected original store") } } @@ -678,7 +664,7 @@ func TestSnapshotClusterWatchRules(t *testing.T) { } rule1.Name = "cluster-rule1" - store.AddOrUpdateClusterWatchRule(rule1, "dest1", "gitops-system", "repo1", "gitops-system", "main", "cluster") + store.AddOrUpdateClusterWatchRule(rule1, "dest1", "gitops-system", "repo1", "gitops-system", "cluster") snapshot := store.SnapshotClusterWatchRules() @@ -687,9 +673,9 @@ func TestSnapshotClusterWatchRules(t *testing.T) { } // Verify deep copy - snapshot[0].Branch = "modified" + snapshot[0].Path = "modified" snapshot2 := store.SnapshotClusterWatchRules() - if snapshot2[0].Branch == "modified" { + if snapshot2[0].Path == "modified" { t.Error("Snapshot modification affected original store") } } @@ -719,7 +705,7 @@ func TestConcurrentAccess(t *testing.T) { rule.Namespace = "default" store.AddOrUpdateWatchRule( - rule, ownNamespaceScope(rule), "dest", "default", "repo", "gitops-system", "main", "test") + rule, ownNamespaceScope(rule), "dest", "default", "repo", "gitops-system", "test") } }(i) } @@ -765,7 +751,7 @@ func TestMultipleResourceRules(t *testing.T) { rule.Namespace = "default" store.AddOrUpdateWatchRule( - rule, ownNamespaceScope(rule), "dest", "default", "repo", "gitops-system", "main", "test") + rule, ownNamespaceScope(rule), "dest", "default", "repo", "gitops-system", "test") // Either item selects its resource: the items are a logical OR. for _, resource := range []string{"pods", "services"} { @@ -801,7 +787,7 @@ func TestGetMatchingRules_MustFilterByNamespaceForNamespacedWatchRule(t *testing rule.Namespace = "tilt-playground" store.AddOrUpdateWatchRule( rule, ownNamespaceScope(rule), - "target", "tilt-playground", "provider", "tilt-playground", "main", "live") + "target", "tilt-playground", "provider", "tilt-playground", "live") sameNSObject := &unstructured.Unstructured{} sameNSObject.SetNamespace("tilt-playground") @@ -838,7 +824,7 @@ func TestGetMatchingRules_NamespacedWatchRule_NamespaceContract(t *testing.T) { rule.Name = "ns-wr" rule.Namespace = "tilt-playground" store.AddOrUpdateWatchRule( - rule, ownNamespaceScope(rule), "tgt", "tilt-playground", "prov", "tilt-playground", "main", "live") + rule, ownNamespaceScope(rule), "tgt", "tilt-playground", "prov", "tilt-playground", "live") store.AddOrUpdateClusterWatchRule( configv1alpha3.ClusterWatchRule{ @@ -852,7 +838,7 @@ func TestGetMatchingRules_NamespacedWatchRule_NamespaceContract(t *testing.T) { }}, }, }, - "cluster-target", "ops", "cluster-provider", "ops", "main", "cluster", + "cluster-target", "ops", "cluster-provider", "ops", "cluster", ) t.Run("same namespace matches", func(t *testing.T) { @@ -911,7 +897,7 @@ func TestGetMatchingRules_PerItemSourceNamespaces(t *testing.T) { store.AddOrUpdateWatchRule( rule, [][]string{{"tenant-acme"}, {"repo-config"}}, - "tgt", "tenant-acme", "prov", "tenant-acme", "main", "live", + "tgt", "tenant-acme", "prov", "tenant-acme", "live", ) object := func(namespace string) *unstructured.Unstructured { @@ -961,7 +947,7 @@ func TestGetMatchingRules_WildcardItemMatchesEveryResolvedNamespace(t *testing.T store.AddOrUpdateWatchRule( rule, [][]string{{"repo-config", "team-payments"}}, - "tgt", "tenant-acme", "prov", "tenant-acme", "main", "live", + "tgt", "tenant-acme", "prov", "tenant-acme", "live", ) for _, ns := range []string{"repo-config", "team-payments"} { @@ -1009,7 +995,7 @@ func TestSharedSelector_RoutesAlikeThroughBothEntryPoints(t *testing.T) { watchRule.Name = "rule" watchRule.Namespace = "tenant-acme" store.AddOrUpdateWatchRule(watchRule, ownNamespaceScope(watchRule), "target", "tenant-acme", - "provider", "tenant-acme", "main", "clusters") + "provider", "tenant-acme", "clusters") // The same selector on the cluster-scoped kind. A ClusterWatchRule may legally name a // namespaced plural (see TestGetMatchingClusterRules); what bounds it is the OBJECT's scope. @@ -1023,8 +1009,7 @@ func TestSharedSelector_RoutesAlikeThroughBothEntryPoints(t *testing.T) { }, } clusterRule.Name = "cluster-rule" - store.AddOrUpdateClusterWatchRule(clusterRule, "target", "tenant-acme", "provider", "tenant-acme", - "main", "clusters") + store.AddOrUpdateClusterWatchRule(clusterRule, "target", "tenant-acme", "provider", "tenant-acme", "clusters") inScope := &unstructured.Unstructured{} inScope.SetNamespace("tenant-acme") @@ -1096,8 +1081,15 @@ func TestSnapshotWatchRules_SourceNamespacesStayDefensivelyCopied(t *testing.T) } rule.Name = "rule" rule.Namespace = "tenant-acme" - store.AddOrUpdateWatchRule(rule, ownNamespaceScope(rule), "target", "tenant-acme", "provider", "tenant-acme", - "main", "clusters") + store.AddOrUpdateWatchRule( + rule, + ownNamespaceScope(rule), + "target", + "tenant-acme", + "provider", + "tenant-acme", + "clusters", + ) snapshot := store.SnapshotWatchRules() if len(snapshot) != 1 || len(snapshot[0].ResourceRules) != 1 { @@ -1150,7 +1142,7 @@ func TestStore_OwnsItsSelectorSlices(t *testing.T) { rule.Name = "rule" rule.Namespace = "tenant-acme" store.AddOrUpdateWatchRule(rule, ownNamespaceScope(rule), "target", "tenant-acme", - "provider", "tenant-acme", "main", "clusters") + "provider", "tenant-acme", "clusters") return func() { item.APIGroups[0] = "batch" @@ -1177,7 +1169,7 @@ func TestStore_OwnsItsSelectorSlices(t *testing.T) { } rule.Name = "cluster-rule" store.AddOrUpdateClusterWatchRule(rule, "target", "tenant-acme", "provider", - "tenant-acme", "main", "clusters") + "tenant-acme", "clusters") return func() { item.APIGroups[0] = "batch" diff --git a/internal/watch/api_resource_catalog_test.go b/internal/watch/api_resource_catalog_test.go index 76055af7..fdea83bd 100644 --- a/internal/watch/api_resource_catalog_test.go +++ b/internal/watch/api_resource_catalog_test.go @@ -143,7 +143,7 @@ func TestNotServedResourceProducesNoGVR(t *testing.T) { }, } store.AddOrUpdateWatchRule( - rule, ownNamespaceScope(rule), "target", "default", "provider", "default", "main", "live") + rule, ownNamespaceScope(rule), "target", "default", "provider", "default", "live") manager := &Manager{RuleStore: store, resourceCatalog: newCommonTestCatalog(t)} assert.Empty(t, manager.ComputeRequestedGVRs()) diff --git a/internal/watch/bootstrap.go b/internal/watch/bootstrap.go index d599b7b4..2c5be40d 100644 --- a/internal/watch/bootstrap.go +++ b/internal/watch/bootstrap.go @@ -49,7 +49,7 @@ func (m *Manager) bootstrapRuleStore(ctx context.Context, log logr.Logger) error func (m *Manager) bootstrapWatchRule(ctx context.Context, rule configv1alpha3.WatchRule) error { targetNS := rule.Namespace - target, provider, err := m.resolveTargetAndProvider(ctx, client.ObjectKey{ + target, err := m.resolveTarget(ctx, client.ObjectKey{ Name: rule.Spec.GitTargetRef.Name, Namespace: targetNS, }) @@ -63,7 +63,7 @@ func (m *Manager) bootstrapWatchRule(ctx context.Context, rule configv1alpha3.Wa // override and watch a namespace the policy refuses. A denial is not fatal to startup: the // rule is simply left out of the store (bootstrap has no controllers yet and cannot publish // status), and the first reconcile re-decides and writes the terminal condition. - resolved, err := CompileWatchRule(ctx, m.Client, m.RuleStore, rule, target, provider) + resolved, err := CompileWatchRule(ctx, m.Client, m.RuleStore, rule, target) if err != nil { return fmt.Errorf("evaluating source-namespace authorization for WatchRule %s/%s: %w", rule.Namespace, rule.Name, err) @@ -79,7 +79,7 @@ func (m *Manager) bootstrapWatchRule(ctx context.Context, rule configv1alpha3.Wa func (m *Manager) bootstrapClusterWatchRule(ctx context.Context, rule configv1alpha3.ClusterWatchRule) error { targetNS := rule.Spec.GitTargetRef.Namespace - target, provider, err := m.resolveTargetAndProvider(ctx, client.ObjectKey{ + target, err := m.resolveTarget(ctx, client.ObjectKey{ Name: rule.Spec.GitTargetRef.Name, Namespace: targetNS, }) @@ -93,7 +93,7 @@ func (m *Manager) bootstrapClusterWatchRule(ctx context.Context, rule configv1al // target the provider never admitted. A refusal is not fatal to startup: the // rule is simply left out of the store, and the reconciler's own gate re-decides (and can grant // it) as soon as the controller's initial sync reaches this rule. - decision, err := CompileClusterWatchRule(ctx, m.Client, m.RuleStore, rule, target, provider) + decision, err := CompileClusterWatchRule(ctx, m.Client, m.RuleStore, rule, target) if err != nil { return fmt.Errorf("evaluating admission for ClusterWatchRule %q: %w", rule.Name, err) } @@ -104,22 +104,11 @@ func (m *Manager) bootstrapClusterWatchRule(ctx context.Context, rule configv1al return nil } -func (m *Manager) resolveTargetAndProvider( - ctx context.Context, - targetKey client.ObjectKey, -) (configv1alpha3.GitTarget, configv1alpha3.GitProvider, error) { +func (m *Manager) resolveTarget(ctx context.Context, targetKey client.ObjectKey) (configv1alpha3.GitTarget, error) { var target configv1alpha3.GitTarget if err := m.Client.Get(ctx, targetKey, &target); err != nil { - return configv1alpha3.GitTarget{}, configv1alpha3.GitProvider{}, - fmt.Errorf("resolving GitTarget %s/%s: %w", targetKey.Namespace, targetKey.Name, err) + return configv1alpha3.GitTarget{}, fmt.Errorf("resolving GitTarget %s/%s: %w", + targetKey.Namespace, targetKey.Name, err) } - - providerKey := target.GitProviderKey() - var provider configv1alpha3.GitProvider - if err := m.Client.Get(ctx, providerKey, &provider); err != nil { - return configv1alpha3.GitTarget{}, configv1alpha3.GitProvider{}, - fmt.Errorf("resolving GitProvider %s/%s: %w", providerKey.Namespace, providerKey.Name, err) - } - - return target, provider, nil + return target, nil } diff --git a/internal/watch/bootstrap_admission_test.go b/internal/watch/bootstrap_admission_test.go index 6a8857a6..b1e23465 100644 --- a/internal/watch/bootstrap_admission_test.go +++ b/internal/watch/bootstrap_admission_test.go @@ -35,9 +35,8 @@ func bootGitTarget() *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: bootTargetName, Namespace: bootTargetNS}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "git"}, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: bootProviderName}, - Branch: "main", Path: "clusters/prod", }, } @@ -132,7 +131,6 @@ func TestBootstrapClusterWatchRule_SeedsAdmittedRule(t *testing.T) { assert.Equal(t, bootRuleName, compiled[0].Source.Name) assert.Equal(t, bootTargetName, compiled[0].GitTargetRef) assert.Equal(t, bootTargetNS, compiled[0].GitTargetNamespace) - assert.Equal(t, "main", compiled[0].Branch) assert.Equal(t, "clusters/prod", compiled[0].Path) } @@ -143,9 +141,8 @@ func TestBootstrapRuleStore_SkipsUnauthorizedRuleButStillReady(t *testing.T) { admittedTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "ok-mirror", Namespace: "team-ok"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "git"}, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: bootProviderName}, - Branch: "main", Path: "clusters/ok", }, } @@ -209,7 +206,7 @@ func TestCompileClusterWatchRule_RefusalRemovesAnAlreadyCompiledRule(t *testing. ) decision, err := CompileClusterWatchRule( - ctx, m.Client, m.RuleStore, *bootClusterWatchRule(), *bootGitTarget(), *bootGitProvider()) + ctx, m.Client, m.RuleStore, *bootClusterWatchRule(), *bootGitTarget()) require.NoError(t, err) require.True(t, decision.Admitted) require.Len(t, bootCompiledNames(m), 1, "precondition: the rule is compiled") @@ -221,7 +218,7 @@ func TestCompileClusterWatchRule_RefusalRemovesAnAlreadyCompiledRule(t *testing. require.NoError(t, m.Client.Update(ctx, &provider)) decision, err = CompileClusterWatchRule( - ctx, m.Client, m.RuleStore, *bootClusterWatchRule(), *bootGitTarget(), *bootGitProvider()) + ctx, m.Client, m.RuleStore, *bootClusterWatchRule(), *bootGitTarget()) require.NoError(t, err) assert.False(t, decision.Admitted) diff --git a/internal/watch/collection_overlap_test.go b/internal/watch/collection_overlap_test.go index da3e871d..48d3d7b8 100644 --- a/internal/watch/collection_overlap_test.go +++ b/internal/watch/collection_overlap_test.go @@ -135,7 +135,7 @@ func TestResolveWatchedTypeTables_SeparateGitTargetsMayOverlap(t *testing.T) { wide := selectorRule("wide", "test-ns", now, nil, configv1alpha3.SourceNamespaceWildcard) wide.Spec.GitTargetRef.Name = "wide-target" store.AddOrUpdateWatchRule(wide, [][]string{{""}}, - "wide-target", "test-ns", "test-provider", "test-ns", "main", "wide") + "wide-target", "test-ns", "test-provider", "test-ns", "wide") addSelectorRule(store, selectorRule("named", "team-a", now.Add(time.Minute), nil, ""), "team-a") manager.refreshWatchedTypeTables() diff --git a/internal/watch/config_plane_split_review_fixes_test.go b/internal/watch/config_plane_split_review_fixes_test.go index 675cff39..eb7b9d56 100644 --- a/internal/watch/config_plane_split_review_fixes_test.go +++ b/internal/watch/config_plane_split_review_fixes_test.go @@ -272,7 +272,7 @@ func TestRefreshWatchedTypeTables_RetargetReResolvesAtEqualRevisions(t *testing. store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "t", "test-ns", "test-provider", "test-ns", "main", "test-path", + "t", "test-ns", "test-provider", "test-ns", "test-path", ) m.rememberGitTargetCluster(gitDestRef("t"), clusterA) m.refreshWatchedTypeTables() diff --git a/internal/watch/event_router.go b/internal/watch/event_router.go index 96778b8b..e5b22f06 100644 --- a/internal/watch/event_router.go +++ b/internal/watch/event_router.go @@ -196,12 +196,20 @@ func (r *EventRouter) commitRequestWorker( attach.GitTargetNamespace, attach.GitTargetName, err) } - provider := gitTarget.GitProviderKey() - worker, exists := r.WorkerManager.GetWorkerForTarget(provider.Name, provider.Namespace, gitTarget.Spec.Branch) - if !exists { - return nil, gitTarget.Spec.Branch, nil + worker, exists := r.WorkerManager.WorkerForTarget(&gitTarget) + if exists { + return worker, worker.Branch, nil + } + // No worker yet: the branch is still the GitBranch's to name, read from the cache. A GitBranch + // that is not there names none. + var gitBranch configv1alpha3.GitBranch + if err := r.Client.Get(ctx, gitTarget.GitBranchKey(), &gitBranch); err != nil { + if apierrors.IsNotFound(err) { + return nil, "", nil + } + return nil, "", fmt.Errorf("get GitBranch %s: %w", gitTarget.GitBranchKey(), err) } - return worker, gitTarget.Spec.Branch, nil + return nil, gitBranch.Spec.Branch, nil } // recordBackgroundResyncFailure counts a fire-and-forget resync whose apply failed or @@ -244,8 +252,7 @@ func (r *EventRouter) resolveWorkerForGitDest( } return nil, fmt.Errorf("get GitTarget %s: %w", gitDest.String(), err) } - provider := gitTarget.GitProviderKey() - worker, exists := r.WorkerManager.GetWorkerForTarget(provider.Name, provider.Namespace, gitTarget.Spec.Branch) + worker, exists := r.WorkerManager.WorkerForTarget(&gitTarget) if !exists { return nil, fmt.Errorf("no worker for %s", gitDest.String()) } diff --git a/internal/watch/event_router_test.go b/internal/watch/event_router_test.go index 7372f9a8..5197f2c6 100644 --- a/internal/watch/event_router_test.go +++ b/internal/watch/event_router_test.go @@ -86,8 +86,7 @@ func TestEnqueueScopedResync_ReportsMissingWorker(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget).Build() @@ -301,11 +300,10 @@ func TestServiceCommitRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } - client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget).Build() + client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget, testGitBranchObject()).Build() workerManager := git.NewWorkerManager( client, logr.Discard(), @@ -332,8 +330,7 @@ func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, gitTarget).Build() @@ -350,8 +347,16 @@ func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) time.Sleep(100 * time.Millisecond) // allow the manager to record its context ensureErr := workerManager.EnsureWorker( - ctx, "team-a-provider", "team-a", "main", - git.RepoIdentity{URL: "file:///tmp/does-not-need-to-exist"}, "") + ctx, + claimWorkerForTest( + workerManager, + "team-a-provider", + "team-a", + "main", + git.RepoIdentity{URL: "file:///tmp/does-not-need-to-exist"}, + "", + ), + ) require.NoError(t, ensureErr) @@ -381,11 +386,10 @@ func TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } - client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget).Build() + client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget, testGitBranchObject()).Build() workerManager := git.NewWorkerManager( client, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) router := NewEventRouter(workerManager, nil, client, logr.Discard()) @@ -416,8 +420,7 @@ func TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, gitTarget).Build() @@ -428,8 +431,20 @@ func TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { defer cancel() go func() { _ = workerManager.Start(ctx) }() time.Sleep(100 * time.Millisecond) // allow the manager to record its context - require.NoError(t, workerManager.EnsureWorker( - ctx, "team-a-provider", "team-a", "main", git.RepoIdentity{URL: "file:///tmp/does-not-need-to-exist"}, "")) + require.NoError( + t, + workerManager.EnsureWorker( + ctx, + claimWorkerForTest( + workerManager, + "team-a-provider", + "team-a", + "main", + git.RepoIdentity{URL: "file:///tmp/does-not-need-to-exist"}, + "", + ), + ), + ) router := NewEventRouter(workerManager, nil, client, logr.Discard()) attach := saveAttach("team-a-config", "team-a") @@ -466,8 +481,7 @@ func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "team-a-provider"}, - Branch: "main", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, gitTarget).Build() @@ -478,9 +492,21 @@ func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T defer cancel() go func() { _ = workerManager.Start(ctx) }() time.Sleep(100 * time.Millisecond) // allow the manager to record its context - require.NoError(t, workerManager.EnsureWorker( - ctx, "team-a-provider", "team-a", "main", git.RepoIdentity{URL: "file:///tmp/does-not-need-to-exist"}, "")) - worker, ok := workerManager.GetWorkerForTarget("team-a-provider", "team-a", "main") + require.NoError( + t, + workerManager.EnsureWorker( + ctx, + claimWorkerForTest( + workerManager, + "team-a-provider", + "team-a", + "main", + git.RepoIdentity{URL: "file:///tmp/does-not-need-to-exist"}, + "", + ), + ), + ) + worker, ok := workerManager.WorkerForGitBranch(gitBranchForTest("team-a-provider", "team-a", "main")) require.True(t, ok) router := NewEventRouter(workerManager, nil, client, logr.Discard()) @@ -554,3 +580,14 @@ func TestDrainScopedResync_QueueFullIsDrainedNotOrphaned(t *testing.T) { assert.Empty(t, mgr.watchPlane().streams, "a resync that never queued wrote nothing, so it must not move stream readiness") } + +// testGitBranchObject is the GitBranch team-a's GitTargets reference: team-a-provider's main. +func testGitBranchObject() *configv1alpha3.GitBranch { + return &configv1alpha3.GitBranch{ + ObjectMeta: metav1.ObjectMeta{Name: gitBranchNameForTest("team-a-provider", "main"), Namespace: "team-a"}, + Spec: configv1alpha3.GitBranchSpec{ + GitProviderRef: meta.LocalObjectReference{Name: "team-a-provider"}, + Branch: "main", + }, + } +} diff --git a/internal/watch/git_branch_helpers_test.go b/internal/watch/git_branch_helpers_test.go new file mode 100644 index 00000000..b7fd675f --- /dev/null +++ b/internal/watch/git_branch_helpers_test.go @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "time" + + k8stypes "k8s.io/apimachinery/pkg/types" + + "github.com/ConfigButler/gitops-reverser/internal/git" +) + +// gitBranchNameForTest is the GitBranch a test GitTarget references: one per provider and branch. +func gitBranchNameForTest(provider, branch string) string { return provider + "-" + branch } + +// gitBranchForTest is that GitBranch, in the provider's namespace. +func gitBranchForTest(provider, namespace, branch string) k8stypes.NamespacedName { + return k8stypes.NamespacedName{Namespace: namespace, Name: gitBranchNameForTest(provider, branch)} +} + +// claimWorkerForTest makes gitBranchForTest(provider, namespace, branch) hold the repository branch, +// as its reconcile does once its credential is proved, and returns the request its GitTargets make. +func claimWorkerForTest( + m *git.WorkerManager, provider, namespace, branch string, repo git.RepoIdentity, parentBranch string, +) git.WorkerRequest { + gitBranch := gitBranchForTest(provider, namespace, branch) + m.Claims().Observe(git.BranchKeyFor(repo.URL, branch), git.BranchClaimant{ + GitBranch: gitBranch, + UID: k8stypes.UID(gitBranch.String()), + Created: time.Unix(0, 0), + }, true) + return git.WorkerRequest{ + GitBranch: gitBranch, Provider: provider, Branch: branch, Repo: repo, ParentBranch: parentBranch, + } +} diff --git a/internal/watch/manager_snapshot_test.go b/internal/watch/manager_snapshot_test.go index b9977654..4615a880 100644 --- a/internal/watch/manager_snapshot_test.go +++ b/internal/watch/manager_snapshot_test.go @@ -78,7 +78,7 @@ func addSecretsWatchRule(store *rulestore.RuleStore) { store.AddOrUpdateWatchRule( rule, ownNamespaceScope(rule), - "my-target", "gitops-reverser", "provider", "gitops-reverser", "main", "live", + "my-target", "gitops-reverser", "provider", "gitops-reverser", "live", ) } diff --git a/internal/watch/manager_startup_test.go b/internal/watch/manager_startup_test.go index 96537f18..fc90f6b0 100644 --- a/internal/watch/manager_startup_test.go +++ b/internal/watch/manager_startup_test.go @@ -40,9 +40,8 @@ func TestManagerStart_MustSeedRuleStoreFromExistingWatchRules(t *testing.T) { existingGitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "playground-target", Namespace: "tilt-playground"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "playground-provider"}, - Branch: "main", - Path: "live-cluster", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("playground-provider", "main")}, + Path: "live-cluster", }, } existingGitProvider := &configv1alpha3.GitProvider{ @@ -107,9 +106,8 @@ func TestManagerStart_MustSeedRuleStoreFromExistingClusterWatchRules(t *testing. existingGitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "ops-target", Namespace: "ops"}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "ops-provider"}, - Branch: "main", - Path: "cluster-state", + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("ops-provider", "main")}, + Path: "cluster-state", }, } existingGitProvider := &configv1alpha3.GitProvider{ diff --git a/internal/watch/object_selector_test.go b/internal/watch/object_selector_test.go index 271e7105..6cc4422f 100644 --- a/internal/watch/object_selector_test.go +++ b/internal/watch/object_selector_test.go @@ -176,7 +176,7 @@ func selectorRule(name, namespace string, created time.Time, selector *metav1.La func addSelectorRule(store *rulestore.RuleStore, rule configv1alpha3.WatchRule, namespaces ...string) { store.AddOrUpdateWatchRule(rule, [][]string{namespaces}, - "sel-target", "test-ns", "test-provider", "test-ns", "main", "test-path") + "sel-target", "test-ns", "test-provider", "test-ns", "test-path") } func labels(kv ...string) *metav1.LabelSelector { @@ -239,7 +239,7 @@ func TestCompileWatchRule_RefusesAnInvalidObjectSelector(t *testing.T) { } _, err := CompileWatchRule(context.Background(), nil, store, rule, - configv1alpha3.GitTarget{}, configv1alpha3.GitProvider{}) + configv1alpha3.GitTarget{}) var selectorErr *ObjectSelectorError require.ErrorAs(t, err, &selectorErr) @@ -251,11 +251,11 @@ func TestCompileWatchRule_RefusesAnInvalidObjectSelector(t *testing.T) { func TestCompileClusterWatchRule_RefusesAnInvalidObjectSelector(t *testing.T) { store := rulestore.NewStore() rule := clusterRuleForResource("bad", "namespaces") - store.AddOrUpdateClusterWatchRule(rule, "t", "ns", "p", "ns", "main", "path") + store.AddOrUpdateClusterWatchRule(rule, "t", "ns", "p", "ns", "path") rule.Spec.Rules[0].ObjectSelector = labels("bad key", "x") _, err := CompileClusterWatchRule(context.Background(), nil, store, rule, - configv1alpha3.GitTarget{}, configv1alpha3.GitProvider{}) + configv1alpha3.GitTarget{}) var selectorErr *ObjectSelectorError require.ErrorAs(t, err, &selectorErr) diff --git a/internal/watch/source_namespace_planning_test.go b/internal/watch/source_namespace_planning_test.go index 822ed62e..2d3a7da2 100644 --- a/internal/watch/source_namespace_planning_test.go +++ b/internal/watch/source_namespace_planning_test.go @@ -34,7 +34,7 @@ func watchRuleWithSource(name, gitTargetName, sourceNamespace string) configv1al // would hand the store — so the planning assertions run against the real shape. func addRule(store *rulestore.RuleStore, rule configv1alpha3.WatchRule, scope [][]string) { store.AddOrUpdateWatchRule( - rule, scope, rule.Spec.GitTargetRef.Name, "test-ns", "test-provider", "test-ns", "main", "test-path") + rule, scope, rule.Spec.GitTargetRef.Name, "test-ns", "test-provider", "test-ns", "test-path") } // makeStoreWithScope compiles one WatchRule with a given resolved scope into a fresh store, for the diff --git a/internal/watch/source_namespace_stream_summary_test.go b/internal/watch/source_namespace_stream_summary_test.go index 4324cde9..2ef7db7e 100644 --- a/internal/watch/source_namespace_stream_summary_test.go +++ b/internal/watch/source_namespace_stream_summary_test.go @@ -59,7 +59,7 @@ func srcnsOverrideRule(sourceNamespace string) configv1alpha3.WatchRule { // compileForSummary seeds the store the way CompileWatchRule would, with an explicit resolved scope. func compileForSummary(m *Manager, rule configv1alpha3.WatchRule, scope [][]string) { m.RuleStore.AddOrUpdateWatchRule( - rule, scope, "acme", "tenant-acme", "git", "tenant-acme", "main", "tenants/acme") + rule, scope, "acme", "tenant-acme", "git", "tenant-acme", "tenants/acme") } func srcnsGitDest() types.ResourceReference { diff --git a/internal/watch/source_namespace_test.go b/internal/watch/source_namespace_test.go index 5cf7e37a..16f99077 100644 --- a/internal/watch/source_namespace_test.go +++ b/internal/watch/source_namespace_test.go @@ -32,9 +32,8 @@ func snbGitTarget() *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: snbTarget, Namespace: snbTenantNS}, Spec: configv1alpha3.GitTargetSpec{ - GitProviderRef: meta.NamespacedObjectReference{Name: "git"}, + GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: snbProvider}, - Branch: "main", Path: "tenants/acme", }, } @@ -129,7 +128,6 @@ func TestBootstrap_LegacyWatchRuleStillCompiles(t *testing.T) { assert.Equal(t, snbRule, compiled[0].Source.Name) assert.Equal(t, []string{snbTenantNS}, compiled[0].ResourceRules[0].SourceNamespaces, "a legacy item's source namespace is the rule's own namespace") - assert.Equal(t, "main", compiled[0].Branch) } // The admitted override path seeds the EFFECTIVE namespace, not the rule's own. @@ -160,7 +158,7 @@ func TestCompileWatchRule_WildcardCompilesToTheClusterWideCollection(t *testing. ) resolved, err := CompileWatchRule(ctx, m.Client, m.RuleStore, - *snbWatchRule(configv1alpha3.SourceNamespaceWildcard), *snbGitTarget(), *snbGitProvider()) + *snbWatchRule(configv1alpha3.SourceNamespaceWildcard), *snbGitTarget()) require.NoError(t, err) require.True(t, resolved.Admitted()) @@ -180,7 +178,7 @@ func TestCompileWatchRule_WildcardIsRefusedWithoutDelegation(t *testing.T) { ) resolved, err := CompileWatchRule(ctx, m.Client, m.RuleStore, - *snbWatchRule(configv1alpha3.SourceNamespaceWildcard), *snbGitTarget(), *snbGitProvider()) + *snbWatchRule(configv1alpha3.SourceNamespaceWildcard), *snbGitTarget()) require.NoError(t, err) assert.False(t, resolved.Admitted()) @@ -201,9 +199,8 @@ func TestCompileWatchRule_TerminalRefusalRemovesAnAlreadyCompiledRule(t *testing rule := *snbWatchRule(snbSourceNS) target := *snbGitTarget() - provider := *snbGitProvider() - resolved, err := CompileWatchRule(ctx, m.Client, m.RuleStore, rule, target, provider) + resolved, err := CompileWatchRule(ctx, m.Client, m.RuleStore, rule, target) require.NoError(t, err) require.True(t, resolved.Admitted()) require.Len(t, m.RuleStore.SnapshotWatchRules(), 1, "precondition: the rule is compiled") @@ -214,7 +211,7 @@ func TestCompileWatchRule_TerminalRefusalRemovesAnAlreadyCompiledRule(t *testing stored.Spec.AllowCrossNamespace = nil require.NoError(t, m.Client.Update(ctx, &stored)) - resolved, err = CompileWatchRule(ctx, m.Client, m.RuleStore, rule, target, provider) + resolved, err = CompileWatchRule(ctx, m.Client, m.RuleStore, rule, target) require.NoError(t, err) assert.False(t, resolved.Admitted()) diff --git a/internal/watch/watched_type_helpers_test.go b/internal/watch/watched_type_helpers_test.go index c44ed8e0..649be200 100644 --- a/internal/watch/watched_type_helpers_test.go +++ b/internal/watch/watched_type_helpers_test.go @@ -58,7 +58,7 @@ func TestRefreshWatchedTypeTables_ConcurrentRefreshesConverge(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) var wg sync.WaitGroup @@ -75,7 +75,7 @@ func TestRefreshWatchedTypeTables_ConcurrentRefreshesConverge(t *testing.T) { // Concurrently add a second rule mid-flight. store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-2", "nodes"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) wg.Wait() diff --git a/internal/watch/watched_type_metrics_test.go b/internal/watch/watched_type_metrics_test.go index dfa81989..4621e51a 100644 --- a/internal/watch/watched_type_metrics_test.go +++ b/internal/watch/watched_type_metrics_test.go @@ -24,7 +24,7 @@ func TestWatchTypeSamples_SumIsTheResolvedTypeCount(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() @@ -59,7 +59,7 @@ func TestWatchTypeSamples_DepartedTargetStopsPublishing(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() manager.installWatchTypeGaugeSource() @@ -94,7 +94,7 @@ func TestWatchTypeSamples_DoesNotResolveOnTheScrapePath(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() manager.installWatchTypeGaugeSource() @@ -106,7 +106,7 @@ func TestWatchTypeSamples_DoesNotResolveOnTheScrapePath(t *testing.T) { // A second type is declared, and nothing refreshes the tables. store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-2", "nodes"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) after, ok := telemetry.CollectInt64Sum(reader, watchTypesMetric, typeStateLabels("replaying")) diff --git a/internal/watch/watched_type_resolver.go b/internal/watch/watched_type_resolver.go index acdadd55..7f7a0294 100644 --- a/internal/watch/watched_type_resolver.go +++ b/internal/watch/watched_type_resolver.go @@ -287,14 +287,14 @@ func (m *Manager) resolveWatchedTypeTables() (map[string]WatchedTypeTable, map[s } byTarget := map[string]*targetSelections{} - get := func(ref types.ResourceReference, providerNS, provider, branch, path string) *targetSelections { + get := func(ref types.ResourceReference, gitBranchNS, gitBranch, path string) *targetSelections { key := ref.Key() ts := byTarget[key] if ts == nil { ts = &targetSelections{gitDest: ref} byTarget[key] = ts } - ts.dest = watchPlanDest(providerNS, provider, branch, path) + ts.dest = watchPlanDest(gitBranchNS, gitBranch, path) return ts } @@ -330,12 +330,12 @@ func (m *Manager) resolveWatchedTypeTables() (map[string]WatchedTypeTable, map[s // before being dropped. func (m *Manager) collectWatchRuleSelections( recordsFor func(clusterID string) []typeset.TypeRecord, - get func(types.ResourceReference, string, string, string, string) *targetSelections, + get func(types.ResourceReference, string, string, string) *targetSelections, ) { for _, rule := range m.RuleStore.SnapshotWatchRules() { gitTargetRef := types.NewResourceReference(rule.GitTargetRef, rule.GitTargetNamespace) records := recordsFor(m.clusterIDForGitTarget(gitTargetRef)) - ts := get(gitTargetRef, rule.GitProviderNamespace, rule.GitProviderRef, rule.Branch, rule.Path) + ts := get(gitTargetRef, rule.GitBranchNamespace, rule.GitBranchRef, rule.Path) selecting := selectingRule{ kind: ruleKindWatchRule, namespace: rule.Source.Namespace, @@ -378,12 +378,12 @@ func (m *Manager) collectWatchRuleSelections( // cannot widen a stream. Selections keep namespace "", now only for genuinely cluster-scoped types. func (m *Manager) collectClusterWatchRuleSelections( recordsFor func(clusterID string) []typeset.TypeRecord, - get func(types.ResourceReference, string, string, string, string) *targetSelections, + get func(types.ResourceReference, string, string, string) *targetSelections, ) { for _, rule := range m.RuleStore.SnapshotClusterWatchRules() { gitTargetRef := types.NewResourceReference(rule.GitTargetRef, rule.GitTargetNamespace) records := recordsFor(m.clusterIDForGitTarget(gitTargetRef)) - ts := get(gitTargetRef, rule.GitProviderNamespace, rule.GitProviderRef, rule.Branch, rule.Path) + ts := get(gitTargetRef, rule.GitBranchNamespace, rule.GitBranchRef, rule.Path) selecting := selectingRule{kind: ruleKindClusterWatchRule, name: rule.Source.Name, createdAt: rule.CreatedAt} for _, rr := range rule.Rules { matched := matchFollowableRecords( @@ -526,8 +526,8 @@ func preferredRecord(records []typeset.TypeRecord) typeset.TypeRecord { // watchPlanDest renders a GitTarget's write destination fingerprint in the exact form // the effective-plan hash uses, so the hash is byte-identical whether built from the // table or inline. -func watchPlanDest(providerNS, provider, branch, path string) string { - return fmt.Sprintf("provider=%s/%s|branch=%q|path=%q", providerNS, provider, branch, path) +func watchPlanDest(gitBranchNS, gitBranch, path string) string { + return fmt.Sprintf("gitbranch=%s/%s|path=%q", gitBranchNS, gitBranch, path) } // rulesFingerprint is a cheap, resolution-free hash of the raw rule inputs — the @@ -566,7 +566,7 @@ func watchRuleFingerprint(rule rulestore.CompiledRule) string { fmt.Fprintf(&b, "wr|%s/%s|created=%d|gt=%s/%s|dest=%s", rule.Source.Namespace, rule.Source.Name, rule.CreatedAt.UnixNano(), rule.GitTargetNamespace, rule.GitTargetRef, - watchPlanDest(rule.GitProviderNamespace, rule.GitProviderRef, rule.Branch, rule.Path)) + watchPlanDest(rule.GitBranchNamespace, rule.GitBranchRef, rule.Path)) for _, rr := range rule.ResourceRules { fmt.Fprintf(&b, "|rr[g=%s;v=%s;r=%s;src=%s;sel=%q]", strings.Join(rr.APIGroups, ","), strings.Join(rr.APIVersions, ","), @@ -583,7 +583,7 @@ func clusterWatchRuleFingerprint(rule rulestore.CompiledClusterRule) string { fmt.Fprintf(&b, "cwr|%s|created=%d|gt=%s/%s|dest=%s", rule.Source.Name, rule.CreatedAt.UnixNano(), rule.GitTargetNamespace, rule.GitTargetRef, - watchPlanDest(rule.GitProviderNamespace, rule.GitProviderRef, rule.Branch, rule.Path)) + watchPlanDest(rule.GitBranchNamespace, rule.GitBranchRef, rule.Path)) for _, rr := range rule.Rules { fmt.Fprintf(&b, "|rr[g=%s;v=%s;r=%s;sel=%q]", strings.Join(rr.APIGroups, ","), strings.Join(rr.APIVersions, ","), diff --git a/internal/watch/watched_type_resolver_test.go b/internal/watch/watched_type_resolver_test.go index 9ca24e04..f5ea4320 100644 --- a/internal/watch/watched_type_resolver_test.go +++ b/internal/watch/watched_type_resolver_test.go @@ -39,7 +39,7 @@ func TestRefreshWatchedTypeTables_ClusterWatchRuleResolvesClusterWideType(t *tes manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() @@ -51,7 +51,7 @@ func TestRefreshWatchedTypeTables_ClusterWatchRuleResolvesClusterWideType(t *tes assert.Equal(t, "Namespace", wt.GVK.Kind) assert.True(t, wt.ClusterWide(), "a ClusterWatchRule streams its cluster-scoped types cluster-wide") assert.Equal(t, []string{""}, wt.WatchScopes()) - assert.Equal(t, `provider=test-ns/test-provider|branch="main"|path="test-path"`, table.Dest) + assert.Equal(t, `gitbranch=test-ns/test-provider|path="test-path"`, table.Dest) } func TestRefreshWatchedTypeTables_WatchRuleScopesTypeToItsNamespace(t *testing.T) { @@ -59,12 +59,12 @@ func TestRefreshWatchedTypeTables_WatchRuleScopesTypeToItsNamespace(t *testing.T store.AddOrUpdateWatchRule( watchRuleForTarget("rule-a", "wt-ns-target", "ns-a"), ownNamespaceScope(watchRuleForTarget("rule-a", "wt-ns-target", "ns-a")), - "wt-ns-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "wt-ns-target", "test-ns", "test-provider", "test-ns", "test-path", ) store.AddOrUpdateWatchRule( watchRuleForTarget("rule-b", "wt-ns-target", "ns-b"), ownNamespaceScope(watchRuleForTarget("rule-b", "wt-ns-target", "ns-b")), - "wt-ns-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "wt-ns-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() @@ -80,7 +80,7 @@ func TestRefreshWatchedTypeTables_RuleChangeReResolves(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() first, _ := manager.watchedTypeTableForGitDest(gitDestRef("test-target")) @@ -89,7 +89,7 @@ func TestRefreshWatchedTypeTables_RuleChangeReResolves(t *testing.T) { // A second rule selecting a different resource is reflected on the next refresh. store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-2", "nodes"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() @@ -109,7 +109,7 @@ func TestRefreshWatchedTypeTables_NoChangeReusesResolvedTables(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() manager.watchedTypes.mu.Lock() @@ -129,14 +129,14 @@ func TestRulesFingerprint_StableUntilRuleChanges(t *testing.T) { manager, store := makeWatchedTypeManager(t) store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "namespaces"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) fp1 := manager.rulesFingerprint() assert.Equal(t, fp1, manager.rulesFingerprint(), "fingerprint must be stable for unchanged rules") store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-2", "nodes"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) assert.NotEqual(t, fp1, manager.rulesFingerprint(), "a new rule must move the fingerprint") } @@ -147,7 +147,7 @@ func TestRefreshWatchedTypeTables_KeepsTargetWithUnresolvableRulesAsEmptyTable(t // but the GitTarget must still appear as an empty table, not vanish. store.AddOrUpdateClusterWatchRule( clusterRuleForResource("rule-1", "ghosts"), - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() @@ -174,7 +174,7 @@ func TestRefreshWatchedTypeTables_ExcludesAmbiguousGVK(t *testing.T) { }}, }, }, - "test-target", "test-ns", "test-provider", "test-ns", "main", "test-path", + "test-target", "test-ns", "test-provider", "test-ns", "test-path", ) manager.refreshWatchedTypeTables() diff --git a/internal/watch/watchrule_compile.go b/internal/watch/watchrule_compile.go index 7da29304..1043f395 100644 --- a/internal/watch/watchrule_compile.go +++ b/internal/watch/watchrule_compile.go @@ -51,7 +51,6 @@ func CompileWatchRule( store *rulestore.RuleStore, rule configv1alpha3.WatchRule, target configv1alpha3.GitTarget, - provider configv1alpha3.GitProvider, ) (authz.ResolvedSourceScope, error) { key := k8stypes.NamespacedName{Name: rule.Name, Namespace: rule.Namespace} @@ -72,8 +71,7 @@ func CompileWatchRule( rule, itemNamespaces(resolved), target.Name, target.Namespace, - provider.Name, provider.Namespace, - target.Spec.Branch, + target.GitBranchKey().Name, target.GitBranchKey().Namespace, target.Spec.Path, ) return resolved, nil @@ -113,7 +111,6 @@ func CompileClusterWatchRule( store *rulestore.RuleStore, rule configv1alpha3.ClusterWatchRule, target configv1alpha3.GitTarget, - provider configv1alpha3.GitProvider, ) (ClusterWatchRuleDecision, error) { key := k8stypes.NamespacedName{Name: rule.Name} @@ -139,8 +136,7 @@ func CompileClusterWatchRule( store.AddOrUpdateClusterWatchRule( rule, target.Name, target.Namespace, - provider.Name, provider.Namespace, - target.Spec.Branch, + target.GitBranchKey().Name, target.GitBranchKey().Namespace, target.Spec.Path, ) return ClusterWatchRuleDecision{Admitted: true}, nil From 548ffa1d762332c370f3c42e6e73c4da83855a11 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Wed, 21 Oct 2026 20:24:00 +0000 Subject: [PATCH 4/6] feat(api)!: move the branch fields from GitTarget to GitBranch GitTarget.spec.gitBranchRef is required. spec.gitProviderRef, spec.branch and spec.parentBranch stay in the schema only so that setting one is refused, naming gitBranchRef. GitTarget.status.remote is gone (it is on the GitBranch), and so is GitProvider.status.branches: listing GitBranches answers the same question. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/gitprovider_types.go | 32 ----- api/v1alpha3/gittarget_types.go | 79 ++++-------- api/v1alpha3/zz_generated.deepcopy.go | 31 +---- .../bases/configbutler.ai_gitproviders.yaml | 37 ------ .../crd/bases/configbutler.ai_gittargets.yaml | 112 ++---------------- config/samples/README.md | 4 +- config/samples/quickstart-gitbranch.yaml | 14 +++ config/samples/quickstart-gittarget.yaml | 7 +- docs/bi-directional-argocd.md | 13 +- docs/bi-directional-flux.md | 13 +- docs/configuration.md | 7 +- hack/crdfields/fields.yaml | 17 +-- internal/applier/argocd_guide_test.go | 15 ++- internal/applier/dispatcher_test.go | 2 +- internal/applier/flux_guide_test.go | 17 ++- internal/authz/gitbranch_admission_test.go | 2 +- internal/authz/source_namespace_test.go | 2 +- .../applytrigger_controller_test.go | 2 +- .../controller/applytrigger_schema_test.go | 4 +- .../clusterwatchrule_admission_test.go | 2 +- .../commitrequest_outcome_metric_test.go | 2 +- .../controller/dependency_watches_test.go | 2 +- .../duration_fields_admission_test.go | 21 ++-- .../controller/gitbranch_controller_test.go | 4 +- .../gittarget_apply_trigger_test.go | 2 +- .../controller/gittarget_controller_test.go | 28 ++--- .../gittarget_controller_unit_test.go | 2 +- .../controller/gittarget_gitbranch_test.go | 8 +- .../controller/gittarget_immutability_test.go | 4 +- .../gittarget_placement_validation_test.go | 2 +- .../gittarget_shared_branch_test.go | 2 +- .../gittarget_source_cluster_test.go | 2 +- .../gittarget_worker_cleanup_test.go | 2 +- .../gittarget_worker_repoint_test.go | 6 +- .../controller/live_message_migration_test.go | 5 +- .../superseded_fields_admission_test.go | 45 ++++++- .../controller/watchrule_controller_test.go | 4 +- .../watchrule_source_namespace_test.go | 2 +- .../wiring/readiness_wiring_test.go | 2 +- internal/git/apply_trigger_test.go | 2 +- internal/git/branch_worker_loop_test.go | 2 +- internal/git/branch_worker_split_test.go | 2 +- internal/git/branch_worker_test.go | 12 +- internal/git/git_branch_ref_test.go | 2 +- internal/git/git_roundtrip_ledger_test.go | 2 +- internal/git/publication_test.go | 2 +- internal/git/refusal_reapply_test.go | 4 +- internal/git/secret_write_test.go | 2 +- internal/git/types_test.go | 4 +- internal/git/worker_branch_owner_test.go | 2 +- internal/git/worker_manager_test.go | 2 +- .../git/write_boundary_precondition_test.go | 2 +- internal/watch/bootstrap_admission_test.go | 4 +- internal/watch/event_router_test.go | 12 +- internal/watch/manager_startup_test.go | 4 +- internal/watch/source_namespace_test.go | 2 +- 56 files changed, 241 insertions(+), 378 deletions(-) create mode 100644 config/samples/quickstart-gitbranch.yaml diff --git a/api/v1alpha3/gitprovider_types.go b/api/v1alpha3/gitprovider_types.go index 5c33117b..0f9c9314 100644 --- a/api/v1alpha3/gitprovider_types.go +++ b/api/v1alpha3/gitprovider_types.go @@ -213,22 +213,6 @@ type GitProviderStatus struct { // Only populated when commit.signing is configured and a signing key is available. // +optional SigningPublicKey string `json:"signingPublicKey,omitempty"` - - // Branches lists the branches this repository's GitTargets are configured to write, with how - // many reference each. It answers what the repository is being used for, so it includes a - // branch whose targets are blocked or suspended and excludes a branch nothing references any - // more. - // - // It is not the branches that exist on the remote, and not the workers currently running: it - // moves only when somebody edits a GitTarget. An EMPTY list beside Ready=True is the state - // this exists to make legible — configured and unused, which looks nothing like broken and - // previously took a GitTarget listing to tell apart. An ABSENT list is the third state and a - // different one: no inventory has been read yet, so nothing here has been measured. - // +optional - // +listType=map - // +listMapKey=name - Branches []GitProviderBranchStatus `json:"branches"` - // LastVerifiedAt is when the credential and the repository were last proved together: the // connectivity check listed the remote's refs with this provider's credential and the remote // answered. It is never cleared — a failure is already carried by Ready=False with a reason — @@ -247,22 +231,6 @@ type GitProviderStatus struct { WebhookPath string `json:"webhookPath,omitempty"` } -// GitProviderBranchStatus is one branch of this repository that GitTargets are configured to -// write, and how many of them write it. -type GitProviderBranchStatus struct { - // Name is the branch. - // +required - // +kubebuilder:validation:MinLength=1 - Name string `json:"name"` - - // GitTargetCount is how many GitTargets are configured to write this branch. More than one means - // they share a branch worker, and each still reports its own folder's health on its own - // conditions. - // +required - // +kubebuilder:validation:Minimum=0 - GitTargetCount int32 `json:"gitTargetCount"` -} - // CommitSpec configures the commit identity and signing a GitProvider uses. Message formatting // lives on the GitTarget (spec.commit.message), because it describes the folder rather than the // connection. diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 9d09457f..008a1ddd 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -11,7 +11,7 @@ import ( // GitTargetSpec defines the desired state of GitTarget. // // The destination fields — gitBranchRef and path — are immutable. A -// GitTarget materializes the watched resources at exactly one (provider, branch, +// GitTarget materializes the watched resources at exactly one (repository branch, // folder); changing where it writes would orphan the old materialization and require // migrating manifests between repositories/branches/folders. Instead of reconciling // that move, the destination is fixed: to relocate a GitTarget, delete it and create a @@ -19,15 +19,18 @@ import ( // simple — a successful snapshot can never be silently invalidated by a destination // change. // -// +kubebuilder:validation:XValidation:rule="has(self.gitBranchRef) == has(oldSelf.gitBranchRef) && (!has(self.gitBranchRef) || self.gitBranchRef == oldSelf.gitBranchRef)",message="spec.gitBranchRef is immutable; delete and recreate the GitTarget to change its destination" -// +kubebuilder:validation:XValidation:rule="!has(oldSelf.gitProviderRef) || self.gitProviderRef == oldSelf.gitProviderRef",message="spec.gitProviderRef is immutable; delete and recreate the GitTarget to change its destination" -// +kubebuilder:validation:XValidation:rule="has(self.branch) == has(oldSelf.branch) && (!has(self.branch) || self.branch == oldSelf.branch)",message="spec.branch is immutable; delete and recreate the GitTarget to change its destination" +// +kubebuilder:validation:XValidation:rule="self.gitBranchRef == oldSelf.gitBranchRef",message="spec.gitBranchRef is immutable; delete and recreate the GitTarget to change its destination" // +kubebuilder:validation:XValidation:rule="self.path == oldSelf.path",message="spec.path is immutable; delete and recreate the GitTarget to change its destination" -// +kubebuilder:validation:XValidation:rule="has(self.parentBranch) == has(oldSelf.parentBranch) && (!has(self.parentBranch) || self.parentBranch == oldSelf.parentBranch)",message="spec.parentBranch is immutable; delete and recreate the GitTarget to change it" +// +// spec.gitProviderRef, spec.branch and spec.parentBranch moved to a GitBranch. A pruned field would +// apply cleanly and leave the GitTarget naming no branch, so the old spellings stay in the schema +// only to be refused by name. The rule sits on spec, not on the fields, so CRD validation +// ratcheting cannot wave a stored value through on the next edit. +// +kubebuilder:validation:XValidation:rule="!has(self.gitProviderRef) && !has(self.branch) && !has(self.parentBranch)",message="spec.gitProviderRef, spec.branch and spec.parentBranch moved to a GitBranch; set spec.gitBranchRef to it" // // spec.clusterProviderRef names the SOURCE cluster a GitTarget mirrors FROM (see its field doc). It // is immutable — a folder's source cluster is part of what the folder means, like -// gitProviderRef/branch/path above — and defaults to a ClusterProvider named "default", so it is +// gitBranchRef/path above — and defaults to a ClusterProvider named "default", so it is // always populated (never nil) and always jumpable. // +kubebuilder:validation:XValidation:rule="self.clusterProviderRef == oldSelf.clusterProviderRef",message="spec.clusterProviderRef is immutable; delete and recreate the GitTarget to change the cluster it mirrors" // +kubebuilder:validation:XValidation:rule="!has(self.onRefusal) || self.onRefusal != 'Reapply' || has(self.applyTriggerRef)",message="spec.onRefusal: Reapply needs spec.applyTriggerRef, the trigger asked to reapply" @@ -37,42 +40,24 @@ type GitTargetSpec struct { // another namespace is used only while its GitProvider's spec.accessFrom admits this GitTarget's // namespace. Many GitTargets may reference one GitBranch; they share its branch worker. // Immutable: delete and recreate the GitTarget to change its destination. - // +optional + // +required // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.gitBranchRef.name must not be empty" - GitBranchRef *meta.NamespacedObjectReference `json:"gitBranchRef,omitempty"` + GitBranchRef meta.NamespacedObjectReference `json:"gitBranchRef"` - // GitProviderRef names the GitProvider that backs this target. Namespace omitted means this - // GitTarget's own namespace. A GitProvider in another namespace is used only while its - // spec.accessFrom admits this GitTarget's namespace. Many GitTargets may name the same - // GitProvider, and every GitTarget writing one repository branch must name the same one. - // Immutable: delete and recreate the GitTarget to change its destination. + // RemovedGitProviderRef is the old place of GitBranch.spec.gitProviderRef. It is never read; + // setting it is refused. // +optional - GitProviderRef meta.NamespacedObjectReference `json:"gitProviderRef,omitzero"` + RemovedGitProviderRef *meta.NamespacedObjectReference `json:"gitProviderRef,omitempty"` - // Branch to use for this target. - // Must be one of the allowed branches in the provider. - // Immutable: delete and recreate the GitTarget to change its destination. + // RemovedBranch is the old place of GitBranch.spec.branch. It is never read; setting it is + // refused. // +optional - Branch string `json:"branch,omitempty"` - - // Why "parent" and not base/source/revision: docs/definitions.md reserves those words, and the - // field is flat because it is one fact. It is immutable like branch and path: it decides what - // history a new write branch gets, and a parent changed under a worker holding writes is the - // one case that needed its own replay. Delete and recreate is cheap, because the destination is - // unchanged. It can never hold a commit or tag: a pinned starting point would be a sibling field. + RemovedBranch string `json:"branch,omitempty"` - // ParentBranch is the branch the write branch (spec.branch) is created from, and the branch the - // target compares against while the write branch does not exist on the remote. Omitted: the - // branch the remote's HEAD points at (its default branch), or a branch with no history when - // the repository is empty. When set, it must exist on the remote for the write branch to be - // created; otherwise the GitTarget reports ParentBranchNotFound and writes nothing. Once the - // write branch exists, it is used as it is and the parent is not followed. It is only read, so - // it does not need to be in the GitProvider's allowedBranches. All GitTargets on one GitProvider - // and branch must name the same parent branch (omitted counts as its own value). - // Immutable, including adding or removing it: delete and recreate the GitTarget to change it. + // RemovedParentBranch is the old place of GitBranch.spec.parentBranch. It is never read; + // setting it is refused. // +optional - // +kubebuilder:validation:MinLength=1 - ParentBranch string `json:"parentBranch,omitempty"` + RemovedParentBranch string `json:"parentBranch,omitempty"` // Path within the repository to write resources to, relative to the repository // root. Required and must be non-empty — there is no default, so a GitTarget can @@ -440,14 +425,6 @@ type GitTargetStatus struct { // condition carries the verdict. // +optional Placement *GitTargetPlacementStatus `json:"placement,omitempty"` - - // Remote is where this target's branch is on the Git remote, and when that was last proved. - // It is renewed by every confirmed look: a successful push as well as a fetch. Absent means - // nothing has proved anything yet — either nothing has looked, or the GitProvider was - // recreated against a different repository and what was published no longer describes the - // one this target points at. - // +optional - Remote *GitBranchRemoteStatus `json:"remote,omitempty"` } // Two rules for anything added here. A field earns its place only if a reader cannot get it from @@ -559,7 +536,6 @@ type GitTargetRetentionStatus struct { // +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason` // +kubebuilder:printcolumn:name="Streams",type=string,JSONPath=`.status.streams.summary` // +kubebuilder:printcolumn:name="Suspended",type=boolean,JSONPath=`.spec.suspend`,priority=1 -// +kubebuilder:printcolumn:name="Verified",type=date,JSONPath=`.status.remote.lastVerifiedAt`,priority=1 // +kubebuilder:printcolumn:name="Layout",type=string,JSONPath=`.status.placement.mode`,priority=1 // +kubebuilder:printcolumn:name="RenderRoot",type=string,JSONPath=`.status.placement.renderRoot`,priority=1 // +kubebuilder:printcolumn:name="LayoutResolved",type=string,JSONPath=`.status.conditions[?(@.type=="LayoutResolved")].reason`,priority=1 @@ -569,8 +545,7 @@ type GitTargetRetentionStatus struct { // +kubebuilder:printcolumn:name="StreamsRunning",type=string,JSONPath=`.status.conditions[?(@.type=="StreamsRunning")].status`,priority=1 // +kubebuilder:printcolumn:name="SourceReachable",type=string,JSONPath=`.status.conditions[?(@.type=="SourceClusterReachable")].reason`,priority=1 // +kubebuilder:printcolumn:name="Encryption",type=string,JSONPath=`.spec.encryption.provider`,priority=1 -// +kubebuilder:printcolumn:name="Provider",type=string,JSONPath=`.spec.gitProviderRef.name`,priority=1 -// +kubebuilder:printcolumn:name="Branch",type=string,JSONPath=`.spec.branch`,priority=1 +// +kubebuilder:printcolumn:name="GitBranch",type=string,JSONPath=`.spec.gitBranchRef.name`,priority=1 // +kubebuilder:printcolumn:name="Path",type=string,JSONPath=`.spec.path`,priority=1 // +kubebuilder:printcolumn:name="Message",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].message`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` @@ -606,21 +581,11 @@ func (g *GitTarget) SourceCluster() string { return g.Spec.ClusterProviderRef.Name } -// GitProviderKey is the GitProvider this GitTarget writes through: spec.gitProviderRef, in this -// GitTarget's own namespace unless the reference names another one. -func (g *GitTarget) GitProviderKey() types.NamespacedName { - namespace := g.Spec.GitProviderRef.Namespace - if namespace == "" { - namespace = g.Namespace - } - return types.NamespacedName{Namespace: namespace, Name: g.Spec.GitProviderRef.Name} -} - // GitBranchKey is the GitBranch this GitTarget writes: spec.gitBranchRef, in this GitTarget's own // namespace unless the reference names another one. An unset reference names nothing, and the zero // key says exactly that. func (g *GitTarget) GitBranchKey() types.NamespacedName { - if g.Spec.GitBranchRef == nil { + if g.Spec.GitBranchRef.Name == "" { return types.NamespacedName{} } namespace := g.Spec.GitBranchRef.Namespace diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index d01319ab..0092f6b3 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -1095,21 +1095,6 @@ func (in *GitProvider) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *GitProviderBranchStatus) DeepCopyInto(out *GitProviderBranchStatus) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitProviderBranchStatus. -func (in *GitProviderBranchStatus) DeepCopy() *GitProviderBranchStatus { - if in == nil { - return nil - } - out := new(GitProviderBranchStatus) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitProviderList) DeepCopyInto(out *GitProviderList) { *out = *in @@ -1213,11 +1198,6 @@ func (in *GitProviderStatus) DeepCopyInto(out *GitProviderStatus) { (*in)[i].DeepCopyInto(&(*out)[i]) } } - if in.Branches != nil { - in, out := &in.Branches, &out.Branches - *out = make([]GitProviderBranchStatus, len(*in)) - copy(*out, *in) - } if in.LastVerifiedAt != nil { in, out := &in.LastVerifiedAt, &out.LastVerifiedAt *out = (*in).DeepCopy() @@ -1386,12 +1366,12 @@ func (in *GitTargetRetentionStatus) DeepCopy() *GitTargetRetentionStatus { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = *in - if in.GitBranchRef != nil { - in, out := &in.GitBranchRef, &out.GitBranchRef + out.GitBranchRef = in.GitBranchRef + if in.RemovedGitProviderRef != nil { + in, out := &in.RemovedGitProviderRef, &out.RemovedGitProviderRef *out = new(meta.NamespacedObjectReference) **out = **in } - out.GitProviderRef = in.GitProviderRef if in.Encryption != nil { in, out := &in.Encryption, &out.Encryption *out = new(EncryptionSpec) @@ -1464,11 +1444,6 @@ func (in *GitTargetStatus) DeepCopyInto(out *GitTargetStatus) { *out = new(GitTargetPlacementStatus) (*in).DeepCopyInto(*out) } - if in.Remote != nil { - in, out := &in.Remote, &out.Remote - *out = new(GitBranchRemoteStatus) - (*in).DeepCopyInto(*out) - } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetStatus. diff --git a/config/crd/bases/configbutler.ai_gitproviders.yaml b/config/crd/bases/configbutler.ai_gitproviders.yaml index 9f4a8869..e85f5e17 100644 --- a/config/crd/bases/configbutler.ai_gitproviders.yaml +++ b/config/crd/bases/configbutler.ai_gitproviders.yaml @@ -278,43 +278,6 @@ spec: status: description: status defines the observed state of GitProvider properties: - branches: - description: |- - Branches lists the branches this repository's GitTargets are configured to write, with how - many reference each. It answers what the repository is being used for, so it includes a - branch whose targets are blocked or suspended and excludes a branch nothing references any - more. - - It is not the branches that exist on the remote, and not the workers currently running: it - moves only when somebody edits a GitTarget. An EMPTY list beside Ready=True is the state - this exists to make legible — configured and unused, which looks nothing like broken and - previously took a GitTarget listing to tell apart. An ABSENT list is the third state and a - different one: no inventory has been read yet, so nothing here has been measured. - items: - description: |- - GitProviderBranchStatus is one branch of this repository that GitTargets are configured to - write, and how many of them write it. - properties: - gitTargetCount: - description: |- - GitTargetCount is how many GitTargets are configured to write this branch. More than one means - they share a branch worker, and each still reports its own folder's health on its own - conditions. - format: int32 - minimum: 0 - type: integer - name: - description: Name is the branch. - minLength: 1 - type: string - required: - - gitTargetCount - - name - type: object - type: array - x-kubernetes-list-map-keys: - - name - x-kubernetes-list-type: map conditions: description: |- Conditions report repository validation and commit configuration readiness: diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index b724c713..64fc0574 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -28,10 +28,6 @@ spec: name: Suspended priority: 1 type: boolean - - jsonPath: .status.remote.lastVerifiedAt - name: Verified - priority: 1 - type: date - jsonPath: .status.placement.mode name: Layout priority: 1 @@ -68,12 +64,8 @@ spec: name: Encryption priority: 1 type: string - - jsonPath: .spec.gitProviderRef.name - name: Provider - priority: 1 - type: string - - jsonPath: .spec.branch - name: Branch + - jsonPath: .spec.gitBranchRef.name + name: GitBranch priority: 1 type: string - jsonPath: .spec.path @@ -141,9 +133,8 @@ spec: type: object branch: description: |- - Branch to use for this target. - Must be one of the allowed branches in the provider. - Immutable: delete and recreate the GitTarget to change its destination. + RemovedBranch is the old place of GitBranch.spec.branch. It is never read; setting it is + refused. type: string clusterProviderRef: default: @@ -343,11 +334,8 @@ spec: rule: self.name != '' gitProviderRef: description: |- - GitProviderRef names the GitProvider that backs this target. Namespace omitted means this - GitTarget's own namespace. A GitProvider in another namespace is used only while its - spec.accessFrom admits this GitTarget's namespace. Many GitTargets may name the same - GitProvider, and every GitTarget writing one repository branch must name the same one. - Immutable: delete and recreate the GitTarget to change its destination. + RemovedGitProviderRef is the old place of GitBranch.spec.gitProviderRef. It is never read; + setting it is refused. properties: name: description: Name of the referent. @@ -388,16 +376,8 @@ spec: type: string parentBranch: description: |- - ParentBranch is the branch the write branch (spec.branch) is created from, and the branch the - target compares against while the write branch does not exist on the remote. Omitted: the - branch the remote's HEAD points at (its default branch), or a branch with no history when - the repository is empty. When set, it must exist on the remote for the write branch to be - created; otherwise the GitTarget reports ParentBranchNotFound and writes nothing. Once the - write branch exists, it is used as it is and the parent is not followed. It is only read, so - it does not need to be in the GitProvider's allowedBranches. All GitTargets on one GitProvider - and branch must name the same parent branch (omitted counts as its own value). - Immutable, including adding or removing it: delete and recreate the GitTarget to change it. - minLength: 1 + RemovedParentBranch is the old place of GitBranch.spec.parentBranch. It is never read; + setting it is refused. type: string path: description: |- @@ -558,27 +538,19 @@ spec: resync; the writes suppressed while suspended are not replayed. type: boolean required: + - gitBranchRef - path type: object x-kubernetes-validations: - message: spec.gitBranchRef is immutable; delete and recreate the GitTarget to change its destination - rule: has(self.gitBranchRef) == has(oldSelf.gitBranchRef) && (!has(self.gitBranchRef) - || self.gitBranchRef == oldSelf.gitBranchRef) - - message: spec.gitProviderRef is immutable; delete and recreate the GitTarget - to change its destination - rule: '!has(oldSelf.gitProviderRef) || self.gitProviderRef == oldSelf.gitProviderRef' - - message: spec.branch is immutable; delete and recreate the GitTarget - to change its destination - rule: has(self.branch) == has(oldSelf.branch) && (!has(self.branch) - || self.branch == oldSelf.branch) + rule: self.gitBranchRef == oldSelf.gitBranchRef - message: spec.path is immutable; delete and recreate the GitTarget to change its destination rule: self.path == oldSelf.path - - message: spec.parentBranch is immutable; delete and recreate the GitTarget - to change it - rule: has(self.parentBranch) == has(oldSelf.parentBranch) && (!has(self.parentBranch) - || self.parentBranch == oldSelf.parentBranch) + - message: spec.gitProviderRef, spec.branch and spec.parentBranch moved + to a GitBranch; set spec.gitBranchRef to it + rule: '!has(self.gitProviderRef) && !has(self.branch) && !has(self.parentBranch)' - message: spec.clusterProviderRef is immutable; delete and recreate the GitTarget to change the cluster it mirrors rule: self.clusterProviderRef == oldSelf.clusterProviderRef @@ -722,64 +694,6 @@ spec: and is filled in by the first scan that finds one. type: string type: object - remote: - description: |- - Remote is where this target's branch is on the Git remote, and when that was last proved. - It is renewed by every confirmed look: a successful push as well as a fetch. Absent means - nothing has proved anything yet — either nothing has looked, or the GitProvider was - recreated against a different repository and what was published no longer describes the - one this target points at. - properties: - commit: - description: |- - Commit is the commit hash the branch is at on the remote. EMPTY means the branch is not on - the remote at all, which is not an error: a branch does not exist without a commit, and a - branch nothing has written to has nothing there yet. - type: string - lastVerifiedAt: - description: LastVerifiedAt is when the remote was last observed. - format: date-time - type: string - parent: - description: |- - Parent is what the branch would be created from, observed with it: present only while the - remote does not carry the branch. Once the branch exists its parent is not followed, so this - is omitted. - properties: - branch: - description: |- - Branch is the parent branch: spec.parentBranch, or the branch the remote's HEAD names when - that is omitted. Empty when State is Unborn, and when the remote names no default branch. - type: string - commit: - description: |- - Commit is the commit hash the parent branch is at on the remote: the commit the branch's - first commit would build on. Set only when State is Found. - type: string - state: - description: |- - State is what the branch would be created from: Found (the parent's tip), Missing (the parent - is not on the remote; nothing is written) or Unborn (an empty repository, with no refs at - all, and no parent configured; the first commit has no history). - enum: - - Found - - Missing - - Unborn - type: string - required: - - state - type: object - verifiedBy: - description: |- - VerifiedBy is what proved it: `Push` means the server accepted a ref update of ours, so - this commit is our own work; `Fetch` means we went and looked, and this is what was - there. A `Fetch` next to a commit no publication of yours produced is how a foreign push - to the branch is read off kubectl. - enum: - - Push - - Fetch - type: string - type: object retention: description: |- Retention reports documents a resync kept because this target's spec.prune.mode suppressed diff --git a/config/samples/README.md b/config/samples/README.md index 3871297b..25e66244 100644 --- a/config/samples/README.md +++ b/config/samples/README.md @@ -3,6 +3,8 @@ These samples are quick starting points for common GitOps Reverser setups. - `quickstart-gitprovider.yaml`: Minimal `GitProvider` with credentials. +- `quickstart-gitbranch.yaml`: Minimal `GitBranch`: the `main` branch of `example-provider`'s + repository. - `quickstart-gittarget.yaml`: Minimal `GitTarget` using a non-root `spec.path` and SOPS encryption auto-generation. - `quickstart-watchrule.yaml`: Minimal `WatchRule` for ConfigMaps. - `clusterprovider.yaml`: The conventional `default` provider (in-cluster) and a remote one, showing @@ -14,5 +16,5 @@ These samples are quick starting points for common GitOps Reverser setups. `GitTarget`'s open commit window. It uses `metadata.generateName`, so apply it with `kubectl create -f`: `kubectl apply` refuses a generated name. -Every sample references `example-target` / `example-provider`, the same names the chart's +Every sample references `example-target` / `example-branch` / `example-provider`, the same names the chart's `quickstart` values use, so the set is internally consistent and can be applied together. diff --git a/config/samples/quickstart-gitbranch.yaml b/config/samples/quickstart-gitbranch.yaml new file mode 100644 index 00000000..cc6e524e --- /dev/null +++ b/config/samples/quickstart-gitbranch.yaml @@ -0,0 +1,14 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: example-branch + namespace: default +spec: + # A GitProvider in this GitBranch's namespace. Its credential has to be able to open a push + # session on the repository, or this GitBranch never holds the branch. + gitProviderRef: + name: example-provider + # Must match one of the GitProvider's allowedBranches. Only one GitBranch writes a repository branch. + branch: main + # Optional: the branch `main` is created from while the remote does not have it. + # parentBranch: develop diff --git a/config/samples/quickstart-gittarget.yaml b/config/samples/quickstart-gittarget.yaml index 01537b90..15a59d3f 100644 --- a/config/samples/quickstart-gittarget.yaml +++ b/config/samples/quickstart-gittarget.yaml @@ -4,9 +4,10 @@ metadata: name: example-target namespace: default spec: - gitProviderRef: - name: example-provider - branch: main + # The GitBranch this folder is written to. Add `namespace:` to share one from another namespace, + # which its GitProvider's accessFrom has to admit. + gitBranchRef: + name: example-branch # Required relative repository path. Use "." only to deliberately write at repo root. path: live-cluster # How this folder's writes become commits. A commit window closes after diff --git a/docs/bi-directional-argocd.md b/docs/bi-directional-argocd.md index 2d248301..5c4a13af 100644 --- a/docs/bi-directional-argocd.md +++ b/docs/bi-directional-argocd.md @@ -136,14 +136,23 @@ spec: name: platform-receiver --- apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout + name: platform-main namespace: team-a spec: gitProviderRef: name: platform branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout + namespace: team-a +spec: + gitBranchRef: + name: platform-main path: apps/checkout applyTriggerRef: kind: ArgoCDTrigger diff --git a/docs/bi-directional-flux.md b/docs/bi-directional-flux.md index a9373006..f6e828cd 100644 --- a/docs/bi-directional-flux.md +++ b/docs/bi-directional-flux.md @@ -110,14 +110,23 @@ spec: name: platform-receiver --- apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout + name: platform-main namespace: team-a spec: gitProviderRef: name: platform branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout + namespace: team-a +spec: + gitBranchRef: + name: platform-main path: apps/checkout applyTriggerRef: kind: FluxTrigger diff --git a/docs/configuration.md b/docs/configuration.md index cd264eea..45de4b7b 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -51,7 +51,7 @@ multiple watch rules. ## Field overview -Ready-made manifests for all eight objects live in [`config/samples/`](../config/samples/). They +Ready-made manifests for all nine objects live in [`config/samples/`](../config/samples/). They cross-reference the same `example-provider` / `example-target` names, so the set can be applied together and edited down, rather than assembled from the fragments below. @@ -99,10 +99,7 @@ The destination fields are immutable: to move a target, delete it and create a n | Field | Default | What it does | |---|---|---| -| `gitBranchRef` | `gitProviderRef` and `branch` instead | The `GitBranch` this target writes: its own namespace, or `namespace` names one whose `GitProvider`'s `accessFrom` admits it | -| `gitProviderRef` | `gitBranchRef` instead | The `GitProvider` backing this target: its own namespace, or `namespace` names one whose `accessFrom` admits it. Every target on one repository branch names the same one | -| `branch` | `gitBranchRef` instead | Branch to write. Must be in the provider's `allowedBranches` | -| `parentBranch` | the remote's default branch | Branch a `branch` the remote does not have yet is created from. Immutable. Only read, so it need not be allowed. See [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch) | +| `gitBranchRef` | **required** | The `GitBranch` this target writes: its own namespace, or `namespace` names one whose `GitProvider`'s `accessFrom` admits it | | `path` | **required** | Folder within the repository. `.` targets the root; empty is rejected | | `clusterProviderRef` | `{"name":"default"}` | Source cluster to mirror from. The default names a `ClusterProvider` called `default` | | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index a772e139..d7c49d9d 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -17,7 +17,7 @@ preamble: | ## Field overview - Ready-made manifests for all eight objects live in [`config/samples/`](../config/samples/). They + Ready-made manifests for all nine objects live in [`config/samples/`](../config/samples/). They cross-reference the same `example-provider` / `example-target` names, so the set can be applied together and edited down, rather than assembled from the fragments below. @@ -123,24 +123,15 @@ kinds: intro: "The destination fields are immutable: to move a target, delete it and create a new one." fields: - path: gitBranchRef - default: "`gitProviderRef` and `branch` instead" doc: >- The `GitBranch` this target writes: its own namespace, or `namespace` names one whose `GitProvider`'s `accessFrom` admits it - path: gitProviderRef - default: "`gitBranchRef` instead" - doc: >- - The `GitProvider` backing this target: its own namespace, or `namespace` names one whose - `accessFrom` admits it. Every target on one repository branch names the same one + skip: moved to `GitBranch`, kept in the schema only so admission refuses it, naming `gitBranchRef` - path: branch - default: "`gitBranchRef` instead" - doc: Branch to write. Must be in the provider's `allowedBranches` + skip: moved to `GitBranch`, kept in the schema only so admission refuses it, naming `gitBranchRef` - path: parentBranch - default: the remote's default branch - doc: >- - Branch a `branch` the remote does not have yet is created from. Immutable. Only read, so - it need not be allowed. See - [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch) + skip: moved to `GitBranch`, kept in the schema only so admission refuses it, naming `gitBranchRef` - path: path doc: Folder within the repository. `.` targets the root; empty is rejected - path: clusterProviderRef diff --git a/internal/applier/argocd_guide_test.go b/internal/applier/argocd_guide_test.go index db6396ee..980d2679 100644 --- a/internal/applier/argocd_guide_test.go +++ b/internal/applier/argocd_guide_test.go @@ -29,10 +29,12 @@ func TestArgoCDGuide_TheCompleteExampleIsAccepted(t *testing.T) { objects := guideExampleObjects(t, argoCDGuide) var provider v1alpha3.GitProvider + var branch v1alpha3.GitBranch var target v1alpha3.GitTarget var trigger v1alpha3.ArgoCDTrigger var secret corev1.Secret decodeStrict(t, objects, "GitProvider", &provider) + decodeStrict(t, objects, "GitBranch", &branch) decodeStrict(t, objects, "GitTarget", &target) decodeStrict(t, objects, v1alpha3.ApplyTriggerKindArgoCD, &trigger) decodeStrict(t, objects, "Secret", &secret) @@ -40,8 +42,15 @@ func TestArgoCDGuide_TheCompleteExampleIsAccepted(t *testing.T) { app := one(t, objects, kindApplication) // The GitOps Reverser objects are wired to each other. - assert.Equal(t, provider.Name, target.Spec.GitProviderRef.Name) - assert.Equal(t, provider.Namespace, target.Namespace, "a GitTarget names a GitProvider in its own namespace") + assert.Equal(t, provider.Name, branch.Spec.GitProviderRef.Name) + assert.Equal(t, provider.Namespace, branch.Namespace, "a GitBranch names a GitProvider in its own namespace") + assert.Equal(t, branch.Name, target.Spec.GitBranchRef.Name) + assert.Equal( + t, + branch.Namespace, + target.Namespace, + "the example's GitTarget names a GitBranch in its own namespace", + ) require.NotNil(t, provider.Spec.Receiver, "the example serves a push receiver") assert.Equal(t, secret.Name, provider.Spec.Receiver.SecretRef.Name) assert.NotEmpty(t, secret.StringData[v1alpha3.ReceiverTokenKey], "the receiver reads the token key") @@ -60,7 +69,7 @@ func TestArgoCDGuide_TheCompleteExampleIsAccepted(t *testing.T) { // The applier, as written, is accepted, binds the GitTarget, and is driven. ref := trigger.Applier() assert.Equal(t, types.NamespacedName{Namespace: app.GetNamespace(), Name: app.GetName()}, ref) - want := Binding{RepositoryURL: provider.Spec.URL, Branch: target.Spec.Branch} + want := Binding{RepositoryURL: provider.Spec.URL, Branch: branch.Spec.Branch} client := newSourceClient(app) argo := pinnedArgo(client) verdict := argo.Validate(context.Background(), ref) diff --git a/internal/applier/dispatcher_test.go b/internal/applier/dispatcher_test.go index ec5d6506..016fd042 100644 --- a/internal/applier/dispatcher_test.go +++ b/internal/applier/dispatcher_test.go @@ -62,7 +62,7 @@ func gitTarget(name string, kustomizationRef *fluxmeta.NamespacedObjectReference target := &v1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: targetNS, Name: name, UID: uid(name)}, Spec: v1alpha3.GitTargetSpec{ - GitBranchRef: &fluxmeta.NamespacedObjectReference{Name: testGitBranch.Name}, + GitBranchRef: fluxmeta.NamespacedObjectReference{Name: testGitBranch.Name}, Path: name, }, } diff --git a/internal/applier/flux_guide_test.go b/internal/applier/flux_guide_test.go index 919d8f9c..7978a111 100644 --- a/internal/applier/flux_guide_test.go +++ b/internal/applier/flux_guide_test.go @@ -43,10 +43,12 @@ func TestFluxGuide_TheCompleteExampleIsAccepted(t *testing.T) { objects := guideExampleObjects(t, fluxGuide) var provider v1alpha3.GitProvider + var branch v1alpha3.GitBranch var target v1alpha3.GitTarget var trigger v1alpha3.FluxTrigger var secret corev1.Secret decodeStrict(t, objects, "GitProvider", &provider) + decodeStrict(t, objects, "GitBranch", &branch) decodeStrict(t, objects, "GitTarget", &target) decodeStrict(t, objects, v1alpha3.ApplyTriggerKindFlux, &trigger) decodeStrict(t, objects, "Secret", &secret) @@ -56,8 +58,15 @@ func TestFluxGuide_TheCompleteExampleIsAccepted(t *testing.T) { assert.NotContains(t, objects, "Receiver", "the guide tells the reader to remove the Flux Receiver") // The GitOps Reverser objects are wired to each other. - assert.Equal(t, provider.Name, target.Spec.GitProviderRef.Name) - assert.Equal(t, provider.Namespace, target.Namespace, "a GitTarget names a GitProvider in its own namespace") + assert.Equal(t, provider.Name, branch.Spec.GitProviderRef.Name) + assert.Equal(t, provider.Namespace, branch.Namespace, "a GitBranch names a GitProvider in its own namespace") + assert.Equal(t, branch.Name, target.Spec.GitBranchRef.Name) + assert.Equal( + t, + branch.Namespace, + target.Namespace, + "the example's GitTarget names a GitBranch in its own namespace", + ) require.NotNil(t, provider.Spec.Receiver, "the example serves a push receiver") assert.Equal(t, secret.Name, provider.Spec.Receiver.SecretRef.Name) assert.Equal(t, provider.Namespace, secret.Namespace) @@ -80,7 +89,7 @@ func TestFluxGuide_TheCompleteExampleIsAccepted(t *testing.T) { // The applier, as written, is accepted, binds the GitTarget, and is driven. ref := trigger.Applier() assert.Equal(t, types.NamespacedName{Namespace: kustomization.GetNamespace(), Name: kustomization.GetName()}, ref) - want := Binding{RepositoryURL: provider.Spec.URL, Branch: target.Spec.Branch} + want := Binding{RepositoryURL: provider.Spec.URL, Branch: branch.Spec.Branch} client := newSourceClient(kustomization, repository) flux := pinnedFlux(client) @@ -93,7 +102,7 @@ func TestFluxGuide_TheCompleteExampleIsAccepted(t *testing.T) { sim := fluxsim.New(t, client) sim.Fetches(types.NamespacedName{Namespace: repository.GetNamespace(), Name: repository.GetName()}, - target.Spec.Branch+"@sha1:"+testCommit) + branch.Spec.Branch+"@sha1:"+testCommit) flux.Now = fluxsim.TickingClock(stampTime) started, err := flux.Request(context.Background(), ref, Ask{ Want: want, Commit: testCommit, Reason: v1alpha3.ApplyReasonForeignCommit, Token: "T1", diff --git a/internal/authz/gitbranch_admission_test.go b/internal/authz/gitbranch_admission_test.go index 76da3a04..b576e235 100644 --- a/internal/authz/gitbranch_admission_test.go +++ b/internal/authz/gitbranch_admission_test.go @@ -38,7 +38,7 @@ func sharedGitBranch(provider string) *configv1alpha3.GitBranch { // namespace means its own. func targetWriting(namespace, name string) *configv1alpha3.GitTarget { target := targetIn("") - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: name, Namespace: namespace} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: name, Namespace: namespace} return target } diff --git a/internal/authz/source_namespace_test.go b/internal/authz/source_namespace_test.go index 135a70c2..2e90ee77 100644 --- a/internal/authz/source_namespace_test.go +++ b/internal/authz/source_namespace_test.go @@ -44,7 +44,7 @@ func snTarget() *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: snTargetName, Namespace: snTenantNS}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "acme-git-main"}, + GitBranchRef: meta.NamespacedObjectReference{Name: "acme-git-main"}, ClusterProviderRef: &meta.LocalObjectReference{Name: snProvider}, Path: "tenants/acme", }, diff --git a/internal/controller/applytrigger_controller_test.go b/internal/controller/applytrigger_controller_test.go index db0f5bd5..2d4d8612 100644 --- a/internal/controller/applytrigger_controller_test.go +++ b/internal/controller/applytrigger_controller_test.go @@ -137,7 +137,7 @@ func referencingTarget(name, kind string) *configbutleraiv1alpha3.GitTarget { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "team-a"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("platform", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("platform", "main")}, Path: name, ApplyTriggerRef: &configbutleraiv1alpha3.ApplyTriggerReference{Kind: kind, Name: "checkout"}, }, diff --git a/internal/controller/applytrigger_schema_test.go b/internal/controller/applytrigger_schema_test.go index fdf06633..bfaaf96d 100644 --- a/internal/controller/applytrigger_schema_test.go +++ b/internal/controller/applytrigger_schema_test.go @@ -117,7 +117,7 @@ var _ = Describe("Apply trigger schema", func() { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov", "main")}, Path: "apps/" + name, ApplyTriggerRef: ref, }, @@ -171,7 +171,7 @@ var _ = Describe("Apply trigger reconcile", func() { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "names-reconciled", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("prov", "main"), }, Path: "apps/reconciled", diff --git a/internal/controller/clusterwatchrule_admission_test.go b/internal/controller/clusterwatchrule_admission_test.go index cdf5794f..5b8fce10 100644 --- a/internal/controller/clusterwatchrule_admission_test.go +++ b/internal/controller/clusterwatchrule_admission_test.go @@ -112,7 +112,7 @@ func cwaGitTarget() *configbutleraiv1alpha3.GitTarget { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: cwaTargetName, Namespace: cwaTargetNS}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: cwaProviderName}, Path: "clusters/prod", }, diff --git a/internal/controller/commitrequest_outcome_metric_test.go b/internal/controller/commitrequest_outcome_metric_test.go index 52e7c666..9a722eb7 100644 --- a/internal/controller/commitrequest_outcome_metric_test.go +++ b/internal/controller/commitrequest_outcome_metric_test.go @@ -356,7 +356,7 @@ func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } fresh := newCommitRequest("save-fresh") diff --git a/internal/controller/dependency_watches_test.go b/internal/controller/dependency_watches_test.go index 283242a8..780ea0cd 100644 --- a/internal/controller/dependency_watches_test.go +++ b/internal/controller/dependency_watches_test.go @@ -66,7 +66,7 @@ func gitTarget(name, namespace, providerName string) *configbutleraiv1alpha3.Git return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest(providerName, "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest(providerName, "main")}, }, } } diff --git a/internal/controller/duration_fields_admission_test.go b/internal/controller/duration_fields_admission_test.go index 6c0d96ec..cc637e4d 100644 --- a/internal/controller/duration_fields_admission_test.go +++ b/internal/controller/duration_fields_admission_test.go @@ -29,10 +29,9 @@ var _ = Describe("Duration fields", func() { // newTargetWindow builds a GitTarget with the given commit.window block. newTargetWindow := func(name string, window map[string]any) *unstructured.Unstructured { spec := map[string]any{ - "gitProviderRef": map[string]any{"name": "any-provider"}, - "branch": "main", - "path": "clusters/prod", - "commit": map[string]any{"window": window}, + "gitBranchRef": map[string]any{"name": "any-provider-main"}, + "path": "clusters/prod", + "commit": map[string]any{"window": window}, } return &unstructured.Unstructured{Object: map[string]any{ "apiVersion": "configbutler.ai/v1alpha3", @@ -143,10 +142,9 @@ var _ = Describe("Duration fields", func() { "kind": "GitTarget", "metadata": map[string]any{"name": "window-empty-block", "namespace": "default"}, "spec": map[string]any{ - "gitProviderRef": map[string]any{"name": "any-provider"}, - "branch": "main", - "path": "clusters/prod", - "commit": map[string]any{"window": map[string]any{}}, + "gitBranchRef": map[string]any{"name": "any-provider-main"}, + "path": "clusters/prod", + "commit": map[string]any{"window": map[string]any{}}, }, }} Expect(k8sClient.Create(ctx, empty)).To(Succeed()) @@ -171,10 +169,9 @@ var _ = Describe("Duration fields", func() { "kind": "GitTarget", "metadata": map[string]any{"name": "window-legacy-string", "namespace": "default"}, "spec": map[string]any{ - "gitProviderRef": map[string]any{"name": "any-provider"}, - "branch": "main", - "path": "clusters/prod", - "commit": map[string]any{"window": "5s"}, + "gitBranchRef": map[string]any{"name": "any-provider-main"}, + "path": "clusters/prod", + "commit": map[string]any{"window": "5s"}, }, }} err := k8sClient.Create(ctx, legacy) diff --git a/internal/controller/gitbranch_controller_test.go b/internal/controller/gitbranch_controller_test.go index efd9bd26..cc2d6765 100644 --- a/internal/controller/gitbranch_controller_test.go +++ b/internal/controller/gitbranch_controller_test.go @@ -109,7 +109,7 @@ func TestGitBranch_AProvenGitBranchHoldsItsBranch(t *testing.T) { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: namespace, Name: name}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, Path: name, + GitBranchRef: meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, Path: name, }, } } @@ -119,7 +119,7 @@ func TestGitBranch_AProvenGitBranchHoldsItsBranch(t *testing.T) { &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "other"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "main"}, Path: "other", + GitBranchRef: meta.NamespacedObjectReference{Name: "main"}, Path: "other", }, }) diff --git a/internal/controller/gittarget_apply_trigger_test.go b/internal/controller/gittarget_apply_trigger_test.go index 087fb548..e8519a94 100644 --- a/internal/controller/gittarget_apply_trigger_test.go +++ b/internal/controller/gittarget_apply_trigger_test.go @@ -25,7 +25,7 @@ func triggerTarget(named bool) *configbutleraiv1alpha3.GitTarget { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "checkout", Namespace: "team-a"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("platform", "main")}, Path: "apps", + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("platform", "main")}, Path: "apps", }, } if named { diff --git a/internal/controller/gittarget_controller_test.go b/internal/controller/gittarget_controller_test.go index f654098d..f1b5fb3e 100644 --- a/internal/controller/gittarget_controller_test.go +++ b/internal/controller/gittarget_controller_test.go @@ -35,7 +35,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "missing-branch"}, + GitBranchRef: meta.NamespacedObjectReference{Name: "missing-branch"}, Path: "test-folder", }, } @@ -85,7 +85,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-two-axis", "main"), }, Path: "two-axis-folder", @@ -145,7 +145,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-conflict", "main"), }, Path: "conflict-folder", @@ -179,7 +179,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-conflict", "main"), }, Path: "conflict-folder", @@ -263,7 +263,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-no-conflict", "main"), }, Path: "folder-a", @@ -278,7 +278,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-no-conflict", "main"), }, Path: "folder-b", // Different! @@ -350,7 +350,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-nested", "main"), }, Path: "team", @@ -383,7 +383,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-nested", "main"), }, Path: "team/app", @@ -463,7 +463,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-generate-enc-secret", "main"), }, Path: "test-path", @@ -528,7 +528,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-no-generate-enc-secret", "main"), }, Path: "test-path", @@ -605,7 +605,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-no-enc-recipients", "main"), }, Path: "test-path", @@ -698,7 +698,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-update-enc-secret", "main"), }, Path: "test-path", @@ -758,7 +758,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-recreate-enc-secret", "main"), }, Path: "test-path", @@ -853,7 +853,7 @@ var _ = Describe("GitTarget Controller Security", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("test-provider-existing-agekey", "main"), }, Path: "test-path", diff --git a/internal/controller/gittarget_controller_unit_test.go b/internal/controller/gittarget_controller_unit_test.go index 4f00b2b7..dff93569 100644 --- a/internal/controller/gittarget_controller_unit_test.go +++ b/internal/controller/gittarget_controller_unit_test.go @@ -94,7 +94,7 @@ func TestCheckForConflicts_ListErrorFailsClosed(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "target-a", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("provider-a", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("provider-a", "main")}, Path: "apps", }, } diff --git a/internal/controller/gittarget_gitbranch_test.go b/internal/controller/gittarget_gitbranch_test.go index 421bcc99..854d95ac 100644 --- a/internal/controller/gittarget_gitbranch_test.go +++ b/internal/controller/gittarget_gitbranch_test.go @@ -48,7 +48,7 @@ func newDestinationFixture(t *testing.T) *destinationFixture { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, + GitBranchRef: meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, Path: "apps", }, } @@ -157,11 +157,11 @@ func TestGitBranchWatches_MapToWhatTheyChange(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, + GitBranchRef: meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, }, } assert.Equal(t, []string{"platform/main"}, requestsToKeys(gitTargetToGitBranch(context.Background(), target))) - target.Spec.GitBranchRef = nil + target.Spec.GitBranchRef = meta.NamespacedObjectReference{} assert.Empty(t, gitTargetToGitBranch(context.Background(), target)) assert.Empty(t, gitTargetToGitBranch(context.Background(), branchProvider("platform", "acme"))) } @@ -200,7 +200,7 @@ func TestGitTargetsOn_FindsTheTargetsOfABranchOrItsProvider(t *testing.T) { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: namespace, Name: name}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: branch, Namespace: branchNS}, + GitBranchRef: meta.NamespacedObjectReference{Name: branch, Namespace: branchNS}, }, } } diff --git a/internal/controller/gittarget_immutability_test.go b/internal/controller/gittarget_immutability_test.go index 49ca194e..dbe73568 100644 --- a/internal/controller/gittarget_immutability_test.go +++ b/internal/controller/gittarget_immutability_test.go @@ -48,7 +48,7 @@ var _ = Describe("GitTarget Destination Immutability", func() { gitTarget := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: key.Name, Namespace: key.Namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov-a", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov-a", "main")}, Path: "apps", }, } @@ -124,7 +124,7 @@ var _ = Describe("GitTarget Destination Immutability", func() { base := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: key.Name, Namespace: key.Namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov-a", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("prov-a", "main")}, }, } diff --git a/internal/controller/gittarget_placement_validation_test.go b/internal/controller/gittarget_placement_validation_test.go index 0dd986a4..e15a09a6 100644 --- a/internal/controller/gittarget_placement_validation_test.go +++ b/internal/controller/gittarget_placement_validation_test.go @@ -268,7 +268,7 @@ func TestEvaluateValidatedGate_InvalidPlacementPolicy(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "target-a", Namespace: ns}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("provider-a", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("provider-a", "main")}, Path: "apps", Placement: &configbutleraiv1alpha3.GitTargetPlacementSpec{ Default: "{bogus}/all.yaml", diff --git a/internal/controller/gittarget_shared_branch_test.go b/internal/controller/gittarget_shared_branch_test.go index 708b0772..2ed65824 100644 --- a/internal/controller/gittarget_shared_branch_test.go +++ b/internal/controller/gittarget_shared_branch_test.go @@ -31,7 +31,7 @@ func sharedProviderTarget( CreationTimestamp: metav1.NewTime(time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC).Add(-age)), }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: gitBranchNameForTest("monorepo", "main"), Namespace: providerNS, }, diff --git a/internal/controller/gittarget_source_cluster_test.go b/internal/controller/gittarget_source_cluster_test.go index 980ec939..529b6cea 100644 --- a/internal/controller/gittarget_source_cluster_test.go +++ b/internal/controller/gittarget_source_cluster_test.go @@ -335,7 +335,7 @@ func TestReconcile_UnauthorizedNamespaceStartsNoWatch(t *testing.T) { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "gt", Namespace: ns, UID: "gt-uid"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("gp", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("gp", "main")}, Path: "apps", ClusterProviderRef: &meta.LocalObjectReference{Name: providerName}, }, diff --git a/internal/controller/gittarget_worker_cleanup_test.go b/internal/controller/gittarget_worker_cleanup_test.go index 48c44dba..cac82242 100644 --- a/internal/controller/gittarget_worker_cleanup_test.go +++ b/internal/controller/gittarget_worker_cleanup_test.go @@ -75,7 +75,7 @@ func TestCleanupDeletedGitTarget_LeavesAWorkerAnotherGitTargetStillUses(t *testi sibling := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "invoices", Namespace: "shop"}, } - sibling.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")} + sibling.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")} k8sClient := fake.NewClientBuilder().WithScheme(scheme).WithObjects(sibling).Build() workers := git.NewWorkerManager(k8sClient, logr.Discard(), git.BranchWorkerLimits{}, diff --git a/internal/controller/gittarget_worker_repoint_test.go b/internal/controller/gittarget_worker_repoint_test.go index 52840632..78580c85 100644 --- a/internal/controller/gittarget_worker_repoint_test.go +++ b/internal/controller/gittarget_worker_repoint_test.go @@ -52,7 +52,7 @@ func TestEnsureEventStream_RepointsTheStreamAtTheReplacementWorker(t *testing.T) target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "apps", Namespace: "shop"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")}, Path: "apps", }, } @@ -112,7 +112,7 @@ func TestRecoverBranchAfterReplacement_ReachesEverySiblingOnTheBranch(t *testing return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "shop", UID: k8stypes.UID("uid-" + name)}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest(provider, branch)}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest(provider, branch)}, Path: name, }, } @@ -174,7 +174,7 @@ func TestEnsureEventStream_ARecoveryThatCouldNotBeDeliveredIsRetried(t *testing. target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "apps", Namespace: "shop", UID: k8stypes.UID("uid-apps")}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("repo1", "main")}, Path: "apps", }, } diff --git a/internal/controller/live_message_migration_test.go b/internal/controller/live_message_migration_test.go index 9d2b36dc..81c03905 100644 --- a/internal/controller/live_message_migration_test.go +++ b/internal/controller/live_message_migration_test.go @@ -43,9 +43,8 @@ func legacyGitTarget(name string) *unstructured.Unstructured { "apiVersion": "configbutler.ai/v1alpha3", "kind": "GitTarget", "metadata": map[string]any{"name": name, "namespace": "default"}, "spec": map[string]any{ - "gitProviderRef": map[string]any{"name": "provider"}, - "branch": "main", - "path": name, + "gitBranchRef": map[string]any{"name": "provider-main"}, + "path": name, "commit": map[string]any{ "message": map[string]any{"eventTemplate": "old event", "groupTemplate": "old group"}, }, diff --git a/internal/controller/superseded_fields_admission_test.go b/internal/controller/superseded_fields_admission_test.go index 669657e8..5fdc7074 100644 --- a/internal/controller/superseded_fields_admission_test.go +++ b/internal/controller/superseded_fields_admission_test.go @@ -87,7 +87,7 @@ var _ = Describe("Superseded source-scope fields", func() { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "renamed-commit", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("any-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("any-provider", "main")}, Path: "clusters/prod", Commit: &configbutleraiv1alpha3.GitTargetCommitSpec{ Window: &configbutleraiv1alpha3.CommitWindow{ @@ -202,7 +202,7 @@ var _ = Describe("Reference names must not be empty", func() { target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "empty-branch-name", Namespace: "default"}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: ""}, + GitBranchRef: meta.NamespacedObjectReference{Name: ""}, Path: "clusters/prod", }, } @@ -268,3 +268,44 @@ var _ = Describe("Retained and refused throttle fields", func() { DeferCleanup(func() { _ = k8sClient.Delete(ctx, provider) }) }) }) + +// GitTarget.spec.gitProviderRef, spec.branch and spec.parentBranch moved to a GitBranch. A pruned +// field would apply cleanly and leave the GitTarget naming no branch, so the old spellings stay in +// the schema and a CEL rule refuses them, naming gitBranchRef. +var _ = Describe("Retained and refused GitTarget branch fields", func() { + legacyTarget := func(name string, spec map[string]any) *unstructured.Unstructured { + spec["gitBranchRef"] = map[string]any{"name": "main"} + spec["path"] = "apps" + return &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "configbutler.ai/v1alpha3", + "kind": "GitTarget", + "metadata": map[string]any{"name": name, "namespace": "default"}, + "spec": spec, + }} + } + + It("refuses spec.gitProviderRef, spec.branch and spec.parentBranch, naming spec.gitBranchRef", func() { + ctx := context.Background() + + for name, spec := range map[string]map[string]any{ + "legacy-provider": {"gitProviderRef": map[string]any{"name": "platform"}}, + "legacy-branch": {"branch": "main"}, + "legacy-parent": {"parentBranch": "develop"}, + } { + err := k8sClient.Create(ctx, legacyTarget(name, spec)) + Expect(err).To(HaveOccurred(), "%s: the old field must not apply cleanly", name) + Expect(err.Error()).To(ContainSubstring("moved to a GitBranch; set spec.gitBranchRef"), name) + } + }) + + It("requires spec.gitBranchRef", func() { + ctx := context.Background() + + target := legacyTarget("no-branch-ref", map[string]any{}) + Expect(unstructured.SetNestedField(target.Object, nil, "spec", "gitBranchRef")).To(Succeed()) + delete(target.Object["spec"].(map[string]any), "gitBranchRef") + err := k8sClient.Create(ctx, target) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("gitBranchRef")) + }) +}) diff --git a/internal/controller/watchrule_controller_test.go b/internal/controller/watchrule_controller_test.go index 2be6b478..6db8678c 100644 --- a/internal/controller/watchrule_controller_test.go +++ b/internal/controller/watchrule_controller_test.go @@ -79,7 +79,7 @@ var _ = Describe("WatchRule Controller", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("test-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("test-provider", "main")}, Path: "default/test", }, } @@ -192,7 +192,7 @@ var _ = Describe("WatchRule Controller", func() { Namespace: "default", }, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("local-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("local-provider", "main")}, Path: "ns/default", }, } diff --git a/internal/controller/watchrule_source_namespace_test.go b/internal/controller/watchrule_source_namespace_test.go index 7863e0f7..df88b903 100644 --- a/internal/controller/watchrule_source_namespace_test.go +++ b/internal/controller/watchrule_source_namespace_test.go @@ -36,7 +36,7 @@ func wrsnGitTarget() *configbutleraiv1alpha3.GitTarget { return &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: wrsnTarget, Namespace: wrsnTenantNS}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: wrsnProvider}, Path: "tenants/acme", }, diff --git a/internal/controller/wiring/readiness_wiring_test.go b/internal/controller/wiring/readiness_wiring_test.go index 6230be3f..0df215eb 100644 --- a/internal/controller/wiring/readiness_wiring_test.go +++ b/internal/controller/wiring/readiness_wiring_test.go @@ -130,7 +130,7 @@ func seedTarget(ctx context.Context, t *testing.T, name, clusterProvider string) target := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "target", Namespace: namespace}, Spec: configbutleraiv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "provider-main"}, + GitBranchRef: meta.NamespacedObjectReference{Name: "provider-main"}, Path: "mirror", }, } diff --git a/internal/git/apply_trigger_test.go b/internal/git/apply_trigger_test.go index 802c3d1f..b786eb2b 100644 --- a/internal/git/apply_trigger_test.go +++ b/internal/git/apply_trigger_test.go @@ -109,7 +109,7 @@ func applierTestTarget(name string, applier bool, mutate ...func(*configv1alpha3 target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "team-a", UID: k8stypes.UID("uid-" + name)}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")}, Path: "apps/" + name, }, } diff --git a/internal/git/branch_worker_loop_test.go b/internal/git/branch_worker_loop_test.go index 50f19665..e7b5346e 100644 --- a/internal/git/branch_worker_loop_test.go +++ b/internal/git/branch_worker_loop_test.go @@ -29,7 +29,7 @@ func TestCommitWindowFor_DefaultsAndParsing(t *testing.T) { d := func(v time.Duration) *metav1.Duration { return &metav1.Duration{Duration: v} } target := func(name string, window *configv1alpha3.CommitWindow) *configv1alpha3.GitTarget { spec := configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName("p", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: testGitBranchName("p", "main")}, Path: "clusters/prod", } if window != nil { diff --git a/internal/git/branch_worker_split_test.go b/internal/git/branch_worker_split_test.go index 3f57e38a..45061b9f 100644 --- a/internal/git/branch_worker_split_test.go +++ b/internal/git/branch_worker_split_test.go @@ -97,7 +97,7 @@ func createGitTarget( Namespace: "default", }, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: testGitBranchName(worker.GitProviderRef, worker.Branch), }, Path: path, diff --git a/internal/git/branch_worker_test.go b/internal/git/branch_worker_test.go index 749dc1a8..c8cc7a01 100644 --- a/internal/git/branch_worker_test.go +++ b/internal/git/branch_worker_test.go @@ -513,7 +513,7 @@ func TestBranchWorker_EnsurePathBootstrapped_RendersAllResolvedRecipients(t *tes target := &configv1alpha3.GitTarget{} target.Name = "bootstrap-target" target.Namespace = "default" - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} target.Spec.Path = "clusters/dev" target.Spec.Encryption = &configv1alpha3.EncryptionSpec{ Provider: "sops", @@ -830,7 +830,7 @@ func TestBranchWorker_CommitAndPushRequest_UsesProviderCommitterAndTargetMessage target := &configv1alpha3.GitTarget{} target.Name = "audit-target" target.Namespace = "default" - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} target.Spec.Path = "clusters/dev" target.Spec.Commit = &configv1alpha3.GitTargetCommitSpec{ Message: &configv1alpha3.CommitMessageSpec{ @@ -928,7 +928,7 @@ func TestBranchWorker_CommitAndPushRequest_UsesBatchTemplateForAtomicRequest(t * target := &configv1alpha3.GitTarget{} target.Name = "demo-target" target.Namespace = "default" - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName("test-repo", "main")} target.Spec.Path = "clusters/dev" target.Spec.Commit = &configv1alpha3.GitTargetCommitSpec{ Message: &configv1alpha3.CommitMessageSpec{ @@ -1243,7 +1243,7 @@ func createTargetWithEncryption( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = namespace - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path target.Spec.Encryption = &configv1alpha3.EncryptionSpec{ Provider: "sops", @@ -1270,7 +1270,7 @@ func createTargetWithoutEncryption( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = namespace - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path require.NoError(t, k8sClient.Create(ctx, target)) } @@ -1293,7 +1293,7 @@ func createTargetWithEncryptionSecretData( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = namespace - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path target.Spec.Encryption = &configv1alpha3.EncryptionSpec{ Provider: "sops", diff --git a/internal/git/git_branch_ref_test.go b/internal/git/git_branch_ref_test.go index fe228df8..d099c11d 100644 --- a/internal/git/git_branch_ref_test.go +++ b/internal/git/git_branch_ref_test.go @@ -21,7 +21,7 @@ import ( func refTarget(namespace, gitBranch string) *configv1alpha3.GitTarget { target := &configv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}} if gitBranch != "" { - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: gitBranch, Namespace: namespace} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: gitBranch, Namespace: namespace} } return target } diff --git a/internal/git/git_roundtrip_ledger_test.go b/internal/git/git_roundtrip_ledger_test.go index 5fa46371..7dc2023b 100644 --- a/internal/git/git_roundtrip_ledger_test.go +++ b/internal/git/git_roundtrip_ledger_test.go @@ -175,7 +175,7 @@ func (f *ledgerFixture) createLedgerTarget(path string, prune *configv1alpha3.Pr require.NoError(f.t, f.worker.Client.Create(f.worker.ctx, &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: ledgerTargetName, Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{ + GitBranchRef: meta.NamespacedObjectReference{ Name: testGitBranchName(f.worker.GitProviderRef, f.worker.Branch), }, Path: path, diff --git a/internal/git/publication_test.go b/internal/git/publication_test.go index d8adb0ec..ceede003 100644 --- a/internal/git/publication_test.go +++ b/internal/git/publication_test.go @@ -138,7 +138,7 @@ func TestPublication_ChangesReachEveryGitTargetOnTheBranch(t *testing.T) { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranch, Namespace: "shop"}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranch, Namespace: "shop"}, }, } } diff --git a/internal/git/refusal_reapply_test.go b/internal/git/refusal_reapply_test.go index b5667974..01d939ac 100644 --- a/internal/git/refusal_reapply_test.go +++ b/internal/git/refusal_reapply_test.go @@ -44,7 +44,7 @@ func refusalTouchWorker(t *testing.T, spec configv1alpha3.GitTargetSpec) *Branch require.NoError(t, clientgoscheme.AddToScheme(scheme)) require.NoError(t, configv1alpha3.AddToScheme(scheme)) - spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")} + spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")} if spec.Path == "" { spec.Path = "apps" } @@ -412,7 +412,7 @@ func TestRefusalReapply_OneTargetDoesNotDropAnothersPendingRequest(t *testing.T) Suspend: suspend, }, } - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName("test-provider", "main")} return target } // B is suspended, so its delayed request must be declined when the timer fires. A is not. diff --git a/internal/git/secret_write_test.go b/internal/git/secret_write_test.go index ac2fec2c..fe4d15e6 100644 --- a/internal/git/secret_write_test.go +++ b/internal/git/secret_write_test.go @@ -68,7 +68,7 @@ func secretTargetObjects(t *testing.T, providerName, branch, path string) []clie Namespace: "default", }, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)}, + GitBranchRef: meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)}, Path: path, Encryption: &configv1alpha3.EncryptionSpec{ Provider: "sops", diff --git a/internal/git/types_test.go b/internal/git/types_test.go index b9244aa0..402d799a 100644 --- a/internal/git/types_test.go +++ b/internal/git/types_test.go @@ -35,7 +35,7 @@ func TestWritesThrough_IsTheGitBranch(t *testing.T) { target := &v1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: "team-a", Name: "apps"}, Spec: v1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, + GitBranchRef: meta.NamespacedObjectReference{Name: "main", Namespace: "platform"}, }, } assert.True(t, WritesThrough(target, shared)) @@ -45,7 +45,7 @@ func TestWritesThrough_IsTheGitBranch(t *testing.T) { target.Spec.GitBranchRef.Namespace = "" assert.True(t, WritesThrough(target, k8stypes.NamespacedName{Namespace: "team-a", Name: "main"}), "no namespace means the GitTarget's own") - target.Spec.GitBranchRef = nil + target.Spec.GitBranchRef = meta.NamespacedObjectReference{} assert.False(t, WritesThrough(target, k8stypes.NamespacedName{}), "no reference writes through nothing") } diff --git a/internal/git/worker_branch_owner_test.go b/internal/git/worker_branch_owner_test.go index af13ebde..5a2393db 100644 --- a/internal/git/worker_branch_owner_test.go +++ b/internal/git/worker_branch_owner_test.go @@ -55,7 +55,7 @@ func TestReconcileWorkers_StopsTheWorkerOfAGitBranchThatLostItsBranch(t *testing require.NoError(t, m.Client.Create(ctx, &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: gitBranch.Namespace, Name: "apps"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranch.Name}, Path: gitBranch.Namespace, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranch.Name}, Path: gitBranch.Namespace, }, })) } diff --git a/internal/git/worker_manager_test.go b/internal/git/worker_manager_test.go index 83b1e951..62597e95 100644 --- a/internal/git/worker_manager_test.go +++ b/internal/git/worker_manager_test.go @@ -117,7 +117,7 @@ func createTargetForRegister( target := &configv1alpha3.GitTarget{} target.Name = name target.Namespace = testTargetNamespace - target.Spec.GitBranchRef = &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} + target.Spec.GitBranchRef = meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)} target.Spec.Path = path require.NoError(t, k8sClient.Create(ctx, target)) } diff --git a/internal/git/write_boundary_precondition_test.go b/internal/git/write_boundary_precondition_test.go index bd01cc58..201c7982 100644 --- a/internal/git/write_boundary_precondition_test.go +++ b/internal/git/write_boundary_precondition_test.go @@ -203,7 +203,7 @@ func diamondGitTarget(providerName, branch string) *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "podinfo-test", Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)}, + GitBranchRef: meta.NamespacedObjectReference{Name: testGitBranchName(providerName, branch)}, Path: "", }, } diff --git a/internal/watch/bootstrap_admission_test.go b/internal/watch/bootstrap_admission_test.go index b1e23465..bb3cd345 100644 --- a/internal/watch/bootstrap_admission_test.go +++ b/internal/watch/bootstrap_admission_test.go @@ -35,7 +35,7 @@ func bootGitTarget() *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: bootTargetName, Namespace: bootTargetNS}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: bootProviderName}, Path: "clusters/prod", }, @@ -141,7 +141,7 @@ func TestBootstrapRuleStore_SkipsUnauthorizedRuleButStillReady(t *testing.T) { admittedTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "ok-mirror", Namespace: "team-ok"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: bootProviderName}, Path: "clusters/ok", }, diff --git a/internal/watch/event_router_test.go b/internal/watch/event_router_test.go index 5197f2c6..66c228b1 100644 --- a/internal/watch/event_router_test.go +++ b/internal/watch/event_router_test.go @@ -86,7 +86,7 @@ func TestEnqueueScopedResync_ReportsMissingWorker(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget).Build() @@ -300,7 +300,7 @@ func TestServiceCommitRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget, testGitBranchObject()).Build() @@ -330,7 +330,7 @@ func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, gitTarget).Build() @@ -386,7 +386,7 @@ func TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget, testGitBranchObject()).Build() @@ -420,7 +420,7 @@ func TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, gitTarget).Build() @@ -481,7 +481,7 @@ func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, gitTarget).Build() diff --git a/internal/watch/manager_startup_test.go b/internal/watch/manager_startup_test.go index fc90f6b0..be0ee03f 100644 --- a/internal/watch/manager_startup_test.go +++ b/internal/watch/manager_startup_test.go @@ -40,7 +40,7 @@ func TestManagerStart_MustSeedRuleStoreFromExistingWatchRules(t *testing.T) { existingGitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "playground-target", Namespace: "tilt-playground"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("playground-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("playground-provider", "main")}, Path: "live-cluster", }, } @@ -106,7 +106,7 @@ func TestManagerStart_MustSeedRuleStoreFromExistingClusterWatchRules(t *testing. existingGitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "ops-target", Namespace: "ops"}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("ops-provider", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("ops-provider", "main")}, Path: "cluster-state", }, } diff --git a/internal/watch/source_namespace_test.go b/internal/watch/source_namespace_test.go index 16f99077..eec2e4af 100644 --- a/internal/watch/source_namespace_test.go +++ b/internal/watch/source_namespace_test.go @@ -32,7 +32,7 @@ func snbGitTarget() *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: snbTarget, Namespace: snbTenantNS}, Spec: configv1alpha3.GitTargetSpec{ - GitBranchRef: &meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("git", "main")}, ClusterProviderRef: &meta.LocalObjectReference{Name: snbProvider}, Path: "tenants/acme", }, From cc06ea4aaec1e0cd5b6c18577090d4e99d285712 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Wed, 21 Oct 2026 20:34:50 +0000 Subject: [PATCH 5/6] docs: describe GitBranch, and move the chart, examples and e2e suite to it The chart's quick start renders a GitBranch (quickstart.gitBranch), the samples and corpus fixtures name one, and the reference docs describe the branch claim, its reasons, and where status.remote now lives. The e2e suite creates a GitBranch per provider branch; the shared-provider spec becomes a shared-GitBranch spec covering the claim, and the parent-branch and remote refresh specs read the GitBranch. Co-Authored-By: Claude Opus 5.5 (1M context) --- README.md | 9 +- charts/gitops-reverser/README.md | 3 +- charts/gitops-reverser/templates/NOTES.txt | 1 + .../gitops-reverser/templates/quickstart.yaml | 18 +- .../templates/validate-quickstart.yaml | 4 +- charts/gitops-reverser/values.schema.json | 9 +- charts/gitops-reverser/values.yaml | 8 +- docs/UPGRADING.md | 97 +++++-- docs/architecture.md | 156 ++++++----- docs/azure-devops-getting-started.md | 13 +- docs/bi-directional-argocd.md | 2 +- docs/bi-directional-flux.md | 2 +- docs/configuration.md | 251 ++++++++++++------ docs/definitions.md | 27 +- docs/demo/commit-window.md | 13 +- docs/facts/subresources.md | 13 +- docs/interpreting-metrics.md | 6 +- docs/layout/model.md | 9 +- docs/layout/new-file-placement-rules.md | 14 +- docs/quickstart.md | 8 +- docs/spec/status-conditions-guide.md | 22 +- hack/crdfields/fields.yaml | 2 +- internal/git/layout_corpus_test.go | 22 +- test/e2e/audit_route_attribution_e2e_test.go | 7 +- test/e2e/gitprovider_validation_e2e_test.go | 10 +- test/e2e/helpers.go | 36 ++- test/e2e/new_branch_parent_e2e_test.go | 29 +- test/e2e/prune_mode_e2e_test.go | 31 +-- test/e2e/quickstart_framework_e2e_test.go | 1 + test/e2e/refresh_remote_e2e_test.go | 36 +-- .../podinfos-intent/2-reverse-gitops.yaml | 12 +- .../test/coffeeconfig-reverse-gitops.yaml | 12 +- test/e2e/shared_gitbranch_e2e_test.go | 219 +++++++++++++++ test/e2e/shared_gitprovider_e2e_test.go | 139 ---------- test/e2e/source_cluster_e2e_test.go | 14 +- test/e2e/suspend_e2e_test.go | 7 +- test/e2e/templates/gittarget.tmpl | 13 +- .../templates/manager/gittarget-prune.tmpl | 13 +- .../1-flat-serialized/config/gittarget.yaml | 15 +- .../config/gittarget-second-namespace.yaml | 15 +- .../config/gittarget.yaml | 13 +- .../3-tree-serialized/config/gittarget.yaml | 15 +- .../config/gittarget.yaml | 13 +- .../config/gittarget-empty-folder.yaml | 15 +- .../config/gittarget.yaml | 13 +- .../config/gittarget-app-root.yaml | 15 +- .../config/gittarget-prod.yaml | 13 +- .../config/gittarget-test.yaml | 15 +- .../config/gittarget-prod.yaml | 13 +- .../config/gittarget-prod.yaml | 13 +- .../homelab-argocd/config/gittarget.yaml | 13 +- .../homelab-flux/config/gittarget-media.yaml | 15 +- .../homelab-flux/config/gittarget.yaml | 13 +- test/playground/config/gittarget.yaml | 13 +- 54 files changed, 1023 insertions(+), 487 deletions(-) create mode 100644 test/e2e/shared_gitbranch_e2e_test.go delete mode 100644 test/e2e/shared_gitprovider_e2e_test.go diff --git a/README.md b/README.md index ce40ebc9..2af3c6d8 100644 --- a/README.md +++ b/README.md @@ -46,10 +46,11 @@ agent over MCP. 2. **Sanitize** the change and compare it against what Git already holds. 3. **Write** stable YAML to the target folder and push, grouping a burst of changes into one commit. -Custom resources configure it: `GitProvider` for repository and credentials, `GitTarget` for branch -and folder, `WatchRule` or `ClusterWatchRule` for what to capture, and `ClusterProvider` for the -source cluster, created as `default` on install. A rule selects whole resource collections, including -their deletions; the `GitTarget`'s prune mode decides whether a deletion removes the file from Git. +Custom resources configure it: `GitProvider` for repository and credentials, `GitBranch` for one +branch of that repository, `GitTarget` for a folder on it, `WatchRule` or `ClusterWatchRule` for what +to capture, and `ClusterProvider` for the source cluster, created as `default` on install. A rule +selects whole resource collections, including their deletions; the `GitTarget`'s prune mode decides +whether a deletion removes the file from Git. See [Configuration](docs/configuration.md). By default it watches its own cluster. Add more of each as you need them: one operator can serve diff --git a/charts/gitops-reverser/README.md b/charts/gitops-reverser/README.md index a3ba1243..99db0140 100644 --- a/charts/gitops-reverser/README.md +++ b/charts/gitops-reverser/README.md @@ -188,11 +188,12 @@ nodeSelector: | `auditService.type` | Service type for the dedicated audit Service | `NodePort` | | `auditService.nodePort` | Fixed NodePort for the audit Service when `auditService.type=NodePort` | `30444` | | `auditService.clusterIP` | Optional fixed ClusterIP for the dedicated audit Service | `""` | -| `quickstart.enabled` | Create starter `GitProvider`, `GitTarget`, and `WatchRule` resources | `false` | +| `quickstart.enabled` | Create starter `GitProvider`, `GitBranch`, `GitTarget`, and `WatchRule` resources | `false` | | `quickstart.namespace` | Namespace for the starter quickstart resources (create it, and the git-creds Secret, before install) | `gitops-reverser-quickstart-demo` | | `quickstart.createNamespace` | Let the chart create `quickstart.namespace` (Helm then owns it) | `false` | | `quickstart.gitProvider.url` | Repository URL used by the starter `GitProvider` | `""` | | `quickstart.gitProvider.secretRef.name` | Existing Secret name used by the starter `GitProvider` | `git-creds` | +| `quickstart.gitBranch.branch` | Branch the starter `GitBranch` writes. Must match `quickstart.gitProvider.allowedBranches` | `main` | | `quickstart.gitTarget.path` | Repository path used by the starter `GitTarget`; set `.` only to deliberately target the repo root | `live-cluster` | | `quickstart.watchRule.rules` | Rules used by the starter `WatchRule` | `configmaps create/update/delete` | | `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors and for the admission webhook to actually record CommitRequest authors (admission runs as a no-op without it); **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | diff --git a/charts/gitops-reverser/templates/NOTES.txt b/charts/gitops-reverser/templates/NOTES.txt index b195f2ed..7cf5f332 100644 --- a/charts/gitops-reverser/templates/NOTES.txt +++ b/charts/gitops-reverser/templates/NOTES.txt @@ -37,6 +37,7 @@ What the chart did for you: {{- end }} - created quickstart resources in namespace `{{ .Values.quickstart.namespace }}` - GitProvider: `{{ .Values.quickstart.gitProvider.name }}` + - GitBranch: `{{ .Values.quickstart.gitBranch.name }}` - GitTarget: `{{ .Values.quickstart.gitTarget.name }}` - WatchRule: `{{ .Values.quickstart.watchRule.name }}` {{- end }} diff --git a/charts/gitops-reverser/templates/quickstart.yaml b/charts/gitops-reverser/templates/quickstart.yaml index f3fe92cf..c53ce60d 100644 --- a/charts/gitops-reverser/templates/quickstart.yaml +++ b/charts/gitops-reverser/templates/quickstart.yaml @@ -26,9 +26,9 @@ spec: name: {{ .Values.quickstart.gitProvider.secretRef.name }} --- apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: {{ .Values.quickstart.gitTarget.name }} + name: {{ .Values.quickstart.gitBranch.name }} namespace: {{ .Values.quickstart.namespace }} labels: {{- include "gitops-reverser.labels" . | nindent 4 }} @@ -36,7 +36,19 @@ metadata: spec: gitProviderRef: name: {{ .Values.quickstart.gitProvider.name }} - branch: {{ .Values.quickstart.gitTarget.branch | quote }} + branch: {{ .Values.quickstart.gitBranch.branch | quote }} +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: {{ .Values.quickstart.gitTarget.name }} + namespace: {{ .Values.quickstart.namespace }} + labels: + {{- include "gitops-reverser.labels" . | nindent 4 }} + app.kubernetes.io/component: quickstart +spec: + gitBranchRef: + name: {{ .Values.quickstart.gitBranch.name }} path: {{ .Values.quickstart.gitTarget.path | quote }} commit: window: diff --git a/charts/gitops-reverser/templates/validate-quickstart.yaml b/charts/gitops-reverser/templates/validate-quickstart.yaml index c3168aba..b74430d1 100644 --- a/charts/gitops-reverser/templates/validate-quickstart.yaml +++ b/charts/gitops-reverser/templates/validate-quickstart.yaml @@ -8,8 +8,8 @@ {{- if eq (trim .Values.quickstart.gitProvider.secretRef.name) "" -}} {{- fail "quickstart.gitProvider.secretRef.name must be set when quickstart.enabled=true" -}} {{- end -}} -{{- if eq (trim .Values.quickstart.gitTarget.branch) "" -}} -{{- fail "quickstart.gitTarget.branch must be set when quickstart.enabled=true" -}} +{{- if eq (trim .Values.quickstart.gitBranch.branch) "" -}} +{{- fail "quickstart.gitBranch.branch must be set when quickstart.enabled=true" -}} {{- end -}} {{- if eq (trim .Values.quickstart.gitTarget.path) "" -}} {{- fail "quickstart.gitTarget.path must be set when quickstart.enabled=true" -}} diff --git a/charts/gitops-reverser/values.schema.json b/charts/gitops-reverser/values.schema.json index 979db558..ccf3d559 100644 --- a/charts/gitops-reverser/values.schema.json +++ b/charts/gitops-reverser/values.schema.json @@ -576,12 +576,19 @@ } } }, + "gitBranch": { + "type": "object", + "additionalProperties": false, + "properties": { + "name": { "type": "string" }, + "branch": { "type": "string" } + } + }, "gitTarget": { "type": "object", "additionalProperties": false, "properties": { "name": { "type": "string" }, - "branch": { "type": "string" }, "path": { "type": "string", "minLength": 1 }, "commit": { "type": "object", diff --git a/charts/gitops-reverser/values.yaml b/charts/gitops-reverser/values.yaml index 3959000b..6d714c34 100644 --- a/charts/gitops-reverser/values.yaml +++ b/charts/gitops-reverser/values.yaml @@ -470,7 +470,7 @@ auditKubeconfig: # you explicitly opt in. quickstart: enabled: false - # Namespace for the starter GitProvider, GitTarget, and WatchRule. Create it (and the git-creds + # Namespace for the starter GitProvider, GitBranch, GitTarget, and WatchRule. Create it (and the git-creds # Secret) before installing, so the starter GitProvider is Ready on the first reconcile instead of # waiting up to ~5 minutes (RequeueSteadyInterval) for the Secret to appear. namespace: gitops-reverser-quickstart-demo @@ -487,9 +487,13 @@ quickstart: secretRef: # Required when quickstart.enabled=true. name: git-creds + # The repository branch the starter GitTarget writes. It holds the branch once the GitProvider's + # credential can open a push session on it. + gitBranch: + name: example-branch + branch: main gitTarget: name: example-target - branch: main commit: # When this target's commit windows close: after idleTimeout of silence, or # maxDuration after the window opened, whichever comes first. It is a GitTarget diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 21dc2468..13b156d2 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -7,48 +7,91 @@ guidance that the changelog's breaking-change entries link to. We are pre-1.0, so breaking changes bump the **minor** version (release-please is configured with `bump-minor-pre-major`) rather than the major. Read the relevant entry before upgrading across it. -## One `GitProvider` per repository branch; a `GitTarget` may name one in another namespace +## `GitBranch` holds a repository branch; a `GitTarget` names one with `spec.gitBranchRef` -**Breaking.** A branch worker is now keyed by repository and branch, and pushes with one -`GitProvider`'s credential. So every `GitTarget` that writes one repository branch must name the same -`GitProvider`. Where two `GitProvider`s name one repository (compared as URLs, so an `https://` and an -SSH spelling of the same repository are one) and their `GitTarget`s write the same branch, the -`GitProvider` of the oldest `GitTarget` that may write there keeps the branch (one its providers do -not admit decides nothing). Every `GitTarget` writing it through another -`GitProvider` stops writing and reports `Validated=False` (`TargetConflict`), with a message that -names the `GitProvider` to share. Two `GitProvider`s for one repository on different branches are -unaffected. +**Breaking.** A repository branch is its own object, `GitBranch`, in the namespace of the +`GitProvider` whose credential writes it. A `GitTarget` names one with the required, immutable +`spec.gitBranchRef`, and `spec.gitProviderRef`, `spec.branch` and `spec.parentBranch` are gone from +`GitTarget`: setting one is refused at admission, with a message naming `spec.gitBranchRef`. +`GitTarget.status.remote` is now `GitBranch.status.remote`, and `GitProvider.status.branches` is +gone; `kubectl get gitbranch -A` answers what it did. -To share one `GitProvider` across namespaces, admit the other namespaces on it, and point their -`GitTarget`s at it: +Replace each GitTarget's three fields with a `GitBranch` and a reference to it: ```yaml -kind: GitProvider -metadata: {name: monorepo, namespace: platform} +kind: GitBranch +metadata: {name: platform-main, namespace: platform} spec: - accessFrom: - names: [team-a, team-b] # or selector:, matched against Namespace labels + gitProviderRef: {name: platform} # a GitProvider in this namespace + branch: main + # parentBranch: develop # was GitTarget.spec.parentBranch --- kind: GitTarget -metadata: {name: apps, namespace: team-a} +metadata: {name: apps, namespace: platform} spec: - gitProviderRef: {name: monorepo, namespace: platform} + gitBranchRef: {name: platform-main} + path: apps ``` -`spec.gitProviderRef` is immutable, so a `GitTarget` that has to move to the shared `GitProvider` -is deleted and recreated. A `GitTarget` that names a `GitProvider` in another namespace whose -`spec.accessFrom` does not admit it reports `Validated=False` (`NamespaceNotAuthorized`). Before -upgrading, list the repositories more than one `GitProvider` names: +`spec.gitBranchRef` is immutable and the old fields are refused, so delete each `GitTarget` and +create it again with the new shape. Its folder in Git is untouched: the new object takes it over +on its first snapshot. List the GitTargets to recreate, with what each `GitBranch` needs: ```bash -kubectl get gitproviders -A -o jsonpath='{range .items[*]}{.spec.url}{"\t"}{.metadata.namespace}/{.metadata.name}{"\n"}{end}' | sort +kubectl get gittargets -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\t"}{.spec.gitProviderRef.name}{"\t"}{.spec.branch}{"\t"}{.spec.parentBranch}{"\n"}{end}' ``` -The list sorts the URLs as written, so an `https://` and an SSH spelling of one repository land -apart: compare host and path, ignoring the scheme, the user, a trailing `.git` and case. +Run it before upgrading the CRDs: the new schema no longer serves the old fields. + +**One `GitBranch` holds a repository branch.** It holds it once its `GitProvider`'s credential +opens a push session on the remote; the oldest such `GitBranch` holds it, keeps it through a later +credential failure, and every other `GitBranch` for the same repository branch (compared as URLs, +so an `https://` and an SSH spelling of one repository are one) reports `Stalled=True` +(`BranchConflict`) and writes nothing. Create one `GitBranch` per repository branch, and point every +`GitTarget` on that branch at it. + +**The push session is a new check, and it needs a credential that may push.** A `GitProvider` with +no `secretRef` on a public repository used to be enough to write; its `GitBranch` now reports +`Ready=False` (`PushSessionFailed`) and holds nothing. On GitHub and GitLab opening the session +takes push permission. On Gitea and Forgejo it takes only a valid credential that can read the +repository, because their AGit flow accepts pushes from readers and checks a branch write when it +arrives. On a host that offers push sessions anonymously it proves only that the repository is +reachable. + +**Sharing a repository branch across namespaces.** A `GitTarget` names a `GitBranch` in another +namespace with `spec.gitBranchRef.namespace`; that `GitBranch`'s `GitProvider` must admit the +`GitTarget`'s namespace in `spec.accessFrom`, or the target reports `Validated=False` +(`NamespaceNotAuthorized`). The tenant namespace holds no copy of the credential: + +```yaml +kind: GitProvider +metadata: {name: monorepo, namespace: platform} +spec: + accessFrom: + names: [team-a, team-b] # or selector:, matched against Namespace labels +--- +kind: GitTarget +metadata: {name: apps, namespace: team-a} +spec: + gitBranchRef: {name: monorepo-main, namespace: platform} +``` -The metrics keep their `provider_namespace` and `provider_name` labels, which now name the one -`GitProvider` that owns the branch. Log lines name a branch as `repository@branch`. +New reasons follow from it. A `GitTarget` whose `GitBranch` does not exist reports +`Validated=False` (`GitBranchNotFound`); one whose `GitBranch` does not hold its branch, or is not +`Ready`, reports `GitBranchReady=False` (`GitBranchNotReady`) and waits. That condition replaces +`GitProviderReady`. `BranchNotAllowed`, `ParentBranchNotFound` and `RecoveringParentBranch` are now +reported by the `GitBranch`, and `GitTarget`s on it show them in the `GitBranchReady` message. +`TargetConflict` now means only overlapping paths on one `GitBranch`. A `WatchRule` or +`ClusterWatchRule` no longer reads the GitProvider and no longer reports `GitProviderNotFound`; it +follows its `GitTarget`'s readiness. The chart's quick start renders a `GitBranch` +(`quickstart.gitBranch.name`, `quickstart.gitBranch.branch`), and `quickstart.gitTarget.branch` is +gone. + +Deleting a `GitBranch` stops its branch worker after it has pushed or refused every pending write and +settled every `CommitRequest` it held. It never deletes the branch on the remote. + +The metrics keep their `provider_namespace` and `provider_name` labels, which name the `GitProvider` +of the `GitBranch` holding the branch. Log lines name a branch as `repository@branch`. ## `FluxTrigger` `spec.quietPeriod` and `status.source.prefetch` are removed diff --git a/docs/architecture.md b/docs/architecture.md index 305a927d..182666a8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -27,13 +27,14 @@ server deliberately remain separate source partitions. The asymmetry is not an oversight. A source cluster is an inherent singleton; a Git destination is not, and a namespaced `GitProvider` keeps a repository's write credential in the namespace of whoever owns that repository, which is what makes a team able to start mirroring without a platform -admin. A repository written from several namespaces needs no copy of the credential in each: one -`GitProvider` admits the other namespaces through its own `spec.accessFrom` (the `ClusterProvider` -pattern, applied to the destination), and their `GitTarget`s name it with -`spec.gitProviderRef.namespace`. Its own namespace is always admitted. **Every `GitTarget` writing -one repository branch must name the same `GitProvider`**, because one branch worker writes the -branch, with one credential: a `GitTarget` that writes it through another `GitProvider` than the -oldest one's is refused (`TargetConflict`). Converting `GitProvider` to cluster scope is not an +admin. Each branch of that repository is a `GitBranch` in the credential's namespace, and **one +`GitBranch` holds a repository branch**, because one branch worker writes the branch, with one +credential: the oldest `GitBranch` whose credential opens a push session on the remote holds it, and +any other reports `BranchConflict`. A repository written from several namespaces needs no copy of +the credential in each: their `GitTarget`s name the shared `GitBranch` with +`spec.gitBranchRef.namespace`, admitted by its `GitProvider`'s `spec.accessFrom` (the +`ClusterProvider` pattern, applied to the destination). Its own namespace is always admitted. +Converting `GitProvider` to cluster scope is not an option: a CRD's `spec.scope` is immutable once the definition is `Established`, so the change would mean deleting the definition (and every `GitProvider` in the cluster with it) and recreating both. @@ -98,8 +99,8 @@ for free: the push session reads the remote's advertisement and the server names accepted, so an active branch never schedules a look of its own. One that has gone quiet is asked on its 5-minute reconcile tick, and spends one ref advertisement per `--git-refresh-interval` (10m by default); only a branch that moved costs a fetch after that. What it learns is published -as `status.remote` (the commit, when it was last proved, and whether a push or a fetch proved -it), and a folder somebody changed in Git is re-read and republished as `status.placement`. +as the `GitBranch`'s `status.remote` (the commit, when it was last proved, and whether a push or a +fetch proved it), and a folder somebody changed in Git is re-read and republished as `status.placement`. **A refresh never writes content.** It plans nothing and commits nothing of its own. What it does publish, besides `status.remote` and `status.placement`, is whether the folder can be written at @@ -281,7 +282,8 @@ graph LR CR[CommitRequest] -->|gitTargetRef| GT GT -->|applyTriggerRef| TR["FluxTrigger /\nArgoCDTrigger"] GT -->|clusterProviderRef| CP[ClusterProvider] - GT -->|gitProviderRef| GP[GitProvider] + GT -->|gitBranchRef| GB[GitBranch] + GB -->|gitProviderRef| GP[GitProvider] KCFG[("Remote kubeconfig Secret\noperator namespace")] WM["Watch manager\nper-provider source context"] end @@ -406,18 +408,12 @@ How attribution and finalization interact is described under - **Source**: [api/v1alpha3/gittarget_types.go](../api/v1alpha3/gittarget_types.go) - **Controller**: [internal/controller/gittarget_controller.go](../internal/controller/gittarget_controller.go) -One materialization from a source provider to a Git destination: `(cluster provider, provider, branch, path)`. +One materialization from a source provider to a Git destination: `(cluster provider, GitBranch, path)`. Key fields: -- `spec.gitProviderRef`: a `GitProvider` by name, in the `GitTarget`'s own namespace unless - `namespace` names another one, whose `spec.accessFrom` must then admit it. Every `GitTarget` - writing one repository branch names the same `GitProvider`. +- `spec.gitBranchRef`: required; a `GitBranch` by name, in the `GitTarget`'s own namespace unless + `namespace` names another one, whose `GitProvider`'s `spec.accessFrom` must then admit it. - `spec.clusterProviderRef`: a cluster-scoped source `ClusterProvider`; it defaults to `{name: default}`. -- `spec.branch`: immutable branch, validated against `GitProvider.spec.allowedBranches`. -- `spec.parentBranch`: optional, immutable branch the write branch is created from and compared - against while it does not exist. Omitted means the remote's default branch. An explicit parent - must exist when creating the write branch. It does not need to be in `allowedBranches` because - it is only read. - `spec.path`: immutable, required path under the repo (`MinLength=1`; `.` means repo root and must be chosen explicitly). - `spec.encryption`: optional SOPS/age encryption settings for sensitive resources. @@ -431,17 +427,21 @@ Key fields: - `spec.onRefusal`: `Ignore` (default) or `Reapply`, which asks the applier to reapply the tip when an edit to an existing document is refused. `Reapply` requires `spec.applyTriggerRef`. -`spec.commit` describes the folder, not the connection, so two `GitTarget`s sharing one `GitProvider` +`spec.commit` describes the folder, not the connection, so two `GitTarget`s sharing one `GitBranch` may batch and phrase their commits differently. A branch worker serves a repository branch, through -its one `GitProvider`, and resolves the window per open window, since a window is bound to exactly -one target. +its `GitBranch`'s `GitProvider`, and resolves the window per open window, since a window is bound to +exactly one target. -`gitProviderRef`, `clusterProviderRef`, `branch`, and `path` are immutable so a target cannot silently -orphan an old materialization or change its source cluster. `parentBranch` is immutable too, -including adding or removing it, because it decides what history a new write branch gets. The -controller rejects a GitTarget that writes a repository branch through another `GitProvider` than -the oldest GitTarget there. Among GitTargets sharing a provider and branch, from any namespace, it -rejects overlapping paths and different parent branches. +`gitBranchRef`, `clusterProviderRef`, and `path` are immutable so a target cannot silently orphan an +old materialization or change its source cluster. Among GitTargets on one `GitBranch`, from any +namespace, the controller rejects overlapping paths. `spec.gitProviderRef`, `spec.branch` and +`spec.parentBranch` left the `GitTarget` for the `GitBranch`; the schema keeps them only so that +setting one is refused, naming `gitBranchRef`. + +A `GitTarget` whose `GitBranch` does not exist reports `Validated=False` (`GitBranchNotFound`); one +whose `GitBranch` does not hold its branch, or is not `Ready`, reports `GitBranchReady=False` and +waits as progress (`GitBranchNotReady`). References are checked in status, not at admission, so Flux +and Argo CD can apply a folder of manifests in any order. Status has a kstatus-compatible summary layer plus domain conditions: @@ -477,10 +477,10 @@ Represents a Git repository and the credentials/configuration used to write it. - `spec.url`: immutable repository URL. - `spec.secretRef`: optional Secret in the same namespace for HTTP/SSH authentication. -- `spec.accessFrom`: which other namespaces' `GitTarget`s may write through it, by name or label - selector, as on a `ClusterProvider`. Its own namespace is always admitted; omitted admits no - other. A `GitTarget` admitted from another namespace writes with this provider's credential, - identity and `allowedBranches`. Checked on every reconcile, so a tightened policy revokes. +- `spec.accessFrom`: which other namespaces' `GitTarget`s may write through this provider's + `GitBranch`es, by name or label selector, as on a `ClusterProvider`. Its own namespace is always + admitted; omitted admits no other. A `GitTarget` admitted from another namespace writes with this + provider's credential and identity. Checked on every reconcile, so a tightened policy revokes. - `spec.knownHostsRef`: optional SSH known hosts source. - `spec.allowedBranches`: glob patterns that gate writable branches. - `spec.commit.committer`: committer identity (defaults to `GitOps Reverser` / `noreply@configbutler.ai`). @@ -497,6 +497,29 @@ ecosystems is the credentials Secret, not a foreign repository object. The crede Kubernetes native, Flux, and Argo CD Secret key dialects (see [finished/git-credentials-interop.md](finished/git-credentials-interop.md)). +### GitBranch + +- **Source**: [api/v1alpha3/gitbranch_types.go](../api/v1alpha3/gitbranch_types.go) +- **Controller**: [internal/controller/gitbranch_controller.go](../internal/controller/gitbranch_controller.go) + +One branch of the repository a `GitProvider` names, holding it for every `GitTarget` that references +it. Every field is immutable: + +- `spec.gitProviderRef`: a `GitProvider` in the `GitBranch`'s own namespace only, so creating one + takes the right to create objects where the credential lives. +- `spec.branch`: validated against the provider's `allowedBranches` (`BranchNotAllowed`). +- `spec.parentBranch`: optional branch the write branch is created from and compared against while + it does not exist. Omitted means the remote's default branch. An explicit parent must exist when + creating the write branch (`ParentBranchNotFound`). It does not need to be in `allowedBranches` + because it is only read. +- `status.repository`: the normalized repository URL, half of the key one `GitBranch` holds. +- `status.remote`: where the branch is on the remote, and when that was last proved. +- `status.gitTargetCount`: how many `GitTarget`s reference it, from any namespace. + +The controller proves the provider's credential by opening a receive-pack session (at most once per +steady interval while it succeeds), keeps the branch claim, and publishes the worker's observation. +See [Git write architecture](#git-write-architecture) for the branch claim. + ### ClusterProvider `ClusterProvider` is the read-side peer of `GitProvider`. A `GitTarget` references it by the immutable @@ -1333,41 +1356,46 @@ stops a fact being published by something that cannot actually prove it. | --- | --- | --- | | Can this repository be reached with this credential? | the `GitProvider` controller's connectivity check | `GitProvider.status`: `Ready`, `lastVerifiedAt` | | Which repository is this? | the `GitProvider` object's identity (`metadata.uid` with `spec.url`) | nothing: it is carried on the branch worker and compared, not reported | -| Where is branch B on the remote, and when was that proved? | the branch worker, on its queue | `GitTarget.status.remote`, sampled by each `GitTarget` on the branch | -| What is this repository being used for? | the `GitTarget`s that reference it | `GitProvider.status.branches` | +| May this `GitBranch` write branch B? | the `GitBranch` controller, opening a push session with the `GitProvider`'s credential | the `GitBranch`'s `Ready` (`BranchConflict`, `PushSessionFailed`) | +| Where is branch B on the remote, and when was that proved? | the branch worker, on its queue | `GitBranch.status.remote`, written at most once a minute | +| What is this repository being used for? | the `GitBranch`es that name its `GitProvider` | `kubectl get gitbranch`, and each one's `status.gitTargetCount` | | Is this folder accepted, streaming and matching live? | the watch plane, per `GitTarget` | the `GitTarget`'s conditions and `status.placement` | | Does the applier need to apply, and is now safe? | the branch worker, on its queue | the trigger's `status.operation` and an `ApplyRequested` Event | | Can the applier be asked at all? | the trigger controller, reading the applier in its cluster | the trigger's `Configured`, and the `GitTarget`'s `ApplyTriggerReady` | -A branch worker sits at the intersection and belongs wholly to neither object: `GitTarget`s decide -which branches are needed, `GitProvider` decides which repository those branches are in. So the -worker is **keyed** by the repository branch, normalized as `internal/repourl` compares URLs, and -writes with the one `GitProvider` that owns it: the GitProvider of the oldest `GitTarget` on the -branch that may write it (`internal/git/branch_owner.go`). A `GitTarget` its `ClusterProvider` or -its `GitProvider` does not admit decides nothing, so it cannot keep the branch from one that may. -The `GitTarget` reconciler, the worker sweep and -`EnsureWorker` decide ownership the same way, so a second `GitProvider` naming the repository gets no -worker on that branch, however close together the two `GitTarget`s were created. A provider -recreated under its name does not move a live worker; it replaces it, and the replacement starts -with no clone, no base trust and no observation. - -`GitTarget.status.remote` is a **branch** fact published per target, and it is worth being exact -about that, because the reverse reading is tempting and wrong. The commit and its `lastVerifiedAt` -are the branch worker's single observation, shared by every `GitTarget` on that branch: a push by -one target proves it for all of them. So a fresh `lastVerifiedAt` says the branch tip was recently -proved. It does **not** say that this target's folder was scanned, that its documents were -published, or that its streams are running. Those are the conditions' job. - -**Delivery is not publication**, and the split is deliberate. Delivery is immediate and costs -nothing: the observation is recorded once, where it is proved, and every target on the branch reads -the same tuple from that moment. Publication is a status write (an etcd write that invalidates -every watcher's cached copy of the type), so it is sampled: each target publishes what it holds -when it reconciles, writes the stanza at most once a minute however often that is, and nothing in -the data plane wakes a target because a branch moved. - -So two targets on one branch can show different times. That is publication lag, not a difference in -what was proved, and each tuple is internally consistent: one observation from one moment, never -the newest commit beside a stale clock. +A branch worker is **keyed** by the repository branch, normalized as `internal/repourl` compares +URLs, and writes for the `GitBranch` that holds it, with that `GitBranch`'s `GitProvider` +credential, signing key and metric labels. The branch claim (`internal/git/branch_owner.go`) lives +in the operator's memory: + +1. A `GitBranch` becomes a candidate once its credential opens a receive-pack session on the + remote. One that cannot claims nothing, so creating it first blocks nobody. +2. Among the candidates for one repository branch, the oldest by `creationTimestamp` (then UID) + holds it. +3. The holder stays a candidate through a later credential or network failure, so it keeps the + branch and its pending writes; any other candidate whose proof fails stops being one. +4. After a restart the claim is decided again by 1 and 2. + +The `GitTarget` reconciler, the worker sweep and `EnsureWorker` all read the claim, so a `GitBranch` +that does not hold its branch gets no worker, however close together two were created. A worker +serves exactly the `GitTarget`s whose `spec.gitBranchRef` names its `GitBranch`, which is how the +event router finds it with no read on the event path. A provider recreated under its name does not +move a live worker; it replaces it, and the replacement starts with no clone, no base trust and no +observation. + +A `GitBranch` carries a finalizer. Deleting it releases the claim, then stops its worker, which +closes the worker's intake, pushes or refuses every pending write and settles every `CommitRequest` +it holds; only then does the finalizer go. Nothing deletes the branch on the remote. + +`GitBranch.status.remote` is a **branch** fact, and it is worth being exact about that, because the +reverse reading is tempting and wrong. A fresh `lastVerifiedAt` says the branch tip was recently +proved. It does **not** say that any one target's folder was scanned, that its documents were +published, or that its streams are running. Those are the `GitTarget` conditions' job. + +**Delivery is not publication**, and the split is deliberate. The worker wakes its `GitBranch` every +time it proves something, a push or a fetch. Publication is a status write (an etcd write that +invalidates every watcher's cached copy of the type), so the `GitBranch` writes the stanza at most +once a minute, and comes back when that floor lifts. ### Everything about a branch happens on that branch's queue @@ -1435,7 +1463,7 @@ A branch whose remote has been **deleted** takes the same path. The advertisemen branch at all, which is reported as a moved remote with a zero hash, so the replay re-roots on the remote's default branch and the retry re-creates the branch with the retained writes on top. -**A branch that does not exist yet** is created from its parent (`spec.parentBranch`, or the +**A branch that does not exist yet** is created from its parent (the `GitBranch`'s `spec.parentBranch`, or the remote's default branch when that is omitted) only when a write has something to commit, so an idle or in-sync target never creates it. Until then the worker compares the folder against the parent. Periodic refresh, ten minutes by default, follows the diff --git a/docs/azure-devops-getting-started.md b/docs/azure-devops-getting-started.md index 293ab692..f72f7a2e 100644 --- a/docs/azure-devops-getting-started.md +++ b/docs/azure-devops-getting-started.md @@ -48,14 +48,23 @@ spec: - --- apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: ado-target + name: ado-branch namespace: my-namespace spec: gitProviderRef: name: ado-provider branch: +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: ado-target + namespace: my-namespace +spec: + gitBranchRef: + name: ado-branch path: clusters/my-cluster --- apiVersion: configbutler.ai/v1alpha3 diff --git a/docs/bi-directional-argocd.md b/docs/bi-directional-argocd.md index 5c4a13af..42bda8c7 100644 --- a/docs/bi-directional-argocd.md +++ b/docs/bi-directional-argocd.md @@ -187,7 +187,7 @@ against them. | Rule | Reported when broken | | --- | --- | | `spec.source.repoURL` is the `GitProvider`'s repository. The forms compare as one repository: `https://` and `ssh://`, `git@host:org/repo` (SCP style), a trailing `.git` or `/`, letter case, and the default port | `SourceMismatch`: `the applier of ArgoCDTrigger checkout applies , not `, on the `GitTarget`, and on the trigger when no `GitTarget` naming it matches | -| `spec.source.targetRevision` is the `GitTarget`'s `spec.branch`, as the name or as `refs/heads/`. `HEAD` (and an empty `targetRevision`), a tag and a commit never follow the branch | `SourceMismatch`: `the applier of ArgoCDTrigger checkout applies branch "HEAD", not main`, on the same objects | +| `spec.source.targetRevision` is the `spec.branch` of the `GitTarget`'s `GitBranch`, as the name or as `refs/heads/`. `HEAD` (and an empty `targetRevision`), a tag and a commit never follow the branch | `SourceMismatch`: `the applier of ArgoCDTrigger checkout applies branch "HEAD", not main`, on the same objects | | A single `spec.source`: no `spec.sources`, no `spec.sourceHydrator` | trigger `UnsupportedSource`: `syncs only a single-source Application` | | Automated sync is off: `spec.syncPolicy.automated` is absent, or has `enabled: false`. This is Argo CD's own rule; an `automated` block with no `enabled` is on | trigger `AutomatedSyncEnabled` | diff --git a/docs/bi-directional-flux.md b/docs/bi-directional-flux.md index f6e828cd..d6e78060 100644 --- a/docs/bi-directional-flux.md +++ b/docs/bi-directional-flux.md @@ -161,7 +161,7 @@ of them are checked again before every apply. | Rule | Reported when broken | | --- | --- | | `spec.url` is the `GitProvider`'s repository. The forms compare as one repository: `https://` and `ssh://`, `git@host:org/repo` (SCP style), a trailing `.git` or `/`, letter case, and the default port | `SourceMismatch`: `the applier of FluxTrigger checkout applies , not `, on the `GitTarget`, and on the trigger when no `GitTarget` naming it matches | -| `spec.ref.branch` is exactly the `GitTarget`'s `spec.branch` | `SourceMismatch`: `the applier of FluxTrigger checkout applies branch "release", not main`, on the same objects | +| `spec.ref.branch` is exactly the `spec.branch` of the `GitTarget`'s `GitBranch` | `SourceMismatch`: `the applier of FluxTrigger checkout applies branch "release", not main`, on the same objects | | `spec.ref` pins nothing stronger: no `commit`, `name`, `semver` or `tag`, even alongside the branch. Flux resolves those before the branch, so the artifact would never follow it | trigger `UnsupportedSource`: `pins spec.ref.tag "v1.0.0", so it never follows a branch` | | `spec.ref.branch` is set | trigger `UnsupportedSource`: `names no spec.ref.branch` | diff --git a/docs/configuration.md b/docs/configuration.md index 45de4b7b..f677375a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -78,7 +78,7 @@ Every field is immutable: to change one, delete the `GitBranch` and create a new | Field | Default | What it does | |---|---|---| | `gitProviderRef` | **required** | The `GitProvider`, in this namespace, whose repository and credential write this branch | -| `branch` | **required** | Branch to write. Must be in the provider's `allowedBranches`. Only one `GitBranch` holds a repository branch | +| `branch` | **required** | Branch to write. Must be in the provider's `allowedBranches`. Only one `GitBranch` holds a repository branch; see [who writes a repository branch](#who-writes-a-repository-branch) | | `parentBranch` | the remote's default branch | Branch a `branch` the remote does not have yet is created from. Only read, so it need not be allowed. See [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch) | ### `ClusterProvider` (cluster-scoped): the source cluster @@ -218,36 +218,6 @@ spec: - main ``` -### What this repository is being used for (`status.branches`) - -`status.branches` lists the branches the `GitTarget`s naming this provider are configured to write, -with how many reference each: - -```yaml -status: - branches: - - name: main - gitTargetCount: 2 - - name: release - gitTargetCount: 1 -``` - -An absent `branches` is a third state, and a different one: nothing has read the `GitTarget`s yet, -so nothing here has been measured. An empty list is a measurement. - -It is a statement about **configuration**, not about the remote and not about what is running: a -suspended or blocked `GitTarget` still says what the repository is for, and a branch that exists on -the remote but nothing writes never appears. It is re-derived on the provider's own reconcile tick, -so a `GitTarget` edit reaches it within one steady interval. - -`Ready=True` beside `branches: []` is the state it exists to make readable: **configured and -unused**, which looks nothing like broken, and which otherwise takes a `GitTarget` listing to tell -apart. - -A `gitTargetCount` above one means that many folders share one branch worker, one clone and one -queue. They still report their own health on their own conditions: one folder being refused -says nothing about its siblings. - ### `GitProvider.spec.secretRef`: the credentials Secret The referenced Secret holds the Git credentials. The examples use the **Kubernetes-native** keys, @@ -656,18 +626,115 @@ when audit attribution is enabled, for joining an audit event to the correspondi Changing a target's source cluster changes what its folder means, so `clusterProviderRef` is immutable. +## `GitBranch` + +`GitBranch` is one branch of the repository a `GitProvider` names. It holds that branch for every +`GitTarget` that references it: one branch worker, one clone, one queue, and the `GitProvider`'s +credential, commit identity and signing key. + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: example-branch + namespace: default +spec: + gitProviderRef: + name: example-provider # a GitProvider in this namespace + branch: main # must match the provider's allowedBranches + parentBranch: develop # optional; only used to create a missing branch +``` + +- `spec.gitProviderRef` names a `GitProvider` in the `GitBranch`'s own namespace, never another one. + Creating a `GitBranch` therefore takes the right to create objects where the credential lives. +- `spec.branch` must match one of the provider's `allowedBranches`, or the `GitBranch` reports + `Stalled=True` with reason `BranchNotAllowed`. +- `spec.parentBranch` is what the branch is created from while the remote does not have it. See + [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch). +- Every field is immutable. To write another branch, or through another `GitProvider`, create a new + `GitBranch` and delete the old one. + +### Who writes a repository branch + +Only one `GitBranch` writes a repository branch. The key is the provider's repository URL, +normalized (so `https://github.com/acme/platform.git` and `git@github.com:acme/platform` are one +repository), plus the branch. + +1. A `GitBranch` becomes a **candidate** once its `GitProvider`'s credential opens a push + (receive-pack) session on the remote. Nothing is pushed: the session is opened and closed. A + `GitBranch` whose credential cannot open one reports `Ready=False` with reason + `PushSessionFailed` and claims nothing, so creating it first blocks nobody. +2. Among the candidates for one repository branch, the **oldest** (by `creationTimestamp`) holds it + and gets the branch worker. Every other one reports `Stalled=True` with reason `BranchConflict` + and a message naming the holder. Reference the holder from your `GitTarget`s, or delete it first. +3. The holder keeps the branch through a later credential or network failure. It reports + `Ready=False` (`PushSessionFailed`) and keeps its pending writes, and the branch does not pass to + another `GitBranch` while it is down. +4. The claim lives in the operator's memory. After a restart it is decided again by 1 and 2. + +What the push session proves depends on the Git host: + +| Host | What opening a push session takes | +|---|---| +| GitHub, GitLab | Push permission on the repository | +| Gitea, Forgejo | A valid credential that can read the repository (Gitea's AGit flow accepts pushes to `refs/for/*` from readers; branch writes are checked when they arrive) | +| A host that offers push sessions anonymously | Nothing: the proof is only that the repository is reachable | + +Nothing refuses a second `GitBranch` at admission: references are checked in status, so a folder of +manifests can be applied in any order. + +### `GitBranch` status + +```yaml +status: + repository: github.com/acme/platform # normalized: half of the key only one GitBranch holds + remote: + commit: 4f1c... + lastVerifiedAt: "2026-10-21T09:12:00Z" + verifiedBy: Push + gitTargetCount: 2 +``` + +`status.remote` is where the branch is on the remote; see +[where the branch is](#where-the-branch-is-and-when-that-was-last-proved-statusremote). +`status.gitTargetCount` is how many `GitTarget`s reference this `GitBranch`, from any namespace. More +than one means they share its branch worker; each still reports its own folder's health on its own +conditions. + +`kubectl get gitbranch` shows the provider, the branch, the commit the branch is at, and `Ready`; +`-o wide` adds the reason, the repository, the target count and when the remote was last verified. + +| Reason | What it means | +|---|---| +| `GitProviderNotFound` | No `GitProvider` of that name in this namespace (`Stalled=True`) | +| `GitProviderNotReady` | The `GitProvider` exists and is not `Ready` | +| `BranchNotAllowed` | `spec.branch` is outside the provider's `allowedBranches` (`Stalled=True`) | +| `PushSessionFailed` | The credential cannot open a push session; a holder keeps the branch meanwhile | +| `BranchConflict` | Another `GitBranch` holds this repository branch (`Stalled=True`) | +| `ParentBranchNotFound` | Neither the branch nor its parent is on the remote (`Stalled=True`) | +| `RecoveringParentBranch` | The parent is back and the worker is publishing what it held | + +### Deleting a `GitBranch` + +A `GitBranch` carries a finalizer. Deleting it stops new intake for every `GitTarget` on the +branch, pushes or refuses each pending write, gives each attached `CommitRequest` its outcome, and +only then removes the finalizer and lets another `GitBranch` hold the branch. **Nothing deletes the +branch on the remote.** + +Deleting a `GitBranch` that `GitTarget`s still reference is allowed; they report `Validated=False` +with reason `GitBranchNotFound` afterwards. A new `GitBranch` with the same name is a new object and +does not pick up the old one's pending work. + ## `GitTarget` `GitTarget` decides where inside the repository resources are written. The important fields are: -- `spec.gitProviderRef`: which `GitProvider` backs this target +- `spec.gitBranchRef`: which `GitBranch` this target writes, and so which repository branch and + credential - `spec.clusterProviderRef`: which `ClusterProvider` supplies resources; omit it to reference the user-created `default` provider -- `spec.branch`: which allowed branch to write to -- `spec.parentBranch`: which branch `spec.branch` is created from while it does not exist yet; omit it - for the remote's default branch - `spec.path`: required relative path inside the repository; use `.` only when you deliberately want the repository root - `spec.encryption`: how `Secret` resources should be encrypted before commit @@ -693,16 +760,25 @@ Example: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: example-target + name: example-branch namespace: default spec: gitProviderRef: name: example-provider + branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: example-target + namespace: default +spec: + gitBranchRef: + name: example-branch # Omit clusterProviderRef to reference the user-created ClusterProvider named "default". # clusterProviderRef: {name: prod-eu-1} selects a different source provider. - branch: main path: live-cluster ``` @@ -717,16 +793,23 @@ dedicated to this target. If you enable `spec.encryption`, that applies to `Secret` resource writes for this target. For SOPS and age details, see [sops-age-guide.md](sops-age-guide.md). -`spec.gitProviderRef` references a `GitProvider` by `name`, in the `GitTarget`'s own namespace. Set -`namespace` to write through a `GitProvider` in another namespace; that `GitProvider`'s -`spec.accessFrom` must admit this `GitTarget`'s namespace, or the `GitTarget` is refused with -`Validated=False` (`NamespaceNotAuthorized`). Every `GitTarget` writing one repository branch must -name the same `GitProvider`, because one branch worker writes it with one credential. One that -writes the branch through another `GitProvider` than the oldest `GitTarget` there is refused with -`TargetConflict`, and the message names the `GitProvider` to share instead. +`spec.gitBranchRef` references a `GitBranch` by `name`, in the `GitTarget`'s own namespace, and is +required and immutable. Set `namespace` to write through a `GitBranch` in another namespace; the +`GitProvider` that `GitBranch` names must admit this `GitTarget`'s namespace in its +`spec.accessFrom`, or the `GitTarget` is refused with `Validated=False` (`NamespaceNotAuthorized`). +One policy governs both hops. A `GitBranch` that does not exist is `Validated=False` +(`GitBranchNotFound`), and one that does not hold its branch, or is not `Ready`, keeps the target at +`Ready=False` with reason `GitBranchNotReady` until it does. Either clears on its own once the +`GitBranch` appears or proves itself, so manifests can be applied in any order. + +`spec.gitProviderRef`, `spec.branch` and `spec.parentBranch` are no longer `GitTarget` fields: +setting one is refused at admission with a message naming `spec.gitBranchRef`. + +Two `GitTarget`s on one `GitBranch` must not write the same or nested paths: the later-created one +reports `Validated=False` with reason `TargetConflict`. `spec.clusterProviderRef` references a cluster-scoped `ClusterProvider`. It defaults to -`{name: default}` when omitted. That is intentionally different from `gitProviderRef`: a source cluster +`{name: default}` when omitted. That is intentionally different from `gitBranchRef`: a source cluster is a shared physical identity, while a Git destination and its credential normally belong to the target's namespace. The default name can represent either an in-cluster or remote source according to the `ClusterProvider` the user created. @@ -768,9 +851,9 @@ summary reason, while the domain conditions name the axis to fix: | Condition | What to check | |---|---| -| `Validated=False` | The referenced `GitProvider`, branch policy, `ClusterProvider`, and namespace authorization. | +| `Validated=False` | The referenced `GitBranch`, path conflicts, `ClusterProvider`, and namespace authorization. | | `EncryptionConfigured=False` | The age/SOPS Secret and whether key generation is allowed. | -| `GitProviderReady=False` | Repository URL, credential Secret, host keys, and push permissions. | +| `GitBranchReady=False` | The `GitBranch`: whether it holds its branch, and its `GitProvider`'s URL, credential Secret, host keys, and push permissions. | | `ClusterProviderReady=False` or `SourceClusterReachable=False` | Source-cluster credentials and access. | | `GitPathAccepted=False` | Repository files under `spec.path`; the message includes the issue kind, path, and actor when known. | | `RenderMatchesLive=False` | Fields that no longer match the rendered source, often transformer-owned values. | @@ -872,23 +955,30 @@ validation a bad template fails. **Point one at a scratch branch.** ```yaml +kind: GitBranch +metadata: {name: gitops-preview} spec: gitProviderRef: {name: homelab} branch: gitops-preview # not main +--- +kind: GitTarget +spec: + gitBranchRef: {name: gitops-preview} path: apps/checkout ``` It commits, and you read the commits: the real files, the real `resources:` registrations, the real deletes, in a diff you can review. Nothing is simulated, so nothing can be wrong about it. When you -are happy, delete the preview target and declare the real one. `spec.branch` is immutable, so the -two are always separate objects, and deleting the preview cannot disturb the real folder. +are happy, delete the preview target and declare the real one. `spec.gitBranchRef` is immutable, so +the two are always separate objects, and deleting the preview cannot disturb the real folder. For inspecting a repository without a cluster at all, use the `manifest-analyzer` CLI. ### Starting the write branch from another branch (`spec.parentBranch`) -A target whose `spec.branch` does not exist on the remote waits on standby: it compares the cluster -against the **parent branch** and creates nothing while they agree. The first write that has +`spec.parentBranch` is a `GitBranch` field. A `GitTarget` on a `GitBranch` whose `spec.branch` does +not exist on the remote waits on standby: it compares the cluster against the **parent branch** and +creates nothing while they agree. The first write that has something to commit creates the branch from the parent's current tip, checked when it pushes, so a target that waited a week does not publish onto a week-old checkout. A write that finds nothing to change creates no branch at all. @@ -896,20 +986,22 @@ change creates no branch at all. The parent is the remote's default branch unless you name one: ```yaml +kind: GitBranch spec: gitProviderRef: {name: homelab} branch: reverser/prod-edits # the write branch: created on the first commit parentBranch: env/prod # what it is created from - path: apps/checkout ``` -- An explicit parent must exist when the write branch is created. Otherwise the target reports - `Ready=False` with reason `ParentBranchNotFound` and writes nothing. +- An explicit parent must exist when the write branch is created. Otherwise the `GitBranch` reports + `Ready=False` with reason `ParentBranchNotFound`, its `GitTarget`s report `GitBranchNotReady`, and + nothing is written. - It recovers on its own once the parent is pushed, with no edit to the cluster and with periodic refresh on or off. The branch worker looks for the parent with one ref advertisement after 10s, then after twice as long each time, up to every 5 minutes, shared by every target on the branch. When the parent is back it publishes the writes it held, with their own authors and messages; - until those are published the target reports `Ready=False` with reason `RecoveringParentBranch`. + until those are published the `GitBranch` reports `Ready=False` with reason + `RecoveringParentBranch`. - An omitted parent is the remote's default branch, as last discovered. In an empty repository (no refs at all, tags included) the first commit starts a branch with no history. A repository that is not empty but whose `HEAD` names no branch it carries also reports `ParentBranchNotFound`, and @@ -922,14 +1014,12 @@ spec: from the previous default branch until then. - Once the write branch exists, it is used as it is: new commits on the parent do not reach it. Delete the write branch after its changes are merged, and the next edit starts a fresh one. -- `parentBranch` is immutable, including adding or removing it: delete and recreate the target to - change it, which loses nothing because the destination is unchanged. It is only read, so it does - not need to be in `allowedBranches`. -- While the write branch is absent, `status.remote.parent` shows the parent, whether it was - `Found`, `Missing` or `Unborn`, and the commit the first commit would build on. With +- `parentBranch` is immutable, including adding or removing it: delete and recreate the `GitBranch` + to change it. It is only read, so it does not need to be in `allowedBranches`. +- While the write branch is absent, the `GitBranch`'s `status.remote.parent` shows the parent, + whether it was `Found`, `Missing` or `Unborn`, and the commit the first commit would build on. With `parentBranch` omitted, that is how you see which default branch the remote resolved. -- All `GitTarget` objects on one `GitProvider` and branch must name the same parent branch; omitted - counts as its own value. The later-created one reports `TargetConflict`. +- One `GitBranch` holds a repository branch, so every `GitTarget` on it has the same parent. `spec.suspend` is **not** this. It is the next section. @@ -1047,9 +1137,9 @@ one target is one environment is one write partition. ### Where the branch is, and when that was last proved (`status.remote`) -`status.remote` is what the target last **proved** about its branch on the Git remote. Like -`status.placement` it is an observation rather than a record of work, so it is there before the -target has written anything: +`status.remote` is on the `GitBranch`: what its branch worker last **proved** about the branch on +the Git remote. It is an observation rather than a record of work, so it is there before anything +has been written: ```yaml status: @@ -1075,11 +1165,12 @@ status: - `parent` appears only while the branch is not on the remote: it says what the branch would be created from, observed together with the branch's absence. `state` is `Found` (the first commit builds on `commit`, the parent's tip), `Missing` (the parent is not on the remote either; the - target reports `ParentBranchNotFound` and writes nothing) or `Unborn` (an empty repository, with + `GitBranch` reports `ParentBranchNotFound` and nothing is written) or `Unborn` (an empty repository, with no refs at all, and no parent configured; the first commit starts the branch with no history). See [starting the write branch from another branch](#starting-the-write-branch-from-another-branch-specparentbranch). -`kubectl get gittarget -o wide` shows `lastVerifiedAt` as an age, in the `Verified` column. +`kubectl get gitbranch` shows `commit` in the `Commit` column, and `-o wide` shows +`lastVerifiedAt` as an age, in the `Verified` column. Two things renew it. A **push** does, for free: the push session reads the remote's advertisement and the server names the hash it accepted, on the connection the push was making anyway. So a @@ -1087,17 +1178,15 @@ target that is publishing never needs anything else. An **idle** target is asked its reconcile tick; see [`--git-refresh-interval`](#keeping-an-idle-target-fresh---git-refresh-interval) below. -**What is proved and what is published are two different rates.** An observation of a branch is -one fact, and it reaches every `GitTarget` on that branch the moment it is proved, in memory and -for nothing. Writing it to status is not free: ten folders sharing a branch would turn one commit -into ten status writes, each invalidating every watcher's cached copy of the type. So each target -publishes what it holds **when it reconciles** (no commit wakes anybody), and writes this stanza -at most once a minute however often it reconciles. Push and read status immediately, and you may -see the previous commit until that target's next tick. - -Two corrections skip the once-a-minute floor, though not the reconcile that carries them: the -**first** observation, and **taking back** a commit that names a repository the target no -longer points at (a recreated `GitProvider` with a different URL). And none of this promises +**What is proved and what is published are two different rates.** Every push and every fetch +proves where the branch is, and the worker wakes the `GitBranch` each time. Writing status on every +push would invalidate every watcher's cached copy of the type at the pace the branch moves, so the +`GitBranch` writes this stanza at most once a minute, and comes back when that floor lifts. Push and +read status immediately, and you may see the previous commit for up to a minute. + +Two corrections skip the once-a-minute floor: the **first** observation, and **taking back** a +commit that names a repository the `GitBranch` no longer points at (a recreated `GitProvider` with +a different URL). And none of this promises anything about the branch itself: an unreachable remote yields no observation at all, which is exactly what the pair `commit` + `lastVerifiedAt` is there to say. @@ -1196,7 +1285,7 @@ metadata: name: checkout namespace: team-a spec: - # gitProviderRef, branch, path ... + # gitBranchRef, path ... applyTriggerRef: kind: FluxTrigger # or ArgoCDTrigger name: checkout @@ -1500,7 +1589,7 @@ For "are resyncs still running", read `gitopsreverser_git_resync_failures_total` evaluated and found something, but a flat one proves nothing: a healthy resync retaining zero produces no increment either. -`spec.prune` is mutable (unlike `gitProviderRef`, `branch`, and `path`), so a target can be moved to +`spec.prune` is mutable (unlike `gitBranchRef` and `path`), so a target can be moved to `Always` once its watch scope is confirmed, without recreating it. Widening it to `Always` re-lists the target's watched scopes, so the cleanup runs on the edit instead of waiting for the next replay. Tightening it applies to the next write and leaves the streams alone, which is what makes it usable diff --git a/docs/definitions.md b/docs/definitions.md index 48ce8c97..353f5c4e 100644 --- a/docs/definitions.md +++ b/docs/definitions.md @@ -52,8 +52,8 @@ omitted means "the control cluster, acting as a source". and a verb for the act. The operator mirrors; it does not "sync", "export" or "back up", and those three words are not used. -**Folder.** The Git directory one `GitTarget` owns, at `spec.path` on `spec.branch`. A folder has -exactly one `GitTarget`, and one `GitTarget` has exactly one folder. +**Folder.** The Git directory one `GitTarget` owns, at `spec.path` on its `GitBranch`'s branch. A +folder has exactly one `GitTarget`, and one `GitTarget` has exactly one folder. The two directions, which share the word "source" and must not share anything else: @@ -129,16 +129,21 @@ Observation failure supplies no evidence of absence. The code calls this write-s **Branch worker.** The single event loop that owns one repository branch: the repository's URL, as `internal/repourl` normalizes it, and the branch. Every write to that branch goes through it, with -the credential of the one `GitProvider` that owns the branch, the `GitProvider` of the oldest -`GitTarget` writing it. Two words, lowercase in prose, `BranchWorker` as the Go type. +the credential of the `GitProvider` named by the `GitBranch` that holds the branch. Two words, +lowercase in prose, `BranchWorker` as the Go type. -**Write branch.** The branch a `GitTarget` writes to: `spec.branch`. It must be in the -`GitProvider`'s `allowedBranches`. +**Branch claim.** Which `GitBranch` holds a repository branch: the oldest one whose credential opened +a push session on the remote. Only the holder gets a branch worker; another `GitBranch` for the same +repository branch reports `BranchConflict`. Say "branch claim", or that a `GitBranch` "holds" its +branch; never a bare "claim", which names a rule's demand on the read side. `BranchClaims` in Go. + +**Write branch.** The branch a `GitTarget` writes to: its `GitBranch`'s `spec.branch`. It must be in +the `GitProvider`'s `allowedBranches`. **Parent branch.** The branch a write branch is created from, and compared against while the write -branch does not exist on the remote: `spec.parentBranch`, or the remote's default branch when that -is omitted. Say "parent branch"; a bare "parent" is fine where the subject is already a branch, such -as the `status.remote.parent` stanza or a sentence that has just named the parent branch. Use the +branch does not exist on the remote: the `GitBranch`'s `spec.parentBranch`, or the remote's default +branch when that is omitted. Say "parent branch"; a bare "parent" is fine where the subject is +already a branch, such as the `status.remote.parent` stanza or a sentence that has just named the parent branch. Use the full term wherever a parent *resource* could be meant. Not "base", which names a kustomize read-only base and base trust, and not "source", which names the source cluster. Once the write branch exists, the parent branch is not followed. @@ -164,7 +169,7 @@ applies those actions to the newer tip instead of planning the cluster's picture `PendingWrite` and `pendingWrites` in Go. Not "log", which also names diagnostic logs and suggests a durable journal, which this is not. -**Publication.** One commit-and-push cycle by a branch worker. The thing a `GitTarget` reports in +**Publication.** One commit-and-push cycle by a branch worker. The thing a `GitBranch` reports in `status.remote`. **Commit.** A bare Git commit hash, wherever one appears in the API: `status.remote.commit`, @@ -314,7 +319,7 @@ fault. `AuditFactsReceived` is the only one. **Observation.** A status field that reports what was measured rather than asserting health: `status.streams`, `status.retention`, `status.placement`, `status.remote`, -`GitProvider.status.branches`. Observations are bounded (counts and capped samples, never a list +`GitBranch.status.gitTargetCount`. Observations are bounded (counts and capped samples, never a list that grows with the cluster) and never drive a condition on their own. **Reason.** The short answer to "why is this condition in this state". It is not a restatement of diff --git a/docs/demo/commit-window.md b/docs/demo/commit-window.md index c704fc5d..c6be77b8 100644 --- a/docs/demo/commit-window.md +++ b/docs/demo/commit-window.md @@ -21,14 +21,23 @@ Save as `watch.yaml`. All resources in this example use the same namespace and t ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: window-demo + name: example-provider-commit-window-demo namespace: gitops-reverser-quickstart-demo spec: gitProviderRef: name: example-provider branch: commit-window-demo +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: window-demo + namespace: gitops-reverser-quickstart-demo +spec: + gitBranchRef: + name: example-provider-commit-window-demo path: apps/demo commit: window: diff --git a/docs/facts/subresources.md b/docs/facts/subresources.md index 898167b0..8285d103 100644 --- a/docs/facts/subresources.md +++ b/docs/facts/subresources.md @@ -192,14 +192,23 @@ Every `configbutler.ai` CRD enables `/status`: `GitProvider`, `GitTarget`, ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: platform + name: platform-config-main namespace: default spec: gitProviderRef: name: platform-config branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: platform + namespace: default +spec: + gitBranchRef: + name: platform-config-main path: clusters/dev status: observedGeneration: 4 diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index 9b20c513..006665f0 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -81,13 +81,13 @@ without a kubeconfig. | --- | --- | --- | | `resource_condition` | gauge | `kind`, `resource_namespace`, `resource_name`, `type`, `status`, `reason` | -- `kind` is `GitTarget`, `WatchRule`, `ClusterWatchRule`, `GitProvider`, `ClusterProvider`, - `FluxTrigger` or `ArgoCDTrigger`: the objects a human declares. +- `kind` is `GitTarget`, `WatchRule`, `ClusterWatchRule`, `GitProvider`, `GitBranch`, + `ClusterProvider`, `FluxTrigger` or `ArgoCDTrigger`: the objects a human declares. - `type` is the kstatus trio: `Ready` (the summary), `Reconciling` (working on it) and `Stalled` (given up, and nothing will retry without a human). - `status` is `True`, `False` or `Unknown`. - `reason` is the condition's reason as it stands — `ValidationFailed`, `UnsupportedContent`, - `ProviderNotFound` — which is what turns a count panel into a first guess. It is the API's own + `BranchConflict` — which is what turns a count panel into a first guess. It is the API's own string, so a release that renames a reason changes the series a selector matches; [`UPGRADING.md`](UPGRADING.md) lists every rename. diff --git a/docs/layout/model.md b/docs/layout/model.md index 5e3b982c..084a942c 100644 --- a/docs/layout/model.md +++ b/docs/layout/model.md @@ -342,8 +342,7 @@ folder. The way to see what a target would do is to let one do it, somewhere har ```yaml spec: - gitProviderRef: {name: homelab} - branch: gitops-preview # not main + gitBranchRef: {name: gitops-preview} # a GitBranch for branch gitops-preview, not main path: apps/checkout ``` @@ -530,6 +529,6 @@ one asserts a rule PR 1 ships; the second-namespace one is written and skipped n unrendered files? The remaining objection is legibility, not correctness. - Namespace-local `GitProvider` (answered): the homelab examples put a `GitTarget` in `argocd`, `flux-system` and `homelab-config`, each with its own `GitProvider`, three copies of one - credential in a single-owner cluster. One `GitProvider` now serves all three: - `spec.gitProviderRef.namespace` names it, and its `spec.accessFrom` admits the other namespaces. - Three `GitProvider`s writing one branch are refused. + credential in a single-owner cluster. One `GitBranch` now serves all three: + `spec.gitBranchRef.namespace` names it, and its `GitProvider`'s `spec.accessFrom` admits the other + namespaces. A second `GitBranch` for the same repository branch holds nothing (`BranchConflict`). diff --git a/docs/layout/new-file-placement-rules.md b/docs/layout/new-file-placement-rules.md index 080474e9..caa232fb 100644 --- a/docs/layout/new-file-placement-rules.md +++ b/docs/layout/new-file-placement-rules.md @@ -86,11 +86,19 @@ per-type policy explicit: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch +metadata: + name: platform-main spec: gitProviderRef: name: platform branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +spec: + gitBranchRef: + name: platform-main path: clusters/prod placement: byType: @@ -119,8 +127,8 @@ escape hatch if users later need scope-wide or metadata-aware placement. ## Current implementation, as reviewed -The current `GitTargetSpec` has `gitProviderRef`, `branch`, `path`, and optional -`encryption`; it has no placement policy yet +The `GitTargetSpec` reviewed here had `gitProviderRef`, `branch`, `path`, and optional +`encryption` (the first two are on `GitBranch` now); it had no placement policy yet ([api/v1alpha3/gittarget_types.go](../../api/v1alpha3/gittarget_types.go)). The writer already uses the materialized-model direction described in diff --git a/docs/quickstart.md b/docs/quickstart.md index 07384ade..d9b29e93 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -1,7 +1,7 @@ # Quick start Point this at a disposable repository: the demo commits real cluster state, and the fastest way to -throw it away afterwards is to delete the repo. It creates a starter `GitProvider`, +throw it away afterwards is to delete the repo. It creates a starter `GitProvider`, `GitBranch`, `GitTarget`, and `WatchRule` in the `gitops-reverser-quickstart-demo` namespace, watching ConfigMaps there and writing them to `/live-cluster` on `main`. It runs in `configured-author` mode (no Redis) by default. The @@ -104,10 +104,12 @@ kubectl rollout status deployment/gitops-reverser -n gitops-reverser --timeout=3 Then check the starter resources: ```bash -kubectl get gitprovider,gittarget,watchrule -n gitops-reverser-quickstart-demo +kubectl get gitprovider,gitbranch,gittarget,watchrule -n gitops-reverser-quickstart-demo ``` -The `GitProvider` and `WatchRule` should report `Ready=True`. The `GitTarget`'s aggregate `Ready` +The `GitProvider`, `GitBranch` and `WatchRule` should report `Ready=True`. A `GitBranch` holds `main` +once the credential can open a push session on the repository; if it reports `PushSessionFailed`, +the Secret is missing, wrong, or may not push. The `GitTarget`'s aggregate `Ready` stays `Unknown` until its first source discovery, which is the expected state and not a fault, so the condition to check on it is `Validated`. It is not a printed column, so ask for it directly: diff --git a/docs/spec/status-conditions-guide.md b/docs/spec/status-conditions-guide.md index e7afab37..b6ccee02 100644 --- a/docs/spec/status-conditions-guide.md +++ b/docs/spec/status-conditions-guide.md @@ -46,8 +46,8 @@ The rule is already visible in the fields this project ships, and they are the w | `status.streams` | counts that move when a stream's readiness changes | status | | `status.retention` | counts and a roll-up time that move when a **resync** reports, not per event | status | | placements, placement refusals | every placed or refused document | metric (`gitopsreverser_placements_total`, `_placement_refusals_total`) | -| `GitProvider.status.branches` | somebody edits a `GitTarget` | status | -| `GitTarget.status.remote` | the branch moves, so at workload rate | status, but **sampled** — see below | +| `GitBranch.status.gitTargetCount` | a `GitTarget` naming it is created or deleted | status | +| `GitBranch.status.remote` | the branch moves, so at workload rate | status, but **floored** — see below | | a "last reconcile attempt" timestamp | every pass | removed, see below | `resolvedAtCommit` is the one worth reading twice, because it looks like a timestamp field and is @@ -64,20 +64,18 @@ than this rule requires. `status.remote` is the exception the rule has to be able to answer, because the question it answers — where is my branch on the remote — is genuinely worth a status field and genuinely moves -on every commit. Publishing it unsampled would be the fan-out this whole section forbids: a branch -ten `GitTarget`s share turns one commit into ten status writes. +on every commit. It lives on the `GitBranch`, one object per repository branch, so a branch that +ten `GitTarget`s share is still one status write, not ten. What makes it publishable is splitting the two rates, and the split is worth copying for any field in the same shape: - **Delivery** is in memory and immediate. One branch worker proves where its branch is, by a push - or by a fetch, and every `GitTarget` on that branch reads the same tuple from that moment. - Distributing a fact is not re-proving it, and it costs nothing. -- **Publication** is the status write, and it is sampled: each target writes what it holds WHEN IT - RECONCILES, on its own cadence, with a floor of one write per minute measured from the last - write. Nothing in the data plane wakes a target because a branch moved. The exceptions are a - reconcile that has ASKED the remote something, and one holding an answer the floor has not - let out yet: each comes back for the answer rather than leaving it undelivered for a tick. + or by a fetch, records it, and wakes its `GitBranch`. Distributing a fact is not re-proving it, + and it costs nothing. +- **Publication** is the status write, and it is floored: the `GitBranch` writes at most once a + minute, measured from the last write, and a reconcile that meets news the floor holds back comes + back when it lifts rather than leaving it undelivered for a tick. - **The tuple stays internally consistent**: one observation from one moment, never the newest commit beside a stale clock. That is what makes a sampled value readable, and it is why `commit` and `lastVerifiedAt` are published together or not at all. @@ -215,7 +213,7 @@ const ( TypeValidated = "Validated" TypeEncryptionConfigured = "EncryptionConfigured" TypeSourceClusterReachable = "SourceClusterReachable" - TypeGitProviderReady = "GitProviderReady" + TypeGitBranchReady = "GitBranchReady" TypeClusterProviderReady = "ClusterProviderReady" TypeStreamsRunning = "StreamsRunning" TypeGitPathAccepted = "GitPathAccepted" diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index d7c49d9d..f5e919db 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -71,7 +71,7 @@ kinds: - path: branch doc: >- Branch to write. Must be in the provider's `allowedBranches`. Only one `GitBranch` holds a - repository branch + repository branch; see [who writes a repository branch](#who-writes-a-repository-branch) - path: parentBranch default: the remote's default branch doc: >- diff --git a/internal/git/layout_corpus_test.go b/internal/git/layout_corpus_test.go index a6c49b03..873f84d9 100644 --- a/internal/git/layout_corpus_test.go +++ b/internal/git/layout_corpus_test.go @@ -361,17 +361,25 @@ func assertCorpusPatch(t *testing.T, path, got string) { "the write path and %s disagree; re-run with -update if the new diff is the intended one", path) } -// readCorpusGitTarget decodes one scenario's GitTarget. It is the SHIPPED type: a config naming a -// field the API does not have fails here, which is the check the harness-local struct used to buy -// with a whole second parser and an assertion test beside it. +// readCorpusGitTarget decodes one scenario's GitTarget: the GitTarget document of the config file, +// which also holds the GitBranch it references. It is the SHIPPED type: a config naming a field the +// API does not have fails here, which is the check the harness-local struct used to buy with a +// whole second parser and an assertion test beside it. func readCorpusGitTarget(t *testing.T, path string) v1alpha3.GitTarget { t.Helper() raw, err := os.ReadFile(path) require.NoError(t, err) - var target v1alpha3.GitTarget - require.NoError(t, yaml.UnmarshalStrict(raw, &target), "parsing %s", path) - require.NotEmpty(t, target.Spec.Path, "%s: spec.path is what the corpus writes into", path) - return target + for _, doc := range strings.Split(string(raw), "\n---\n") { + if !strings.Contains(doc, "kind: GitTarget") { + continue + } + var target v1alpha3.GitTarget + require.NoError(t, yaml.UnmarshalStrict([]byte(doc), &target), "parsing %s", path) + require.NotEmpty(t, target.Spec.Path, "%s: spec.path is what the corpus writes into", path) + return target + } + require.Failf(t, "no GitTarget", "%s holds no GitTarget document", path) + return v1alpha3.GitTarget{} } // readCorpusSourceNamespaces derives the source namespaces a scenario's target is reached by, from diff --git a/test/e2e/audit_route_attribution_e2e_test.go b/test/e2e/audit_route_attribution_e2e_test.go index 66911485..0d933311 100644 --- a/test/e2e/audit_route_attribution_e2e_test.go +++ b/test/e2e/audit_route_attribution_e2e_test.go @@ -255,22 +255,21 @@ spec: func applyGitTargetForClusterProvider( ns, name, gitProvider, targetPath, clusterProvider string, ) (string, error) { - manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 + manifest := gitBranchManifest(ns, gitProvider) + fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 kind: GitTarget metadata: name: %s namespace: %s spec: - gitProviderRef: + gitBranchRef: name: %s - branch: main path: %s clusterProviderRef: name: %s commit: window: idleTimeout: "0s" -`, name, ns, gitProvider, targetPath, clusterProvider) +`, name, ns, gitBranchName(gitProvider, "main"), targetPath, clusterProvider) return kubectlRunWithStdin(ns, manifest, "apply", "-f", "-") } diff --git a/test/e2e/gitprovider_validation_e2e_test.go b/test/e2e/gitprovider_validation_e2e_test.go index 11263870..1d165e80 100644 --- a/test/e2e/gitprovider_validation_e2e_test.go +++ b/test/e2e/gitprovider_validation_e2e_test.go @@ -80,7 +80,7 @@ var _ = Describe("Manager GitProvider Validation", Label("manager"), Ordered, fu _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", gitProviderName, "--ignore-not-found=true") }) - It("should handle GitTarget with nonexistent branch pattern", func() { + It("should refuse a GitBranch whose branch the GitProvider does not allow", func() { gitProviderName := "gitprovider-branch-test" // GitProvider should be Ready=True (validates connectivity, not branch existence) @@ -94,12 +94,14 @@ var _ = Describe("Manager GitProvider Validation", Label("manager"), Ordered, fu "gitprovider", gitProviderName, testNs, "True", "Succeeded", "Repository connectivity validated", ) - // GitTarget with branch not matching any pattern should fail + // A GitBranch for a branch matching no pattern is refused, and its GitTarget waits for it. destName := "dest-invalid-branch" createGitTarget(destName, testNs, gitProviderName, "test/invalid", "different-branch") - By("verifying GitTarget fails branch validation") - verifyResourceStatus("gittarget", destName, testNs, "False", "ValidationFailed", "BranchNotAllowed") + By("verifying the GitBranch fails branch validation") + verifyResourceStatus("gitbranch", gitBranchName(gitProviderName, "different-branch"), testNs, + "False", "BranchNotAllowed", "allowedBranches") + verifyResourceStatus("gittarget", destName, testNs, "False", "GitBranchNotReady", "BranchNotAllowed") cleanupGitTarget(destName, testNs) _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", gitProviderName, "--ignore-not-found=true") diff --git a/test/e2e/helpers.go b/test/e2e/helpers.go index 2aa474cd..27297481 100644 --- a/test/e2e/helpers.go +++ b/test/e2e/helpers.go @@ -197,7 +197,32 @@ const e2eCommitWindow = "0s" // cap would be exactly such a timer. const e2eCommitMaxDuration = "24h" -// createGitTarget creates a GitTarget that binds a GitProvider, branch and path. +// gitBranchName is the GitBranch the e2e helpers create for a GitProvider's branch: one per +// provider and branch, in the provider's namespace, so every GitTarget the helpers create on that +// branch shares it. +func gitBranchName(providerName, branch string) string { + return providerName + "-" + strings.ReplaceAll(strings.ToLower(branch), "/", "-") +} + +// gitBranchManifest is the GitBranch gitBranchName(providerName, "main") in namespace, as a YAML +// document followed by a separator, for a spec that applies its GitTarget as an inline manifest. +func gitBranchManifest(namespace, providerName string) string { + const branch = "main" + return fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: %s + namespace: %s +spec: + gitProviderRef: + name: %s + branch: %s +--- +`, gitBranchName(providerName, branch), namespace, providerName, branch) +} + +// createGitTarget creates a GitTarget on the GitBranch of a GitProvider's branch, creating that +// GitBranch too (applying it again is a no-op), with a path. func createGitTarget(name, namespace, providerName, path, branch string) { createGitTargetWithCommitWindow(name, namespace, providerName, path, branch, e2eCommitWindow) } @@ -256,7 +281,8 @@ func createGitTargetWithOptions( applyGitTarget(name, namespace, providerName, path, branch, "", encryptionSecretName, generateWhenMissing, commit) } -// createGitTargetWithParentBranch is createGitTarget with spec.parentBranch; empty omits it. +// createGitTargetWithParentBranch is createGitTarget whose GitBranch has spec.parentBranch; empty +// omits it. func createGitTargetWithParentBranch(name, namespace, providerName, path, branch, parentBranch string) { applyGitTarget(name, namespace, providerName, path, branch, parentBranch, e2eEncryptionRefName, false, gitTargetCommitOptions{Window: e2eCommitWindow}) @@ -273,12 +299,13 @@ func applyGitTarget( generateWhenMissing bool, commit gitTargetCommitOptions, ) { - By(fmt.Sprintf("creating GitTarget '%s' in ns '%s' for GitProvider '%s' with path '%s'", - name, namespace, providerName, path)) + By(fmt.Sprintf("creating GitTarget '%s' in ns '%s' on GitBranch '%s' with path '%s'", + name, namespace, gitBranchName(providerName, branch), path)) data := struct { Name string Namespace string + GitBranchName string ProviderName string Branch string ParentBranch string @@ -292,6 +319,7 @@ func applyGitTarget( }{ Name: name, Namespace: namespace, + GitBranchName: gitBranchName(providerName, branch), ProviderName: providerName, Branch: branch, ParentBranch: parentBranch, diff --git a/test/e2e/new_branch_parent_e2e_test.go b/test/e2e/new_branch_parent_e2e_test.go index 2ac4c757..0ab9097a 100644 --- a/test/e2e/new_branch_parent_e2e_test.go +++ b/test/e2e/new_branch_parent_e2e_test.go @@ -153,8 +153,12 @@ func describeMissingParentRecoverySpec() { By("an edit made while release does not exist is not written") applyParentConfigMap(testNs, "made-while-missing") + branchName := gitBranchName(providerName, writeBranch) Eventually(func(g Gomega) { - g.Expect(readyReasonOf(g, destName, testNs)).To(Equal("ParentBranchNotFound")) + g.Expect(gitBranchReadyReasonOf(g, branchName, testNs)).To(Equal("ParentBranchNotFound"), + "the GitBranch reports the missing parent") + g.Expect(readyReasonOf(g, destName, testNs)).To(Equal("GitBranchNotReady"), + "and its GitTarget waits for it") }, 3*time.Minute, 3*time.Second).Should(Succeed()) Expect(remoteBranchHeadOf(Default, repo.CheckoutDir, writeBranch)).To(BeEmpty(), "no orphan branch") @@ -183,8 +187,9 @@ func describeMissingParentRecoverySpec() { }, 5*time.Minute, 5*time.Second).Should(Succeed()) Eventually(func(g Gomega) { - g.Expect(readyReasonOf(g, destName, testNs)).NotTo( + g.Expect(gitBranchReadyReasonOf(g, branchName, testNs)).NotTo( BeElementOf("ParentBranchNotFound", "RecoveringParentBranch")) + g.Expect(readyReasonOf(g, destName, testNs)).NotTo(Equal("GitBranchNotReady")) }, 2*time.Minute, 3*time.Second).Should(Succeed()) }) } @@ -277,13 +282,14 @@ data: func(v float64) bool { return v > 0 }, "the worker for the write branch checked out the parent", 2*time.Minute) - By("status.remote names the parent the write branch would be created from") + By("the GitBranch's status.remote names the parent the write branch would be created from") + branchName := gitBranchName(providerName, writeBranch) Eventually(func(g Gomega) { - g.Expect(gitTargetRemoteField(g, destName, testNs, "commit")).To(BeEmpty(), "the write branch is absent") - g.Expect(gitTargetRemoteField(g, destName, testNs, "parent.state")).To(Equal("Found")) - g.Expect(gitTargetRemoteField(g, destName, testNs, "parent.branch")).To(Equal(parent), + g.Expect(gitBranchRemoteField(g, branchName, testNs, "commit")).To(BeEmpty(), "the write branch is absent") + g.Expect(gitBranchRemoteField(g, branchName, testNs, "parent.state")).To(Equal("Found")) + g.Expect(gitBranchRemoteField(g, branchName, testNs, "parent.branch")).To(Equal(parent), "with spec.parentBranch omitted this is the default branch the remote resolved") - g.Expect(gitTargetRemoteField(g, destName, testNs, "parent.commit")).To(Equal(hashA)) + g.Expect(gitBranchRemoteField(g, branchName, testNs, "parent.commit")).To(Equal(hashA)) }, 3*time.Minute, 5*time.Second).Should(Succeed()) By("an idle, in-sync target creates no branch") @@ -405,3 +411,12 @@ func checkoutRemoteTipOf(repo *RepoArtifacts, branch string, runGit func(...stri _, _ = gitRun(repo.CheckoutDir, "rm", "-rf", ".") return nil } + +// gitBranchReadyReasonOf reads a GitBranch's Ready reason. +func gitBranchReadyReasonOf(g Gomega, name, namespace string) string { + GinkgoHelper() + out, err := kubectlRunInNamespace(namespace, "get", "gitbranch", name, + "-o", `jsonpath={.status.conditions[?(@.type=="Ready")].reason}`) + g.Expect(err).NotTo(HaveOccurred(), "failed to read Ready reason of GitBranch %q", name) + return strings.TrimSpace(out) +} diff --git a/test/e2e/prune_mode_e2e_test.go b/test/e2e/prune_mode_e2e_test.go index 38d6d1a5..4fa0c799 100644 --- a/test/e2e/prune_mode_e2e_test.go +++ b/test/e2e/prune_mode_e2e_test.go @@ -325,19 +325,21 @@ func retentionModeOf(g Gomega, name, namespace string) string { func applyPruneGitTarget(name, namespace, providerName, targetPath, mode string) { GinkgoHelper() data := struct { - Name string - Namespace string - ProviderName string - Branch string - Path string - PruneMode string + Name string + Namespace string + GitBranchName string + ProviderName string + Branch string + Path string + PruneMode string }{ - Name: name, - Namespace: namespace, - ProviderName: providerName, - Branch: "main", - Path: targetPath, - PruneMode: mode, + Name: name, + Namespace: namespace, + GitBranchName: gitBranchName(providerName, "main"), + ProviderName: providerName, + Branch: "main", + Path: targetPath, + PruneMode: mode, } Expect(applyFromTemplate("test/e2e/templates/manager/gittarget-prune.tmpl", data, namespace)). To(Succeed(), "failed to apply GitTarget %q with prune mode %q", name, mode) @@ -354,15 +356,14 @@ metadata: name: %s namespace: %s spec: - gitProviderRef: + gitBranchRef: name: %s - branch: main path: %s commit: window: idleTimeout: "0s" %s -`, name, namespace, providerName, targetPath, pruneBlock) +`, name, namespace, gitBranchName(providerName, "main"), targetPath, pruneBlock) out, err := kubectlRunWithStdin(namespace, manifest, "apply", "-f", "-") if err != nil { return fmt.Errorf("%w: %s", err, out) diff --git a/test/e2e/quickstart_framework_e2e_test.go b/test/e2e/quickstart_framework_e2e_test.go index 59e7e5d1..7ae16f85 100644 --- a/test/e2e/quickstart_framework_e2e_test.go +++ b/test/e2e/quickstart_framework_e2e_test.go @@ -112,6 +112,7 @@ func (r *quickstartFrameworkRun) runReadmeQuickstartFlow() { By("verifying the chart's starter resources become Ready") verifyResourceStatus("gitprovider", "example-provider", r.namespace, "True", "Succeeded", "") + verifyResourceStatus("gitbranch", "example-branch", r.namespace, "True", "Succeeded", "holds") verifyResourceCondition("gittarget", "example-target", r.namespace, "Validated", "True", "Succeeded", "") verifyResourceStatus("watchrule", "example-watchrule", r.namespace, "True", "Succeeded", "") waitForStreamsRunning("example-target", r.namespace) diff --git a/test/e2e/refresh_remote_e2e_test.go b/test/e2e/refresh_remote_e2e_test.go index a56f77fe..3ae8bed2 100644 --- a/test/e2e/refresh_remote_e2e_test.go +++ b/test/e2e/refresh_remote_e2e_test.go @@ -28,14 +28,12 @@ import ( // Timing. The refresh rides the GitTarget reconcile, and a converged target requeues on the // 5-minute steady interval, so the bound on noticing the move is one steady tick plus the refresh // itself — not the 90s --git-refresh-interval config/deployment.yaml sets, which only decides how -// old an observation may be when that tick arrives. What the refresh learns is published in the -// same round: the reconcile that asked comes back for the answer, and one the publication floor -// stops comes back when it lifts (requeueForRemoteAnswer). +// old an observation may be when that tick arrives. What the refresh learns wakes the GitBranch, +// whose status.remote publishes it, or comes back for it when the publication floor lifts. // -// status.remote is SAMPLED: proving where the branch is and publishing it are two different -// rates, and nothing in the data plane wakes a target because a branch moved. Our OWN push is -// published once RemotePublicationInterval has passed since the last write, which was the unborn -// branch moments before; a minute, not a tick. That is the contract, not slack: see +// status.remote is on the GitBranch, and written at most once a minute: our OWN push is published +// once RemotePublicationInterval has passed since the last write, which was the unborn branch +// moments before; a minute, not a tick. That is the contract, not slack: see // docs/spec/status-conditions-guide.md. // // No reconcile-request annotation is used anywhere in this spec: an operator asking by hand is @@ -48,6 +46,7 @@ var _ = Describe("Manager Remote Refresh", Label("manager", "refresh"), Ordered, destName = "refresh-dest" ruleName = "refresh-rule" gitPath = "apps/refresh" + branchName = gitBranchName(providerName, "main") ) BeforeAll(func() { @@ -100,17 +99,17 @@ var _ = Describe("Manager Remote Refresh", Label("manager", "refresh"), Ordered, // The push it just made IS an observation of the remote, made on the connection the push // was opening anyway. That is the whole of decision 2, and it is visible here: status // names the commit the server accepted, and says a Push proved it. - By("status.remote names the commit our own push established") + By("the GitBranch's status.remote names the commit our own push established") var pushedCommit string Eventually(func(g Gomega) { - pushedCommit = gitTargetRemoteField(g, destName, testNs, "commit") + pushedCommit = gitBranchRemoteField(g, branchName, testNs, "commit") g.Expect(pushedCommit).NotTo(BeEmpty(), "a target that has pushed knows where its branch is") - g.Expect(gitTargetRemoteField(g, destName, testNs, "verifiedBy")).To(Equal("Push")) + g.Expect(gitBranchRemoteField(g, branchName, testNs, "verifiedBy")).To(Equal("Push")) g.Expect(pushedCommit).To(Equal(remoteHeadSHA(g, repo.CheckoutDir))) }, 3*time.Minute, 5*time.Second).Should(Succeed(), "published when the floor lifts, well before a steady tick") - verifiedBefore := gitTargetRemoteField(Default, destName, testNs, "lastVerifiedAt") + verifiedBefore := gitBranchRemoteField(Default, branchName, testNs, "lastVerifiedAt") Expect(verifiedBefore).NotTo(BeEmpty(), "the observation is dated, or nothing can report its own staleness") commitsBefore := remoteCommitCount(repo.CheckoutDir) @@ -126,11 +125,11 @@ var _ = Describe("Manager Remote Refresh", Label("manager", "refresh"), Ordered, // advertisement, and what it saw reaches status. By("the idle target notices, with nobody asking it to") Eventually(func(g Gomega) { - g.Expect(gitTargetRemoteField(g, destName, testNs, "commit")).To(Equal(movedCommit), + g.Expect(gitBranchRemoteField(g, branchName, testNs, "commit")).To(Equal(movedCommit), "an idle target must follow the branch somebody else moved") - g.Expect(gitTargetRemoteField(g, destName, testNs, "verifiedBy")).To(Equal("Fetch"), + g.Expect(gitBranchRemoteField(g, branchName, testNs, "verifiedBy")).To(Equal("Fetch"), "we went and looked, and that is how a foreign push is read off kubectl") - g.Expect(gitTargetRemoteField(g, destName, testNs, "lastVerifiedAt")).NotTo(Equal(verifiedBefore), + g.Expect(gitBranchRemoteField(g, branchName, testNs, "lastVerifiedAt")).NotTo(Equal(verifiedBefore), "the clock has to move with the fact it dates") }, 7*time.Minute, 5*time.Second).Should(Succeed()) @@ -157,9 +156,12 @@ var _ = Describe("Manager Remote Refresh", Label("manager", "refresh"), Ordered, }) }) -// gitTargetRemoteField reads one field of status.remote. -func gitTargetRemoteField(g Gomega, name, namespace, field string) string { - return gitTargetField(g, name, namespace, fmt.Sprintf("{.status.remote.%s}", field)) +// gitBranchRemoteField reads one field of a GitBranch's status.remote. +func gitBranchRemoteField(g Gomega, name, namespace, field string) string { + out, err := kubectlRunInNamespace(namespace, "get", "gitbranch", name, + "-o", fmt.Sprintf("jsonpath={.status.remote.%s}", field)) + g.Expect(err).NotTo(HaveOccurred(), "failed to read status.remote.%s of GitBranch %s", field, name) + return strings.TrimSpace(out) } func gitTargetField(g Gomega, name, namespace, jsonPath string) string { diff --git a/test/e2e/setup/demo-only/podinfos-intent/2-reverse-gitops.yaml b/test/e2e/setup/demo-only/podinfos-intent/2-reverse-gitops.yaml index efe92517..7f0aa1f0 100644 --- a/test/e2e/setup/demo-only/podinfos-intent/2-reverse-gitops.yaml +++ b/test/e2e/setup/demo-only/podinfos-intent/2-reverse-gitops.yaml @@ -11,13 +11,21 @@ spec: name: git-creds-demo --- apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: demo-podinfos + name: demo-nl-stuff spec: gitProviderRef: name: demo branch: nl-stuff +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: demo-podinfos +spec: + gitBranchRef: + name: demo-nl-stuff path: podinfos commit: window: diff --git a/test/e2e/setup/demo-only/voter-gitops/test/coffeeconfig-reverse-gitops.yaml b/test/e2e/setup/demo-only/voter-gitops/test/coffeeconfig-reverse-gitops.yaml index 63745d1c..dbe24cef 100644 --- a/test/e2e/setup/demo-only/voter-gitops/test/coffeeconfig-reverse-gitops.yaml +++ b/test/e2e/setup/demo-only/voter-gitops/test/coffeeconfig-reverse-gitops.yaml @@ -11,13 +11,21 @@ spec: name: git-creds-demo --- apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: demo-coffeeconfig + name: demo-coffeeconfig-demo-test spec: gitProviderRef: name: demo-coffeeconfig branch: demo-test +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: demo-coffeeconfig +spec: + gitBranchRef: + name: demo-coffeeconfig-demo-test path: voter-coffee commit: window: diff --git a/test/e2e/shared_gitbranch_e2e_test.go b/test/e2e/shared_gitbranch_e2e_test.go new file mode 100644 index 00000000..e09c379a --- /dev/null +++ b/test/e2e/shared_gitbranch_e2e_test.go @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: Apache-2.0 + +package e2e + +import ( + "fmt" + "os" + "path" + "path/filepath" + "strings" + "time" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +// One GitBranch per repository branch, shared across namespaces (docs/architecture.md, +// "GitBranch"). A GitBranch claims its branch once its GitProvider's credential opens a push +// session, and holds it for every GitTarget that references it: those in other namespaces are +// admitted by the GitProvider's spec.accessFrom and write with its credential, so the tenant +// namespace holds no copy of it. A second GitBranch for the same repository branch, however it +// spells the repository, holds nothing and reports BranchConflict; one with a bad credential +// claims nothing at all, so creating it first blocks nobody. +var _ = Describe("Manager shared GitBranch", Label("manager"), Ordered, func() { + const ( + providerName = "shared-monorepo" + branchName = "monorepo-main" + ) + + var ( + platformNs, tenantNs, outsiderNs string + repo *RepoArtifacts + ) + + BeforeAll(func() { + platformNs = testNamespaceFor("manager-shared-platform") + tenantNs = testNamespaceFor("manager-shared-tenant") + outsiderNs = testNamespaceFor("manager-shared-outsider") + for _, ns := range []string{platformNs, tenantNs, outsiderNs} { + _, _ = kubectlRun("create", "namespace", ns) // idempotent; ignore AlreadyExists + } + + By("setting up the repository, with its credential in the platform namespace only") + repo = SetupRepo(resolveE2EContext(), platformNs, fmt.Sprintf("e2e-manager-shared-%d", GinkgoRandomSeed())) + _, err := kubectlRunInNamespace(platformNs, "apply", "-f", repo.SecretsYAML) + Expect(err).NotTo(HaveOccurred(), "failed to apply git secrets to the platform namespace") + + By("creating the GitProvider, admitting the tenant namespace, and its GitBranch") + applySharedGitProvider(providerName, platformNs, repo.RepoURLHTTP, repo.GitSecretHTTP, tenantNs) + verifyResourceStatus("gitprovider", providerName, platformNs, + "True", "Succeeded", "Repository connectivity validated") + applySharedGitBranch(branchName, platformNs, providerName, "main") + verifyResourceStatus("gitbranch", branchName, platformNs, "True", "Succeeded", "holds") + }) + + AfterAll(func() { + for _, ns := range []string{tenantNs, outsiderNs, platformNs} { + cleanupNamespace(ns) + } + }) + + SetDefaultEventuallyTimeout(60 * time.Second) + SetDefaultEventuallyPollingInterval(2 * time.Second) + + It("mirrors GitTargets in two namespaces through one GitBranch, without copying the Secret", func() { + applySharedGitTarget("platform-apps", platformNs, branchName, "", "platform/apps") + applySharedGitTarget("apps", tenantNs, branchName, platformNs, "tenant/apps") + verifyResourceCondition("gittarget", "apps", tenantNs, "Validated", "True", "Succeeded", "") + verifyResourceCondition("gittarget", "apps", tenantNs, "GitBranchReady", "True", "Succeeded", "") + + out, err := kubectlRunInNamespace(tenantNs, "get", "secrets", "-o", "name") + Expect(err).NotTo(HaveOccurred()) + Expect(out).NotTo(ContainSubstring(repo.GitSecretHTTP), "the tenant namespace holds no copy of the credential") + + applyIsolationWatchRule("platform-configmaps", platformNs, "platform-apps", `"configmaps"`) + applyIsolationWatchRule("apps-configmaps", tenantNs, "apps", `"configmaps"`) + waitForStreamsRunning("platform-apps", platformNs) + waitForStreamsRunning("apps", tenantNs) + + By("creating a ConfigMap in each namespace") + applyIsolationConfigMap("shared-cm", platformNs) + applyIsolationConfigMap("shared-cm", tenantNs) + + files := []string{ + filepath.Join(repo.CheckoutDir, path.Join("platform/apps", platformNs, "configmaps", "shared-cm.yaml")), + filepath.Join(repo.CheckoutDir, path.Join("tenant/apps", tenantNs, "configmaps", "shared-cm.yaml")), + } + Eventually(func(g Gomega) { + pullLatestRepoState(g, repo.CheckoutDir) + for _, file := range files { + _, err := os.Stat(file) + g.Expect(err).NotTo(HaveOccurred(), "both folders are written through the one GitBranch") + } + }).Should(Succeed()) + + Eventually(func(g Gomega) { + out, err := kubectlRunInNamespace(platformNs, "get", "gitbranch", branchName, + "-o", "jsonpath={.status.gitTargetCount}") + g.Expect(err).NotTo(HaveOccurred()) + g.Expect(strings.TrimSpace(out)).To(Equal("2")) + }).Should(Succeed()) + }) + + It("refuses a namespace the GitBranch's GitProvider does not admit", func() { + applySharedGitTarget("intruder", outsiderNs, branchName, platformNs, "outsider") + verifyResourceCondition("gittarget", "intruder", outsiderNs, + "Validated", "False", "NamespaceNotAuthorized", "spec.accessFrom") + }) + + It("holds the branch against a second GitBranch, even through another GitProvider", func() { + By("giving the tenant a GitProvider of its own, with a valid credential, for the same repository") + _, err := kubectlRunInNamespace(tenantNs, "apply", "-f", repo.SecretsYAML) + Expect(err).NotTo(HaveOccurred()) + applySharedGitProvider("own-monorepo", tenantNs, repo.RepoURLHTTP, repo.GitSecretHTTP, "") + applySharedGitBranch("own-main", tenantNs, "own-monorepo", "main") + + verifyResourceCondition("gitbranch", "own-main", tenantNs, + "Stalled", "True", "BranchConflict", fmt.Sprintf("%s/%s", platformNs, branchName)) + verifyResourceStatus("gitbranch", branchName, platformNs, "True", "Succeeded", "holds") + + applySharedGitTarget("own", tenantNs, "own-main", "", "tenant/own") + verifyResourceCondition("gittarget", "own", tenantNs, + "GitBranchReady", "False", "GitBranchNotReady", "does not hold its branch") + }) + + It("does not let a GitBranch with a bad credential, created first, block a valid one", func() { + By("creating a GitBranch for the release branch whose GitProvider's credential is wrong") + _, err := kubectlRunInNamespace(outsiderNs, "apply", "-f", repo.SecretsYAML) + Expect(err).NotTo(HaveOccurred()) + applySharedGitProvider("squat", outsiderNs, repo.RepoURLHTTP, repo.GitSecretInvalid, "") + applySharedGitBranch("squat-release", outsiderNs, "squat", "release") + verifyResourceStatus("gitbranch", "squat-release", outsiderNs, "False", "PushSessionFailed", "") + + By("creating a valid GitBranch for the same branch afterwards") + applySharedGitBranch("monorepo-release", platformNs, providerName, "release") + verifyResourceStatus("gitbranch", "monorepo-release", platformNs, "True", "Succeeded", "holds") + }) + + It("lets a deleted GitBranch go, and leaves its GitTargets reporting it missing", func() { + _, err := kubectlRunInNamespace(platformNs, "delete", "gitbranch", "monorepo-release", "--wait=true", + "--timeout=60s") + Expect(err).NotTo(HaveOccurred(), "the finalizer gives the branch up once its worker has stopped") + + By("the squatter, still unproven, does not inherit the branch") + verifyResourceStatus("gitbranch", "squat-release", outsiderNs, "False", "PushSessionFailed", "") + + _, err = kubectlRunInNamespace(platformNs, "delete", "gitbranch", branchName, "--wait=true", "--timeout=120s") + Expect(err).NotTo(HaveOccurred()) + verifyResourceCondition("gittarget", "platform-apps", platformNs, + "Validated", "False", "GitBranchNotFound", "not found") + Eventually(func(g Gomega) { + pullLatestRepoState(g, repo.CheckoutDir) + g.Expect(remoteHeadSHA(g, repo.CheckoutDir)).NotTo(BeEmpty(), "the remote branch is never deleted") + }).Should(Succeed()) + }) +}) + +// applySharedGitProvider applies a GitProvider for url with secret, admitting accessFrom when set. +func applySharedGitProvider(name, namespace, url, secret, accessFrom string) { + By(fmt.Sprintf("creating GitProvider '%s/%s'", namespace, name)) + access := "" + if accessFrom != "" { + access = fmt.Sprintf("\n accessFrom:\n names: [%s]", accessFrom) + } + _, err := kubectlRunWithStdin("", fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: GitProvider +metadata: + name: %s + namespace: %s +spec: + url: %s + allowedBranches: ["main", "release"] + secretRef: + name: %s%s +`, name, namespace, url, secret, access), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred(), "failed to apply GitProvider %s/%s", namespace, name) +} + +// applySharedGitBranch applies a GitBranch writing branch through the GitProvider provider. +func applySharedGitBranch(name, namespace, provider, branch string) { + By(fmt.Sprintf("creating GitBranch '%s/%s' for branch '%s' through GitProvider '%s'", + namespace, name, branch, provider)) + _, err := kubectlRunWithStdin("", fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: %s + namespace: %s +spec: + gitProviderRef: + name: %s + branch: %s +`, name, namespace, provider, branch), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred(), "failed to apply GitBranch %s/%s", namespace, name) +} + +// applySharedGitTarget applies a GitTarget writing path through the GitBranch branchNS/branchName; +// an empty branchNS names its own namespace. +func applySharedGitTarget(name, namespace, branchName, branchNS, targetPath string) { + By(fmt.Sprintf("creating GitTarget '%s/%s' on GitBranch '%s/%s'", namespace, name, branchNS, branchName)) + ref := "name: " + branchName + if branchNS != "" { + ref += "\n namespace: " + branchNS + } + _, err := kubectlRunWithStdin("", fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: %s + namespace: %s +spec: + gitBranchRef: + %s + path: %s + commit: + window: + idleTimeout: "%s" + maxDuration: "%s" +`, name, namespace, ref, targetPath, e2eCommitWindow, e2eCommitMaxDuration), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred(), "failed to apply GitTarget %s/%s", namespace, name) +} diff --git a/test/e2e/shared_gitprovider_e2e_test.go b/test/e2e/shared_gitprovider_e2e_test.go deleted file mode 100644 index f06308bf..00000000 --- a/test/e2e/shared_gitprovider_e2e_test.go +++ /dev/null @@ -1,139 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package e2e - -import ( - "fmt" - "os" - "path" - "path/filepath" - "time" - - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" -) - -// One GitProvider per repository branch, shared across namespaces (docs/architecture.md, -// "GitProvider"). A GitProvider admits other namespaces through spec.accessFrom, and their -// GitTargets name it with spec.gitProviderRef.namespace: they write with its credential, so the -// tenant namespace holds no copy of it. A namespace it does not admit is refused, and so is a -// GitTarget that writes the same repository branch through a GitProvider of its own: one branch -// worker writes the branch, with one credential. -var _ = Describe("Manager shared GitProvider", Label("manager"), Ordered, func() { - const providerName = "shared-monorepo" - - var ( - platformNs, tenantNs, outsiderNs string - repo *RepoArtifacts - ) - - BeforeAll(func() { - platformNs = testNamespaceFor("manager-shared-platform") - tenantNs = testNamespaceFor("manager-shared-tenant") - outsiderNs = testNamespaceFor("manager-shared-outsider") - for _, ns := range []string{platformNs, tenantNs, outsiderNs} { - _, _ = kubectlRun("create", "namespace", ns) // idempotent; ignore AlreadyExists - } - - By("setting up the repository, with its credential in the platform namespace only") - repo = SetupRepo(resolveE2EContext(), platformNs, fmt.Sprintf("e2e-manager-shared-%d", GinkgoRandomSeed())) - _, err := kubectlRunInNamespace(platformNs, "apply", "-f", repo.SecretsYAML) - Expect(err).NotTo(HaveOccurred(), "failed to apply git secrets to the platform namespace") - - By("creating the GitProvider, admitting the tenant namespace") - _, err = kubectlRunWithStdin("", fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: GitProvider -metadata: - name: %s - namespace: %s -spec: - url: %s - allowedBranches: ["main"] - secretRef: - name: %s - accessFrom: - names: [%s] -`, providerName, platformNs, repo.RepoURLHTTP, repo.GitSecretHTTP, tenantNs), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to apply the shared GitProvider") - verifyResourceStatus("gitprovider", providerName, platformNs, - "True", "Succeeded", "Repository connectivity validated") - }) - - AfterAll(func() { - for _, ns := range []string{tenantNs, outsiderNs, platformNs} { - cleanupNamespace(ns) - } - }) - - SetDefaultEventuallyTimeout(60 * time.Second) - SetDefaultEventuallyPollingInterval(2 * time.Second) - - It("mirrors a GitTarget in another namespace through the shared GitProvider", func() { - applySharedGitTarget("apps", tenantNs, providerName, platformNs, "tenant/apps") - verifyResourceCondition("gittarget", "apps", tenantNs, "Validated", "True", "Succeeded", "") - - applyIsolationWatchRule("apps-configmaps", tenantNs, "apps", `"configmaps"`) - waitForStreamsRunning("apps", tenantNs) - - By("creating a ConfigMap in the tenant namespace") - applyIsolationConfigMap("shared-cm", tenantNs) - - file := filepath.Join(repo.CheckoutDir, path.Join("tenant/apps", tenantNs, "configmaps", "shared-cm.yaml")) - Eventually(func(g Gomega) { - pullLatestRepoState(g, repo.CheckoutDir) - _, err := os.Stat(file) - g.Expect(err).NotTo(HaveOccurred(), "the ConfigMap is written with the shared GitProvider's credential") - }).Should(Succeed()) - }) - - It("refuses a namespace the GitProvider does not admit", func() { - applySharedGitTarget("intruder", outsiderNs, providerName, platformNs, "outsider") - verifyResourceCondition("gittarget", "intruder", outsiderNs, - "Validated", "False", "NamespaceNotAuthorized", "spec.accessFrom") - }) - - It("refuses a second GitProvider on the same repository branch", func() { - By("giving the tenant a GitProvider of its own for the same repository") - // No credential: which GitProvider owns the branch is decided from the URL alone. - _, err := kubectlRunWithStdin("", fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: GitProvider -metadata: - name: own-monorepo - namespace: %s -spec: - url: %s - allowedBranches: ["main"] -`, tenantNs, repo.RepoURLHTTP), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to apply the tenant's own GitProvider") - - applySharedGitTarget("own", tenantNs, "own-monorepo", "", "tenant/own") - verifyResourceCondition("gittarget", "own", tenantNs, - "Validated", "False", "TargetConflict", fmt.Sprintf("GitProvider '%s/%s'", platformNs, providerName)) - }) -}) - -// applySharedGitTarget applies a GitTarget writing path on main through the GitProvider -// providerNS/providerName; an empty providerNS names its own namespace. -func applySharedGitTarget(name, namespace, providerName, providerNS, targetPath string) { - By(fmt.Sprintf("creating GitTarget '%s/%s' through GitProvider '%s/%s'", namespace, name, providerNS, providerName)) - ref := "name: " + providerName - if providerNS != "" { - ref += "\n namespace: " + providerNS - } - _, err := kubectlRunWithStdin("", fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget -metadata: - name: %s - namespace: %s -spec: - gitProviderRef: - %s - branch: main - path: %s - commit: - window: - idleTimeout: "%s" - maxDuration: "%s" -`, name, namespace, ref, targetPath, e2eCommitWindow, e2eCommitMaxDuration), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to apply GitTarget %s/%s", namespace, name) -} diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index bd384d62..8d4b6110 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -197,22 +197,21 @@ func deleteClusterProvider(name string) { // //nolint:unparam // gitProvider is kept an explicit argument for readability; the corner uses one. func applyGitTargetWithClusterProvider(ns, name, gitProvider, path, clusterProvider string) (string, error) { - manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 + manifest := gitBranchManifest(ns, gitProvider) + fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 kind: GitTarget metadata: name: %s namespace: %s spec: - gitProviderRef: + gitBranchRef: name: %s - branch: main path: %s commit: window: idleTimeout: "0s" clusterProviderRef: name: %s -`, name, ns, gitProvider, path, clusterProvider) +`, name, ns, gitBranchName(gitProvider, "main"), path, clusterProvider) return kubectlRunWithStdin(ns, manifest, "apply", "-f", "-") } @@ -354,17 +353,16 @@ var _ = Describe("Manager source cluster / ClusterProvider attribution", Label(" // {name: default} and mirrors the operator's own cluster. It("treats an omitted clusterProviderRef as the default (local) cluster", func() { target := "sc-local-target" - manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 + manifest := gitBranchManifest(testNs, providerName) + fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 kind: GitTarget metadata: {name: %s, namespace: %s} spec: - gitProviderRef: {name: %s} - branch: main + gitBranchRef: {name: %s} path: clusters/local commit: window: idleTimeout: "0s" -`, target, testNs, providerName) +`, target, testNs, gitBranchName(providerName, "main")) _, err := kubectlRunWithStdin(testNs, manifest, "apply", "-f", "-") Expect(err).NotTo(HaveOccurred()) diff --git a/test/e2e/suspend_e2e_test.go b/test/e2e/suspend_e2e_test.go index 2d92595c..2bc749c3 100644 --- a/test/e2e/suspend_e2e_test.go +++ b/test/e2e/suspend_e2e_test.go @@ -207,21 +207,20 @@ var _ = Describe("Manager GitTarget suspend", Label("manager", "suspend"), Order // applySuspendGitTarget applies a GitTarget with spec.suspend set explicitly. func applySuspendGitTarget(name, namespace, providerName, targetPath string, suspend bool) { GinkgoHelper() - manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 + manifest := gitBranchManifest(namespace, providerName) + fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 kind: GitTarget metadata: name: %s namespace: %s spec: - gitProviderRef: + gitBranchRef: name: %s - branch: main path: %s commit: window: idleTimeout: "0s" suspend: %t -`, name, namespace, providerName, targetPath, suspend) +`, name, namespace, gitBranchName(providerName, "main"), targetPath, suspend) out, err := kubectlRunWithStdin(namespace, manifest, "apply", "-f", "-") Expect(err).NotTo(HaveOccurred(), "failed to apply GitTarget %q with suspend %t: %s", name, suspend, out) diff --git a/test/e2e/templates/gittarget.tmpl b/test/e2e/templates/gittarget.tmpl index 3a5fccc9..9c991be2 100644 --- a/test/e2e/templates/gittarget.tmpl +++ b/test/e2e/templates/gittarget.tmpl @@ -1,7 +1,7 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: {{ .Name }} + name: {{ .GitBranchName }} namespace: {{ .Namespace }} spec: gitProviderRef: @@ -10,6 +10,15 @@ spec: {{- if .ParentBranch }} parentBranch: {{ .ParentBranch }} {{- end }} +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: {{ .Name }} + namespace: {{ .Namespace }} +spec: + gitBranchRef: + name: {{ .GitBranchName }} path: {{ .Path }} commit: window: diff --git a/test/e2e/templates/manager/gittarget-prune.tmpl b/test/e2e/templates/manager/gittarget-prune.tmpl index ab696f56..6710059d 100644 --- a/test/e2e/templates/manager/gittarget-prune.tmpl +++ b/test/e2e/templates/manager/gittarget-prune.tmpl @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: {{ .Name }} + name: {{ .GitBranchName }} namespace: {{ .Namespace }} spec: gitProviderRef: name: {{ .ProviderName }} branch: {{ .Branch }} +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: {{ .Name }} + namespace: {{ .Namespace }} +spec: + gitBranchRef: + name: {{ .GitBranchName }} path: {{ .Path }} commit: window: diff --git a/test/fixtures/layout-corpus/shapes/1-flat-serialized/config/gittarget.yaml b/test/fixtures/layout-corpus/shapes/1-flat-serialized/config/gittarget.yaml index 00241d04..94855c4d 100644 --- a/test/fixtures/layout-corpus/shapes/1-flat-serialized/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/shapes/1-flat-serialized/config/gittarget.yaml @@ -1,14 +1,23 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: prod-mirror + name: mirror-repository-main namespace: homelab-config spec: gitProviderRef: name: mirror-repository + branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: prod-mirror + namespace: homelab-config +spec: + gitBranchRef: + name: mirror-repository-main clusterProviderRef: name: prod - branch: main path: mirror/prod # The WatchRule beside this file names source namespaces other than its own. The # ClusterProvider's allowCrossNamespace.sources is the only grant that needs, and the diff --git a/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget-second-namespace.yaml b/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget-second-namespace.yaml index 447612a4..eee2e2e2 100644 --- a/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget-second-namespace.yaml +++ b/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget-second-namespace.yaml @@ -1,3 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: artifacts-repository-main + namespace: shop +spec: + gitProviderRef: + name: artifacts-repository + branch: main +--- # The fence around "one namespace", as a fixture. Same folder and same flag as # gittarget.yaml; what differs is that TWO WatchRules reach it, from two source namespaces. # @@ -12,11 +22,10 @@ metadata: name: checkout-artifact namespace: shop spec: - gitProviderRef: - name: artifacts-repository + gitBranchRef: + name: artifacts-repository-main clusterProviderRef: name: apps - branch: main path: apps/checkout placement: default: "{name}.yaml" diff --git a/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget.yaml b/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget.yaml index 5bfded53..03390b44 100644 --- a/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/shapes/2-flat-namespace-free/config/gittarget.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout-artifact + name: artifacts-repository-main namespace: shop spec: gitProviderRef: name: artifacts-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout-artifact + namespace: shop +spec: + gitBranchRef: + name: artifacts-repository-main path: apps/checkout # One file per object, flat, and no {namespace} in the template: the folder is # single-namespace by construction. diff --git a/test/fixtures/layout-corpus/shapes/3-tree-serialized/config/gittarget.yaml b/test/fixtures/layout-corpus/shapes/3-tree-serialized/config/gittarget.yaml index f71024eb..7b0c22bf 100644 --- a/test/fixtures/layout-corpus/shapes/3-tree-serialized/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/shapes/3-tree-serialized/config/gittarget.yaml @@ -1,14 +1,23 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: home-cluster-state + name: mirror-repository-main namespace: homelab-config spec: gitProviderRef: name: mirror-repository + branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: home-cluster-state + namespace: homelab-config +spec: + gitBranchRef: + name: mirror-repository-main clusterProviderRef: name: home - branch: main path: clusters/home # The WatchRule beside this file names source namespaces other than its own. The # ClusterProvider's allowCrossNamespace.sources is the only grant that needs, and the diff --git a/test/fixtures/layout-corpus/shapes/4-tree-namespace-free/config/gittarget.yaml b/test/fixtures/layout-corpus/shapes/4-tree-namespace-free/config/gittarget.yaml index c537c6c7..8e6bb2b1 100644 --- a/test/fixtures/layout-corpus/shapes/4-tree-namespace-free/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/shapes/4-tree-namespace-free/config/gittarget.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout-artifact + name: artifacts-repository-main namespace: shop spec: gitProviderRef: name: artifacts-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout-artifact + namespace: shop +spec: + gitBranchRef: + name: artifacts-repository-main path: apps/checkout # A tree WITHOUT {namespace}: the subfolders group by type, not by namespace, # so the folder stays single-namespace. That is the precondition for the flag diff --git a/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget-empty-folder.yaml b/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget-empty-folder.yaml index 72c93e8a..1e700abb 100644 --- a/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget-empty-folder.yaml +++ b/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget-empty-folder.yaml @@ -1,3 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: app-repository-main + namespace: shop +spec: + gitProviderRef: + name: app-repository + branch: main +--- # The same folder, when it does not exist yet. This is the ONLY difference # between adopting a kustomize folder and creating one. apiVersion: configbutler.ai/v1alpha3 @@ -6,9 +16,8 @@ metadata: name: checkout namespace: shop spec: - gitProviderRef: - name: app-repository - branch: main + gitBranchRef: + name: app-repository-main path: apps/checkout # There is nothing to infer from, so both halves are declared. useKustomize # writes the root; serializeNamespace: false is honest BECAUSE the operator owns diff --git a/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget.yaml b/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget.yaml index 91ea47a9..0a141e41 100644 --- a/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/shapes/5-kustomize-single-folder/config/gittarget.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout + name: app-repository-main namespace: shop spec: gitProviderRef: name: app-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout + namespace: shop +spec: + gitBranchRef: + name: app-repository-main path: apps/checkout # Nothing else is declared, and that is the shape: the folder already has one # kustomization.yaml, so a new file is placed beside it, registered in its diff --git a/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-app-root.yaml b/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-app-root.yaml index 7763f965..1ca657f2 100644 --- a/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-app-root.yaml +++ b/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-app-root.yaml @@ -1,3 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: app-repository-main + namespace: shop-prod +spec: + gitProviderRef: + name: app-repository + branch: main +--- # The MISTAKE this shape exists to name, kept as a fixture because a set where every write # succeeds is advertising rather than specification. Same repository as gittarget-prod.yaml, # one field different: the path is the app root rather than a leaf overlay. @@ -13,7 +23,6 @@ metadata: name: checkout-app namespace: shop-prod spec: - gitProviderRef: - name: app-repository - branch: main + gitBranchRef: + name: app-repository-main path: apps/checkout diff --git a/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-prod.yaml b/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-prod.yaml index 879559c7..5dcba34b 100644 --- a/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-prod.yaml +++ b/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-prod.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout-prod + name: app-repository-main namespace: shop-prod spec: gitProviderRef: name: app-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout-prod + namespace: shop-prod +spec: + gitBranchRef: + name: app-repository-main # A LEAF overlay, not the app root. One target = one overlay = one environment = # one write partition. apps/checkout would cover four roots at once, which is # LayoutResolved=Ambiguous, and would make the shared base writable from prod. diff --git a/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-test.yaml b/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-test.yaml index 2e5ad526..f9e603ba 100644 --- a/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-test.yaml +++ b/test/fixtures/layout-corpus/shapes/6-kustomize-base-and-overlays/config/gittarget-test.yaml @@ -1,3 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: app-repository-main + namespace: shop-test +spec: + gitProviderRef: + name: app-repository + branch: main +--- # The second and third targets are the same object with the environment swapped. # Nothing is shared between them except the byType map, if one were declared. apiVersion: configbutler.ai/v1alpha3 @@ -6,7 +16,6 @@ metadata: name: checkout-test namespace: shop-test spec: - gitProviderRef: - name: app-repository - branch: main + gitBranchRef: + name: app-repository-main path: apps/checkout/overlays/test diff --git a/test/fixtures/layout-corpus/shapes/7-kustomize-layered/config/gittarget-prod.yaml b/test/fixtures/layout-corpus/shapes/7-kustomize-layered/config/gittarget-prod.yaml index f40558a7..0031c40f 100644 --- a/test/fixtures/layout-corpus/shapes/7-kustomize-layered/config/gittarget-prod.yaml +++ b/test/fixtures/layout-corpus/shapes/7-kustomize-layered/config/gittarget-prod.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout-prod + name: app-repository-main namespace: shop-prod spec: gitProviderRef: name: app-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout-prod + namespace: shop-prod +spec: + gitBranchRef: + name: app-repository-main # The leaf again, and here it is not a preference. envs/prod and envs/test both # include layers/observability, so every file in that layer has fan-in 2 and is # refused as a write target (L2). The leaf is the only folder in this tree with a diff --git a/test/fixtures/layout-corpus/shapes/8-base-owned-field-edit/config/gittarget-prod.yaml b/test/fixtures/layout-corpus/shapes/8-base-owned-field-edit/config/gittarget-prod.yaml index 7fa9bdb3..0dfdadcc 100644 --- a/test/fixtures/layout-corpus/shapes/8-base-owned-field-edit/config/gittarget-prod.yaml +++ b/test/fixtures/layout-corpus/shapes/8-base-owned-field-edit/config/gittarget-prod.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: checkout-prod + name: app-repository-main namespace: shop-prod spec: gitProviderRef: name: app-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: checkout-prod + namespace: shop-prod +spec: + gitBranchRef: + name: app-repository-main # The leaf overlay. The base one directory up is READ to render this folder and is # never written: read scope is wider than write scope, always. path: apps/checkout/overlays/prod diff --git a/test/fixtures/layout-corpus/specific-examples/homelab-argocd/config/gittarget.yaml b/test/fixtures/layout-corpus/specific-examples/homelab-argocd/config/gittarget.yaml index 66618705..dadfc192 100644 --- a/test/fixtures/layout-corpus/specific-examples/homelab-argocd/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/specific-examples/homelab-argocd/config/gittarget.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: argocd-applications + name: homelab-repository-main namespace: argocd spec: gitProviderRef: name: homelab-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: argocd-applications + namespace: argocd +spec: + gitBranchRef: + name: homelab-repository-main path: bootstrap/argocd-applications # Safe here only because the folder's kustomization.yaml supplies # namespace: argocd — and that file belongs to the repository owner, not to us. diff --git a/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget-media.yaml b/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget-media.yaml index 164f3aa6..ecc0f058 100644 --- a/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget-media.yaml +++ b/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget-media.yaml @@ -1,3 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitBranch +metadata: + name: homelab-repository-main + namespace: flux-system +spec: + gitProviderRef: + name: homelab-repository + branch: main +--- # The second target. HelmReleases are a different layer from the sources they pull # from, so they get their own folder and their own GitTarget rather than sharing one # with the sources — which is the shape Flux's documented repository structures exist @@ -8,8 +18,7 @@ metadata: name: flux-media namespace: flux-system spec: - gitProviderRef: - name: homelab-repository - branch: main + gitBranchRef: + name: homelab-repository-main path: apps/home/media serializeNamespace: false # the folder's own kustomization.yaml supplies it diff --git a/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget.yaml b/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget.yaml index 9c0b88ce..f480b594 100644 --- a/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget.yaml +++ b/test/fixtures/layout-corpus/specific-examples/homelab-flux/config/gittarget.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: flux-sources + name: homelab-repository-main namespace: flux-system spec: gitProviderRef: name: homelab-repository branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: flux-sources + namespace: flux-system +spec: + gitBranchRef: + name: homelab-repository-main # Not clusters/home/flux-system. That directory belongs to `flux bootstrap`, and a # GitTarget must not point at a path another controller writes. path: infrastructure/home/sources diff --git a/test/playground/config/gittarget.yaml b/test/playground/config/gittarget.yaml index 3abada2e..1ab20547 100644 --- a/test/playground/config/gittarget.yaml +++ b/test/playground/config/gittarget.yaml @@ -1,12 +1,21 @@ apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: GitBranch metadata: - name: playground-target + name: playground-provider-main namespace: tilt-playground spec: gitProviderRef: name: playground-provider branch: main +--- +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: playground-target + namespace: tilt-playground +spec: + gitBranchRef: + name: playground-provider-main # Keep playground writes under a folder. Use "." only when testing repo-root ownership. path: live-cluster # Every event is its own commit, so a playground change shows up in Git immediately. From a508b92340ec1b9f1e70f4e3a71134f34ee86272 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Wed, 21 Oct 2026 21:01:04 +0000 Subject: [PATCH 6/6] fix(controller): wake the next GitBranch when a holder is deleted A GitBranch stalled on BranchConflict is not reconciled again before its steady requeue, so when the holder's finalizer released the branch the next candidate held it in memory and reported BranchConflict for up to five minutes. The finalizer now wakes it. The e2e helpers label the GitBranches they create with their GitProvider, and deleting a GitProvider in a spec's cleanup deletes them, so a spec's GitBranch no longer holds the branch the next spec in the namespace writes. The plan no longer links to a page this branch does not carry. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../design/gitbranch-and-kubeprovider-plan.md | 2 +- internal/controller/gitbranch_controller.go | 9 +++++++- .../controller/gitbranch_controller_test.go | 3 +++ internal/git/worker_manager.go | 5 +++++ test/e2e/flux_applier_e2e_test.go | 2 +- test/e2e/flux_restart_recovery_e2e_test.go | 2 +- test/e2e/foreign_content_e2e_test.go | 2 +- test/e2e/gitprovider_validation_e2e_test.go | 10 ++++----- test/e2e/helmrelease_mirror_edit_e2e_test.go | 2 +- test/e2e/helpers.go | 22 ++++++++++++++++++- test/e2e/inplace_edit_e2e_test.go | 2 +- test/e2e/new_branch_parent_e2e_test.go | 4 ++-- test/e2e/new_file_placement_e2e_test.go | 2 +- test/e2e/refresh_remote_e2e_test.go | 2 +- test/e2e/refusal_reapply_e2e_test.go | 2 +- test/e2e/templates/gittarget.tmpl | 2 ++ .../templates/manager/gittarget-prune.tmpl | 2 ++ test/e2e/unsupported_folder_e2e_test.go | 2 +- test/e2e/values_file_acceptance_e2e_test.go | 2 +- 19 files changed, 59 insertions(+), 20 deletions(-) diff --git a/docs/design/gitbranch-and-kubeprovider-plan.md b/docs/design/gitbranch-and-kubeprovider-plan.md index d05ce1a1..c544e289 100644 --- a/docs/design/gitbranch-and-kubeprovider-plan.md +++ b/docs/design/gitbranch-and-kubeprovider-plan.md @@ -2,7 +2,7 @@ Status: **plan**, 2026-10-21. Nothing here is built. The current contract stays in [architecture.md](../architecture.md) and [configuration.md](../configuration.md) until each phase -ships. [ideal-shared-stuff.md](ideal-shared-stuff.md) is the analysis behind this plan; this page +ships. `ideal-shared-stuff.md` is the analysis behind this plan; this page records what we decided from it and in what order we build it. The plan breaks the API on purpose, more than once. There are no production users to carry across, diff --git a/internal/controller/gitbranch_controller.go b/internal/controller/gitbranch_controller.go index 5162e956..3532d462 100644 --- a/internal/controller/gitbranch_controller.go +++ b/internal/controller/gitbranch_controller.go @@ -344,8 +344,15 @@ func (r *GitBranchReconciler) finalize(ctx context.Context, branch *configbutler return nil } if r.WorkerManager != nil { - r.WorkerManager.Claims().Release(branch.UID) + claims := r.WorkerManager.Claims() + key, _, held := claims.HeldBy(client.ObjectKeyFromObject(branch)) + claims.Release(branch.UID) r.WorkerManager.RetireGitBranch(client.ObjectKeyFromObject(branch)) + // The next candidate holds the branch now. It may be a GitBranch stalled on BranchConflict, + // which nothing else would reconcile before its steady requeue. + if next, taken := claims.Holder(key); held && taken { + r.WorkerManager.WakeGitBranch(next.GitBranch) + } } r.proofsMu.Lock() delete(r.proofs, branch.UID) diff --git a/internal/controller/gitbranch_controller_test.go b/internal/controller/gitbranch_controller_test.go index cc2d6765..03c77af9 100644 --- a/internal/controller/gitbranch_controller_test.go +++ b/internal/controller/gitbranch_controller_test.go @@ -274,6 +274,9 @@ func TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch(t *testing.T) { assert.False(t, live, "the worker stopped before the finalizer went") require.Error(t, f.client.Get(context.Background(), holder, stored), "the finalizer is gone, and so is the object") + require.Len(t, f.workers.GitBranchEvents(), 1, "the next candidate is woken, not left stalled") + woken := (<-f.workers.GitBranchEvents()).Object + assert.Equal(t, "team-b/main", woken.GetNamespace()+"/"+woken.GetName()) next := f.reconcile(t, "team-b", "main") assert.Equal(t, metav1.ConditionTrue, readyOf(next).Status, readyOf(next).Message) } diff --git a/internal/git/worker_manager.go b/internal/git/worker_manager.go index 90e7670a..7084ac62 100644 --- a/internal/git/worker_manager.go +++ b/internal/git/worker_manager.go @@ -560,6 +560,11 @@ func (m *WorkerManager) notifyGitBranchLocked(gitBranch k8stypes.NamespacedName) } } +// WakeGitBranch asks the GitBranch reconciler to look at gitBranch now: a branch claim moved to it. +func (m *WorkerManager) WakeGitBranch(gitBranch k8stypes.NamespacedName) { + m.notifyGitBranchLocked(gitBranch) +} + // GitBranchEvents carries a GitBranch whose branch worker proved something new about the remote or // whose publication report changed, so the GitBranch controller publishes status.remote within one // reconcile instead of at its next requeue. Nil for a manager not built by NewWorkerManager. diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index fa5c0911..db615bd9 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -68,7 +68,7 @@ var _ = Describe("Manager Flux Applier", Label("manager", "flux-applier"), Order cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) cleanupNamespacedResource(testNs, fluxTriggerResource, fluxName) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespacedResource(testNs, "secret", receiverSecretName) cleanupNamespacedResource("flux-system", "kustomization", fluxName) cleanupNamespacedResource("flux-system", "gitrepository", fluxName) diff --git a/test/e2e/flux_restart_recovery_e2e_test.go b/test/e2e/flux_restart_recovery_e2e_test.go index f8d4b186..e1c625ca 100644 --- a/test/e2e/flux_restart_recovery_e2e_test.go +++ b/test/e2e/flux_restart_recovery_e2e_test.go @@ -81,7 +81,7 @@ var _ = Describe("Flux Restart Recovery", Label("flux-restart-recovery"), Serial cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) cleanupNamespacedResource(testNs, fluxTriggerResource, fluxName) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespacedResource("flux-system", "kustomization", fluxName) cleanupNamespacedResource("flux-system", "gitrepository", fluxName) cleanupNamespacedResource("flux-system", "secret", fluxName+"-git") diff --git a/test/e2e/foreign_content_e2e_test.go b/test/e2e/foreign_content_e2e_test.go index 7ead1b77..2da78cf0 100644 --- a/test/e2e/foreign_content_e2e_test.go +++ b/test/e2e/foreign_content_e2e_test.go @@ -57,7 +57,7 @@ var _ = Describe("Manager Foreign Content Refusal", Label("manager", "foreign-co AfterAll(func() { cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/gitprovider_validation_e2e_test.go b/test/e2e/gitprovider_validation_e2e_test.go index 1d165e80..5c2ddc24 100644 --- a/test/e2e/gitprovider_validation_e2e_test.go +++ b/test/e2e/gitprovider_validation_e2e_test.go @@ -65,7 +65,7 @@ var _ = Describe("Manager GitProvider Validation", Label("manager"), Ordered, fu By("showing final controller logs") showControllerLogs("after status verification") - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", gitProviderName, "--ignore-not-found=true") + deleteGitProvider(testNs, gitProviderName) }) It("should handle GitProvider with invalid credentials", func() { @@ -77,7 +77,7 @@ var _ = Describe("Manager GitProvider Validation", Label("manager"), Ordered, fu validationRepo.RepoURLHTTP, ) verifyResourceStatus("gitprovider", gitProviderName, testNs, "False", "ConnectionFailed", "") - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", gitProviderName, "--ignore-not-found=true") + deleteGitProvider(testNs, gitProviderName) }) It("should refuse a GitBranch whose branch the GitProvider does not allow", func() { @@ -101,10 +101,10 @@ var _ = Describe("Manager GitProvider Validation", Label("manager"), Ordered, fu By("verifying the GitBranch fails branch validation") verifyResourceStatus("gitbranch", gitBranchName(gitProviderName, "different-branch"), testNs, "False", "BranchNotAllowed", "allowedBranches") - verifyResourceStatus("gittarget", destName, testNs, "False", "GitBranchNotReady", "BranchNotAllowed") + verifyResourceStatus("gittarget", destName, testNs, "False", "GitBranchNotReady", "allowedBranches") cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", gitProviderName, "--ignore-not-found=true") + deleteGitProvider(testNs, gitProviderName) }) It("should validate GitProvider with SSH authentication", func() { @@ -143,6 +143,6 @@ var _ = Describe("Manager GitProvider Validation", Label("manager"), Ordered, fu By("✅ Final SSH test logs") showControllerLogs("SSH test completion") - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", gitProviderName, "--ignore-not-found=true") + deleteGitProvider(testNs, gitProviderName) }) }) diff --git a/test/e2e/helmrelease_mirror_edit_e2e_test.go b/test/e2e/helmrelease_mirror_edit_e2e_test.go index e3a183e2..8971d4cb 100644 --- a/test/e2e/helmrelease_mirror_edit_e2e_test.go +++ b/test/e2e/helmrelease_mirror_edit_e2e_test.go @@ -67,7 +67,7 @@ var _ = Describe("Manager Higher-Level KRM (HelmRelease)", _, _ = kubectlRunInNamespace(testNs, "delete", "helmrelease", hrName, "--ignore-not-found=true") cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/helpers.go b/test/e2e/helpers.go index 27297481..f02466d2 100644 --- a/test/e2e/helpers.go +++ b/test/e2e/helpers.go @@ -204,6 +204,20 @@ func gitBranchName(providerName, branch string) string { return providerName + "-" + strings.ReplaceAll(strings.ToLower(branch), "/", "-") } +// gitBranchProviderLabel marks the GitBranches the helpers create with the GitProvider they name, so +// deleting that GitProvider in a spec's cleanup deletes them too (deleteGitProvider). A GitBranch +// left behind keeps holding its repository branch, and the next spec's GitBranch on that branch +// would report BranchConflict. +const gitBranchProviderLabel = "e2e.configbutler.ai/git-provider" + +// deleteGitProvider deletes a GitProvider and the GitBranches the helpers created for it, waiting +// for the GitBranches' finalizers so their branch is free when the next spec claims it. +func deleteGitProvider(namespace, name string) { + _, _ = kubectlRunInNamespace(namespace, "delete", "gitbranch", "-l", gitBranchProviderLabel+"="+name, + "--ignore-not-found=true", "--wait=true", "--timeout=60s") + _, _ = kubectlRunInNamespace(namespace, "delete", "gitprovider", name, "--ignore-not-found=true") +} + // gitBranchManifest is the GitBranch gitBranchName(providerName, "main") in namespace, as a YAML // document followed by a separator, for a spec that applies its GitTarget as an inline manifest. func gitBranchManifest(namespace, providerName string) string { @@ -213,12 +227,14 @@ kind: GitBranch metadata: name: %s namespace: %s + labels: + %s: %s spec: gitProviderRef: name: %s branch: %s --- -`, gitBranchName(providerName, branch), namespace, providerName, branch) +`, gitBranchName(providerName, branch), namespace, gitBranchProviderLabel, providerName, providerName, branch) } // createGitTarget creates a GitTarget on the GitBranch of a GitProvider's branch, creating that @@ -393,6 +409,10 @@ func cleanupNamespacedResource(namespace, resource, name string) { if skipCleanupBecauseResourcesArePreserved(fmt.Sprintf("%s %s/%s", resource, namespace, name), namespace) { return } + if resource == "gitprovider" { + deleteGitProvider(namespace, name) + return + } _, _ = kubectlRunInNamespace(namespace, "delete", resource, name, "--ignore-not-found=true") } diff --git a/test/e2e/inplace_edit_e2e_test.go b/test/e2e/inplace_edit_e2e_test.go index 6a5fe34f..6edf5dbb 100644 --- a/test/e2e/inplace_edit_e2e_test.go +++ b/test/e2e/inplace_edit_e2e_test.go @@ -188,7 +188,7 @@ var _ = Describe( } cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/new_branch_parent_e2e_test.go b/test/e2e/new_branch_parent_e2e_test.go index 0ab9097a..9b03bec0 100644 --- a/test/e2e/new_branch_parent_e2e_test.go +++ b/test/e2e/new_branch_parent_e2e_test.go @@ -140,7 +140,7 @@ func describeMissingParentRecoverySpec() { AfterAll(func() { cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) @@ -254,7 +254,7 @@ data: AfterAll(func() { cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/new_file_placement_e2e_test.go b/test/e2e/new_file_placement_e2e_test.go index dbdbb3ae..99b9eb09 100644 --- a/test/e2e/new_file_placement_e2e_test.go +++ b/test/e2e/new_file_placement_e2e_test.go @@ -58,7 +58,7 @@ var _ = Describe("Manager New-File Placement", Label("manager", "new-file-placem _, _ = kubectlRunInNamespace(testNs, "delete", "configmap", newConfigMap, "--ignore-not-found=true") cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/refresh_remote_e2e_test.go b/test/e2e/refresh_remote_e2e_test.go index 3ae8bed2..e89c33e7 100644 --- a/test/e2e/refresh_remote_e2e_test.go +++ b/test/e2e/refresh_remote_e2e_test.go @@ -69,7 +69,7 @@ var _ = Describe("Manager Remote Refresh", Label("manager", "refresh"), Ordered, AfterAll(func() { cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/refusal_reapply_e2e_test.go b/test/e2e/refusal_reapply_e2e_test.go index b4e61157..67f374bb 100644 --- a/test/e2e/refusal_reapply_e2e_test.go +++ b/test/e2e/refusal_reapply_e2e_test.go @@ -67,7 +67,7 @@ var _ = Describe("Manager Refusal Reapply", Label("manager", "refusal-reapply"), AfterAll(func() { cleanupWatchRule(ruleName, testNs) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespacedResource("flux-system", "kustomization", fluxName) cleanupNamespacedResource("flux-system", "gitrepository", fluxName) cleanupNamespacedResource("flux-system", "secret", fluxSecretName) diff --git a/test/e2e/templates/gittarget.tmpl b/test/e2e/templates/gittarget.tmpl index 9c991be2..fefb9023 100644 --- a/test/e2e/templates/gittarget.tmpl +++ b/test/e2e/templates/gittarget.tmpl @@ -3,6 +3,8 @@ kind: GitBranch metadata: name: {{ .GitBranchName }} namespace: {{ .Namespace }} + labels: + e2e.configbutler.ai/git-provider: {{ .ProviderName }} spec: gitProviderRef: name: {{ .ProviderName }} diff --git a/test/e2e/templates/manager/gittarget-prune.tmpl b/test/e2e/templates/manager/gittarget-prune.tmpl index 6710059d..3671d471 100644 --- a/test/e2e/templates/manager/gittarget-prune.tmpl +++ b/test/e2e/templates/manager/gittarget-prune.tmpl @@ -3,6 +3,8 @@ kind: GitBranch metadata: name: {{ .GitBranchName }} namespace: {{ .Namespace }} + labels: + e2e.configbutler.ai/git-provider: {{ .ProviderName }} spec: gitProviderRef: name: {{ .ProviderName }} diff --git a/test/e2e/unsupported_folder_e2e_test.go b/test/e2e/unsupported_folder_e2e_test.go index 3e61a0d6..b5e43f8c 100644 --- a/test/e2e/unsupported_folder_e2e_test.go +++ b/test/e2e/unsupported_folder_e2e_test.go @@ -57,7 +57,7 @@ var _ = Describe("Manager Unsupported Folder Refusal", Label("manager", "unsuppo cleanupWatchRule(ruleName, testNs) cleanupClusterWatchRule(clusterRuleName) cleanupGitTarget(destName, testNs) - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) }) diff --git a/test/e2e/values_file_acceptance_e2e_test.go b/test/e2e/values_file_acceptance_e2e_test.go index ad0fee59..81ef4927 100644 --- a/test/e2e/values_file_acceptance_e2e_test.go +++ b/test/e2e/values_file_acceptance_e2e_test.go @@ -100,7 +100,7 @@ func provisionValuesFileEnv(caseName string) valuesFileEnv { verifyResourceStatus("gitprovider", providerName, testNs, "True", "Succeeded", "") DeferCleanup(func() { - _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + deleteGitProvider(testNs, providerName) cleanupNamespace(testNs) })