diff --git a/docs/developer-guide/01-mental-model.md b/docs/developer-guide/01-mental-model.md new file mode 100644 index 0000000..4bed4ac --- /dev/null +++ b/docs/developer-guide/01-mental-model.md @@ -0,0 +1,40 @@ +# Mental model + +The lifecycle the library runs for you, and the operations you implement to plug into it. + +## The managed-resource lifecycle + +You hand the reconciler a typed custom resource and an implementation of a few operations. On every reconcile the framework asks your code to **observe** the external world, then — based on what you report — **creates**, **updates**, **deletes**, or does nothing, persisting conditions and managing the finalizer along the way. + +```mermaid +stateDiagram-v2 + [*] --> Get + Get --> Paused: paused + Get --> Connect: active + Connect --> Observe + Observe --> Create: missing + Observe --> Update: drifted + Observe --> Delete: being deleted + Observe --> UpToDate: matches + Create --> [*] + Update --> [*] + Delete --> [*] + UpToDate --> [*] + Paused --> [*] +``` + +You implement the boxes; the library implements everything else — getting the object, the pause check, finalizer add/remove, the safety steps around creation, persisting conditions and status, and requeue timing. The full sequence is in [`02-architecture.md`](./02-architecture.md). + +## The operations you implement + +- **Connect** — build whatever talks to the external system for *this* resource (an API client, a Kubernetes client, …), typically reading credentials from a referenced secret. It returns the client the four operations below will use. +- **Observe** — look at the external world and report two things: whether the resource **exists**, and whether it is **up to date**. It must not change the external resource. It may also report that it filled in some defaults (so the framework persists them) and, for debugging, a description of the drift it found. +- **Create / Update / Delete** — make the external world match the desired state. + +The contract that keeps this safe: + +- **Idempotent and non-blocking.** The framework can re-run any operation, so create must tolerate an already-existing resource and delete a missing one. +- **Observe is read-only on the outside world.** It can adjust the managed object's status in memory (the loop persists it), but it must not mutate the external resource. +- **The framework persists conditions and status, not your code.** You report observations and errors; the loop writes the `Ready`/`Synced` conditions and updates status. + +> What "exists" and "up to date" mean is entirely yours to define — that's where a provider encodes how it compares desired state against the external system. diff --git a/docs/developer-guide/02-architecture.md b/docs/developer-guide/02-architecture.md new file mode 100644 index 0000000..d42bbef --- /dev/null +++ b/docs/developer-guide/02-architecture.md @@ -0,0 +1,56 @@ +# Architecture + +What the library provides, and how it sequences a reconcile. + +## What's in the box + +- **The reconciler** — the heart of the library: the loop, and the small set of operations a provider plugs into it. +- **The resource model** — the shape a managed resource must have (its conditions live on its status, its list can enumerate its items), finalizer handling, and helpers to read credentials from secrets and config maps. +- **Standard conditions and status types** — the `Ready` and `Synced` conditions and their reasons. +- **Cross-cutting helpers** — rate limiters, controller options, a logging abstraction, optional telemetry, an errors helper, and a test toolkit (a mock client and comparison helpers). + +The event recorder comes from the shared Krateo libraries, not from this repo. + +## How the reconcile loop is sequenced + +The loop runs in a fixed order on every reconcile. The order is the important part — read it top to bottom: + +1. **Set timeouts** for the reconcile and for the external calls. +2. **Get the managed resource**; a not-found is treated as "nothing to do". +3. **Pause** — if the resource is paused, record a paused condition and stop. +4. **Orphan fast-delete** — if the resource is being deleted and its policy says not to delete the external resource, just drop the finalizer and stop. +5. **Create-incomplete guard** — if a previous create may have started without confirming, refuse to proceed (this prevents leaking an external resource). +6. **Connect**, then **Observe**. +7. **Creation grace period** — if the resource doesn't exist yet but was created very recently, wait and retry (tolerates eventually-consistent backends). +8. **Delete path** — if being deleted and the external resource exists, delete it, then drop the finalizer once it's gone. +9. **Add the finalizer** for live resources. +10. **Create path** — mark the create as pending, create, then record success or failure. +11. **Persist any defaults** Observe filled in. +12. **Up to date** — record success and requeue after the poll interval. +13. **Update path** — update, record success, requeue. + +Two behaviors run through nearly every step: conditions and status are **persisted after each branch**, and a **conflict is treated as a requeue, not an error** (so concurrent writers simply retry). Mirror that conflict handling in any custom finalizer or updater you add. + +> Constructing the reconciler requires the resource's kind to be registered in the manager's scheme — if it isn't, construction fails immediately rather than misbehaving later. Register your scheme before wiring the reconciler. + +## Management and deletion policies + +Two annotations on the managed resource let an operator narrow what the loop is allowed to do. They're set by whoever applies the resource, but they directly decide **which of your operations the loop actually calls** — so it matters when you're building a provider (don't assume your `Create` runs just because the resource is missing). + +**Management policy** — the annotation `krateo.io/management-policy` — gates the allowed actions: + +| Value | Observe | Create | Update | Delete | +| --- | :---: | :---: | :---: | :---: | +| `default` (when the annotation is absent) | ✓ | ✓ | ✓ | ✓ | +| `observe-create-update` | ✓ | ✓ | ✓ | — | +| `observe-delete` | ✓ | — | — | ✓ | +| `observe` | ✓ | — | — | — | + +`default` is full management. `observe` is the read-only case — the resource is owned by something else and the loop only observes it. Under any non-`default` value, the disallowed operations are simply never invoked, even if `Observe` reports the resource as missing or drifted. + +**Deletion policy** — the annotation `krateo.io/deletion-policy` — decides what happens to the *external* resource when the managed resource is deleted: + +- `delete` (the default when absent) — the external resource is deleted too. +- `orphan` — the external resource is left in place. + +**How they combine on delete.** When the managed resource is being deleted, the loop deletes the external resource only when the management policy permits it **and** the deletion policy asks for it — concretely, when management is `default` and deletion is `delete` (the default), or when management is `observe-delete`. In every other case the external resource is orphaned. One subtlety to keep in mind: `observe-delete` deletes regardless of the deletion policy. diff --git a/docs/developer-guide/03-building-a-provider.md b/docs/developer-guide/03-building-a-provider.md new file mode 100644 index 0000000..7ea2eff --- /dev/null +++ b/docs/developer-guide/03-building-a-provider.md @@ -0,0 +1,23 @@ +# Building a provider + +The steps to stand up a provider on this library, using **core-provider** as the worked reference. + +## 1. Define the typed resource + +Define your custom resource so it fits the managed-resource shape: put the standard conditions on its status (and delegate the condition getters/setters to them), let its list type enumerate its items, and register it in a scheme. The provider supplies its own scheme registration — the library doesn't provide one. + +## 2. Implement the operations + +Implement **Connect** (build the client that talks to the external system for a given resource) and the four operations — **Observe**, **Create**, **Update**, **Delete** — following the contract in [`01-mental-model.md`](./01-mental-model.md). `Observe` reports whether the external resource exists and is up to date; the others make it so. + +## 3. Wire the reconciler into a manager + +Construct the reconciler for your resource's kind, passing your `Connect` implementation and options (timeout, poll interval, logger, recorder, metrics). Then register a controller for your resource, wrapping the reconciler with a rate limiter. core-provider does exactly this for its `CompositionDefinition`. + +## 4. Bootstrap the process + +Register your scheme **before** wiring the reconciler (construction fails fast if the kind isn't in the scheme), set up the controller options (concurrency, poll interval, the global rate limiter, and — if used — the telemetry recorder), and start the manager. + +## Testing + +The library ships a test toolkit so you can exercise the loop without a cluster: a mock Kubernetes client with per-method hooks, fakes for the managed resource, and comparison helpers for errors and conditions. You can also implement the operations inline for a table-driven test rather than building a full client. The library's own reconciler tests are the best reference for testing your implementation. diff --git a/docs/developer-guide/04-equivalence.md b/docs/developer-guide/04-equivalence.md new file mode 100644 index 0000000..b9bdb23 --- /dev/null +++ b/docs/developer-guide/04-equivalence.md @@ -0,0 +1,43 @@ +# Equivalence: provider-runtime ⟷ unstructured-runtime + +> These two libraries deliberately implement the **same managed-resource lifecycle**. `provider-runtime` (this library) drives it for **typed** custom resources; `unstructured-runtime` drives it for **untyped** objects at a resource type chosen at runtime. This appendix lines them up so a change to one can be mirrored in the other. **If you change lifecycle semantics in one, mirror it in the other — divergence is a bug, not a feature.** + +This same appendix appears in the **unstructured-runtime** developer guide. + +## Concept map + +| Concept | provider-runtime (typed) | unstructured-runtime (dynamic) | +| --- | --- | --- | +| What you manage | a typed custom resource, registered in a scheme | untyped objects at a resource type chosen at runtime | +| Operations you implement | Observe / Create / Update / Delete | the same four operations | +| Per-resource setup | a **Connect** step builds a client for each resource | **none** — you register one client for the whole controller | +| What Observe reports | exists, up-to-date, plus "defaults filled in" and a drift description | exists, up-to-date (minimal) | +| How it's wired | a reconciler into a controller-runtime manager | a controller built directly on lower-level primitives — no manager | +| Work queue | the manager's rate-limited queue | a local priority queue (de-duplicating, priority-aware) | +| Concurrency | a max-concurrent-reconciles setting | a fixed number of workers — **no autoscaling** | +| Finalizer | a configurable finalizer | a fixed finalizer name | +| Conditions / status | standard conditions on the typed resource's status | the same conditions written onto the untyped object's status | +| Standard conditions | `Ready` and `Synced`, with the same reasons | the same | +| Pause | a paused annotation short-circuits to a paused condition | the same | +| Create safety | pending / succeeded / failed create-tracking, plus a grace period | the same tracking | +| Lifecycle policies | the loop may skip operations or orphan on delete | the same | +| Type resolution | scheme / RESTMapper (compile-time types) | runtime pluralization | +| Event recorder, logger | shared Krateo helpers | the same | +| Origin | trimmed fork of crossplane-runtime's managed reconciler | the dynamic analog of the same lifecycle | + +## Invariants that must stay equivalent + +- **Branching from Observe** — missing ⇒ create; exists but drifted ⇒ update; otherwise mark success and requeue. +- **Finalizer discipline** — add the finalizer before creating the external resource; remove it only after a confirmed delete. +- **Create safety** — mark the create pending before doing it, record success or failure after, and refuse to proceed while a create is unconfirmed. +- **Pause** — a paused resource short-circuits to a paused condition without touching the external resource. +- **Conditions** — maintain `Ready` and `Synced` with the same reasons; the framework persists them, not your code. +- **Idempotency** — operations must be idempotent and non-blocking, and a conflict is a requeue, not an error. + +## Where they legitimately differ (and why) + +- **The Connect step** exists only in provider-runtime — typed providers often build a per-resource client; the dynamic controller registers a single client instead. +- **What Observe reports is richer** in provider-runtime (it also carries "defaults filled in" and a drift description); the dynamic side keeps the minimal two-signal form. +- **The plumbing differs** — provider-runtime rides a controller-runtime manager; unstructured-runtime wires the lower-level primitives itself and brings its own priority queue. +- **Concurrency** — a max-concurrent setting versus a fixed worker count (the dynamic side explicitly favors sharding over autoscaling). +- **Type handling** — compile-time types versus runtime resolution of untyped objects. diff --git a/docs/developer-guide/README.md b/docs/developer-guide/README.md new file mode 100644 index 0000000..f3f37e1 --- /dev/null +++ b/docs/developer-guide/README.md @@ -0,0 +1,27 @@ +# provider-runtime — Developer Guide + +A contributor-facing guide to the library that gives Krateo's **typed** providers a managed-resource controller: implement a small set of operations, and the framework runs the reconcile loop. + +> Audience: engineers **building a provider on top of this library, or maintaining the library itself**. This guide explains *ideas and flows*, not line-by-line code. For product concepts, see [docs.krateo.io](https://docs.krateo.io). + +## What it is + +`provider-runtime` implements the **managed-resource controller pattern** for Krateo providers (core-provider, git-provider, github-provider, and others). A provider author defines a typed custom resource, implements a handful of operations, and wires a reconciler into a manager. The library owns the reconcile loop: finalizers, the safety steps around creating an external resource, the `Ready`/`Synced` conditions, pause, requeue and rate-limiting, and optional metrics. + +It is a **trimmed, rebranded fork of crossplane-runtime**, with Krateo-specific conventions (its own annotation prefix and finalizer, a shared event recorder) and the removal of crossplane's provider-config and connection-detail machinery. See [`01-mental-model.md`](./01-mental-model.md). + +> **Sibling library.** `unstructured-runtime` is the **dynamic/unstructured analog** of this library — the same lifecycle, applied to untyped objects at a resource type chosen at runtime. The two are meant to stay functionally equivalent; the mapping is in [`04-equivalence.md`](./04-equivalence.md). + +## Documents in this folder + +| Document | What it covers | +| --- | --- | +| [`01-mental-model.md`](./01-mental-model.md) | The managed-resource lifecycle and the operations you implement. | +| [`02-architecture.md`](./02-architecture.md) | What the library provides, and how the reconcile loop is sequenced. | +| [`03-building-a-provider.md`](./03-building-a-provider.md) | The steps to stand up a provider, using core-provider as the worked reference. | +| [`04-equivalence.md`](./04-equivalence.md) | How `provider-runtime` and `unstructured-runtime` line up, concept by concept. | + +## See also + +- **Ecosystem overview (canonical)** — how Krateo Composable Operations fits together lives in the **core-provider** repo: `core-provider/docs/developer-guide/00-ecosystem-overview.md`. +- **The exemplar consumer** — **core-provider** is the reference provider built on this library.