diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1318539..f2e373b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,6 +15,12 @@ Run `task fixtures-check`, `task test`, and `task lint` before opening a pull re ## Design rules +Start with watch streams and authoritative resource views; add editing as an optional layer. +The [README](README.md) explains current capabilities and the native-fetch direction. Current API +examples must match this branch. The next native slice is read-only LIST/WATCH using the existing +lifecycle and state-input helper; editor cleanup, native editing and larger comparisons are separate. +Native transport and save-progress evaluation remain proposals until implemented. + - Keep the core gateway free of `client-go`; Kubernetes integration belongs in `gateway/kube`. - Keep the browser client framework-free and free of runtime dependencies. - Keep credentials, application identity, authorization policy, and writes in the host application. @@ -34,6 +40,16 @@ Run `task fixtures-check`, `task test`, and `task lint` before opening a pull re Protocol compatibility is explicit. Additive optional fields are allowed when consumers can ignore them; incompatible event semantics require a new protocol version rather than a silent reinterpretation. +## Documentation changes + +Keep viewing and transport guidance in [watching resources](docs/why-a-gateway.md), editor behavior in +[the editor model](docs/client-state-model.md) and writes in [saving](docs/saving.md). Examples own +their framework-specific guidance. Link to these contracts rather than repeating them in proposals. +Retain decisions and useful evidence when shortening completed plans; remove superseded request +inventories and duplicate guides. Preserve dated observations and generated release history. +Documentation-only changes need local link/anchor checks, snippet/API review and diagram review; +cluster and runtime suites are required when behavior changes, not for prose alone. + ## Fixtures Fixtures use source YAML in `conformance/bodies/` and `conformance/fixtures/`. Run `task fixtures` to diff --git a/README.md b/README.md index c6fb117..23108f3 100644 --- a/README.md +++ b/README.md @@ -10,215 +10,184 @@ # krm-stream -Live Kubernetes resource updates for browser apps. +Efficient live Kubernetes views for browser applications, with optional editing. -`krm-stream` turns a Kubernetes watch into a small, browser-safe stream. It ships a Go gateway, a -headless TypeScript store with zero runtime dependencies, and shared conformance fixtures, so your -product can show live cluster state while people are editing it. +`krm-stream` helps applications watch Kubernetes resources while controllers and other users keep +changing them. Its gateway delivers a defined resource view, withholds selected values, suppresses +irrelevant updates and optionally shares upstream watches. Its headless TypeScript client adds connection lifecycle, recovery and +live state and reconciles incoming changes with local drafts when a page needs editing. -## Is this for you? +Your application supplies authentication, authorization policy, Kubernetes credentials, UI and writes. +The browser client has zero runtime dependencies and chooses no UI framework. -**Yes, if:** +## Start with the watch -- You want to consume a Kubernetes **watch from a browser**, without handing the browser a cluster - credential or turning on CORS across your API server. -- You want to **live-edit Kubernetes resources**, where a concurrent server change is merged into - what the user is typing rather than clobbering it, and a genuine conflict is surfaced instead of - silently resolved. That is the [three-way merge](docs/glossary.md). -- You want to **bound the number of real watches** on your API server. Ten tabs on one namespace - should be one upstream watch, not ten. -- You are building a **product**, not a Kubernetes dashboard. The store is headless and picks no UI - framework. +Kubernetes already provides a change feed. krm-stream adds a browser lifecycle and resource-view +contract, reducing work at different points: -**Probably not, if:** +| Capability | Benefit | Boundary | +|---|---|---| +| Scope | Watch a resource kind, namespace, name or allowed label selector | The host authorizes the scope; a selector does not grant access | +| Projection and redaction | Deliver the fields a view needs; full/spec views withhold Kubernetes Secret values | Secret key paths and change revisions remain visible; arbitrary sensitive CRD fields are not automatically redacted | +| Update suppression | Avoid downstream events when projected content and redaction records are unchanged | The gateway still receives upstream changes; fewer events do not guarantee a current write version | +| Optional watch sharing | Use one upstream watch for matching scopes on the same shared backend | Every subscriber is authorized separately and receives its own snapshot and updates | -- You just want a **generic three-way merge library**. This store includes KRM identity, projection - and redaction rules, plus optional schema-based keyed-list merging. -- You want a **ready-made Kubernetes dashboard**. Use [Headlamp](https://headlamp.dev/). See - [alternatives](docs/alternatives.md). -- You want to **write to the cluster from the browser**. krm-stream is the read-and-edit half: it - captures a merge patch and version together. Your application validates and performs the write. - See [saving edits safely](docs/saving.md). +Choose a view explicitly: -## What is KRM? +| View | Delivered content | Updates suppressed | +|---|---|---| +| `krm-full/v1` (default) | Resource including status, with Secret values withheld | Bookkeeping-only changes | +| `krm-spec/v1` | Full view with status omitted | Bookkeeping-only and status-only changes | +| `krm-raw/v1` | Secret values included when host policy permits | Bookkeeping-only changes | -**KRM** is the Kubernetes Resource Model: the shape every Kubernetes object has (`apiVersion`, -`kind`, `metadata`, and kind-specific fields such as `spec`, `status` or ConfigMap `data`). Custom -resources use the same conventions, -which is why this works for your product's own objects, a `Database`, a `FeatureFlag`, a `Tenant`, -and not only for cluster infrastructure. +All three remove `metadata.managedFields` and the last-applied-configuration annotation. +`krm-raw/v1` is still a projection. A hidden Secret rotation produces a redaction update in full/spec. -Never touched a cluster? The [glossary for frontend developers](docs/glossary.md). +Each current gateway connection starts with a complete snapshot, then follows visible changes. On reconnect, +a fresh snapshot repairs missed changes and deletes; resources are pruned only when it completes. +This is a live state feed: intermediate updates may be coalesced. See [watching resources](docs/why-a-gateway.md). -## Why a gateway +## Choose a source -Kubernetes already has a good change feed: a watch, documented under -[efficient detection of changes](https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes). -Direct browser access requires exposing cluster credentials and arranging cross-origin access. -A raw watch also carries whole objects, including Secret values. The embedded gateway uses host-owned -credentials, enforces the selected disclosure policy and emits SSE. Hosts can opt into one shared -upstream watch per scope with per-subscriber authorization. +Start with the resource state and guarantees the page needs; framing is an implementation detail. -[Why a gateway](docs/why-a-gateway.md) works through this in full. +| Source | Use it for | Current support | +|---|---|---| +| Native through a host proxy | Original Kubernetes resources without adopting the gateway; credentials stay on the host | A focused read-only fetch connector is [next work](docs/field-reports/third-our-identity.md#native-watch-connector) | +| Gateway | Named views, Secret-value redaction, suppression and optional upstream sharing | Supported today through the fetch/SSE connector | -## How it fits +Native is the straightforward starting point for hosts that already proxy Kubernetes. Gateway SSE +is the delivery format for projected views; its added capabilities remain useful. Both sources should +reuse connection state, errors, cancellation, bounded recovery and state application. +The first native slice lists then watches and re-lists on reconnect. Native editing, resume and a +larger comparison example follow separately. No source fallback may bypass a refused view. -```mermaid -flowchart LR - api["Kubernetes API"] - gateway["Embedded Go gateway
Enforces host authorization
Projection and redaction"] - store["Browser store
Delivered state, local draft and conflicts"] - ui["Your form or live view"] - save["Your Go save handler
Write authorization and patch validation"] - - api -->|"Snapshot and watch updates"| gateway - gateway -->|"One-way SSE"| store - store -->|"Render draft and live status"| ui - ui -->|"Local edits"| store - ui -->|"On Save: capture patch, UID and RV from store"| save - save -->|"Conditional PATCH"| api - - classDef library fill:#dff3ff,stroke:#1677a4,color:#062f45; - classDef application fill:#e4f7e8,stroke:#27834c,color:#113d23; - classDef upstream fill:#f8e0ef,stroke:#a83970,color:#4b1230; - class gateway,store library; - class save,ui application; - class api upstream; -``` - -The library owns the read stream and browser reconciliation. Your application owns identity, -authorization policy, Kubernetes credentials, and writes. The browser never receives a Kubernetes -credential or a raw API-server URL. The blue boxes are the library; the green boxes are your product. -Both Go components run inside your application: the host decides authorization policy, and the -embedded gateway enforces it. A successful write returns through the Kubernetes watch as another -live update. +## Watch a resource view today -Each connection starts with a complete projected snapshot, then follows visible changes. The store -keeps that delivered state separate from local edits. A quiet stream can still hold an older write -version: see [why a quiet stream can reject a save](docs/saving.md#why-a-quiet-stream-can-still-reject-a-save). +```ts +import { + LiveResourceStore, readOnlyPolicy, applyStreamEvent, + connectResourceStream, resourceStreamURL, +} from "@configbutler/krm-stream"; -## Start here +const store = new LiveResourceStore(readOnlyPolicy); +const stopRendering = store.subscribe(() => renderResources(store)); +const connection = connectResourceStream( + resourceStreamURL("/resource-stream/v1", { + target: "production", version: "v1", resource: "configmaps", namespace: "app", + }), + event => applyStreamEvent(store, event), +); +renderConnection(connection.state.status); +const stopConnection = connection.subscribe(state => renderConnection(state.status)); +connection.closed.catch(reportApplicationError); + +// On view disposal: +stopRendering(); +stopConnection(); +connection.close(); +``` -**Want it running in minutes?** [krm-foyer](https://github.com/ConfigButler/krm-foyer) is a -ready-made backend for browser apps on Kubernetes: OIDC sign-in, server-side sessions, a `/k8s` API -proxy, and krm-stream's live resources, with each watch opened as the signed-in user so Kubernetes -RBAC decides what they see. Its `task demo` starts a test cluster and an example app. Start there if -you don't already have a Go backend to embed the gateway in. To build the integration yourself, read -on. +The current viewer uses `LiveResourceStore(readOnlyPolicy)`; a dedicated read-only store is deferred. +The connector delivers state events independently of editing. It uses same-origin cookies by default, +exposes connection state and bounded retries, and stops on terminal refusals. Apply each event +synchronously. The [client README](packages/krm-stream/README.md) explains lifecycle and errors. +For a browser without a bundler, the same API is available in one file through +`@configbutler/krm-stream/bundle`. -There are two halves, and they are usually two different people. +## Add editing when the page needs it -### The browser half +Use `new LiveResourceStore()` for an editor. It keeps the last delivered server object separate from +the person's draft. Incoming changes update untouched fields, preserve local edits and record +conflicts when both sides changed the same editable field differently. -No bundler, no framework, no Kubernetes client. The connector uses fetch and hands each resource -event to a callback; the store is plain ESM: +For example, someone changes a Deployment's image while an autoscaler changes its replicas. The +replicas follow the server and the image edit stays. If another person changes that image to a +different value, the editor keeps the local value and exposes the disagreement for review. ```ts -import { LiveResourceStore, applyStreamEvent, connectResourceStream, resourceStreamURL } from "@configbutler/krm-stream"; +const store = new LiveResourceStore(); // use this store in the connection setup for an editor +store.setValue(uid, ["data", "message"], "hello"); // ConfigMap field edit +store.conflicts(uid); // disagreements to resolve before Save +const intent = store.captureSave(uid); // detached { uid, resourceVersion, patch } +// Your save controller submits this intent after review while the connection is live. +``` -const store = new LiveResourceStore(); +The default policy allows `spec`, labels, annotations, `data` and `stringData`; status, immutable +metadata and redacted paths remain read-only. A host can narrow the policy for its form. -const connection = connectResourceStream( - resourceStreamURL("/resource-stream/v1", { - target: "production", - version: "v1", - resource: "configmaps", - namespace: "app", - }), - (event) => { - const change = applyStreamEvent(store, event); - render(change.uid); // what moved, and which resource it moved on - }, -); -connection.subscribe((state) => renderConnection(state.status)); // connecting, syncing, live, retrying… -connection.closed.catch(reportApplicationError); // the callback threw, and the stream stopped +The intended Save flow is explicit: capture the patch, UID and resource version together; have the +host authorize and validate it; apply a conditional merge PATCH; then observe the projected result +through the stream or a guarded projected read. Preserve typing made after Save. A version rejection +can occur without any field conflict, because suppressed updates still advance Kubernetes versions. +The current recovery is a guarded read, review and another deliberate Save. -// The user edits. The server keeps changing underneath them. Neither wins by accident. -store.setValue(uid, ["spec", "replicas"], 3); -store.conflicts(uid); // paths where the server disagreed with an edit the user actually made -store.patch(uid); // an RFC 7386 merge patch of just their changes, or null +**A dirty draft, an accepted write and application progress are separate states.** A successful PATCH +can precede its watch observation, and neither proves a workload has finished rolling out. -// In your host/view teardown callback: -connection.close(); // stop the stream and pending retries -``` +Use [the editor state model](docs/client-state-model.md) for reconciliation, conflict resolution and +arrays, and [saving edits safely](docs/saving.md) for the complete host-owned write contract. The +[conditional-save example](examples/conditional-save/README.md) executes that contract. + +## How it fits today -If you have no bundler at all and vendor the library by copying it, import -[`@configbutler/krm-stream/bundle`](packages/krm-stream/README.md): the same API in one file. - -### The server half - -Mount a scoped stream endpoint in an existing Go application: - -```go -mux.Handle("/resource-stream/v1", gateway.Handler(gateway.Options{ - Principal: func(r *http.Request) (gateway.Principal, error) { return userFromSession(r) }, - Scopes: gateway.ScopePolicy{ - Targets: []string{"production"}, - Resources: []gateway.GroupResource{ - {Resource: "configmaps", Scope: gateway.ResourceScopeNamespaced}, - }, - }, - StreamConfig: gateway.StreamConfig{ - Authorizer: authorizeScope, - Clients: func(_ context.Context, _ string, p gateway.Principal) (gateway.Backend, error) { - return kube.NewBackendForConfig(restConfigFor(p.(*User))) - }, - Projections: gateway.StaticProjection(gateway.ProjectionFull), - }, -})) +```mermaid +flowchart LR + api["Kubernetes API"] + gateway["Go gateway
Scopes, views and optional sharing"] + connector["Fetch connector
State events and recovery"] + store["Resource store
Live state and optional drafts"] + ui["Your list, viewer or form"] + save["Your save endpoint
Authorize, validate and conditionally PATCH"] + api -->|"Snapshot and watch"| gateway + gateway -->|"SSE"| connector + connector --> store + store --> ui + ui -->|"Local edits"| store + ui -->|"Captured save intent"| save + save --> api ``` -The Go side owns identity, authorization, the Kubernetes credential, and the scope a caller is -allowed to ask for. It never lets the browser choose which cluster to talk to. +The gateway runs inside your Go application. Per-user backends let Kubernetes authorize the caller's +reads; a shared backend uses a service identity and requires checks for each subscriber. Sharing +reduces duplicate upstream work; access controls and host limits govern who can consume it. +Gateway upstream continuation and improved save progress during suppressed churn are proposed work. +New browser connections receive a fresh snapshot under the current protocol. -`LiveResourceStore` keeps server truth and the local draft separate, reconciles live updates with a -three-way merge, records conflicts, and builds RFC 7386 merge patches. Your application applies any -patch through its own save endpoint, which is the one place a write can happen. +## Start here + +If you already have a Go host, follow [adopting krm-stream](docs/adopting.md). For a ready-made host, +[krm-foyer](https://github.com/ConfigButler/krm-foyer) integrates sign-in, sessions, native API proxying +and krm-stream gateway hosting. The host's dependency version determines which APIs are available. -## Packages +KRM means the Kubernetes Resource Model: `apiVersion`, `kind`, `metadata` and kind-specific fields +such as `spec`, `status` or ConfigMap `data`. Custom resources follow the same conventions. See the +[frontend glossary](docs/glossary.md) and [alternatives](docs/alternatives.md) for context. | Package | Purpose | |---|---| -| `github.com/ConfigButler/krm-stream/gateway` | Dependency-free Go stream gateway and SSE handler. | -| `github.com/ConfigButler/krm-stream/gateway/kube` | Optional `client-go` backend and SubjectAccessReview authorizer. | -| `@configbutler/krm-stream` | Official dependency-free ESM client store and transports. | -| [`spec/v1.md`](spec/v1.md) | Normative protocol contract. | -| [`conformance/`](conformance/) | Shared fixtures exercised by the Go gateway and TypeScript client. | - -## Boundaries - -- No browser token handling or raw API-server URLs. -- No authorization system: the host provides `Principal`, `Authorizer`, and `ClientFor`. -- No write endpoint: hosts validate and apply their own patches. -- No framework dependency in the browser client. -- No `client-go` dependency in the core gateway. - -The important safety rule is simple: a projected or redacted field must never be written back by a -browser. Use [`gateway.ValidateMergePatch`](gateway/patch.go) in the host save handler. +| `github.com/ConfigButler/krm-stream/gateway` | Dependency-free Go stream gateway and SSE handler | +| `github.com/ConfigButler/krm-stream/gateway/kube` | Optional client-go backend and SubjectAccessReview authorizer | +| `@configbutler/krm-stream` | Dependency-free ESM connector and resource/editor store | +| [spec/v1.md](spec/v1.md) and [conformance](conformance/README.md) | Shared normative contract and executable fixtures | ## Guides -- [Glossary for frontend developers](docs/glossary.md): the Kubernetes vocabulary you actually need, and where each word shows up in the library. -- [Why a gateway](docs/why-a-gateway.md): why the browser cannot watch the API server, and why watches are shared. -- [Adopting krm-stream](docs/adopting.md): same-origin cookie, bearer-token, and shared-watch setups. -- [Authentication and authorization](docs/auth.md): identity and RBAC boundaries. -- [Saving edits safely](docs/saving.md): patch validation and host write responsibilities. -- [Operating krm-stream](docs/operations.md): metrics, alerts, and runtime controls. -- [Client state model](docs/client-state-model.md): drafts, conflicts, redactions, and keyed lists. -- [Alternatives and prior art](docs/alternatives.md): how this differs from Kubernetes clients, browser dashboards, and config-as-data systems. -- [Releasing](docs/releasing.md): release workflow and publication prerequisites. -- [Upgrading from 0.7](docs/migrating.md): the one browser connector and the gateway's shared stream configuration. - -## Requirements and maturity - -The project is pre-1.0. Protocol and API changes may still be made before 1.0. +- [Watching resources](docs/why-a-gateway.md): scopes, views, suppression, sharing and recovery. +- [Adoption](docs/adopting.md): host and browser wiring. +- [Editor state model](docs/client-state-model.md): drafts, conflicts, redactions and arrays. +- [Saving](docs/saving.md): conditional writes, recovery and user-facing outcomes. +- [Authorization](docs/auth.md) and [operations](docs/operations.md): identity, revocation and runtime limits. +- [Examples](examples/README.md): browser, conditional editor, recovery recipes and Vue integration. +- [Delivery plan](docs/proposals/0006-stream-and-save-implementation-plan.md): completed work, open work, ordering and dependencies. +- [Upgrading from 0.7](docs/migrating.md) and [releasing](docs/releasing.md). -- Go 1.27.1 for the gateway. -- Node 24 for client development and tests. -- Kubernetes 1.35+ for strict resource-version ordering. `OrderingLenient` supports known - non-conformant or aggregated APIs at the cost of per-object monotonic ordering. +## Requirements and development -## Development +The project is pre-1.0; protocol and API changes may still be made before 1.0. +Go 1.27.1 and Node 24 are required for development. Kubernetes 1.35+ supports strict resource-version +ordering; `OrderingLenient` accommodates known non-conformant or aggregated APIs with a reduced +per-object monotonicity guarantee. ```bash task fixtures-check @@ -227,5 +196,4 @@ task lint task build-client ``` -See [CONTRIBUTING.md](CONTRIBUTING.md) for the fixture and test workflow. Licensed under -[Apache-2.0](LICENSE). +See [CONTRIBUTING.md](CONTRIBUTING.md). Licensed under [Apache-2.0](LICENSE). diff --git a/SECURITY.md b/SECURITY.md index 6b52977..4cf3ffe 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -27,7 +27,8 @@ almost all *disclosure* failures. The things we would treat as security bugs: - **A projected or redacted value reaching the browser.** A `Secret` value, `managedFields`, or any path the effective projection withheld appearing in a stream event, an error message, or a save response. The projection is the boundary; a leak through it is the highest-severity bug this - codebase can have. + codebase can have. This applies to the effective projection: explicitly authorized raw/native + access can disclose Secret values, and built-in redaction does not classify arbitrary CRD fields. - **A caller receiving an object outside their authorized scope.** Particularly through [`SharedBackend`](gateway/shared.go), where one upstream watch is fanned out to many subscribers and the host's `Authorizer` is the only thing standing between a caller and the cache. A bug there is @@ -52,4 +53,5 @@ The boundaries this library claims to hold, and where they are enforced: - [docs/auth.md](docs/auth.md): identity, RBAC, and the `SharedBackend` trade. - [docs/saving.md](docs/saving.md): the write path, and why a save answers 204. -- [docs/why-a-gateway.md](docs/why-a-gateway.md): why the browser never holds a cluster credential. +- [docs/why-a-gateway.md](docs/why-a-gateway.md): watch views, Secret disclosure, host-owned credentials + and explicit native/SSE source selection. diff --git a/conformance/README.md b/conformance/README.md index 28782cf..1b01812 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -1,5 +1,9 @@ # conformance — the shared contract, executable +These fixtures verify the current gateway SSE contract and browser state application. Editing cases +add local drafts to that live-state contract; viewing does not require edit controls. Requested native +watch support will need recovery tests of its own, not an assumption that these SSE fixtures prove it. + **One YAML file describes one scenario end to end**: the Kubernetes watch input, the gateway's wire output, and the client's resulting state after applying events and local edits. The Go and TypeScript suites load the same files, so a contract change is checked on both sides in the same commit. diff --git a/docs/adopting.md b/docs/adopting.md index 8886148..bdfe4fd 100644 --- a/docs/adopting.md +++ b/docs/adopting.md @@ -1,8 +1,17 @@ # Adopting krm-stream -This is the recommended route from an existing Go application to a live KRM view in the browser. The -library owns the read path. Your application owns identity, authorization policy, Kubernetes -credentials and writes. +Start with a live resource view, then add editing only where the page needs it. This guide wires the +currently supported gateway stream into an existing Go application. The library owns stream recovery +and browser reconciliation; your application owns identity, authorization policy, credentials and writes. + +Choose scope, projection and sharing separately using [watching resources](why-a-gateway.md). +Choose the source for the guarantees the page needs: native access retains original resources through +an existing host proxy; the gateway adds projected views, redaction, suppression and optional sharing, +delivered over SSE. Both should reuse a fetch-based frontend lifecycle. + +The [first native slice](field-reports/third-our-identity.md#slice-1-a-native-viewer) is a small read-only +LIST/WATCH connector with re-list recovery and a viewer. Native editing is separate and does not block +that slice. Today the supported connector uses the gateway wiring below. ## 1. Mount the stream endpoint @@ -101,12 +110,13 @@ Use the `Authorizer` to pin a user to a namespace or target before any watch ope ```ts import { LiveResourceStore, + readOnlyPolicy, applyStreamEvent, connectResourceStream, resourceStreamURL, } from "@configbutler/krm-stream"; -const store = new LiveResourceStore(); +const store = new LiveResourceStore(readOnlyPolicy); // list or viewer const url = resourceStreamURL("/resource-stream/v1", { target: "production", version: "v1", @@ -115,17 +125,25 @@ const url = resourceStreamURL("/resource-stream/v1", { projection: "krm-full/v1", }); +const stopRendering = store.subscribe(() => render(store)); const connection = connectResourceStream(url, event => applyStreamEvent(store, event)); renderConnection(connection.state.status); -connection.subscribe(state => renderConnection(state.status)); // gaps recover with a fresh snapshot -connection.closed.catch(reportApplicationError); // the callback threw, and the stream stopped -store.subscribe(() => render(store)); +const stopConnection = connection.subscribe(state => renderConnection(state.status)); +connection.closed.catch(reportApplicationError); // host callback failures stop the stream + +// On view disposal: +stopRendering(); +stopConnection(); +connection.close(); ``` Fetch sends the same-origin session cookie, and the connection recovers on its own after network failures and sequence gaps. -Saving is the host's: see [saving](saving.md). A quiet stream can still hold an older write version; +For an editor, choose `new LiveResourceStore()` with the default policy or a form-specific policy. +Render the draft, edit through the store methods and resolve conflicts explicitly; see the +[editor state model](client-state-model.md). Saving is the host's: see [saving](saving.md). +A quiet stream can still hold an older write version; see [why a quiet stream can reject a save](saving.md#why-a-quiet-stream-can-still-reject-a-save). ## 4. Optional: recheck quiet streams on a timer diff --git a/docs/alternatives.md b/docs/alternatives.md index d2096e6..21885db 100644 --- a/docs/alternatives.md +++ b/docs/alternatives.md @@ -1,10 +1,12 @@ # Alternatives -krm-stream combines a scoped KRM read stream with a browser store for live state, local drafts and -conflicts. Choose tools according to which part of that problem your application needs. +krm-stream starts with efficient live Kubernetes views for browser applications. Its gateway adds +projection, redaction, suppression and optional watch sharing; its client adds lifecycle recovery +and optional draft reconciliation. Choose tools according to the part your application needs. | Need | Relevant approach | Where krm-stream fits | |---|---|---| +| Native watch through an existing browser proxy | Host proxy plus a native fetch/watch consumer | A read-only native connector with shared lifecycle is next work; gateway SSE delivers the supported projected source today. | | Kubernetes API access from a server | Kubernetes client libraries and raw watches | The Go adapter uses `client-go`; the gateway adds projections and browser snapshot framing. | | A complete Kubernetes UI | Dashboard applications and their plugin APIs | krm-stream supplies state and transport; the host builds the UI. | | Review and deliver configuration packages | KRM package and GitOps systems | The host can record accepted writes in its delivery workflow. | @@ -24,5 +26,5 @@ conflicts. Choose tools according to which part of that problem your application - [gitops-reverser](https://reversegitops.dev): a complementary write-and-record workflow. The host connects accepted Kubernetes writes to Git; krm-stream supplies the read and edit side. -The [architecture overview](../README.md#how-it-fits) shows the library/host split. The +The [architecture overview](../README.md#how-it-fits-today) shows the library/host split. The [saving guide](saving.md) describes the conditional-write boundary and its limitations. diff --git a/docs/auth.md b/docs/auth.md index 7eb8216..6ead23e 100644 --- a/docs/auth.md +++ b/docs/auth.md @@ -9,7 +9,14 @@ host's scope and projection decisions. How Kubernetes checks a caller depends on | Shared | One service identity | Authorize every subscriber before serving cached objects; use `kube.SubjectAccessReviewAuthorizer` for Kubernetes RBAC decisions. | Prefer a per-user backend: Kubernetes RBAC is then the boundary by construction. The wiring for both -is in [adopting](adopting.md). +is in [adopting](adopting.md). Watch sharing is an efficiency choice; it never grants a subscriber +the service identity's access. Full/spec Secret redaction also does not replace scope authorization. + +These settings describe the current gateway. The requested native fetch connector uses a host proxy +and the same session boundary. Native retains original authorized resources; gateway SSE delivers +projected views. Neither source may bypass refusal or extend session lifetime. The first native +slice is read-only and must stop on terminal authentication failure, with a new connection after +access is restored; native editing is separate. ## Browser sessions diff --git a/docs/client-state-model.md b/docs/client-state-model.md index 08c3a21..6a7ed86 100644 --- a/docs/client-state-model.md +++ b/docs/client-state-model.md @@ -1,14 +1,38 @@ -# Client state model - -`LiveResourceStore` is a headless TypeScript store for a KRM resource stream. It separates the -authoritative object received from Kubernetes from the user's local draft, so a live update never -silently overwrites an edit. +# Editor state model + +Editing is an optional layer on a live resource view. `LiveResourceStore` keeps the last delivered +server object separate from the person's local draft, incorporates incoming changes and exposes +conflicts for review. The host owns the form and every write; see [saving edits safely](saving.md). +For a list or viewer today, use `LiveResourceStore(readOnlyPolicy)` and render `server(uid)`. +The connector consumes and delivers state events independently of the editor. + +## Intended use + +1. Choose one authorized source, scope and view for the store. Choose the form's editable fields. +2. Apply authoritative state events synchronously with `applyStreamEvent(store, event)`. +3. Render detached store reads and make edits through `setValue`, `removeKey` and related methods. +4. Present incoming changes and resolve conflicts explicitly while preserving the draft. +5. On a deliberate Save, capture the patch, UID and resource version together. Let the host authorize, + validate and conditionally write that intent; preserve later typing during the request. +6. Observe the result through the watch or a guarded read under the same view. Track write acceptance + and workload progress separately from draft dirtiness. + +Do not mutate a `draft()` return value directly: reads are detached copies, and that bypasses edit +policy and notifications. Independent sources/scopes need separate stores. Switching identity or view +must not silently reuse drafts, redactions or snapshot state. Native watch integration is +[requested](field-reports/third-our-identity.md#native-watch-connector). The first native slice uses the +store read-only; this guide's editing and save examples currently use gateway views. Native editing +requires a source-bound policy and host read/write contract in a later slice. + +Keep the existing standalone `applyStreamEvent(store, event)` for the native viewer. A bound store +method is proposed editor cleanup in proposal 0009, not a current API or a prerequisite for native +transport. No new store is needed. ## State per resource | Value | Meaning | |---|---| -| `server(id)` | The latest delivered complete projected object. Every upsert replaces it. | +| `server(id)` | The latest complete object delivered by the chosen source. Current gateway objects are projected; native objects retain upstream fields. Every upsert replaces it. | | `draft(id)` | The object rendered and edited by the UI. Editable regions are reconciled with server changes. | | `conflicts(id)` | Server values that changed concurrently with a different local edit. | | `redactions(id)` | Paths known to exist upstream but intentionally withheld by the selected projection. | @@ -40,7 +64,14 @@ objects it has not yet reloaded. The default editable regions are `spec`, `metadata.labels`, `metadata.annotations`, `data`, and `stringData`. `status`, immutable metadata, and redacted paths are read-only. -When a new server object arrives, the store compares three values at each editable path: +Suppose a Deployment starts with image `v1` and replicas `3`. The person types image `v2`, then an +autoscaler changes replicas to `5`. The draft becomes image `v2`, replicas `5`: untouched fields follow +the server while local work stays. If another person changes the image to `v3`, the local image `v2` +stays and a conflict records `v3` for review. + +When a new server object arrives, the store compares three values at each editable path: **base** is +the previous server value, **ours** is the draft, and **theirs** is the incoming server value. The +incoming object then becomes the new server base. | Draft differs from base | Incoming server differs from base | Result | |---|---|---| @@ -50,7 +81,9 @@ When a new server object arrives, the store compares three values at each editab | yes | yes, differing from the draft | keep the draft and record a conflict | `isDirty` and `changes` are derived from `draft` versus `server`; neither is a cache that can drift -after a stream update. `revert` or `takeTheirs` restores the current server value. +after a stream update. `revert` restores the current server value. To keep a local value explicitly, +use the [tested keep-local recipe](../examples/editor-recipes/README.md#keep-the-local-value-in-a-conflict). +The UI owns the choice; the store does not silently choose a winner for differing concurrent edits. ## Patches and redactions @@ -88,27 +121,22 @@ const pendingDeletes = new Set(); // uids marked for removal ### Reflecting the result -The recommended shape is still 204 and let the watch echo it (see [saving edits safely](saving.md)): a -create arrives as an `added` event, a delete as a `deleted` event, and the store converges on its own. -Two primitives exist for a host that cannot wait for the echo, and both are idempotent with it -(`I-IDEMPOTENT`): - -- `adoptSaved(object)` — insert the created object once the save returns it. The echo that follows is - a no-op, not a second card. -- `removeResource(uid)` — drop a deleted object before its `deleted` event arrives. +Prefer 204 or a receipt and let the stream reflect creates and deletes. Keep accepted writes separate +from pending confirmation, so a delayed watch echo cannot cause a duplicate submission. A create +arrives with its server-assigned UID; a delete removes only the original UID. A completed snapshot +can confirm missed observations. Account for an echo arriving before the HTTP response as well as +for an echo arriving afterwards. -Either way, clear the page-local entry — the `pendingCreates` draft or the `pendingDeletes` uid — when -its write succeeds. Those primitives update the store, not your pending lists; a completed mutation -left staged reappears in the review list and can be submitted twice. +Current `adoptSaved` and `removeResource` APIs remain available, but the recommended flow uses the +stream. `adoptSaved` is unguarded and must not receive an asynchronous response that can race newer +watch state; a reconciliation guard for an existing UID does not insert a newly created resource. +Optimistic removal after a failed delete cannot recover until an authoritative event or snapshot +restores the object. Do not invent UIDs or fabricate authoritative stream events to represent intent. -Two caveats keep this honest: - -- **A create reflects only _after_ the server responds.** `adoptSaved` needs the server-assigned uid - and a **projected** object (never a raw Kubernetes object — see [saving edits safely](saving.md)). Do - not fabricate a uid for a pending draft; keep it page-local until the create returns the object. -- **An optimistic delete is not self-healing.** A delete that _fails_ server-side produces no watch - event, so a `removeResource`d object does not reappear until the next snapshot. Re-add it on failure, - or skip the optimism and let the `deleted` echo do it. +Deletion and snapshot pruning discard the resource's draft. If recovery matters, capture a detached +copy as edits change, before removal, using the [recovery-copy recipe](../examples/editor-recipes/README.md#recover-work-after-a-deletion). +Retain it under the original identity and UID for explicit copy-out; never transfer it automatically +to a same-name replacement. ## Arrays and associative lists @@ -131,8 +159,8 @@ retain the atomic behavior. Merge patches still send the final array as one RFC ## UI integration -The store has no rendering dependency. Subscribe once, then query `draft`, `status`, `changes`, -`conflicts`, and `redactions` during rendering: +The store has no rendering dependency. Subscribe once, then query `draft`, `server`, `changes`, +`conflicts` and `redactions` during rendering. Connection state belongs to the connector: ```ts const unsubscribe = store.subscribe(() => render(store)); @@ -146,3 +174,8 @@ if (intent) await hostSave(intent); Use the [conditional editor](../examples/conditional-save/README.md) for conflict checks, serialized saves and guarded asynchronous responses. `adoptSaved` is for synchronous adoption or newly created objects; a delayed response must use a reconciliation guard. See [saving](saving.md). + +A clean draft means it matches the delivered editable values. It does not prove that a captured write +was accepted or that a controller completed its work. A suppressed update can leave the displayed +view unchanged but its save version stale; see [the save-version explanation](saving.md#why-a-quiet-stream-can-still-reject-a-save). +For reactive integration and subscription ownership, see the [Vue example](../examples/vue/README.md). diff --git a/docs/field-reports/first-fieldreport.md b/docs/field-reports/first-fieldreport.md deleted file mode 100644 index ef494f5..0000000 --- a/docs/field-reports/first-fieldreport.md +++ /dev/null @@ -1,262 +0,0 @@ -# krm-stream feedback from krm-foyer - -krm-foyer is about to host the krm-stream gateway with user-authenticated watches -([roadmap step 5](https://github.com/ConfigButler/krm-foyer/blob/main/docs/roadmap.md#order-of-work)). This note asks for five changes, most -important first. None of them blocks us: for each we say what krm-foyer does meanwhile. -Everything was checked against krm-stream v0.4.0 (`gateway`, `gateway/kube` and -`@configbutler/krm-stream`) on 2026-10-02, by reading the source and, for the error -output below, by running the gateway. - -## Status (2026-10-02): asks 1 to 5 are in krm-stream 0.5.0 - -krm-stream 0.5.0 took all five asks below (krm-stream #40), and krm-foyer moved to it: -the gateway maps the API server's answers, keeps their text off the wire, sends it to a -`Diagnostics` hook, leaves retries to the browser's client with the server's hint, admits -any resource with `ScopePolicy.AnyResource`, and lets `Principal` choose its refusal. -krm-foyer's own mapping, status capture and backoff are gone. - -Reviewing krm-foyer's integration found three more, below the original asks as -[asks 6 to 8](#ask-6-refuse-redirects-in-the-kubernetes-backend). krm-foyer works around -each meanwhile. - -## The asks at a glance - -| # | Ask | Priority | What krm-foyer does meanwhile | -| --- | --- | --- | --- | -| 1 | Map Kubernetes errors to the spec's codes, and recover from the retryable ones with backoff | High | Wraps `kube.Backend` and maps errors itself | -| 2 | Send a generic message with an unexpected `INTERNAL`, and give the detail to a diagnostic hook | High | The same wrapper keeps raw errors from reaching the gateway | -| 3 | Let `Handler` serve a host that delegates resource admission to Kubernetes | Medium | Parses the scope itself and calls `Gateway.ServeStream` | -| 4 | Let `Principal` say why it failed, so a missing credential is `UNAUTHENTICATED` | Low | Answers before the gateway runs, so it never sees this | -| 5 | Honour retry hints at the layer that retries | Low | Nothing needed for now | - -## How krm-foyer uses krm-stream - -krm-foyer is a backend for frontend: it signs users in with OIDC, keeps their ID token -server-side behind a session cookie, and proxies `/k8s` to the API server with that -token. **Kubernetes alone decides what a user may do:** krm-foyer has no allowlist of -its own, and no service-account fallback. Its [design](https://github.com/ConfigButler/krm-foyer/blob/main/docs/design.md#access) promises -that for `/stream` as well as `/k8s`. - -So krm-foyer uses the gateway in per-user mode: - -- **Before the gateway,** krm-foyer checks the session and answers 401 itself when there - is none. It parses the scope with `ScopeFromQuery` and accepts only its one target. -- **`Clients`** returns `kube.NewBackend` over a `rest.Config` holding only the API - server's address, its CA and the user's token. Tests that try to get past this boundary - are krm-foyer's to write; no type check in krm-stream could prove it. -- **`Authorizer`** is `AllowAll`, because every watch is opened as the user, so - Kubernetes RBAC decides. -- **When the session ends,** krm-foyer cancels the request context. That aborts the - stream and cancels the upstream watch; the managed connector then gets a 401 and stops. - -Shared watches come later. Everything below is about per-user mode unless it says -otherwise. - -## Ask 1: map Kubernetes errors, and recover from them with backoff - -**The problem.** `kube.Backend.Watch` wraps any error from the API server (except the -`sendInitialEvents` refusal) in `fmt.Errorf`. The gateway turns any error that is not a -`*StreamError` into a terminal `INTERNAL`. So `FORBIDDEN`, `UNAUTHENTICATED` and -`UPSTREAM_UNAVAILABLE` never come from the Kubernetes backend. What a browser receives -from `Handler` with a backend that fails as `kube.Backend` does: - -```text -upstream 403: -data: {"seq":1,"type":"error","code":"INTERNAL","message":"krm-stream/kube: streaming list for https://10.43.0.1:443/apis/hello.krm-foyer.example/v1/namespaces/hello/notes: notes.hello.krm-foyer.example is forbidden: User \"oidc:carol@example.com\" cannot watch resource \"notes\"","terminal":true} - -API server unreachable: -data: {"seq":1,"type":"error","code":"INTERNAL","message":"krm-stream/kube: streaming list for https://10.43.0.1:443/api/v1/namespaces/x/configmaps: dial tcp 10.43.0.1:443: connect: connection refused","terminal":true} -``` - -A user without `watch` permission sees "INTERNAL" where Kubernetes said why. A restart of -the API server ends every open stream permanently, though the spec calls that case -retryable. - -**Mapping alone is not enough.** The stream loop answers every non-terminal -`*StreamError` by starting the next cycle at once (`StreamProjection` in -`gateway/stream.go`). It emits a generic `RESYNC_REQUIRED` in place of the error, so the -code and any `retryAfterMs` are lost, and it does not wait. Mapping an unreachable API -server to a non-terminal `UPSTREAM_UNAVAILABLE` would therefore turn an outage into a -tight loop of reconnect attempts. The mapping and the recovery have to change together. - -What we would expect, by the spec's own table (§4.3): - -| Upstream answer | Today | Expected | -| --- | --- | --- | -| 403 Forbidden | `INTERNAL`, terminal | `FORBIDDEN`, terminal, with the API server's `Status.message` | -| 401 Unauthorized (token expired or rejected) | `INTERNAL`, terminal | `UNAUTHENTICATED`, terminal | -| 404 (resource not served, wrong version) | `INTERNAL`, terminal | `SCOPE_INVALID`, terminal | -| Connection refused, timeout, 5xx | `INTERNAL`, terminal | `UPSTREAM_UNAVAILABLE`, not terminal, with backoff | -| 429 from Priority and Fairness | `INTERNAL`, terminal | `UPSTREAM_UNAVAILABLE`, not terminal, `retryAfterMs` from `Retry-After` | -| 410 Gone / `Expired` on an open watch | `RESYNC_REQUIRED` | Unchanged: a new snapshot cycle | -| Any other `ERROR` event on an open watch | `RESYNC_REQUIRED` | Classified as in the rows above; 401 and 403 terminal | - -The mapping is needed on every path an error can take: opening a streaming list, the -LIST and the WATCH of the list-then-watch fallback, and `ERROR` events on an established -watch (`watchError` in `gateway/kube/backend.go` turns all of these into -`RESYNC_REQUIRED` today). - -**Who retries.** The spec leaves open whether a retryable error is retried by the gateway, -on the same connection, or by the client, on a new one. We would prefer the client: the -gateway sends the non-terminal error, with its code and `retryAfterMs`, then closes the -connection. Our reasons: - -- **One retry policy, not two.** The managed connector already has a bounded budget, - exponential backoff and jitter. A second backoff in the gateway would multiply with - it. -- **Nothing is held during an outage.** A gateway that retries keeps the browser's - connection, a goroutine and the host's subscription slot for an outage of unknown - length. A closed connection costs nothing until the client returns. -- **No continuity is lost.** A retry in the gateway starts a new snapshot cycle, and a - reconnect asks for a fresh snapshot anyway, so the browser sees the same thing. - -If you prefer that the gateway retries, it needs a cancellable backoff that honours the -upstream's `Retry-After`, and it must keep the error's code and hint on the wire rather -than replacing them with `RESYNC_REQUIRED`. Either way, please say which in the spec. - -A shared backend's own upstream watch is a different matter: it has no browser to hand -the retry to, so it must retry with backoff itself. - -**Tests we would find convincing:** - -- An API server that keeps refusing connections: the number of upstream attempts per - minute is bounded, and the browser sees `UPSTREAM_UNAVAILABLE`, never `INTERNAL`. -- Cancellation during a backoff, or a closed connection during one, ends it at once. -- 401 and 403, at opening and as watch `ERROR` events, are terminal with the right code. -- 410 on an open watch still starts a new cycle. - -## Ask 2: keep unexpected error text off the wire - -`asStreamError` uses `err.Error()` as the message of any `INTERNAL` it generates, so -whatever an error says reaches the browser. In the output above that is the cluster's -internal address (`10.43.0.1:443`), and, with `NewBackendForConfig`, the configured host, -which `Backend.upstream` says is "never sent to a browser". `Handler` already declines to -pass on a `Principal` error for the same reason. - -We suggest: - -- **A fixed message for a generated `INTERNAL`,** such as "internal error". -- **Messages the host chose stay.** A `*StreamError` returned by an `Authorizer` or - `ClientFor` keeps its message, as does the API server's `Status.message` on a 403 - under ask 1. That message is Kubernetes telling the user about their own access, and - `/k8s` passes it on verbatim too. It is not the same problem as an internal error. -- **A separate diagnostic hook for the raw error,** rather than `Observer`, whose kinds - deliberately carry no error messages (`gateway/observe.go`). The host decides what to - log and what to redact. krm-foyer, for one, logs no token or session ID anywhere, and - "log the full error" would be too broad a default for it. - -A test: an error carrying a recognisable secret string, returned from a backend, appears -in the diagnostic hook and nowhere in the response. - -## Ask 3: let `Handler` serve a host that delegates resource admission - -`ScopePolicy` answers "may anyone stream this kind of thing here?", separately from -the `Authorizer`'s "may this caller see it". That is a legitimate host policy, and the -right default. krm-foyer has a different, equally deliberate one: what its endpoint -exposes is whatever Kubernetes lets the user read, exactly as `/k8s` does. It -promises its users no allowlist. - -`ServeStream` already serves that policy: krm-foyer parses the scope with -`ScopeFromQuery`, checks the target and calls the gateway. So this is not a blocker. We -ask for it because `Handler`'s own comment is right: every host that writes this glue by -hand gets four chances to get security-relevant code subtly wrong, and leaving -`ScopePolicy` out should not mean leaving `Handler` out. - -**A possible shape.** An explicit opt-in that stands out in review, as `AllowAll` does: - -```go -Scopes: gateway.ScopePolicy{ - Targets: []string{""}, - Resources: gateway.AnyResource{}, // admission is left to the upstream's own authorization -}, -``` - -With it, any scope that `ScopeFromQuery` accepts passes `Validate` if its target is -allowlisted. An omitted namespace means whatever the upstream makes of it: the list of a -cluster-scoped resource, or every namespace for a namespaced one, which RBAC allows only -to users with a cluster-wide grant. The `Authorizer` still runs, so a host can narrow it -there. The target allowlist and the refusal of `server`, `url`, `token` and the like stay -mandatory. - -**What it must not promise.** The option should not claim to prove that the stream uses -the caller's credential. A runtime check for `*SharedBackend` would not prove it: a -`kube.Backend` can carry a service account's client, and a wrapper can hide a shared -backend. Binding the stream to the caller's own credential stays the host's obligation, -and the docs should say so where the option is described. - -**The spec.** §8 says the server maps the request to "an allowlisted target + GVR", and -conformance item 10 says the scope is allowlisted. "An allowlisted target and a -host-approved GVR" would cover hosts that approve by delegating to the upstream's own -authorization. - -## Ask 4: let `Principal` say why it failed - -`Handler` answers every `Principal` error with `FORBIDDEN` "not authenticated". A page -treats the two codes differently: `UNAUTHENTICATED` offers sign-in, `FORBIDDEN` says no. -But not every failure means signing in again will help: when a host's session store is -down, the honest answer is "try later". - -We suggest honouring a `*StreamError` that `Principal` returns, as the gateway already -does for `Authorizer` and `ClientFor`, so a host can choose `UNAUTHENTICATED`, -`UPSTREAM_UNAVAILABLE` or `FORBIDDEN` with a message of its own. Any other error keeps -today's generic answer, with the code changed to `UNAUTHENTICATED`, which is what -"not authenticated" says. - -## Ask 5: honour retry hints at the layer that retries - -The managed connector ignores both hints a host can give: `Retry-After` on an HTTP 429, -and `retryAfterMs` on a non-terminal error event. It uses its own backoff for both. - -- **On reconnecting,** the connector should wait at least as long as the last hint - said, within its own cap. krm-foyer's rate bound sends `Retry-After` with the exact - time until a request would be let through, so retries before then are sure to fail. -- **A non-terminal event on an open connection** should not make the client reconnect: - under the gateway's current behaviour, recovery follows on the same connection. The hint - applies only if the connection then ends. -- **If the gateway retries itself** (ask 1), it should honour the upstream's - `Retry-After` in the same way. - -Smaller, and related: on a non-200 answer the connector reports only `stream: HTTP 403` -and discards the body. When the body is a Kubernetes `Status`, as krm-foyer's refusals -are, its `message` would be more useful to show. A bounded read of a JSON `Status`, or a -decoder the host supplies, with today's text as the fallback, would cover it. - -## Ask 6: refuse redirects in the Kubernetes backend - -**This one is about credentials.** client-go follows redirects, and its transport adds the -bearer token to the redirected request too. A backend built with -`kube.NewBackendForConfig(cfg)` from a `rest.Config` holding the caller's token therefore -sends that token wherever a redirect points: another host, or plain `http`. Pinning the -host and leaving `Proxy` unset does not prevent it. krm-foyer reproduced it against a -test API server that redirects to a recording server, which received the token. An -aggregated API's answer passes back through the API server, so a redirect is not only a -hostile-server case. - -krm-foyer builds its client with `rest.HTTPClientFor`, sets `CheckRedirect` to refuse, -and ends the stream with a terminal error. We suggest `NewBackendForConfig` do the same, -and that the gateway map a refused redirect to a terminal error, since a redirect will -not go away on retry. The docs that show `kube.NewBackend(dynamicClientFor(user))` -should say a host-built client must refuse redirects too. - -## Ask 7: treat a watch that ends before it was of use as a failure - -The stream loop reopens at once, on the same connection, after `ErrWatchClosed`, and -after a `RESYNC_REQUIRED`. That is right for the API server's routine timeout, which -comes after half an hour. But an upstream that ends every watch before its snapshot is -complete, or answers 410 right after it, gets a tight loop: krm-foyer measured about 100 -reopenings a second against a test server that does so, on one browser connection. - -krm-foyer's backend now answers such an end with a non-terminal `UPSTREAM_UNAVAILABLE`, -so the connection closes and the client waits: an end before the `synced` boundary, or -within a second of it. We suggest the gateway do this itself, for every backend. One -detail cost us a bug: `StreamError.Unwrap` returns `Cause`, so an `UPSTREAM_UNAVAILABLE` -whose cause is `ErrWatchClosed` is still read by the loop as a routine close. - -## Ask 8: say that client-go retries a 429 with Retry-After itself - -client-go asks again on its own, up to ten times, when a 429 or 5xx answer carries a -`Retry-After` header, before the backend sees the error. The API server sets the pace, so -this is not a tight loop, but the browser hears nothing meanwhile, and the -`retryAfterMs` mapping of 0.5.0 only applies once client-go has given up. We suggest -saying so in the operations guide, and considering a client with those retries off, -so the browser's client, which shows the wait, owns them as 0.5.0 intends. diff --git a/docs/field-reports/second-fieldreport.md b/docs/field-reports/second-fieldreport.md deleted file mode 100644 index 7678592..0000000 --- a/docs/field-reports/second-fieldreport.md +++ /dev/null @@ -1,167 +0,0 @@ -# krm-stream feedback from krm-foyer: shared watches - -krm-foyer now hosts `SharedBackend` with `kube.SubjectAccessReviewAuthorizer` -([roadmap step 5b](https://github.com/ConfigButler/krm-foyer/blob/main/docs/roadmap.md#order-of-work), [watches](https://github.com/ConfigButler/krm-foyer/blob/main/docs/watches.md)). This note asks -for five changes, numbered on from the [first feedback](first-fieldreport.md), whose -asks 1 to 8 krm-stream 0.5.0 and 0.6.0 took. Two (9 and 10) are about guarantees that -do not hold today in a failure case; three (11 to 13) would cut the authorization load -that sharing costs. For each we say what krm-foyer does meanwhile. Everything was -checked against krm-stream main at `5ca19ed` (gateway 0.6.0) on 2026-10-04, by reading -the source and by tests in krm-foyer that reproduce each problem. - -## Status (2026-10-04): asks 9, 10 and 13 are taken - -[Proposal 0008](../proposals/0008-shared-watch-hardening.md) selected ask 9, the "refuse the -combination" form of ask 10, and the documentation part of ask 13: - -- **Ask 9:** a shared scope now opens its upstream watch without the backend-wide lock, under a - context of its own that its waiting callers share; the last to leave cancels it. -- **Ask 10:** `Handler`, `ServeStream` and `ServeStreamProjection` refuse a positive - `ReauthorizationInterval` without a positive `WriteTimeout`, and the - [revocation budget](../auth.md#revocation-budget) is documented in one place. -- **Ask 13:** the [SubjectAccessReview request](../auth.md#what-the-subjectaccessreview-asks) is - documented on the adapter and in the guide, and tests pin it. The exported attribute builder is - deferred. - -Asks 11 and 12 are deferred; the proposal records what would reopen them. - -## The asks at a glance - -| # | Ask | Priority | What krm-foyer does meanwhile | -| --- | --- | --- | --- | -| 9 | Open a shared watch without holding the backend-wide lock, and cancel an opening nobody waits for | High | Its upstream backend returns at once and opens the watch on the first `Next` | -| 10 | Make timed rechecks independent of a blocked write, or require a write timeout with them | High | Sets `WriteTimeout` on every stream, and counts it in its revocation bound | -| 11 | Let a host trigger a recheck | Medium | Rechecks on the timer only | -| 12 | Recheck once per principal and scope, not once per subscriber | Medium | Reuses decisions for a short time in its own authorizer | -| 13 | Say which attributes the SubjectAccessReview asks about | Low | Keys its decisions on them, with a test that fails if they change | - -## Ask 9: open a shared watch without holding the backend-wide lock - -**The problem.** `SharedBackend.Watch` holds `b.mu`, the lock every scope takes, while -`startScope` calls `b.upstream.Watch`, a network round trip to the API server. The -context it passes is `context.WithCancel(context.Background())`, and nothing cancels it -until the scope exists. So one API server slow to answer one opening: - -- **holds up every other scope:** a stream of an unrelated namespace waits for the lock, - and never reaches the API server; -- **cannot be abandoned:** the browser that asked for it leaving cancels its own request, - but not the opening, so its stream slot stays taken, and so does every other waiting - stream's. - -krm-foyer reproduced both: an API server that never answers a watch of one namespace, -not even with headers, held up a stream of another namespace until the test timed out, -and the first stream's slot stayed taken after its browser left. A client-go watch has -no response-header timeout of its own, so the wait is unbounded. - -**What we suggest.** Open the upstream outside `b.mu`: record an in-flight entry for the -scope under the lock, so later subscribers of the same scope wait on it, then open -without the lock. Give the opening a context that its waiting subscribers can cancel -together: when the last one leaves before the watch opened, cancel the opening and -drop the entry. The backoff bookkeeping can stay under the lock. - -**What krm-foyer does meanwhile.** Its upstream backend's `Watch` returns a watcher at -once, and opens the real watch on the first `Next`, with the context `Watch` was given. -`pump` calls `Next` without the lock, and `leave` cancels that context when the last -subscriber goes, so the opening is the scope's alone and ends with it. An opening -failure arrives through `Next` and `die`, which already records an -`UPSTREAM_UNAVAILABLE` for the backoff. This works, but it relies on the order of calls -inside `SharedBackend`, which is not a contract. - -## Ask 10: keep timed rechecks from waiting on a blocked write - -**The problem.** Timed checks and delivery share one gate per subscriber (docs/auth.md: -"Timed checks run per subscriber and pause that subscriber's object delivery"). A write -to a browser that stopped reading blocks once the buffers between are full, and the -recheck waits behind it. `ReauthorizationTimeout` does not cover that wait. With -`WriteTimeout` zero, the default, the write blocks until something else ends the -request, so a revoked grant is never applied to that stream. krm-foyer reproduced it: a -stream whose reader stopped, its buffers filled, its grant revoked, stayed open with no -further reviews, past a bound of about 10 seconds. - -docs/auth.md does say the bound "assumes ... sinks do not block indefinitely", and the -sharedstream example sets `WriteTimeout`. But nothing ties the two together, and a host -that sets `ReauthorizationInterval` alone gets an interval that does not hold. - -**What we suggest,** either of: - -- **Refuse the combination:** `Handler` panics on `ReauthorizationInterval > 0` with - `WriteTimeout == 0`, as it does for other unsafe options, or defaults `WriteTimeout` - then; or -- **Decouple them:** run timed checks apart from delivery, and on a denial end the - request (close the connection), whatever the write in progress is doing. - -Either way, document the whole revocation budget in one place: interval, plus the check -timeout, plus one write timeout, plus cleanup, and whatever caching the host adds. - -**What krm-foyer does meanwhile.** It sets `WriteTimeout` (10 seconds, configurable) on -every stream, shared or not, and its documented bound adds it in. A test stops reading, -fills the buffers, revokes the grant, and sees the stream end within the bound. - -## Ask 11: let a host trigger a recheck - -**The problem.** Rechecks are timed, per subscriber, and only the interval can be -tuned. A shorter interval revokes sooner and costs more reviews; a longer one the -reverse. A host that learns of a change sooner (it watches RBAC's Roles and Bindings, a -webhook authorizer's policy, or its own session store) has no way to apply it before -the next tick. - -**What we suggest.** An API on the gateway to recheck now, for every subscriber or for -those matching a principal or scope, for example `Gateway.Reauthorize(func(Principal, -Scope) bool)`, through the same path as a timed check, so a denial ends only those -streams. A host could then pair an event-driven recheck (immediate for RBAC changes) -with a long periodic one as a safety net for what it cannot watch, such as group -membership from the issuer. For krm-foyer that would make the common revocation -immediate and cut the periodic review load several times over. - -**What krm-foyer does meanwhile.** Rechecks on the timer; its decision guide explains -the trade-off between the interval and the load ([watches](https://github.com/ConfigButler/krm-foyer/blob/main/docs/watches.md#load-on-the-api-server)). - -## Ask 12: recheck once per principal and scope - -**The problem.** Each subscriber has its own recheck timer. A user with nine tabs on one -scope is nine timers, each asking the same two questions, and the timers drift apart -as the tabs open at different times. The reviews then scale with streams, not with -users: with 200 users each on nine scopes, about 120 reviews a second at a 30-second -interval, against 13 for one scope. - -**What we suggest,** either of: - -- one timed check per (principal, scope), whose answer applies to every subscriber of - that pair; or -- a decision cache in `kube`, keyed on exactly the attributes a review sends, that a - host can wrap `SubjectAccessReviewAuthorizer` with, with a lifetime the host chooses - and counted in the revocation budget. Every shared host needs one, and getting the - key wrong is a disclosure, so it belongs next to the authorizer. - -**What krm-foyer does meanwhile.** Its own authorizer wraps the SubjectAccessReview -authorizer with such a cache: a decision is reused for 10 seconds for exactly the same -subject and asked attributes, two checks at once wait for one answer, and an error is -never kept. A fuzz test checks that the cache never changes an answer. Reuse still -depends on the timers falling within one lifetime of each other, which ask 12's first -form would make unnecessary. - -## Ask 13: say which attributes the review asks about - -**The problem.** A host that caches decisions has to know exactly which parts of a scope -the review asks about. Today `review` sends the group, version, resource, namespace and -name, and not the label selector (Kubernetes' `ResourceAttributes` has a `LabelSelector` -since 1.31, unused here). That is reasonable, since RBAC cannot grant by label, but it is -written down nowhere, and a later version that sends selectors would make a cache keyed -without them wrong. - -**What we suggest.** Document the attributes on `SubjectAccessReviewAuthorizer`, and -mention a change to them in the changelog as a breaking change for hosts that cache. An -exported function returning the `ResourceAttributes` for a scope and verb would let a -host key on exactly what is asked. - -**What krm-foyer does meanwhile.** Keys on those five attributes and the subject, and a -test fails if a review it observes carries a selector. - -## What we did not ask for - -- **A response-header timeout in the Kubernetes backend.** Ask 9 is the real fix; with - it, a slow opening costs only its own scope's streams, which their browsers can - abandon. -- **Sharing by default.** We agree with keeping `SharedBackend` opt-in: an identity that - may read everything, and the review load, are not costs every host should carry - unasked. krm-foyer keeps it opt-in too ([why](https://github.com/ConfigButler/krm-foyer/blob/main/docs/watches.md#why-sharing-stays-opt-in)). diff --git a/docs/field-reports/third-our-identity.md b/docs/field-reports/third-our-identity.md new file mode 100644 index 0000000..0dd9cb9 --- /dev/null +++ b/docs/field-reports/third-our-identity.md @@ -0,0 +1,106 @@ +# Watch streams, optional editing and native access + +**Current request, 2026-10-05. Slice 1 is not implemented.** The implementation starting point includes +connector separation, Go API cleanup and real-API save tests. Use [proposal 0006](../proposals/0006-stream-and-save-implementation-plan.md#open-work-and-delivery-order) +for completed work, delivery order and independent tracks. + +## Identity and ownership + +krm-stream provides live Kubernetes resource state for browser applications, with optional editing. +Its client adds connection lifecycle, recovery and a resource store. Its gateway adds named views, +redaction, suppression and optional upstream watch sharing. Editing adds drafts, reconciliation, +conflict review and captured save intent; the host owns every write. + +| Source | Delivered content | Added value | +|---|---|---| +| Native through a host proxy — requested | Original Kubernetes resources the host authorizes, including Secret values and machinery fields | Reuse connection lifecycle and live state without adopting the gateway | +| Gateway — supported today | Named projected views delivered over SSE | Selected disclosure, fewer downstream events and optional shared upstream watches | + +Native is the straightforward entry point for a host that already proxies Kubernetes. Gateway SSE +is the delivery format of the projected source, not a legacy label for its capabilities. Both sources +use fetch and should share frontend lifecycle and state application. SSE and native JSON can both +carry errors. The existing SSE connector already handles HTTP refusals and in-stream errors. + +Scope, projection, suppression and sharing have separate costs and guarantees; see +[watching resources](../why-a-gateway.md). Full/spec redaction withholds core Secret values while +revealing paths and change revisions. It does not classify arbitrary sensitive fields in other kinds. +The host owns credentials, session validity, scope authorization, UI and resource limits. + +## Native watch connector + +**Adoption reason:** a host such as krm-foyer already exposes native collection URLs through its +session-authenticated `/k8s` proxy. Its page should reuse krm-stream lifecycle and state without first +wrapping native watch frames in SSE. Keep the projected gateway and its current v1 wire contract. + +### Slice 1: a native viewer + +Deliver one small read-only connector, using the existing `ResourceStateEvent` consumer and +`LiveResourceStore(readOnlyPolicy)`. Keep the standalone `applyStreamEvent` API for this slice; +proposal 0009's bound-method/editor cleanup is independent. No second store, framework or native +write endpoint is needed. A short contract, executable example and focused tests belong in this +change; native editing and the larger comparison project do not gate it. + +- Use caller-supplied fetch or global fetch, same-origin credentials by default and cancellable reads. +- Perform an ordinary collection LIST, then WATCH from its collection resourceVersion. Emit `reset`, + an `added` for each member and `synced` for a complete snapshot, including an empty collection. + Declare live only after snapshot application and watch acceptance; do not start the healthy-period + timer while the watch request is still opening. +- Require a complete unpaginated LIST in slice 1. Do not request a page limit. If the response has a + nonempty continuation token, refuse the unsupported partial collection with a clear terminal error + before emitting `reset` or `synced`; it is not safe to prune from one page. +- Validate collection items and identity before delivery. Supply missing item type metadata from a + concrete typed collection where valid, preserving fields already present; never guess kind from a + plural resource URL or label an item with the collection's `List` kind. +- Decode native JSON frames across byte/chunk boundaries, including split UTF-8. Map resource events + to added/modified/deleted and derive deletion identity from the object. Bookmarks affect neither + membership nor an individual object's version; resume checkpoints are unnecessary for slice 1. +- Every reconnect starts a fresh LIST. HTTP/in-stream 410 triggers bounded recovery to a fresh + snapshot. Do not route native 410 through the SSE transport's terminal HTTP classification. +- HTTP/in-stream 401/403 and other 4xx except 408/429/410 are terminal. Retry transient failures, + 408/429, 5xx, malformed/truncated frames and EOF with the existing bounded budget/backoff and hints. + Repeated 410/short EOF must not create an immediate re-list loop. +- Share lifecycle and HTTP/error helpers where useful, without importing native framing through an + SSE encoder/decoder. No SSE sequence or gateway protocol-header check belongs to native transport. +- Close/abort cancels LIST, WATCH, body reads and sleep; no subsequent events or live publication. + Host consumer/subscriber/error-handler exceptions preserve existing cleanup and rejected completion. +- Use separate stores for separate sources, scopes and identities. Never fall back from a refused + projected source to native access. Raw/native objects have no gateway projection or redaction revisions. + +### Acceptance for slice 1 + +Tests cover an empty collection; complete-list validation, unexpected continuation and type metadata; +chunked frames; malformed/truncated input; initial/list/watch failures; HTTP/in-stream 410 and auth +refusal; transient retries, exhaustion and retry hints; close during fetch/read/backoff; host callback +exceptions; UID replacement and missed deletes or selector exits repaired by re-list. Verify live +readiness, fresh snapshot pruning and preserved existing SSE lifecycle behavior. + +Add a minimal native viewer with identity, current fields and connection state. It makes no save +claims and demonstrates cleanup. Show how to target an existing host proxy. If an existing real-API +harness can exercise LIST/WATCH cheaply, add one focused integration case and report its actual run; +a new cluster campaign is not a prerequisite. Existing tests still apply to any changed lifecycle. + +## Follow-ups after the viewer + +Native editing is a separate slice: define the editable policy, exclude machinery and last-applied +annotation changes, and bind host reads/writes to the same source and UID. Preserve later typing, +submitted intent and reconciliation guards. A native response must not recover a projected editor. +Keep [the editor model](../client-state-model.md) and [saving guide](../saving.md) as the detailed contracts. + +Resumable native watches, streaming-list initialization and pagination are later improvements with +their own recovery tests. Slice 1 improves adoption and lifecycle reuse; re-listing on every reconnect +does not claim resume efficiency. Gateway upstream continuation has a different checkpoint owner and +can proceed independently. Neither implies browser replay for gateway SSE v1. + +## Show the value of each path + +After the viewer works, compare native/full/spec and shared/unshared gateway runs using the same +objects and churn. Demonstrate Secret disclosure, downstream bytes/events, notifications/renders, +snapshots, authorization work and upstream watches. Add editing once its native contract exists. +Keep this comparison and the larger benchmark outside slice 1; begin with the small viewer. + +## Save progress under suppressed churn + +[Proposal 0006](../proposals/0006-stream-and-save-implementation-plan.md#5-save-progress-under-suppressed-churn) +owns the independent evaluation of bounded submitted-intent recovery and optional coalesced version +delivery. Current projected recovery remains a guarded read, review and another deliberate Save. +It does not block the native viewer and does not authorize an automatic write policy in slice 1. diff --git a/docs/glossary.md b/docs/glossary.md index 9d1d382..7755909 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -1,133 +1,98 @@ # Glossary for frontend developers -You do not need to know Kubernetes to use krm-stream. You do need about a dozen words. This page -defines them, then shows where each one appears in the library. +Start with [watching resources](why-a-gateway.md); add the [editor](client-state-model.md) when a page +needs drafts. These are the Kubernetes and editing terms used by those guides. -## The data +## Resource identity and content -**KRM (Kubernetes Resource Model)** is the shape every object here has. It is JSON with four -conventions: `apiVersion`, `kind`, `metadata`, and a body. Read it as a schema for a declarative -object with an identity and a desired state. Nothing about it is specific to containers. A -`Database`, a `FeatureFlag` or a `Tenant` can be a KRM resource. +**KRM (Kubernetes Resource Model)** is the JSON shape of a resource: `apiVersion`, `kind`, `metadata` +and kind-specific fields. A ConfigMap uses `data`; many resources have `spec` and `status`; a custom +resource can represent a product's own Database, FeatureFlag or Tenant. -**Resource** is one such object. Identified by `apiVersion`, `kind`, `namespace` and `name`, and by -`metadata.uid`. +**CRD (Custom Resource Definition)** registers a custom resource kind and its API/schema. -**`uid`** is what the store keys on. Kubernetes gives every object created over a cluster's lifetime a -distinct `uid`, so a resource deleted and recreated under the same name is a different object and does -not inherit the old draft. Key on `name` instead and a user's unsaved edits reattach to whatever takes -the name next. +**UID** is the resource's server-assigned identity, used as the store key. A same-name replacement +gets a new UID and never inherits the old draft. Identity is scoped to an upstream **target**: use +one store per stream, or key multiple targets by `(target, uid)`. Names are useful for display/routing, +not for transferring an edit to a replacement. -A `uid` is a UUID, but the guarantee is per cluster, so the protocol scopes it to one upstream target: -hold objects from more than one stream and you key on `(target, uid)`, or keep one store per stream. -See [spec §1](../spec/v1.md). +**Namespace** scopes namespaced resources. Other resource kinds are cluster-scoped. An omitted +namespace on a namespaced collection can mean all namespaces, so the host declares that access. -**CRD (Custom Resource Definition)** is how a team adds their own `kind`. It is why KRM works as an -application configuration API and not only as cluster plumbing: your product's domain objects can be -resources. +**Spec and status** commonly distinguish desired configuration from observed progress. The default +editor policy makes status read-only. A read-only field can still follow the server live; it is +omitted only if the selected projection says so. -**`spec` and `status`** are the split to remember. `spec` is what a human or agent wants. `status` -is what the system observed. Users edit `spec`. Nobody edits `status`, and the store enforces that: -`status` is read-only and never part of a save. +**Resource version** is a server-assigned token. An object's version is a conditional-write +precondition; a collection/watch version is a recovery checkpoint. They serve different purposes. +Suppression can keep an object's visible content correct while leaving its held write version stale. +See [quiet streams and saving](saving.md#why-a-quiet-stream-can-still-reject-a-save). -**`resourceVersion`** is an opaque server-assigned token that changes on every write. It works like -an ETag, and it is how the server detects that you edited a stale copy. +## Watching and views -**Namespace** is a folder. Resources live in one, and some kinds are cluster-wide instead. +**Watch** is Kubernetes' change feed: added, modified and deleted objects, plus control events such +as bookmarks and errors. History can expire and initialization/recovery must establish a complete +collection boundary. The current gateway handles these mechanics for its browser clients. -## The stream +**Scope** selects a target, resource kind, namespace, optional name and allowed labels. The host +still authorizes the request; a selector does not grant permission. -**Watch** is the Kubernetes primitive for "tell me when this changes". It yields a stream of `ADDED`, -`MODIFIED` and `DELETED` events. It is server-side and stateful, and it has edge cases: history -expiry, relist, bookmarks. The gateway handles those. You do not see them. +**Projection** is the named resource view delivered by the gateway. Full view includes status and +withholds core Secret values; spec view also omits status. Raw includes Secret values with host +permission but still strips managed fields and last-applied configuration. It is not native passthrough. -**Snapshot** is the initial run of events describing the world as it currently is, before live -changes arrive. It has a completion point, the `synced` event. Until a snapshot completes, the client -cannot tell whether a resource it remembers is gone or simply not re-sent yet, so an incomplete -snapshot never prunes state. +**Redaction** withholds a value while disclosing its path and change revision. There is no placeholder +in the object to save back accidentally. The UI renders withheld state from redaction metadata and +must not edit it. Built-in Secret redaction does not classify every sensitive CRD field. -**SSE (Server-Sent Events)** is the browser transport: an HTTP response that stays open and streams -text events. `EventSource` is built into every browser. It is one-directional, which is all a read -stream needs. +**Suppression** skips an object event when projected content excluding resourceVersion plus redaction +records are unchanged. It saves downstream work. A hidden Secret rotation still produces an update. -**Gateway** is the server-side piece you mount in your own Go application. It uses host-owned Kubernetes -clients, enforces host authorization and turns a watch into a scoped SSE stream. The browser never -receives a cluster credential or an API-server URL. +**Shared watch** uses one upstream watch for matching scopes in a shared backend. It saves duplicate +upstream work; each subscriber still needs authorization, a snapshot and its own delivery. -**Projection** is the subset of a resource the gateway sends. What the browser receives may be less -than what exists upstream. +**Snapshot** establishes complete scope membership. The current gateway emits `reset`, member upserts +and `synced`; only completion prunes unseen UIDs. A partial snapshot cannot establish absence. -**Redaction** is a path the gateway knows exists but withholds, such as a Secret value. It is not -sent as a placeholder; it is absent from the object, and listed in `redactions(id)` so the UI can -render it as withheld and offer no editor. It follows that a redacted field must never be written -back, or the browser would erase it. +**SSE (Server-Sent Events)** is the current gateway's text-event framing, consumed by the official +connector through **fetch**. Browser `EventSource` is another SSE client with different header/retry +limitations. Fetch can read SSE or native Kubernetes watch JSON and inspect HTTP responses; both +formats can carry errors. A native fetch/watch connector is requested, while SSE remains supported. -## The editing +**Gateway** is the embeddable Go read layer. It enforces host scope/view policy, projects resources +and delivers the current SSE protocol. A host can separately proxy native Kubernetes access with a +session cookie while keeping cluster credentials server-side. -**Draft** is the object your form is bound to. It is separate from the server object, and the stream -cannot overwrite it without telling you. +## Optional editing and saving -**Three-way merge**: while you are editing a resource, new server updates can arrive. A three-way -merge uses the version you started editing, your draft, and the new server version to combine -non-conflicting changes without overwriting your work. +**Server and draft** are separate objects in the editor: the latest delivered authoritative view and +that view with local edits. Store reads are detached copies; render them and edit through store APIs. -At each editable path the store asks two questions: did the server change this field, and did you -change this field? +**Three-way reconciliation** compares the previous server value (**base**), local draft (**ours**) +and incoming server value (**theirs**) at editable paths: -| Server changed it | You changed it | Result | +| Server changed | Local changed | Result | |---|---|---| -| yes | no | the server value flows into your draft | -| no | yes | your edit stands | -| yes | yes, to the same value | converge, no conflict | -| yes | yes, to a different value | your draft stands, and a conflict is recorded | - -Only the last row needs a human. Without this, a controller updating one annotation would discard -the text you were typing in an unrelated field. - -**Conflict** is the fourth row above. The draft is retained, and `conflicts(id)` exposes the server -value for review. `takeTheirs` and `revert` restore server values. The store leaves save policy to the -host; the conditional editor requires conflicts to be resolved before saving. - -**Associative list** is a Kubernetes array that behaves as a map. `spec.containers` is keyed by -`name`, not by index. The store merges these by key only when configured with -`withOpenAPIKeyedLists` and the host-supplied structural schema. Arrays are atomic by default. - -**RFC 7386 merge patch** is the save format: a JSON document containing only what changed, where -`null` means delete. The store builds it by diffing draft against server over editable paths only. -It never diffs the whole projected object, which would turn a field that is absent because of a -projection into a deletion. - -## How this hooks into krm-stream - -The read path, in the order the words appear: - -1. Your Go application mounts the **gateway**. It authenticates the user, decides the **scope**, and - opens a **watch**. -2. The gateway applies a **projection** and its **redactions**, then streams a **snapshot** over - **SSE**, followed by live updates. -3. `LiveResourceStore` consumes those events and keeps, per **`uid`**: `server(id)` for the server - object, `draft(id)` for yours, plus `conflicts(id)` and `redactions(id)`. One store per stream. -4. Every incoming update runs the **three-way merge** over `server` as the base you started from, - `draft` as what you typed, and the new server object. Server changes you did not touch appear in - the form. Your edits survive. Collisions land in `conflicts(id)`. - -The write path is not the library's: - -5. On save, `captureSave(id)` captures an **RFC 7386 merge patch**, UID and base resourceVersion - together, or returns `null` when nothing changed. -6. You send that intent to your own save endpoint. The store never writes to Kubernetes. -7. Your handler calls [`gateway.ValidateMergePatch`](../gateway/patch.go), which rejects a patch - touching anything the effective projection withheld or stripped: a redacted path, - `metadata.managedFields`, the last-applied annotation, and `status` under `ProjectionSpec`. It is - what stops a buggy or hostile browser from destroying what it was never shown. Do not skip it on - the grounds that the store is careful, because the store runs on the caller's machine. -8. The host writes with the captured UID and resourceVersion preconditions. The watch sees it, it - returns down the stream as an ordinary update, - and the merge converges your draft with it. Your own write needs no special handling. - -If you know TanStack Query or SWR, this is the same server cache with local edits, with two -differences: the cache is pushed rather than refetched, and the local edit is not discarded when new -server data arrives. - -Next: [Client state model](client-state-model.md) for the API surface, and -[Saving edits safely](saving.md) for the host's responsibilities on the write path. +| yes | no | Follow server | +| no | yes | Keep local edit | +| yes | yes, same value | Converge | +| yes | yes, different values | Keep local edit and expose conflict | + +The incoming server object becomes the next base. Untouched fields stay live while local work stays +in the draft. A **field conflict** needs explicit review; `revert` takes the current server value. + +**Associative list** is a Kubernetes array with schema-declared item keys. With the exact host-supplied +OpenAPI schema, the editor can merge these by key. Other arrays are atomic. The resulting merge patch +still replaces an array as one value. + +**Save intent** captures a detached merge patch, UID and resourceVersion together before awaiting a +request. The host authorizes, validates and conditionally writes it. A **merge patch** contains only +editable changes; `null` means deletion. Never write a whole projected object back with PUT. + +**Version rejection** means a precondition failed, not necessarily a field disagreement. Current +recovery is a guarded projected read, review and another deliberate Save. A **reconciliation guard** +prevents a delayed read/response from overwriting newer watch state or crossing UID/snapshot recovery. + +**Dirty draft, accepted write and domain progress** are separate facts. Later typing survives Save; +204 or a receipt confirms acceptance, while stream observation and rollout/Git completion have their +own meaning. See [saving edits safely](saving.md) for the complete contract and UI outcomes. diff --git a/docs/migrating.md b/docs/migrating.md index 86f839a..80dfef3 100644 --- a/docs/migrating.md +++ b/docs/migrating.md @@ -3,7 +3,10 @@ The release after 0.7 has one browser connector and one name for each gateway setting. Old names are removed rather than kept as aliases, so the compiler or a failed import finds every call site. The editor store's methods (`setValue`, `captureSave`, `captureReconciliation`, `adoptSaved` and so on) -are unchanged. +are unchanged on this branch. Proposed editor removals in proposal 0009 are not migration requirements +yet. A read-only native fetch/watch connector is separate requested work and uses today's standalone +state-input helper. This upgrade still uses gateway SSE; native viewing does not wait for editor cleanup. +For the overall direction, see [watching resources](why-a-gateway.md). ## Browser client diff --git a/docs/operations.md b/docs/operations.md index 6ffc6c8..6ebf49c 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1,5 +1,10 @@ # Operating krm-stream +Measure the watch path before enabling broader scopes or sharing: upstream watch count, downstream +bytes/events, snapshot size/time, subscriber authorization and browser notifications/renders. +Suppression reduces downstream work; sharing reduces duplicate upstream watches but retains +per-subscriber costs. See [watching resources](why-a-gateway.md) for those boundaries. + The gateway reports low-cardinality lifecycle signals through `Observer`, so hosts can count them without inspecting resources: @@ -24,7 +29,7 @@ browser receives only the protocol code and a message chosen for it. Redact befo | `shared_overflow` | one subscriber exceeded `SharedOptions.QueueDepth` | increase only after checking browser stalls and event rate; resnapshot is intentional | | `terminal_error` | logical-stream failure, observed before attempting its terminal frame; delivery can fail | alert by low-cardinality error code; browsers must not retry terminal errors | | `retryable_error` rising | streams ending with a non-terminal error, usually `UPSTREAM_UNAVAILABLE` | check API-server health and priority-and-fairness rejections; browsers reconnect on their own backoff | -| `event_suppressed` ratio | `krm-spec/v1` is removing expected churn | a sharp drop may mean callers selected `krm-full/v1` or a projection changed | +| `event_suppressed` ratio | the selected projection is removing unchanged-view churn | a sharp drop may mean callers selected `krm-full/v1` or a projection changed | | stream count / snapshot duration | connection pressure or oversized scopes | narrow namespaces/selectors; avoid accidental all-namespaces watches | | unorderable `resourceVersion` terminal errors | an unsupported or aggregated API does not meet strict ordering | use `OrderingLenient` only after accepting the reduced monotonicity guarantee | @@ -105,3 +110,8 @@ are tested; other middleware and proxy combinations need host tests. For authorization bursts and the limits of the 200-subscriber profile, see the [shared-host capacity guide](../gateway/kube/examples/sharedstream/README.md#verification-and-capacity). + +Upstream continuation, native browser watch recovery and save-progress evaluation are +[planned work](proposals/0006-stream-and-save-implementation-plan.md), not current runtime controls. +When measuring editing, distinguish stale-version 409s from field conflicts and include save latency +and successful saves during sustained controller churn, rather than only after it stops. diff --git a/docs/proposals/0005-kubernetes-stream-and-save-semantics.md b/docs/proposals/0005-kubernetes-stream-and-save-semantics.md index 95fd130..df4f754 100644 --- a/docs/proposals/0005-kubernetes-stream-and-save-semantics.md +++ b/docs/proposals/0005-kubernetes-stream-and-save-semantics.md @@ -3,7 +3,8 @@ **Status: design rationale for the remaining stream and save work.** [Proposal 0006](0006-stream-and-save-implementation-plan.md) owns work order and acceptance criteria. -Use the [saving guide](../saving.md) for current host integration. +Use [watching resources](../why-a-gateway.md) for view selection, the [editor model](../client-state-model.md) +for optional reconciliation and the [saving guide](../saving.md) for current host integration. Kubernetes owns identity and conditional writes. The library supplies projected reads and one draft store; the host owns credentials, write policy and presentation. The sections below explain the @@ -122,11 +123,11 @@ losses of trustworthy state can also require initialization. Bookmarks are usefu promises of a fixed heartbeat cadence. [Watch recovery semantics](https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes) **Upstream:** today, routine watch closure can cause a new initial-state cycle even when retained -Kubernetes history could permit continuation. Prefer investigating a resumable Kubernetes backend -first. It could keep the existing `Watcher` seam and SSE protocol, returning a resync only when it -cannot establish continuity. The upstream [client-go reflector](https://github.com/kubernetes/client-go/blob/master/tools/cache/reflector.go) -is the reference to study before implementing more watch machinery. It must track the upstream checkpoint, including appropriate bookmarks, -rather than infer one from a browser's last visible resource. +Kubernetes history could permit continuation. Proposal 0006 assesses client-go options and proposes a +bounded loop behind the existing `Watcher` seam, preserving SSE and returning a resync when continuity +cannot be established. The upstream [client-go reflector](https://github.com/kubernetes/client-go/blob/master/tools/cache/reflector.go) +is useful reference machinery. A continuation checkpoint belongs to the backend, including appropriate +bookmarks; it cannot be inferred from a browser's last visible resource. This is still proposed work. **Downstream:** spec §7 deliberately prohibits SSE resume in v1. SharedBackend can serve a warm snapshot without opening one upstream watch per subscriber, but serialization, network bytes and browser @@ -151,8 +152,11 @@ Upstream continuation is a named follow-up, ahead of any downstream replay desig | Add a write-base/read-ticket abstraction | New host/server protocol | Could coordinate reads and intended writes | State, expiration, identity binding and replay concerns; too much core machinery now | | Move host to SSA or targeted JSON Patch | No SSE change | Different write tradeoffs | Host policy/validation work; not interchangeable concurrency semantics | -**Current choice:** retain existing emissions and explicit host save outcomes. Prefer complete existing event shapes over a new -version-only event if measurements later justify version delivery. If changing a named projection's +**Current choice:** retain existing emissions and explicit host save outcomes. +[Save-progress evaluation in proposal 0006](0006-stream-and-save-implementation-plan.md#5-save-progress-under-suppressed-churn) +evaluates bounded submitted-intent recovery and coalesced version delivery under sustained churn. +This is an evaluation request, not an implemented retry policy or wire event. Prefer complete +existing event shapes over a new version-only event if measurements later justify version delivery. If changing a named projection's promised suppression behavior, use an explicit new contract/projection identity or a coordinated pre-1.0 change; do not silently repurpose `krm-spec/v1`. Pre-release naming flexibility is useful, but it does not excuse ambiguous guarantees. @@ -169,8 +173,8 @@ recreation between those operations must never make the old draft modify the new validates UID immutability, but validation/storage ordering can affect the returned error. [Metadata update validation](https://github.com/kubernetes/apimachinery/blob/master/pkg/api/validation/objectmeta.go) -The UID-race response is not established by the existing stale-RV 409 test. Add a real API-server test -that forces this race, records the structured Kubernetes Status and verifies the replacement is -unchanged. If the host normalizes an identity-mismatch error to a conflict response, classify that -specific case. Keep schema/admission validation failures as validation errors; do not map every 422 -to 409. Preserve useful structured error information without exposing a raw protected object. +The real-API composition tests now force this preflight/PATCH race: Kubernetes checks the captured +resourceVersion before UID, returning a plain 409. The example host follows the rejection with one +GET to classify a replacement while leaving it unchanged. See [the completed baseline](0006-stream-and-save-implementation-plan.md#completed-baseline) +and [saving](../saving.md#on-409-reconcile-and-save-again). Preserve structured Kubernetes Status and +keep schema/admission errors distinct; do not map every 422 to a conflict. diff --git a/docs/proposals/0006-stream-and-save-implementation-plan.md b/docs/proposals/0006-stream-and-save-implementation-plan.md index c5f756d..b79240f 100644 --- a/docs/proposals/0006-stream-and-save-implementation-plan.md +++ b/docs/proposals/0006-stream-and-save-implementation-plan.md @@ -1,115 +1,93 @@ -# Proposal 0006: Remaining stream and save work +# Proposal 0006: Delivery plan for watch streams and optional editing -**Status: active follow-up plan.** +**Status: active delivery plan, updated 2026-10-05.** This is the single inventory of completed work, +open work and ordering. The starting implementation includes connector separation, Go API cleanup +and real-API save-composition tests. Detailed contracts stay in the linked proposals and guides +rather than becoming another roadmap. -Follow the standing [design rules](../../CONTRIBUTING.md#design-rules) and -[release policy](../releasing.md). [Proposal 0005](0005-kubernetes-stream-and-save-semantics.md) -explains the unresolved tradeoffs; this document owns work order and acceptance criteria. +Start with [watching resources](../why-a-gateway.md); add the [editor](../client-state-model.md) only +where a page needs drafts. Prioritize a read-only native fetch connector for hosts with a proxy; +the gateway remains the projected source with redaction, suppression and sharing, delivered over SSE. +Follow the [design rules](../../CONTRIBUTING.md#design-rules) and [release policy](../releasing.md). +[Proposal 0005](0005-kubernetes-stream-and-save-semantics.md) owns the stream/save tradeoffs. -[Proposal 0007](0007-shared-stream-host-integration.md) proposes bounded HTTP delivery, lifecycle -observations and tested shared-host composition. That work can support the authorization bounds -and continuation measurements below, but adds no dependency or acceptance gate to this plan. -Hosts may demonstrate the existing requirements with their own bounded sinks and instrumentation. +## Completed work -[Proposal 0009](0009-stream-and-editor-separation.md) separates browser stream consumption from the -optional editor layer. It owns that API and package-boundary work; this plan retains the existing -behavioral follow-ups and acceptance criteria. Priorities 2–3 below form the editor track, and -priority 4 forms the stream track. Deliver them independently. Complete the browser separation -before expanding the editor's public API; it does not block gateway fixes, real-API evidence or -upstream continuation. +“Implemented” below means present in this checkout; it does not establish that the current stack is +merged, released or adopted by a host. Final-commit validation and consumer dependency upgrades +remain separate delivery steps. -[Proposal 0010](0010-gateway-api-cleanup.md) owns separate Go API cleanup. Use its final serving and -configuration names once implemented; it adds no acceptance gate to the behavioral work here. - -## Baseline and order - -Use the [adoption guide](../adopting.md), [saving guide](../saving.md) and -[conditional editor](../../examples/conditional-save/README.md) for current behavior. - -| Priority | Remaining work | Completion evidence | +| Done now | Result | Reference | |---|---|---| -| 1 | Define convergence precisely — complete | [Evidence](../../conformance/README.md#convergence-evidence) | -| 2 | Publish tested deletion-recovery and keep-local recipes — complete | [Recipes](../../examples/editor-recipes/README.md) and [tests](../../packages/krm-stream/test/recipes.test.ts) | -| 3 | Harden real-API save composition and identity races — implemented | [`TestRealAPI` cases](../../gateway/kube/composition_e2e_test.go), run by `task test-real-api` | -| 4 | Measure and implement upstream continuation | Same-workload comparison proves continuity, bounded recovery and authorization. | - -Review the remaining priorities as separate changes. Baseline measurement can run alongside -priorities 2–3. Unit tests do not establish real-cluster composition, consumer readiness or capacity. - -## 1. Define convergence precisely - -Completed: [contract and executable final-write evidence](../../conformance/README.md#convergence-evidence). - -## 2. Tested adoption recipes - -Completed: the [editor recipes](../../examples/editor-recipes/README.md) keep a recovery copy before -deletion or pruning and keep a local value in a conflict, executed by the client suite through -stream events and linked from the saving and Vue guides. The acceptance criteria below are what -those tests cover. - -Keep the existing [user-facing outcomes](../saving.md#what-the-person-editing-sees) and host-owned -[receipt contract](../saving.md#answer-204-or-a-receipt-and-let-the-watch-echo-it) in the saving guide. - -For proposal 0009's create/delete guidance, use its event-only store input, consolidated connector and -guarded response contract. Successful creates/deletes remain host-owned pending confirmations until -their echo or a completed snapshot confirms state; do not restore unguarded adoption or optimistic -store deletion in a recipe. Keep server acceptance separate from confirmation to avoid duplicate -submissions when an echo is delayed or arrives before the response. - -### Deletion recovery copy - -Demonstrate a host subscription that snapshots detached `store.draft(uid)` on each notification while -that fixed UID exists, including the initial state. When removal or snapshot pruning makes it absent, -retain the last copy for explicit copy-out; do not try to read the removed draft. Capturing only on -Save misses later typing. Specify subscription cleanup, identity-scoped retention and expiry. - -**Acceptance:** edits immediately before deletion and snapshot pruning remain recoverable; a -replacement UID opens separately and never inherits the old draft. Test initial capture, edit-time -updates, null-state handling and disposal in the actual recipe. Keep one active draft store; the -recovery copy is not another reconciler. Link the tested recipe from the saving and Vue guides. - -### Explicit keep-local resolution - -Demonstrate capturing the chosen local value (including absence), resolving that path with -`revert`, then synchronously reapplying it through `setValue` or `removeKey`. Proposal 0009 removes -the equivalent `takeTheirs` alias. Verify the recipe against current behavior before publishing -executable guidance. - -**Acceptance:** cover nested paths, deletion, whole-array replacement and policy/redaction refusal; -preserve unrelated edits and conflicts. Capture a fresh save intent after review. Add a small helper -only if repeated consumer code warrants it; do not create another conflict registry. - -## 3. Real-API composition and identity races - -Implemented as the `TestRealAPI` cases, run by `task test-real-api`. The browser side runs the real store and conditional editor under node. A -real status subresource comes from a test-installed Widget CRD, and a test-only transport barrier -recreates the ConfigMap between the host's preflight GET and its PATCH. The race showed that -Kubernetes checks the captured `resourceVersion` before the UID, so a replacement arrives as a plain -409. The host now classifies it with one GET and answers with structured Kubernetes `Status` -responses. The criteria below are what those cases assert. - -The [existing real-API stale-RV test](../../gateway/kube/e2e_test.go) proves ordinary rejection, not -these compositions. Extend it alongside [store/example tests](../../packages/krm-stream/test/saving.test.ts) -and [host handler tests](../../gateway/kube/examples/conditionalsave/handler_test.go). - -- **Status churn:** use a real status subresource. Prove a persisted status change advances RV, - spec projection suppresses its notification, stale PATCH returns 409, guarded projected GET - advances the base without field conflicts, and a fresh intent succeeds when churn stops. Contrast - full projection and bookkeeping-only suppression. A no-op SSA reapply is not a valid fixture. -- **Overlapping recovery:** save starts live, status advances RV, upstream closure begins a snapshot, - and the post-409 GET arrives before synced. Assert refused reconciliation, retained drafts, - recovery presentation and later progress without blind PATCH retries. -- **UID race:** force deletion/recreation between host preflight GET and PATCH with a test-only - barrier. Preserve structured API Status and prove the replacement is unchanged. Classify the - observed identity mismatch specifically; never normalize all 422 validation errors to conflicts. - -**Acceptance:** observable winner/draft preservation and accurate outcomes, with the exact tested -commit, server version, commands, scenarios and results attached to the PR. Use existing host -validation boundaries; do not generalize the ConfigMap endpoint just to build a test. - -Wire focused real-API cases into CI or an explicitly invoked workflow whose successful run is merge -evidence. Keep broader aggregated-API coverage available through `task test-cluster`. Record skips -and unrun cases separately from fake-client results; update task/workflow comments to match coverage. +| Defined views and suppression | Full/spec Secret-value redaction, status omission in spec, redaction revisions and suppression excluding RV | [Proposal 0004](0004-views-and-bytes.md) | +| Gateway snapshot and shared-watch baseline | Complete snapshot boundaries, UID membership, optional watch sharing, bounded configured HTTP delivery and lifecycle observations | [Watching](../why-a-gateway.md), [proposal 0007](0007-shared-stream-host-integration.md) | +| Shared-watch hardening | Independent/cancellable openings, write-timeout requirement with timed authorization and exact SubjectAccessReview inputs | [Proposal 0008](0008-shared-watch-hardening.md) | +| Browser connector separation and fixes | One fetch/SSE connector delivers state events; bounded recovery, terminal refusal, callback-failure cleanup, protocol diagnostics and adopted-save membership fix | [Proposal 0009](0009-stream-and-editor-separation.md#implemented-connector-and-correctness-fixes) | +| Go API cleanup | Shared `StreamConfig`, consolidated serving/constructor names and internal repository harness | [Proposal 0010](0010-gateway-api-cleanup.md), [migration](../migrating.md) | +| Stream convergence contract | Projected-content guarantee and executable final-write evidence | [Conformance](../../conformance/README.md#convergence-evidence) | +| Optional editor and safe-save baseline | Draft reconciliation, explicit conflicts, atomic intent capture and guarded projected reads; recovery remains review plus another deliberate Save | [Editor model](../client-state-model.md), [saving](../saving.md) | +| Editor integration and recovery recipes | Tested deletion recovery, keep-local resolution and Vue subscription ownership | [Recipes](../../examples/editor-recipes/README.md), [Vue](../../examples/vue/README.md) | +| Real-API save composition | Cases for suppressed churn, guarded-read overlap, structured errors and same-name UID replacement | [Completed baseline](#completed-baseline) | +| Watch-first documentation | README, watch/edit/save guides, native-fetch request and compact decision records; duplicate/superseded guides removed | [README](../../README.md), [current request](../field-reports/third-our-identity.md) | + +## Open work and delivery order + +The numbers express the recommended order of attention, not a requirement to finish every earlier row +before starting a later one. Deliver separately reviewable changes and use the prerequisites below. + +| Order | Open work | Dependencies and completion | +|---|---|---| +| 1 | Native viewer: a short contract, shared lifecycle refactor, LIST/WATCH connector, tests and minimal example in one focused PR | Use today's event consumer and read-only store; re-list on reconnect. Native editing, editor cleanup, resume and benchmarks do not gate this slice. Acceptance lives in the [request](../field-reports/third-our-identity.md#slice-1-a-native-viewer). | +| 2 | Native editing | After the viewer; define source-bound edit policy and host validation, protect machinery fields and preserve save guards. Coordinate any public editor changes with proposal 0009. | +| 3 | Native reconnect efficiency | After the viewer; evaluate resumable watches, streaming-list initialization and pagination with explicit membership/checkpoint tests. Do not claim these efficiencies for slice 1. | +| 4 | Gateway upstream continuation | Independent track with a design below; review reopen/credential bounds, record a baseline, implement and measure. No dependency on native transport. | +| 5 | Save progress during suppressed churn | Independent evaluation using existing real-API fixtures; bounded submitted-intent recovery first, optional version delivery second. Implementation depends on measured benefit and reviewed scope. | +| 6 | Comparative frontend and measurements | Follow working native viewing; add editing after order 2. Compare native/full/spec and shared/unshared workloads; the larger project is outside slice 1. | +| Separate | Editor API cleanup in proposal 0009 | Independently proposed. Keep current standalone `applyStreamEvent` in slice 1; neither bound methods nor a new read-only store are required for native transport. | +| Per change | Validation, release and host adoption | Release ready increments under the existing lockstep policy; do not wait for every track. Check each change's final-commit CI, artifact/API notes and host acceptance. | + +### Next implementation + +Start the native viewer from the branch containing the implemented connector separation and Go API +cleanup. Keep native frame parsing separate from SSE while sharing lifecycle and useful HTTP helpers. +The short read-only contract and test results belong in the implementation PR; no additional +editor-design review or comparison campaign must finish first. + +Preserve existing SSE behavior and public APIs during the lifecycle refactor. Slice 1 requires complete +LIST membership, correct type/UID handling, an accepted WATCH before live readiness, transport-specific +410 recovery, terminal refusals, bounded repeated failure and cancellation. Use unpaginated lists; +reject unexpected continuation rather than pruning a partial snapshot. Add one small runnable native +viewer and document the two sources. See the [request](../field-reports/third-our-identity.md#acceptance-for-slice-1). + +Keep the current standalone state-input helper. Proposed editor removals remain in +[proposal 0009](0009-stream-and-editor-separation.md#proposed-editor-cleanup), with brief API change notes +when implemented. Native editing can follow the viewer without waiting for resume or streaming lists. +Gateway continuation and save measurements can proceed independently. Retain the existing projected +save flow and explicit protocol-version policy; neither needs redesign to deliver a native viewer. + +## Deferred work + +| Item | Trigger for reconsideration | Current approach | +|---|---|---| +| Dedicated `ResourceStore`, shared snapshot-tracker extraction and stream/editor subpaths | Demonstrated viewer or unbundled-loading need with measurable benefit | `LiveResourceStore(readOnlyPolicy)` and the root/combined bundle; no extra store is required for native transport | +| Authorization triggers, grouped checks, decision cache and exported review helpers | Measured duplicate authorization work or a host lifecycle unmet by cancellation/timed checks | Existing per-subscriber checks and host-owned policy; [proposal 0008](0008-shared-watch-hardening.md#deferred-work-and-conditions-for-reconsideration) | +| Version-delivery or automatic save-recovery implementation | Order 5 establishes benefit, safe scope and explicit compatibility | Current suppression and another deliberate Save; evaluating alternatives is scheduled, implementation is conditional | +| Browser replay for gateway SSE, independent delivery switches, write tickets and general SSA abstraction | Separate concrete use case and reviewed protocol/write contract | Fresh browser snapshots, named projections and host-owned conditional merge PATCH | + +## Completed baseline + +Convergence is defined by [the protocol and executable evidence](../../conformance/README.md#convergence-evidence). +The [editor recipes](../../examples/editor-recipes/README.md) cover deletion recovery and explicit +keep-local resolution through store events. Preserve their draft, UID, policy and disposal guarantees +when changing the editor API. + +The `TestRealAPI` composition cases in [composition_e2e_test.go](../../gateway/kube/composition_e2e_test.go) +exercise status-subresource churn, bookkeeping-only suppression, overlapping guarded reads and +same-name UID replacement. They use the real store and conditional editor under node, a Widget CRD +and a test-only preflight/PATCH race barrier. They establish 409 rejection, guarded recovery and a +fresh deliberate save after churn stops; they do not establish usable save progress during sustained +churn. `task test-real-api` runs these cases. Record actual runs, server versions, skips and final +commit separately from fake-client and general CI results. ## 4. Measured upstream continuation @@ -128,7 +106,7 @@ is needed. Downstream v1 remains snapshot-based on a new connection. upstream cannot leave subscribers reporting live forever. Specify the exhaustion transition before implementation; avoid adding public retry knobs without a demonstrated host need. - Recheck the auth implications: fewer cycles mean fewer cycle-only authorization checks and fewer - `ClientFor` refresh calls. Timed subscriber checks must keep running during upstream recovery. + `Clients` refresh calls. Timed subscriber checks must keep running during upstream recovery. Document refreshing credentials and revocation expectations; do not accidentally weaken them. - Keep single-subscriber overflow recovery and the shared cache's ownership intact. Last subscriber departure must cancel an in-progress reconnect. No second shared-watch implementation. @@ -148,10 +126,154 @@ do not label by UID, username or opaque RV. One shared watch still incurs a snap browser reconciliation per subscriber; measure those costs rather than inferring capacity from upstream watch count. +### Proposed design (for review before implementation) + +Nothing below is implemented. It answers the questions this section requires settled first: the +client-go assessment, the checkpoint and boundary rules, the exhaustion transition and the +authorization consequences. + +**Where.** In `gateway/kube`, inside the watcher the backend returns. The gateway core, its snapshot +loop and `SharedBackend` do not change: to them a continued watch is one upstream watch that never +ended. One reopen therefore serves every subscriber of a shared scope, and nothing new reaches the +browser: no event, no reset, no replay protocol. + +**Why not client-go's `RetryWatcher` (v0.36.0).** It has the reopen mechanics but conflicts with +three requirements here. It needs a concrete initial resourceVersion, so it cannot own the streaming +list's initial snapshot and could only take over after the boundary. It retries every failure except +410 and authorization indefinitely, "leaving it up to the user to timeout", where this plan requires +bounded failure. And it turns 401/403 into terminal error events, where this design wants a fresh +cycle so the host's `Clients` can supply refreshed credentials. Wrapping it to impose those rules +would be larger than a small loop in `backend.go` that reuses `translate`, `classify` and the +existing bookmark handling. + +**Checkpoint.** The resourceVersion of the last event the API server delivered on this watch: added, +modified, deleted, a routine bookmark or the initial-events-end bookmark. It is recorded in the +backend, before the gateway projects or suppresses anything, so a suppressed update still advances +it. It is never derived from a browser object, an SSE `seq` or a projected or shared view. Bookmarks +only move it forward sooner; no bookmark cadence is assumed. + +**Boundary.** Continuation is armed only once the snapshot boundary is established: the +initial-events-end bookmark on the streaming-list path, the list's resourceVersion on +list-then-watch. A close before that returns `ErrWatchClosed` as today, so a partial snapshot is +never relabeled as live or complete. + +**Reopen.** When an armed watch closes cleanly (the API server's routine timeout, 30 to 60 minutes +by default), the watcher reopens `Watch` with the scope's selectors, `allowWatchBookmarks` and +`resourceVersion` set to the checkpoint, and without `sendInitialEvents`. Both initialization paths +reopen the same way. The reopened watch's events continue the stream, and the gateway never sees the +seam. + +| Reopen outcome | Result | +|---|---| +| 410 Gone or `Expired`, when opening or as a watch error event | `ResyncRequired`: history is gone, so a fresh snapshot follows, as today | +| 401 or 403 | `ErrWatchClosed`: a fresh cycle reauthorizes and asks `Clients` again, so expired credentials are replaced; a refusal there is terminal, exactly as at opening today | +| Network failure, 429, 5xx or timeout | retry within the bound below | +| Context cancelled, or `Stop` | return at once, abandoning a reopen in progress | + +**Exhaustion transition.** At most three consecutive failed reopens, with jittered backoff of +roughly 250 ms, 1 s and 4 s. These delays total about six seconds; opening/network time is additional +and each reopen needs an explicit cancellable time bound before claiming a total recovery limit. A reopened +watch that closes again before delivering any event, bookmarks included, counts as a failure, so a +server that accepts and immediately drops watches cannot loop. The count resets when a reopened +watch delivers an event. At the bound the watcher returns `ErrWatchClosed`: the gateway starts a +fresh cycle, its existing rule turns a second early end into `UPSTREAM_UNAVAILABLE` and closes the +connection, and the client's own bounded retry budget takes over. Measure time to downstream +recovery/closure including opening delays; the backoff total alone does not bound stale `live` +presentation. No public retry setting is proposed without a demonstrated need. + +**Authorization and credentials.** Each routine upstream close used to start a cycle, and with it a +cycle authorization check and a `Clients` call. With continuation those happen far less often. That +cadence was never a revocation bound: quiet streams already relied on `ReauthorizationInterval`, +whose timed checks run per subscriber and keep running through an upstream reopen, because the cycle +they belong to has not ended. What does change is credential lifetime: a per-user backend is now +used for the whole stream rather than for one routine watch period. The 401 fallback above refreshes +credentials instead of ending the stream, and the [authorization guide](../auth.md) must say that a +backend's credentials should refresh themselves (client-go token sources do) or be renewed through a +fresh cycle. Open question for review: whether to also cap continuation age, so that a fresh cycle +still happens periodically. The proposal is not to, because timed checks and the 401 fallback cover +the cases a cap would; measurements may say otherwise. + +**Shared watches.** `SharedBackend` wraps the upstream backend, so continuation happens beneath its +cache: one reopen per scope and no resnapshot for any subscriber, which also removes a +SubjectAccessReview per subscriber per routine close. Its single-subscriber overflow recovery and +early-end backoff are unchanged and still apply to the fallbacks. The last subscriber leaving stops +the upstream watcher, which cancels a reopen in progress. + +**Evidence plan.** Unit tests with a fake dynamic client cover each row above, the checkpoint on a +suppressed update and on a routine bookmark, deletion during the interruption, a close during the +initial snapshot, exhaustion, cancellation and subscriber departure during backoff, and two shared +subscribers seeing one reopen and no reset. Against the real API server, a package-internal test +sets a short watch `timeoutSeconds` so routine closes happen in seconds, on both the streaming-list +path and the aggregated API's list-then-watch path. Measurements use the same workload before and +after on the spike cluster, with counts from the gateway's Observer and a counting backend: upstream +reopens, downstream resets by cause, snapshot bytes and duration, the browser store's time to apply +a snapshot (the node driver times it), recovery latency, SubjectAccessReview rate and latency, and +the save 409 rate with the share of 409s that carry no field conflicts. Labels stay bounded: no UID, +user or resourceVersion. + **Acceptance:** retained-history recycling preserves downstream continuity, lost history still recovers safely, retries stop correctly, and authorization/credential lifecycle expectations remain explicit. No new SSE events or downstream replay protocol. +## 5. Save progress under suppressed churn + +**Status: requested evaluation, not an implemented retry or delivery feature.** A spec editor should +avoid repeated extra Save clicks when controller changes leave its relevant editable values unchanged. +The adopter has not yet measured frequency in a representative workload. Reuse the real-API fixtures +above and [proposal 0005's tradeoffs](0005-kubernetes-stream-and-save-semantics.md#host-write-strategies). + +Compare today's guarded read plus another deliberate Save with two optional approaches. Evaluate +bounded save recovery first; periodic version delivery still races changes between delivery and PATCH. + +### Bounded recovery of a submitted intent + +After a definite version rejection, refresh and reconcile under the existing UID, revision and +snapshot guards. Determine whether the original submitted intent is still valid by comparing its +relevant base values and dependencies. No field conflicts in the store is insufficient by itself. +A retry must preserve the submitted patch, exclude later typing, protect whole-array replacements +and refuse changed dependencies or replacement UIDs. Each candidate retry captures the validated +original intent and its new precondition together; never just replace the RV on an old patch. + +Keep explicit review for real disagreements. Bound attempts and elapsed recovery, explain exhaustion +and preserve drafts. Distinguish a definite 409 rejection from an unknown write outcome; do not +replay an ambiguous write automatically. Retain host authorization, projection validation and +serialization of writes. Any public recovery policy requires demonstrated use and its own contract. + +### Optional coalesced version delivery + +Retain the latest eligible version per subscriber and UID and compare periodic batches of small +version-only records with batched complete projected updates. Proposal 0005 currently prefers +existing event shapes if measurements justify more version delivery; do not choose a wire extension +in advance. + +Eligibility requires proof that the last delivered authoritative projected content, excluding RV but +including redaction records, still represents that UID at the newer version. The draft is not the +baseline. Bind delivery to the current view and snapshot and apply it in order. A pending record may +not overtake a visible update, resurrect a delete, roll a version backwards or cross a reconnect. +Clear or supersede pending records on those transitions. Collection bookmarks cannot supply an +object's edit version. Hidden Secret changes require normal redaction updates. + +Advance authoritative versions without resetting drafts or waking content-only subscribers. Capture +future save versions and patches together; do not mutate already captured intents when a batch +arrives. Declare which writes can rely on the guarantee, including dependencies outside the view. +Periodic delivery can reduce staleness but cannot eliminate the delivery/PATCH race. + +### Evidence and decision + +Use identical objects, subscriber counts, edits and status rates for the baseline and variants. +Include quiet periods, bursts and sustained churn. Measure stale-version 409s separately from field +conflicts, extra clicks, save completion latency, successful saves and bounded failures during churn, +GET/PATCH attempts, downstream bytes/events, notifications/renders and batch memory/delivery work. +Success after churn stops is insufficient evidence of usable editing during churn. + +Cover later typing, changed array members, concurrent spec edits, deletion/recreation, partial +snapshots, resets/reconnects with pending batches, expired sessions, denied access and hidden Secret +rotations. Preserve current `krm-spec/v1` status-only silence by default. Any new delivery mode needs +an explicit opt-in and compatibility decision; do not silently wake current consumers. + +Document measurements, the decision and adoption guidance. Retaining the baseline is a valid outcome +if added costs outweigh measured benefit. A universal automatic-save policy is outside this request. + ## Verification and scope Use [existing verification tasks](../../CONTRIBUTING.md#test-levels) appropriate to each change, @@ -175,6 +297,7 @@ and sinks have bounded completion times. Exercise quiet and active streams; cycl meet a bounded quiet-stream revocation target. See [authorization lifecycle](../auth.md#session-validity-and-timed-checks) for configuration and host responsibilities. -Version-only events, independent content/delivery switches, downstream replay, write tickets, -automatic conflict-free retry and a general SSA abstraction remain deferred until a concrete use -case and measurements justify them. Preserve current named-projection semantics and host policy. +The save-progress track evaluates submitted-intent recovery and coalesced version delivery without authorizing +implementation or changing default emissions. Independent content/delivery switches, downstream +replay, write tickets and a general SSA abstraction remain deferred. Preserve current named-view +semantics, host policy and the distinction between native browser and gateway upstream recovery. diff --git a/docs/proposals/0007-shared-stream-host-integration.md b/docs/proposals/0007-shared-stream-host-integration.md index a3ec480..3b3ba3f 100644 --- a/docs/proposals/0007-shared-stream-host-integration.md +++ b/docs/proposals/0007-shared-stream-host-integration.md @@ -1,230 +1,41 @@ # Proposal 0007: Shared-stream transport correctness -**Status: historical implementation plan; implemented and released in 0.4.0.** It is kept as the -record of what was decided and why. Its instructions are not adopter guidance, and "current" below -describes the code at the time (`e450334`), not today's. For current guidance see -[authorization](../auth.md), [operations](../operations.md#bounded-http-delivery) and the -[shared ConfigMap host](../../gateway/kube/examples/sharedstream/README.md). +**Status: implemented and released in 0.4.0.** This compact record preserves the decisions; current +configuration and boundaries are in [authorization](../auth.md), [operations](../operations.md) and +the [shared-host example](../../gateway/kube/examples/sharedstream/README.md). -The transport, observation and example increments below have landed. The manual real-cluster scenario has been run at 2 and 200 identities and recorded in -[docs/facts/shared-host-rehearsal.md](../facts/shared-host-rehearsal.md); 200 subscribers cost -exactly one additional API-server watch. That is one run on one disposable cluster, without a proxy, -browsers or resource limits, and is not a capacity guarantee. +## Decisions -## Recommendation - -Centralize bounded HTTP delivery and balanced stream/subscription observations in the library. -Keep identity resolution as a tested example and use existing manual cluster tooling for capacity -measurements. The justification is consistent transport and lifecycle behavior across hosts; -line-count reduction is a possible adoption result, not an acceptance criterion. - -This responds to Voter's 2026-09-11 request against reported release 0.3.0 and the subsequent -scope review. The initial code review used `e450334`. Voter's unpublished implementation and -200-identity rehearsal have not been reproduced here. - -Follow the [design rules](../../CONTRIBUTING.md#design-rules) and -[release policy](../releasing.md). Credentials, sessions, cluster selection, allowlists, RBAC, -writes and metrics export remain host-owned. No wire or browser-store change is planned. - -| Increment | Deliver | Keep out | -|---|---|---| -| 1 | HTTP operation deadlines, capability check, transport-rejection observation and cancellation tests | New transport framework or promise of a universal revocation bound. | -| 2 | Stream close plus shared-subscription open/close observations | New observation fields, HTTP request accounting, authorization telemetry and upstream gauges. | -| 3 | Tested shared-host example with local subject resolution and capacity notes | Public identity helper, new cluster CI workflow or adopter-specific login suite. | - -[Proposal 0006](0006-stream-and-save-implementation-plan.md) remains independently deliverable. -These changes can help satisfy its existing evidence requirements, but are not additional gates. -Hosts can demonstrate bounded sinks and measurements through their own implementations. - -## 1. Bound HTTP delivery - -The current [`sse.go`](../../gateway/sse.go) discards flush errors, ignores the emit context, -and stops only the heartbeat goroutine when its write fails. Teardown waits for that goroutine. -In [`reauthorize.go`](../../gateway/reauthorize.go), downstream delivery holds the authorization -gate, and the periodic callback timeout starts after acquiring it. These paths need testing together. - -Moving `context.WithTimeout` above `gate.mu.Lock()` alone is not a teardown fix: -[`Mutex.Lock`](https://pkg.go.dev/sync#Mutex.Lock) does not accept cancellation. The budget can -expire while the goroutine still waits, and the in-flight sink write still needs to finish. -Do not ship that move as an independent bounded-revocation guarantee. For this increment, retain -`ReauthorizationTimeout` as the periodic callback budget and bound HTTP I/O at its owner. A -cancelable gate can be considered separately if measurements establish a remaining need; it -would still not terminate an arbitrary blocked sink. - -Add `WriteTimeout time.Duration` to `Options` and `Gateway` for HTTP serving only. Zero installs -no library deadline, positive values enable the bound, and negative values are configuration -errors. Validate numeric options at handler construction and before I/O for direct serving calls. -Use five seconds explicitly in the example, not as a universal default. - -Implement within the existing SSE transport and reuse it for normal streams and the identity/scope -`refuse` path in [`handler.go`](../../gateway/handler.go): - -- One serialized operation gets one absolute deadline covering write plus flush. Include initial - headers, heartbeat comments and terminal frames. Use an earlier request deadline where present. -- Clear deadlines after successful operations so quiet streams survive many timeout periods. - Attempt cleanup on exit after transport workers finish. Do not reset a failed transport to - attempt another frame. Document ownership: a previous host deadline cannot be retrieved and restored. -- Use [`http.ResponseController`](https://pkg.go.dev/net/http#ResponseController) for deadline - support and error-aware flushing. Propagate reported flush errors and short writes. An errorless - flusher cannot report every failure; successful flushing is not browser acknowledgement. -- Cancel the internal stream context on header, frame or heartbeat failure and join the heartbeat - worker. Queued operations must not resume after failure. Never abandon a goroutine in a write. -- Preserve ordinary terminal SSE refusals on supported transports, including error propagation - from their initial headers and flush. - -### Helper and compatibility decisions - -Remove exported `WriteSSEHeaders` and replace its serving-path use with an internal error-returning -header operation under the same deadline handling as frames. Update `Handler`, direct serving -entry points and refusal paths together. No compatibility shim; direct HTTP users should use -`Gateway.ServeStream` or `ServeStreamProjection`, which own headers and transport cleanup. - -Keep `NewSSESink(io.Writer)` for generic framing, including the existing buffer-based golden tests. -Its argument need not become `http.ResponseWriter`: the HTTP serving path already has that writer -and can supply controller-backed I/O through internal configuration of the same sink. There must -be one frame encoder. Generic callers receive no HTTP timeout guarantee; propagate flush errors -when their writer exposes an error-returning flush method. Change `SSESink.Heartbeat` to return -an error so its HTTP owner can cancel the stream on failure, and update callers to handle it. - -The increment's release notes must name removal of `WriteSSEHeaders`, the serving-method migration, -the `Heartbeat` return change, the new timeout/check API and error propagation. These are public -API changes under the pre-1.0 policy, even though no new wire behavior is introduced. - -### Unsupported writers - -Check deadline/flush capability before committing SSE headers or opening a subscription. The -request's writer does not exist at `Handler` construction, so middleware capability must be tested -at request time and in the host's integration tests. Transparent wrappers should implement `Unwrap`. - -Add `CheckHTTPStreaming(w http.ResponseWriter) error`, shared by bounded HTTP serving and host -tests. It checks exposed flush support through the controller-compatible wrapper chain and probes -write-deadline support by setting a zero deadline. It must not write headers/body or invoke flush. -Return an error identifying the missing capability, preserving `errors.Is(err, http.ErrNotSupported)` -for unsupported operations. Call only before streaming with no concurrent writer use: the probe -clears any existing write deadline and cannot restore it. It checks exposed capabilities, not -whether arbitrary middleware correctly implements them or whether a proxy buffers responses. - -Add a tested adoption recipe using `httptest.NewServer` with the host's actual middleware chain -around a test-only leaf handler that calls this check and records its result for assertion. Use a -real server writer, not `httptest.ResponseRecorder`, which does not supply network deadlines. -Cover transparent-wrapper success and opaque-wrapper failure. Never mount this probe as a public -production route. Real blocked-client tests remain necessary to establish deadline behavior. - -If the host requests a bound and the writer cannot provide it, abort with `http.ErrAbortHandler`. -Do not attempt an unbounded terminal frame: even a tiny frame can block during flush, so that fallback -would violate the requested guarantee. No new wire error code is needed. - -This failure can cause browser reconnects and must not be described as a clean terminal SSE refusal. -Emit `http_transport_rejected` once before aborting, using the existing observer and internal-error -code, with no raw middleware error or claim of frame delivery. This event has no corresponding -`stream_opened` and does not change a stream gauge. A broken transport cannot guarantee both bounded -completion and an explanatory frame reaching the browser. This tradeoff is deliberate, not a -solution to automatic reconnects. - -### Acceptance - -Use real HTTP/1.1 sockets, TLS HTTP/2 and controlled failing writers. Cover a non-reading peer with -enough data to exhaust buffers, quiet/active healthy streams across multiple timeout periods, -header/refusal and heartbeat-only failures, flush errors, short writes, transparent and opaque -wrappers. Verify HTTP/2 isolation on the same connection. Keep these tests in normal Go CI. - -Measure expiry, cancellation and periodic revocation during blocked delivery while another -subscriber continues. Assert handler/heartbeat completion, subscriber release and eventual last -upstream cleanup with explicit synchronization and declared timing tolerances. - -The write timeout excludes gate waiting, encoding CPU and arbitrary callbacks or backend -`Watch`/`Stop`. Shared opening currently uses a detached context under the backend mutex; do not -claim subscriber expiry bounds it. Generic sinks and host callbacks must cooperate with cancellation. -The total revocation budget still includes timer cadence, in-flight delivery, authorization and cleanup. - -## 2. Balance a small set of lifecycle observations - -Keep the existing `Observer` interface and `Observation` fields. The complete additions are below; -`http_transport_rejected` ships with increment 1, and the three lifetime events with increment 2. - -| Observation | Counting contract | +| Area | Result and boundary | |---|---| -| `http_transport_rejected` | One pre-stream capability rejection when bounded HTTP serving cannot be supported. No logical stream opens, no close is owed, and no SSE error delivery is attempted. | -| `stream_closed` | Pair existing `stream_opened` at `StreamProjection` entry with one close on return, including authorization failure. It excludes HTTP identity/scope refusals before entry. | -| `shared_subscription_opened` | Report successful attachment to a shared scope, including a warm-cache join. This is a new event; no shared-attach observation exists today. | -| `shared_subscription_closed` | Report once when that attachment becomes inactive through overflow, scope death or leave. A later repeated `Stop` must not count again. | - -Retain cycle, overflow, resync, suppression and terminal-error signals. A resnapshot can close and -reopen an attachment on one logical stream without creating a new upstream watch. These counts -are not physical API-server watch counts, HTTP request counts or proof that cleanup has finished. - -Preserve `terminal_error` for logical-stream failure. It already occurs before `sink.Emit` in -`emitTerminal`, so it has never guaranteed frame delivery. In increment 1, clarify this in the -`observe.go` comment and the operations alert row, and add the separate transport-rejection row. -Test that unsupported writers emit exactly one rejection, no `stream_opened` or `terminal_error`, -and open no backend; separately test that a failed terminal-frame write still reports the existing -logical-stream failure. Do not count either diagnostic as a lifetime open or close. - -Callbacks remain synchronous, prompt and concurrency-safe; no lossy library queue, new IDs or -statistics framework. Guarantee open-before-close for each lifetime, not global ordering. Hosts -update counters synchronously and must not block, panic or reenter the gateway from an observer. -Review callback placement under shared locks. Ignore unknown kinds and avoid turning `Scope` -values into metric labels; correct the overly broad safe-to-export wording in `observe.go`. - -**Acceptance:** race-test concurrent/warm-cache joins, denial, revocation, immediate upstream EOF, -recovery, overflow, transport failure and repeated cleanup. Assert intermediate counts and final -balance. Add a small tested counter mapping in the operations example. - -Keep HTTP entry/exit and authorization/SAR instrumentation in host wrappers for now. Defer upstream -handle observations too: they can describe a useful library lifetime, but do not measure physical -API-server watches. This smaller change will not let Voter delete all its watch/metrics wrappers; -retain them until equivalent observations are actually provided. Revisit additions when concrete -remaining integration code demonstrates their value, rather than requiring an arbitrary adopter count. - -## 3. Test the composition and record capacity - -Add `gateway/kube/examples/sharedstream`, following the existing conditional-save example's -placement. Use a local, tested function to issue SelfSubjectReview with a supplied participant -client and copy username, groups, UID and multi-valued extras into `kube.Subject`. Reject missing -username/nil results; preserve API and cancellation errors and independent returned data. No public -resolver API, credential discovery, identity cache, impersonation or guessed OIDC mapping. - -Show one process-wide shared backend, trusted session resolution, one server-selected cluster, -fixed namespace/name policy, participant SSR, service-account SAR/data access, explicit write and -periodic-check timeouts, and the earlier session/token expiry. Keep direct writes participant-owned -and link the existing save example. Browser identity headers must not select the client or override -the resolved subject. A service-account SSR resolves the service account; it is not participant identity. - -Opening SSR and initial/cycle authorization need explicit short callback deadlines in the example: -`ReauthorizationTimeout` covers periodic checks only. Do not impose a short deadline on the whole -healthy stream. Keep backend request timeouts and session deadlines distinct. - -Update [adoption](../adopting.md), [authorization](../auth.md), [operations](../operations.md), -the [gateway README](../../gateway/README.md) and [examples index](../../examples/README.md) -when implementation lands. Explain `2 * N / interval` SARs/second for allowed checks: about 13.3 -at 200 subscribers/30 seconds, plus opening/cycle bursts. Two hundred simultaneous openings can -need 400 SARs and 200 separate SSRs. Early denial can issue fewer; periodic checks may align. -Client throttling, shared rate limiters and API-server capacity all matter. Voter's reported -100-QPS/400-burst setting is evidence for its workload, not a default. - -Compile and test the example in normal CI, including identity copying, cancellation and host -boundary tests. Reuse manual `task test-cluster`/`task cluster-facts` machinery for a focused load -scenario; add an opt-in Task entry only if needed. No new cluster CI workflow. Record actual runs -in `docs/facts/` with commit, versions, workload, proxy/protocols, limits, commands and results. -Keep general guidance in operations/auth docs and unrun scenarios clearly separate from facts. - -Exercise concurrent openings/reconnects, one revoked identity, expiry, a blocked peer, continuing -peers and final cleanup. Use independent API-server evidence for physical watches. The 200-user, -30-second recheck, five-second check and 60-second closure profile from Proposal 0006 is a declared -measurement target, not a guarantee or release-wide load-test requirement. Measure snapshot bytes -and sustained resource/latency behavior; a short rehearsal cannot establish production capacity. -Do not import Voter's login/browser suite or extend Kubernetes support from its k3s 1.31.5 result. - -## Delivery - -Review transport, observations and example changes separately; observations can proceed independently. -For runtime PRs use the [existing checks](../../CONTRIBUTING.md#test-levels): `task fixtures-check`, -`task test`, `task lint`, focused race tests in changed Go modules, and wire/browser tests for HTTP -changes. Attach a manual real-cluster run for claims about actual SSR/SAR composition; do not label -fake-client results or skipped scenarios as cluster evidence. Planning-only changes need link and -whitespace checks. - -Use the lockstep release process and document option defaults, unsupported-writer behavior, flush -errors and event meanings. No release is required just for this proposal. Consumers pin released -artifacts and remove only wrappers whose behavior they have verified is covered; credentials, -scope/session policy and their boundary tests remain in the host. +| HTTP delivery | A configured positive `WriteTimeout` bounds each write-plus-flush operation, including headers, events and heartbeats; a quiet stream can outlive many timeout periods | +| Writer capabilities | Bounded serving requires flushing and write deadlines; unsupported middleware aborts before a logical stream opens | +| Lifecycle observations | Balanced stream and shared-subscription open/close events; bounded labels and synchronous, prompt callbacks | +| Host composition | Tested session/subject resolution, timed checks and write bounds in an example, rather than a public identity framework | + +The library owns HTTP deadlines during serving, clears them after successful operations and at exit, +and cannot restore a previous host deadline. A whole-response server timeout is not a per-operation +bound. HTTP/1.1, TLS HTTP/2, peer isolation and transparent/opaque wrappers have focused tests; mounted +middleware and proxy combinations still need host checks. Generic sinks are the host's to bound. + +Timed authorization and delivery share a subscriber gate. Write bounds matter to revocation, but +encoding, backend opening, gate waiting and arbitrary host callbacks are not bounded by the HTTP +write timeout. [The revocation budget](../auth.md#revocation-budget) owns that explanation. + +Stream observations count `Stream` entry/return, including authorization refusal but excluding earlier +HTTP identity/scope refusals. Shared-subscription observations count attachments, including warm-cache +joins; overflow or scope loss can replace an attachment without a new logical stream. Callbacks may +run concurrently under shared locks, must return promptly and must not reenter the gateway. + +## Capacity and deferred scope + +The [recorded shared-host rehearsal](../facts/shared-host-rehearsal.md) reports 200 subscribers under +a declared workload. Its authorization rate is part of the cost: allowed checks require list and watch +reviews per subscriber. Watch sharing reduces upstream duplication, while snapshot delivery, browser +work and authorization still scale with subscribers. The rehearsal is not a universal capacity claim. + +Public identity helpers, authorization caches, grouped scheduling and new metric APIs were not part +of this change. Use existing host wrappers and bounded labels. [Proposal 0008](0008-shared-watch-hardening.md) +records subsequent hardening; [proposal 0006](0006-stream-and-save-implementation-plan.md) owns remaining +continuation and save-progress evaluations. Native browser transport leaves these gateway boundaries +intact. diff --git a/docs/proposals/0008-shared-watch-hardening.md b/docs/proposals/0008-shared-watch-hardening.md index 80f5997..85ff7e2 100644 --- a/docs/proposals/0008-shared-watch-hardening.md +++ b/docs/proposals/0008-shared-watch-hardening.md @@ -1,214 +1,47 @@ -# Proposal 0008 Shared watch hardening implementation plan +# Proposal 0008: Shared-watch hardening -**Status: historical implementation plan; selected scope implemented, asks 11 and 12 deferred.** -It is kept as the record of what was decided and why; its instructions were for the implementer and -are not adopter guidance. For current guidance see -[session validity and timed checks](../auth.md#session-validity-and-timed-checks), the -[revocation budget](../auth.md#revocation-budget), -[what the SubjectAccessReview asks](../auth.md#what-the-subjectaccessreview-asks) and -[shared backends](../../gateway/README.md#shared-backends). +**Status: selected scope implemented.** This decision record replaces the old adopter issue inventory +and implementation handoff. Use [shared-watch authorization](../auth.md#shared-watch-authorization), +[the revocation budget](../auth.md#revocation-budget) and [operations](../operations.md) for current setup. -This is the implementer handoff for the [second krm-foyer field report](../field-reports/second-fieldreport.md). -Implement ask 9, the explicit HTTP configuration validation option from ask 10, and the -documentation portion of ask 13. These changes repair shared-watch opening and make the -requirements for timed authorization clearer. Asks 11 and 12 are deferred; they are not -acceptance criteria for this work. +## Decisions and guarantees -The local source review used commit `869ea81`. The field report describes a different baseline, -`5ca19ed`, and external krm-foyer reproductions. Those reproductions have not been run in this -repository. Confirm the implementation baseline and add repository-owned tests before changing code. +| Problem | Implemented decision | +|---|---| +| A blocked upstream open held the shared backend lock and unrelated scopes | Open each scope independently outside the backend-wide lock, with a shared opening context and cancellable callers | +| The last waiting caller could leave an opening running | Last-waiter departure cancels and discards the opening; last-subscriber departure stops an established watch | +| A blocked HTTP write could delay timed authorization indefinitely | Positive timed reauthorization requires a positive HTTP write timeout, validated at construction/direct serving | +| Hosts needed exact authorization inputs for safe caching | Document and test the adapter's complete SubjectAccessReview request | -## Decisions and work order +A caller's context bounds its own wait rather than the lifetime of other subscribers' shared watch. +An abandoned opening must not overwrite a replacement or publish its watcher; stop any discarded +watcher. Shared scope backoff and overflow resnapshot behavior remain unchanged. -| Order | Deliver | Acceptance evidence | -| --- | --- | --- | -| 1 | Ask 9: independent, cancellable shared-watch opening | Deterministic concurrency tests and Go race checks | -| 2 | Ask 10: require a positive HTTP write timeout with timed reauthorization | Construction and direct-serving validation, plus blocked-reader HTTP coverage | -| 3 | Ask 13: document the actual SubjectAccessReview request | Complete request assertions and matching API and adoption documentation | +`ReauthorizationTimeout` starts after the periodic check obtains the delivery gate. It bounds that +check's callbacks, not opening/cycle authorization or the entire revocation interval. Host callbacks +must honor contexts and generic sinks must bound delivery. A write timeout is necessary to bound +blocked HTTP I/O but does not make the total revocation budget equal to the callback timeout. -Deliver these as separate reviewable changes. They do not depend on authorization caching, -event-driven policy integration, new wire events, or browser changes. Keep the existing save and -draft-recovery work in [proposal 0006](0006-stream-and-save-implementation-plan.md) independently -deliverable. Follow [design rules](../../CONTRIBUTING.md#design-rules) and -[release policy](../releasing.md). - -## 1 Shared watch opening - -### Required behaviour - -In [shared.go](../../gateway/shared.go), move upstream `Watch` calls outside the backend-wide -mutex. Waiters must be able to abandon an opening through their own contexts. The upstream -opening must have a scope-owned context: cancelling the first caller must not cancel an opening -that another caller still needs. - -Maintain one current opening per scope. Register an opening under the lock, perform network work -without that lock, and publish its result only if that entry is still current. Count waiting callers -so the final departure cancels and removes an unfinished opening. Waiters for unrelated scopes must -continue independently. Keep the current live subscription lifetime: once attached, `Stop` releases -the subscription and the last subscriber cancels the shared upstream. - -Handle the handoff between opening and subscription explicitly. There must be no gap in which the -opening has no waiters or subscribers but retains an upstream watch. If an abandoned upstream -returns a watcher late, stop it and discard it. Its result must not delete a replacement opening, -populate its cache, or change its backoff. - -The implementation may use a private opening entry with a completion signal and waiter accounting; -no public opening API is required. Preserve existing snapshot framing, cache ownership, queue -overflow recovery, failure classification, backoff and lifecycle observations. A cancellation caused -only by all waiters departing is not an upstream failure and must not impose outage backoff. - -### Acceptance tests - -Use channel barriers and a fake upstream that can block inside `Watch`; avoid sleep-based ordering. -Tests must cover: - -- Scope A blocks while scope B opens and delivers its snapshot. -- Concurrent callers for the same scope cause exactly one current upstream opening. -- Cancelling one waiter returns promptly without disrupting remaining waiters. -- Cancelling the final waiter cancels the upstream context, removes the opening and permits a retry. -- A caller whose context is already cancelled does not create or retain an upstream opening. -- Cancellation racing with successful opening leaves neither a watcher nor a subscription orphaned. -- An abandoned opening returning late cannot mutate a newer entry; any returned watcher is stopped. -- A genuine retryable opening failure updates backoff once, and all relevant waiters receive the - classified failure. Existing terminal failures and retry hints retain their behaviour. -- Warm-cache joins, last-subscriber cleanup, overflow and balanced observations remain correct. - -Cancellation guarantees assume the upstream honours context cancellation. Do not promise to forcibly -terminate an arbitrary host backend; ensure its late result cannot affect current state. - -## 2 HTTP configuration for timed reauthorization - -### Chosen implementation - -Require `WriteTimeout > 0` whenever `ReauthorizationInterval > 0` for library-owned HTTP serving. -Reject the invalid combination with a clear configuration panic, consistent with existing option -validation. Do not silently select a timeout. - -Validate at `Handler` construction and before any response I/O in direct `ServeStream` and -`ServeStreamProjection` calls. Share private validation where useful so these paths cannot diverge. -Retain existing rejection of negative write timeouts. A zero write timeout remains valid for HTTP -streams with timed reauthorization disabled. - -Transport-neutral `Stream` and `StreamProjection` must continue to support timed checks with -host-owned bounded sinks. Do not require an HTTP-only option for those methods. Update the -`Options` and `Gateway` comments to distinguish these paths. - -Preserve the delivery gate and existing per-subscriber authorization and projection checks. This -increment does not introduce a cancellable gate, independent authorization scheduler, or force-close -API. Positive timeout capability checks must retain their current fail-before-streaming behaviour. - -### Documentation and limits - -Update [auth.md](../auth.md), [operations.md](../operations.md), [adopting.md](../adopting.md), and -affected examples. The timed-recheck example must set both authorization and write budgets. - -Explain that `ReauthorizationTimeout` starts after gate acquisition and covers the periodic callbacks. -Describe the revocation budget in one place, including timer scheduling, gate waiting, callback -timeout, host cache freshness, and termination/cleanup. Link to it from the other guides. -Do not turn interval + callback timeout + one write timeout into an unconditional bound: -terminal delivery, competing I/O, scheduling and host cleanup can add time. Distinguish cessation of -new object delivery, request return, and browser receipt. Already buffered bytes cannot be recalled. -Opening and cycle authorization still require host callback deadlines. - -### Acceptance tests - -- `Handler` rejects timed reauthorization with zero write timeout at construction. -- Both direct HTTP entry points reject that combination before headers, observations for logical - stream opening, or backend calls. Use a writer spy to prove no response I/O occurred. -- Positive write timeout with timed checks is accepted; zero timeout without timed checks remains - accepted. Generic bounded sinks can use timed checks without an HTTP write timeout. -- Reuse or extend the real HTTP blocked-reader tests in [sse_test.go](../../gateway/sse_test.go). - Fill transport buffers, revoke access, and verify that the request terminates within a declared - test budget and releases its subscription. A healthy subscriber must remain unaffected. -- Preserve HTTP/1.1 and TLS HTTP/2 coverage, unsupported-writer rejection, quiet-stream survival, - and authorization/projection denial tests. Do not require a terminal frame to reach a stalled client. - -Record which coverage is reused and which scenario is new. State the test's callback, write and -scheduling assumptions; a generous test tolerance is not a published production guarantee. - -This is a breaking configuration change. Include a `BREAKING CHANGE:` footer explaining that HTTP -hosts using timed reauthorization must choose a positive per-operation `WriteTimeout` and verify -their mounted middleware supports deadlines and flushing. Update repository callers together. - -## 3 SubjectAccessReview documentation - -Document [SubjectAccessReviewAuthorizer](../../gateway/kube/authz.go) as currently implemented: - -- Subject inputs are username, groups, UID and extras, supplied by the host's `SubjectFor`. -- The adapter checks `list` and `watch`, stopping on the first refusal or failure. -- Resource attributes are verb, group, version, resource, namespace and name. -- Label and field selectors and subresource are not supplied. Target and projection are not SAR - resource attributes; the configured client selects the authority receiving the review. - -Add this contract to the exported API comment and shared-watch authorization guide. State that -selectors currently do not narrow the authorization question. Keep this description specific to the -adapter; do not generalize it into a guarantee about all Kubernetes authorization policies. - -Extend [authz_test.go](../../gateway/kube/authz_test.go) to assert both verbs, every transmitted -subject/resource attribute, and the omitted selector/subresource fields. Include a named scope and -a scope carrying selectors so omissions are deliberate and tested. - -Explain that a host cache must distinguish the complete subject and request, including verb, and -isolate decisions by authority or target binding. Reusing a SAR result must not bypass current host -session validity or projection checks. No cache implementation or exported attribute builder is -part of this increment. - -Future changes to the documented authorization inputs require compatibility review and explicit -release notes for caching hosts. The current documentation change does not itself change SAR -semantics. Do not manually edit generated changelogs; use conventional commits and PR details. +The kube adapter checks `list`, then `watch`, using the host-provided subject and the scope's group, +version, resource, namespace and name. Selectors and subresource are not review attributes. Target +selects the review client rather than a Kubernetes review field. Host caches must distinguish full +subject, request/verb and API-server authority, and retain session/projection checks. ## Deferred work and conditions for reconsideration -**Ask 11:** no gateway-wide `Reauthorize(predicate)` API in this plan. Reconsider a minimal trigger -only with a concrete host integration, stream lifecycle and trigger-coalescing rules, a distinction -between scheduling and completion, and coordination with any authorization cache invalidation. -Policy watches and determining affected subjects remain host-owned. Preserve periodic checks as a -fallback, and do not promise immediate revocation from a trigger alone. - -**Ask 12:** no grouping scheduler and no decision cache in this plan. `Principal` is opaque; matching -users does not establish interchangeable sessions or projection policy. Correct the workload before -evaluating benefit: 200 users with nine tabs on one scope produce about 120 periodic SARs/s at a -30-second interval and could reduce to about 13/s with deduplication. Nine distinct scopes per user -still require about 120/s when grouping by principal and scope. These counts assume both verbs are -allowed and exclude opening and recovery checks. - -An optional kube cache can be reconsidered with measured duplicate requests, a complete key, -authority isolation, bounded memory, explicit TTL and invalidation, error handling and waiter -cancellation semantics. Deduplicating SARs must remain separate from subscriber-specific host -authorization and projection decisions. An exported attribute builder also remains deferred. - -## Verification and completion - -For each implementation PR, add focused tests proving the changed behaviour. Before opening the PR, -run `task fixtures-check`, `task test`, and `task lint` as required by CONTRIBUTING. Run the affected -Go package tests with the race detector for shared opening changes. Inspect existing test/task setup -to run each Go module correctly. No new wire fixture is needed unless implementation unexpectedly -changes the shared protocol contract; such a change requires separate scope review. - -The final handoff from the implementer must identify the tested commit, commands and results, -unrun external scenarios, compatibility change, and remaining limitations. All three selected -increments must be complete; deferred asks must not be reported as implemented. No benchmark, -external krm-foyer reproduction, universal capacity claim, or new real-cluster campaign is a merge -requirement for this bounded work. - -## Draft response to the krm foyer team - -Thanks for the concrete failure cases and the explanation of your workarounds. We are taking on -ask 9, the explicit HTTP validation option in ask 10, and documentation of the current SAR inputs -from ask 13. +Public authorization recheck triggers, grouped per-principal/scope scheduling, decision caches and +exported attribute builders remain deferred. Revisit them only with measured duplicate requests or a +specific host lifecycle that existing request cancellation/timed checks cannot handle. -We will isolate upstream openings by scope and cancel an unfinished opening when its final waiter -leaves. For HTTP streams with timed authorization, we will require an explicit positive write timeout -and document the budget and its limits. We will document the exact review inputs and add tests -protecting them. +Any future cache needs complete subject/request/authority keys, explicit validity and invalidation, +failure behavior and subscriber isolation. Grouping must not let one expired session retain access +because another session still passes. Triggers need defined interaction with delivery gates, pending +checks, teardown and cancellation. Do not infer a cache or immediate revocation guarantee from sharing. -We are deferring the public trigger API, grouped authorization scheduler, cache and exported -attribute builder. These need additional contracts around session identity, policy checks, -invalidation and measured duplicate load. Please distinguish nine tabs on one scope from nine -distinct scopes in the load example; grouping only removes duplicate questions for the same pair. +## Evidence and limits -Please share the opening-cancellation and blocked-reader reproductions with their tested commits and -transport settings. They will help us compare repository tests with your integration. Also include -verb and authority isolation explicitly in the cache-key description. We will provide a configuration -migration note with the timed-authorization change. This scope does not promise immediate revocation -or a universal bound for arbitrary host callbacks and sinks. +Focused tests cover independent openings, caller/last-waiter cancellation, discarded/replaced opens, +shared scope lifetime, unsafe configuration refusal, blocked-reader bounds and exact review attributes. +Adopter reproductions motivated the changes; repository tests establish their own scope. These changes +add no new wire event, authorization framework or capacity claim. Actual runtime PR evidence remains +in history; [proposal 0007](0007-shared-stream-host-integration.md) retains the earlier transport decisions. diff --git a/docs/proposals/0009-stream-and-editor-separation.md b/docs/proposals/0009-stream-and-editor-separation.md index 0ee0396..e3fc5d5 100644 --- a/docs/proposals/0009-stream-and-editor-separation.md +++ b/docs/proposals/0009-stream-and-editor-separation.md @@ -1,505 +1,121 @@ -# Proposal 0009 Stream and editor separation - -**Status: in progress.** Change 1, one connector that delivers state events, is implemented in -[#53](https://github.com/ConfigButler/krm-stream/pull/53); the host-callback exception rule and the -adopted-save `fix:` follow on the same stack. Change 2 is proposed. The read-only store and the -stream/editor entry points are [deferred](#deferred-until-a-read-only-view-needs-them) until a -read-only view needs them. Source review baseline: `a8281c5`, 2026-10-04. - -krm-stream should make live Kubernetes resources easy to consume, with editing available as an -additional layer. Keep one repository, one npm package and the existing lockstep release process. -Separate browser transport and authoritative resource state from local drafts, three-way -reconciliation, conflicts and patch capture. The Go gateway continues to serve the same v1 protocol. - -Follow the [design rules](../../CONTRIBUTING.md#design-rules) and [release policy](../releasing.md). -This proposal owns the separation; existing behavioral follow-ups remain in [proposal -0006](0006-stream-and-save-implementation-plan.md). - -## Why live watches and three-way merge belong together - -This section is background for readers new to the library. The rest of the proposal assumes it. - -### A live copy of a shared object - -A Kubernetes object is shared. Controllers write `status`, an autoscaler changes `spec.replicas`, -GitOps tooling and other people edit labels and spec. A browser page holds a copy of something other -parties keep changing. - -The stream keeps that copy current. The gateway authorizes the user's subscription, trims each -object to what the user may see (the projection), and sends complete projected objects as visible -state changes. It may suppress bookkeeping-only changes or coalesce intermediate updates. The -browser replaces its authoritative copy with each delivered object. After a reconnect the gateway -sends a fresh snapshot (`reset` … `synced`), and the browser drops whatever was not resent only when -that snapshot completes. The copy also recovers from deletes it missed while disconnected. - -A resource list or status page needs nothing more: replace the object, re-render. - -### Editing an object that keeps moving - -Now put a form on the same object. Someone starts changing the image tag. Meanwhile the autoscaler -raises `spec.replicas` from 3 to 5, and the stream delivers the new object. The page can do one of -three things: - -| Approach | What the person sees | Result | -|---|---|---| -| Replace the form with the new object | Their typing disappears | Lost work | -| Ignore updates while the form has edits | `replicas` still shows 3 | A stale form. A guarded save fails with 409; an unguarded full-object save puts `replicas` back to 3 | -| Three-way merge | `replicas` changes to 5 and the image tag stays as typed | Both changes kept. If both sides changed the same field, the form shows a conflict | - -Replacing a form on every live update destroys edits. Freezing its server state while editing leaves -it stale. Three-way reconciliation lets this editor incorporate live state while preserving the -person's draft and exposing conflicts for review. - -### What the merge compares - -The editor keeps two objects per resource: `server`, the last object the stream delivered, and -`draft`, that object plus the person's edits. When a new object arrives, each editable field is -compared three ways: - -- **base**: the field in the current `server` object, which the draft was built against; -- **ours**: the field in the `draft`; -- **theirs**: the field in the incoming object. - -If only theirs differs from base, the draft follows the server and the UI can highlight the change. -If only ours differs, the edit stays. If both changed it to the same value, they have converged. If -both changed it differently, the edit stays and the field is recorded as a conflict. The incoming -object then becomes `server`, and so the next base. The [client state -model](../client-state-model.md#editing-and-conflicts) has the full table. - -```mermaid -sequenceDiagram - participant K as Kubernetes - participant G as Gateway stream - participant E as Browser editor - participant H as Host save endpoint - G->>E: replicas 3, image v1 (becomes the base) - Note over E: person types image v2 (draft) - K->>G: autoscaler sets replicas 5 - G->>E: replicas 5, image v1 - Note over E: replicas follows the server, image v2 is kept,
the incoming object is the new base - E->>H: merge patch with only image v2, plus UID and base resourceVersion - H->>K: conditional PATCH - K->>G: the watch sees the write - G->>E: replicas 5, image v2, so the draft is clean again -``` - -### Why the editor uses the stream's guarantees - -A merge is only as correct as its inputs. The stream supplies an authoritative base under a defined -identity, projection and recovery contract: - -- Each resource update replaces the visible server copy, so removed fields do not linger. -- Resources are keyed by `uid`, so recreating a name never transfers an old draft to a new object. -- A completed snapshot repairs missed updates after reconnect; surviving resources keep their - drafts. -- A captured save binds the person's patch to the identity and version they reviewed. - -Projection and redaction determine what the person may see and edit. Snapshot pruning can remove a -deleted resource's draft, and merge patches replace arrays as whole values. The [client state -model](../client-state-model.md) and [saving guide](../saving.md) explain those limits. - -The live editor consumes the protocol's state; transport does not depend on the editor. The editor -can also reconcile directly supplied server objects without opening a connection. The current -connectors do not show that boundary: a list page must construct an editing store to use them. This -proposal makes transport deliver protocol events, adds a small read-only store for lists and -viewers, and keeps the editor as the layer a live form adds. Both stores use the same stream -contract. - -## Current coupling and scope - -The [protocol](../../spec/v1.md#10-conformance) already makes editing optional. Today all three -connectors take a concrete `LiveResourceStore`, and transport produces editor-specific -`StreamChange` results. The managed connector also detects reset through the editor's `onChange` -callback. The [root entry](../../packages/krm-stream/src/index.ts) exports transport and editing -alongside implementation utilities. - -The connector imports of the store are type-only and disappear from JavaScript. The work is to -separate their contracts and state application, expose a read-only store, and simplify the public -browser API while touching these modules. Preserve merge behavior, gateway behavior and v1 wire -framing. [Proposal 0010](0010-gateway-api-cleanup.md) owns separate Go API cleanup. - -## Ownership and dependencies - -```mermaid -flowchart LR - gateway["Go gateway"] --> transport["Browser connector"] - transport -->|"State events"| resources["ResourceStore"] - transport -->|"State events"| editor["LiveResourceStore"] - resources --> viewer["Live list or viewer"] - editor --> form["Live form with drafts and conflicts"] -``` - -| Layer | Owns | +# Proposal 0009: Stream and editor separation + +**Status: partly implemented.** One event-based connector, +host-callback failure handling and the adopted-save snapshot-membership fix are implemented. +Editor API cleanup remains proposed. A dedicated read-only store and separate entry points remain +deferred. Current APIs are documented in the [client README](../../packages/krm-stream/README.md) and +[upgrade guide](../migrating.md); this proposal does not advertise future APIs as available. + +## Direction and ownership + +Start with watch streams and live resource state; editing is an optional layer. Keep one repository, +one npm package, zero browser runtime dependencies and lockstep releases. Follow the +[design rules](../../CONTRIBUTING.md#design-rules) and [release policy](../releasing.md). + +The connector owns decoding, ordering, connection state, cancellation and bounded recovery. State +application owns complete resources, UID membership and snapshot pruning. The editor adds drafts, +edit policy, three-way reconciliation, conflicts and patch capture. The host owns authentication, +authorization policy, credentials, UI, writes and deletion recovery copies. + +The [editor model](../client-state-model.md) explains why a moving server copy and local draft belong +together; [saving](../saving.md) owns the conditional-write contract. Keep those explanations there +rather than duplicating them in this implementation plan. + +The [native-watch request](../field-reports/third-our-identity.md#native-watch-connector) extends this +separation: native is the original-resource source through a host proxy; the gateway provides +projected views over SSE. Share lifecycle and state application without converting native JSON to SSE. +The first native slice is a read-only viewer using today's event consumer and standalone +`applyStreamEvent`. It does not depend on the editor cleanup below or a new store. Native support +is not implemented here; native editing follows separately. + +## Implemented connector and correctness fixes + +`connectResourceStream(url, consume, options)` is the one current connector. `consume` receives +`ResourceStateEvent` synchronously and independently of a store. It can call the standalone +`applyStreamEvent(store, event)` or another state consumer. The connector handles wire sequence and +errors privately; its handle exposes `state`, `subscribe`, `close` and `closed`. + +- Reset publishes `syncing` before application; `synced` is applied before publishing `live`. +- Sequence gaps discard the event beyond the gap and recover through a fresh snapshot. Gap details + appear on the resulting `retrying` state. Unknown wire events still participate in sequencing. +- HTTP refusals and in-stream errors preserve terminal/retry classifications and retry hints. +- Consumer, state-subscriber or error-handler exceptions stop recovery, finish cleanup and reject + `closed` with the first host exception. They must not become network retry storms. +- Close/abort prevents later events, including remaining events in a decoded chunk. Close during + `synced` prevents a later `live` publication. +- A mismatched optional protocol header is terminal before event application; an absent header is + accepted. No wire negotiation was added. +- Adopted save responses no longer mark snapshot membership. Only authoritative stream upserts + establish membership; a late response cannot preserve a UID absent from a completed snapshot. + +The callback must finish synchronously. An async callback type-checks but cannot establish these +application-before-live and exception guarantees. Callers handle `closed` rejection and dispose all +subscriptions. Native `EventSource` framing/cookie compatibility remains tested separately from the +reference fetch connector's retry policy. + +## Proposed editor cleanup + +Keep reconciliation behavior and the gateway v1 contract. Reduce public input and duplicate exports +as one separately reviewable breaking change: + +| Area | Proposed change | |---|---| -| Gateway | Authorization enforcement, projection, watches, framing and SSE delivery | -| Browser connector | Decoding, sequence checks, connection state, errors and bounded retries | -| Read-only store | Complete projected objects, redactions, UID membership and snapshot pruning | -| Editor store | Authoritative base, drafts, reconciliation, edit policy, conflicts and patch capture | -| Host | Credentials, identity, authorization policy, UI, writes and retention of deleted drafts | - -A view chooses one active store. An editor keeps its own authoritative base; it does not mirror a -`ResourceStore`. Both stores use the same internal snapshot tracker, with a separate instance per -store. Until a read-only view needs `ResourceStore`, the editor is the only store, and a list -renders from a `LiveResourceStore` with a read-only policy. Share small value and dispatch helpers -where useful. Keep framework adapters and a generic merge library outside this work. - -## One public connector - -Make the existing managed fetch connector the only public connector, named `connectResourceStream`. -Move the existing single-connection fetch loop behind it as a private implementation. Remove -`connectManagedResourceStream` and `connectWithEventSource` rather than forwarding their old names. -Fetch already supports both same-origin cookies and bearer headers; a second public transport is -unnecessary for either authentication case. - -The native EventSource helper closes on a sequence gap without arranging a new snapshot connection, -and leaves reconnect timing to the browser instead of enforcing the protocol's retry hints. This is -a limitation of that helper, not a reason to remove native SSE support from the gateway. - -The proposed public signature is: - -```ts -type ResourceEventConsumer = (event: ResourceStateEvent) => void; - -function connectResourceStream( - url: string, - consume: ResourceEventConsumer, - options?: ResourceStreamOptions, -): ResourceStreamHandle; -``` - -`ResourceStreamHandle` keeps `state`, `subscribe`, `close` and `closed`. `ResourceStreamOptions` -keeps `onError`, `signal`, injected fetch, credentials, headers and the existing retry settings. -Remove public `onOpen`, `onSynced`, `onGap`, `onStateChange` and `onChange`. Observe connection -state through `state` plus `subscribe`, and errors through `onError`. Store results or subscriptions -drive rendering. A consumer already receives `synced` when it needs per-snapshot work. - -Keep the existing statuses: `connecting`, `syncing`, `live`, `retrying`, `closed`, `terminal` and -`exhausted`. Defer opening until the handle exists so subscribers can observe the initial -transition. Read `state` for initial presentation; `subscribe` observes subsequent publications. -Preserve a `live` publication after every completed snapshot, including when the previous status was -live. A sequence gap must remain diagnosable: add an optional -`gap: { expected: number; received: number }` to the `retrying` state caused by that gap, clearing -it on the next attempt. Keep other diagnostics in `onError`; do not add another lifecycle callback -or a general retry telemetry API. - -### Delivery and completion - -1. Decode and check the per-connection sequence before delivering state. On a gap, discard the event - beyond the gap, close the connection and request a fresh snapshot through bounded retry. -2. Observe `reset` directly in transport: enter `syncing` and clear the healthy-period timer before - state application. This must not depend on what a store returns. -3. Deliver recognized state events exactly once and synchronously, in stream order. Unknown wire - event types still participate in sequence accounting and are then ignored. Error events go only - to `onError`. Discard an upsert or deletion without its required object or identity; this - preserves current no-op state behavior and is not comprehensive object schema validation. -4. Apply `synced` through the consumer before publishing `live` and starting the healthy-period - timer. Pruning already precedes `live` today; rendering moves from the old `onChange`, which ran - afterwards, into state application before `live`. -5. Preserve retry hints, exhaustion, terminal refusal and in-band `RESYNC_REQUIRED` recovery. A - retryable error does not itself restart an open connection; reconnect when that connection ends. -6. After close or abort, deliver no later events from the same decoded chunk. If the consumer closes - during `synced`, publish no later `live` transition. The existing fetch loop already checks abort - between events; preserve this and test it. - -The consumer must complete state application synchronously. An `async` consumer type-checks, because -its result is ignored, but returns before applying anything and its exceptions never reach `closed`; -the documentation says so rather than adding a runtime check. Both stores expose `applyStreamEvent` -as a bound arrow field, so passing `store.applyStreamEvent` directly is safe and preserves `this`. -Verify direct method passing as well as wrapped consumers that render the returned change. - -An exception from host code is a bug, not a network failure, so it is never retried. The rule covers -the consumer, `subscribe` callbacks and `onError` alike. Review found the first implementation -handled only the consumer: an observer that threw during a connection was swallowed as a network -failure and retried until the budget ran out, so a render bug became a reconnect storm, and one that -threw between connections rejected `closed` without publishing `closed`. Capture the first exception -where it is thrown, even after the callback called `close()`. Finish reader/listener/timer cleanup, -still notify every subscriber of the state being published, publish `closed` unless the stream -already ended `terminal` or `exhausted`, and reject `closed` with that exception. Do not classify it -as a Kubernetes error. Ordinary network failure still follows the existing retry policy. Callers -await `closed` or attach a rejection handler; tests explicitly observe failure, including -close-then-throw. Completion settles only after the callback returns and cleanup finishes. - -### Protocol version diagnostics - -On a successful streaming response, inspect `X-KRM-Stream-Protocol` before consuming events. A -present value other than the supported version is a terminal local `INTERNAL` error with a clear -protocol-mismatch message: cancel the response body, publish `terminal` and do not retry. A missing -header is accepted because [v1 versioning](../../spec/v1.md#0-versioning) makes this diagnostic -header optional. HTTP refusals retain their existing classification. No version negotiation or new -wire error is introduced. Keep the protocol constant internal and its Go/TypeScript consistency -test. Keep public `VERSION`: build provenance has an established use for vendored assets. - -## One public stream input per store - -Export `ResourceStateEvent`, a discriminated union of `reset`, `added`, `modified`, `deleted` and -`synced` state events. Upserts require `object` and may carry wire-shaped `Redaction[]`; deletion -requires `identity`. Reset keeps target, scope and projection metadata. This input does not require -`seq` and carries no transport error fields. The connector retains wire sequencing privately and -passes state events; hosts can supply the same state shapes directly without opening a connection. - -Both stores expose a bound `applyStreamEvent(event)` method. In `LiveResourceStore`, make -`beginSnapshot`, `endSnapshot`, `applyServerEvent` and `removeResource` private. Remove the -standalone `applyStreamEvent(store, event)` export and fold public `ApplyResult` into -`StreamChange`. Remove `ApplyOptions`: direct state input accepts only the wire redaction shape, -while stored/read redactions may still use parsed path segments. Tests must use the public event -input for behavior; private helpers remain available for the editor's guarded reconciliation -implementation. - -### Shared snapshot tracking (deferred) - -Deferred with `ResourceStore`: extract the tracker when a second store needs it. Until then the -editor keeps its private membership state, where the membership rule below already holds: only -stream upserts mark a UID as seen. - -Extract `#seen` and `#snapshotRevision` into one internal `SnapshotTracker` before adding -`ResourceStore`. It owns only the active membership pass, seen UIDs and generation: - -- `begin()` increments generation and starts a fresh seen set, even when replacing a partial pass. -- `markSeen(uid)` records membership only during an active pass. -- `finish(currentUIDs)` returns unseen UIDs and ends the pass before removal and notification. - Without an active pass it returns an empty list. This is the shared implementation of pruning. - -Only `applyStreamEvent` upserts mark membership. Guarded reads follow a separate private update path -and never mark a UID as seen. Reconciliation captures generation and rejects while a pass is active -or after generation changed, including after a completed newer pass. Preserve per-resource revision -and UID checks in the editor. Use both stores to verify interrupted and restarted snapshots; use the -editor to verify late reads before, during and after a recovery cycle. Keep tracker internals -private. - -### Read-only store (deferred) - -Deferred until a read-only view needs it. By this proposal's own account the reference use is the -repository's example, not an external request, and krm-foyer, the one known consumer, edits. The -design stays here so the store can be built as its own change when a list or viewer needs it. - -Add a read-only list to the [vanilla browser example](../../examples/vanilla-browser/README.md), -which currently demonstrates editing. Show resource identity, status and redaction metadata without -edit controls. This is the concrete reference use case for `ResourceStore`; it is not an asserted -external consumer request. Its public surface is: - -```ts -class ResourceStore { - applyStreamEvent: (event: ResourceStateEvent) => ResourceChange; - ids(): string[]; - server(uid: string): KRMObject; - redactions(uid: string): { path: Path; rev: number }[]; - subscribe(callback: () => void): () => void; -} - -interface ResourceChange { - type: ResourceStateEvent["type"]; - uid?: string; - added: boolean; - removed: string[]; -} -``` - -Replace complete projected objects, never deep-merge them. Reset removes nothing, upserts mark seen, -and only synced prunes. Replayed events are idempotent. A new UID under the same name starts a -separate entry. Copy incoming objects/redactions and return detached reads using existing -missing-UID conventions. Notify after object or membership changes; connection state comes from the -handle. Each store belongs to one target and scope. Independent streams must use independent store -instances. - -`LiveResourceStore(readOnlyPolicy)` still retains a draft and runs reconciliation. Remove the named -`readOnlyPolicy` definition/export once `ResourceStore` exists. Preserve all-read-only editor policy -coverage through `regionPolicy([])`; a read-only editor policy and a resource viewer remain -different uses. `ResourceStore` loads no merge engine and exposes no editing API. - -```ts -import { ResourceStore, connectResourceStream } from "@configbutler/krm-stream/stream"; - -const resources = new ResourceStore(); -const stopRendering = resources.subscribe(renderResources); -const connection = connectResourceStream(url, resources.applyStreamEvent); -const stopConnection = connection.subscribe(renderConnection); -void connection.closed.catch(reportApplicationError); - -// On view disposal: -stopRendering(); -stopConnection(); -connection.close(); -``` - -### Editor store and save results - -Keep drafts, policies, conflicts, keyed-list reconciliation, changes, patch capture and guarded -reconciliation. `LiveResourceStore.applyStreamEvent` returns: - -```ts -interface StreamChange extends ResourceChange { - structural: boolean; - flashed: Path[]; - conflicts: Path[]; -} -``` - -Both stores report UIDs actually removed by deleted or synced events. Repetition produces no second -removal. The editor reports `structural: true` on snapshot pruning that removes resources and keeps -its current upsert/highlight/conflict results. Define common change types independently of either -store so the editor does not import `ResourceStore`. - -Remove `adoptSaved`: it is unguarded, so an old save response can still overwrite newer watch state. -It also marked membership during recovery: an old save response adopted after reset survived an -empty synced snapshot, retaining a deleted UID. That half is a bug, fixed separately as a `fix:` -ahead of change 2 with a regression test, so the removal no longer carries a correctness fix. Remove -public optimistic `removeResource` alongside the other direct inputs. - -Successful edits normally return 204 and converge through the watch echo. For an existing UID, a -host returning an object uses `captureReconciliation(uid)` captured before the request. The guard -refuses newer bases, deleted/recreated UIDs and snapshot overlap; it does not insert a new resource. -Creates wait for their added echo. A host tracks successful creates as pending confirmation so they -cannot be submitted twice, correlating an echo with server-assigned identity/receipt when available. -Handle an echo arriving before the create response as well as after it. Pending deletes likewise -wait for stream absence, rather than removing authoritative state optimistically. Snapshot -completion can confirm missed echoes. Host write status and host confirmation state are separate -from store data. Use a small host-state recipe and executed tests; this does not add create/delete -endpoints to the library or turn the replay demo into a CRUD backend. - -Simplify `ReconciliationOptions` to `{ redactedPaths?: string[] }`. Omission preserves current -protections; an explicit list removes absent paths, preserves known revisions and rejects unknown -paths rather than inventing revision counters. Remove the `redacted` response branch from the -conditional editor example and revise its tests. Stream input still carries wire redaction -revisions; a stateless GET does not. Preserve edits made while a save/read was pending. - -Remove `takeTheirs`; `revert` already implements that operation. Remove the unused `status(uid)` -convenience method; read `server(uid).status`. Update the keep-local recipe in [proposal -0006](0006-stream-and-save-implementation-plan.md) to use `revert`. - -```ts -import { connectResourceStream } from "@configbutler/krm-stream/stream"; -import { LiveResourceStore } from "@configbutler/krm-stream/editor"; - -const editor = new LiveResourceStore(); -const connection = connectResourceStream(url, (event) => { - renderEditor(editor.applyStreamEvent(event)); -}); -void connection.closed.catch(reportApplicationError); - -editor.setValue(uid, ["spec", "replicas"], 3); -const intent = editor.captureSave(uid); -// The host validates and applies the captured intent through its save endpoint. -``` - -## Package boundaries and public exports - -The `stream` and `editor` subpaths are deferred with `ResourceStore`: they help an unbundled browser -that loads only the stream layer, and krm-foyer's example vendors the single-file bundle, which -contains both layers anyway. Change 2 still trims the root exports as described at the end of this -section. - -| Import | Exports | -|---|---| -| `@configbutler/krm-stream/stream` | Connector, lifecycle types, resource state/wire types, URL builder, `ResourceStore`, version stamp | -| `@configbutler/krm-stream/editor` | `LiveResourceStore`, change/save/reconciliation types, edit policies and schema helpers | -| `@configbutler/krm-stream` | Combined convenience exports from those implementations | -| `@configbutler/krm-stream/bundle` | Existing single-file combined build | - -Retain stream/editor subpaths. An unbundled browser importing the combined root loads its whole -module graph; bundler tree-shaking does not solve that supported use case. The stream entry loads no -editor store, merge, edit-policy or schema implementation. Editor modules load no connector. Assert -the emitted module graph and declarations rather than claiming a bundle-size improvement. The -combined bundle still contains both layers. Defer a stream-only single-file bundle until needed. - -Remove public exports of `clone`, `deepEqual`, `get`, `has`, `isPrefix`, `parsePointer`, `pathKey`, -`SSEDecoder` and `StreamSequence`; they remain internal helpers and package-test imports as needed. -Retain useful policies, schema helpers and public domain types. Verify repository and available host -imports before implementing removals; unavailable host source is not evidence of no users. Add -`sideEffects: false` only after auditing top-level effects; it can assist bundlers but does not -replace explicit entry boundaries. Keep one npm package, zero runtime dependencies and existing ESM -emission. - -## Implementation order and migration - -Deliver in this order, keeping each intermediate commit buildable. Review split the original change -2, which bundled removals, the tracker, a new store and entry points; only the removals and fixes -remain planned. - -| Change | Deliver | Acceptance evidence | -|---|---|---| -| 1 | One connector, event consumers, consolidated lifecycle observation, host-callback exceptions, version diagnostics, spec §7 and consumer rule 9 wording | Plain callbacks recover on gaps, respect retry policy and expose host exceptions after cleanup without retrying; all connector callers migrated; raw EventSource browser check kept | -| Fix | Adopted save responses never count as snapshot membership, as a `fix:` | A response adopted after a reset does not survive an empty snapshot | -| 2 | Editor removals and fixes: bound `applyStreamEvent` with private snapshot and upsert methods; the standalone helper, `ApplyResult`, `ApplyOptions`, `adoptSaved`, `takeTheirs` and `status()` removed; simplified `ReconciliationOptions`; trimmed root exports; create/delete host guidance | Editor behavior through the public event input; late-read and snapshot regressions; host examples confirm creates and deletes by echo; docs and package metadata match the API | - -### Deferred until a read-only view needs them - -| Item | Trigger | Until then | -|---|---|---| -| `SnapshotTracker` extraction, `ResourceStore`, `readOnlyPolicy` removal and the read-only example | A list or viewer that should not load the merge engine, or a consumer asking for a read-only store | A list renders from `LiveResourceStore` with a read-only policy and carries a merge engine it does not use | -| `stream` and `editor` subpath entry points, the `sideEffects` audit and module-graph assertions | An unbundled consumer that measurably benefits from loading only the stream layer | Hosts import the root or the single-file bundle, which contain both layers | - -Each deferred item keeps the design and acceptance criteria in this document; build it as its own -change when its trigger appears. - -Use `feat!:` and short `BREAKING CHANGE:` notes for public removals. List connector/name/options -changes, store input removals, response-redaction simplification, utility exports and completion -rejections. Remove old names and signatures without overloads or forwarding aliases. Keep the -current lockstep release process; generated versions and changelogs remain Release Please's -responsibility. - -Change 1 migrated tests, the conditional-save, Vue and browser examples, READMEs, guides and -`url.ts` examples to the connector, fixed stale native EventSource descriptions, and added an -[upgrade guide](../migrating.md). Change 2 migrates store callers, package metadata and read-only -policy/type comments, and extends that guide. Lead with streaming and describe editing as optional; -retain useful merge keywords. A save success must not clear edits made while it was in flight. - -Change 1 updated [spec §7](../../spec/v1.md#7-transport--authentication) to say the reference -connector uses fetch for cookies and headers, and [consumer rule 9](../../spec/v1.md#10-conformance) -to require closing the connection on terminal refusal rather than naming a JavaScript class. Keep -gateway native SSE/cookie support and a raw `EventSource` browser check. That check verifies -framing, cookies, fresh snapshots on native reconnect and terminal shutdown through test-owned -handling; it is not another official connector and does not claim browser-native retry timing -satisfies the reference retry policy. These are implementation-neutral wording changes, not a new -wire version. Keep documented v1 event fields and error codes, including `SLOW_CONSUMER`; their -removal is separate protocol scope. - -## Verification and completion - -Extend existing suites with observable outcomes: - -- Plain consumer, including a bound store method: chunked framing, ignored unknown events, gap - discard and recovery, terminal refusal, retry hints/exhaustion, abort and repeated snapshot - cycles. Verify state subscriptions replace lifecycle callbacks and gap context appears on retrying - state. -- Close from synced without a later live publication; close then throw without losing the original - exception; no retries after an exception from the consumer, a subscriber or `onError`; no pending - reader, timer or listener after cleanup. -- Matching, missing and mismatched protocol headers: mismatches apply no events and never retry. -- Both stores, once `ResourceStore` lands: replacement/field removal, detached data, redaction - changes, repeated replay, deletion while disconnected, same-name/new-UID replacement, - partial/restarted snapshots, synced without reset, and matching removal reports. Use - authoritative-state conformance and convergence fixtures for both stores and keep edit/conflict - fixtures specific to the editor. -- Editor: late GETs during and across completed recovery cycles, later typing during saves, refused - replacement UIDs, redaction-path omissions/unknown paths, conflict retention and keyed lists. Keep - the adopted-save regression until `adoptSaved` is removed; show that a guarded late response never - counts as membership and an empty completed snapshot prunes the UID. -- Host examples: no synthetic adoption/deletion, create confirmation in either echo/response order, - pending-confirmation entries cannot be resubmitted, and a failed write leaves authoritative state - unchanged. No reconciliation guard inserts a resource it did not capture. -- Packed exports and declarations: root and combined bundle expose the intended API. Real Go-wire - and Chromium checks cover the editor through the public connector and direct raw EventSource - compatibility. When the deferred entry points land, the emitted graph shows stream imports run - with editor modules absent, and the checks cover both stores. - -For runtime PRs run `task fixtures-check`, `task test`, `task lint`, `task e2e-wire`, -`task e2e-browser` and `task pack-client`, and inspect final-commit CI. Existing goldens must retain -wire behavior. Real-cluster/capacity campaigns are not a browser-separation merge gate. Report the -final tested commit, results, unrun host checks and limitations. - -The planned work is complete when one public connector serves the editor, the editor's public input -is the event stream with no unguarded adoption, a live editor preserves draft and save guarantees, -published artifacts work and migration guidance matches the implementation. A read-only view that -loads no editor modules through a stream entry is the completion criterion of the deferred items. -This document alone is not implementation evidence. - -## Relationship to the remaining roadmap - -[Proposal 0006](0006-stream-and-save-implementation-plan.md) retains its behavioral follow-ups: -editor recovery/keep-local recipes and real-API save races, plus independently deliverable measured -upstream continuation. Align its recipes with this proposal's smaller API. Implement separation -before expanding editor public surface; it does not block gateway fixes or real-API evidence. -[Proposal 0010](0010-gateway-api-cleanup.md) owns Go cleanup and can proceed independently. -[Proposal 0008](0008-shared-watch-hardening.md) remains completed; deferred authorization triggers -and caches are not reopened. Let demonstrated stream and editing needs drive their respective -follow-ups. +| State input | Bound `store.applyStreamEvent(event)`; private snapshot/upsert/removal helpers; remove standalone `applyStreamEvent`, `ApplyOptions` and duplicate result types | +| Save adoption | Remove unguarded `adoptSaved` and public optimistic `removeResource`; existing-resource asynchronous responses use captured reconciliation guards | +| Redaction reads | Simplify `ReconciliationOptions` to `{ redactedPaths?: string[] }`; keep revisions on authoritative stream events | +| Duplicate helpers | Remove `takeTheirs` in favor of `revert`, and `status(uid)` in favor of `server(uid).status` | +| Root exports | Keep domain APIs, policies and schema helpers; make cloning/path/decoder/sequence implementation utilities private | + +Reconciliation captures UID, resource revision and snapshot generation before a request. Refuse a +response after a newer base, during or across snapshot recovery, for a deleted/replaced UID, or with +unknown redaction paths. Preserve typing after capture. Omitted redaction paths keep protections; +explicit paths remove absent protections and preserve known revisions, never inventing counters. + +Successful edits normally converge through the watch or a guarded projected read. Creates/deletes +remain host-owned intents and accepted-but-pending confirmations until an echo or completed snapshot. +Account for either echo/response order and prevent duplicate submissions. A guard does not insert a +new resource. Failed writes leave authoritative state intact; do not fabricate events or UIDs. + +Keep deletion recovery and keep-local recipes on store APIs. `StreamChange` should report actual +removals from delete/snapshot completion, structural changes, highlights and conflicts; repeated +replay must not report a second removal. Retain the adopted-save regression until adoption is removed. + +## Deferred until a read-only view needs them + +A viewer currently uses `LiveResourceStore(readOnlyPolicy)` and carries the editor implementation. +Add `ResourceStore` only for a demonstrated list/viewer need. It would expose event application, +`ids`, detached `server`/`redactions` reads and subscription, without drafts or merge APIs. Complete +upserts replace objects; only a completed snapshot prunes unseen UIDs; replay stays idempotent. + +Extract a shared internal snapshot tracker when the second store needs it. Its generation, seen set +and begin/finish logic must distinguish authoritative stream membership from guarded reads. Each +store owns one source/scope and has its own tracker; an editor does not mirror a second resource store. + +Separate `stream` and `editor` package entry points remain deferred until an unbundled consumer +benefits measurably from avoiding editor modules. Keep the combined root and single-file bundle. +When implemented, assert the emitted graph: stream imports load no editor/merge/schema implementation, +and editor imports load no connector. Audit effects before declaring `sideEffects: false`; no new +stream-only bundle is justified yet. Existing `readOnlyPolicy` stays exported until a dedicated +viewer store replaces its role. + +## Acceptance and migration + +Implement cleanup independently of native transport, gateway fixes and real-API evidence. Inventory +repository and available consumer imports; unavailable source does not establish no users. Remove +superseded APIs before 1.0 without aliases, update all examples and extend [migration](../migrating.md). +Do not change release-generated versions or changelogs by hand. + +Verify observable behavior through public events: complete replacements, redactions, UID changes, +partial/restarted snapshots, deletion during disconnect, later typing, late reads across snapshots, +conflict retention and keyed lists. Verify connection cleanup, callback exceptions, gap recovery, +retry hints/exhaustion and protocol diagnostics. Host recipes must preserve intent, confirmation and +no-duplicate-submission behavior. Packed exports and declarations must match the supported API. + +For runtime changes run `task fixtures-check`, `task test`, `task lint`, `task e2e-wire`, +`task e2e-browser` and `task pack-client`, then inspect final-commit CI. Entry-point graph and dual-store +checks apply only when their deferred changes are implemented. Documentation edits need link and +example/diagram review, not a cluster campaign. + +[Proposal 0006](0006-stream-and-save-implementation-plan.md) owns gateway continuation and save-progress +evaluation. [Proposal 0010](0010-gateway-api-cleanup.md) records implemented Go cleanup. Completed +shared-watch hardening does not reopen authorization triggers or caches without measured demand. diff --git a/docs/proposals/0010-gateway-api-cleanup.md b/docs/proposals/0010-gateway-api-cleanup.md index e2a365e..140653b 100644 --- a/docs/proposals/0010-gateway-api-cleanup.md +++ b/docs/proposals/0010-gateway-api-cleanup.md @@ -1,187 +1,48 @@ -# Proposal 0010 Gateway API cleanup +# Proposal 0010: Gateway API cleanup -**Status: implemented** in [#54](https://github.com/ConfigButler/krm-stream/pull/54), as the two -changes below, pending review. Source review baseline: `a8281c5`; proposal review branch: `0cde13b`, -2026-10-04. +**Status: implemented.** This is the decision record for +PR #54. Use [the gateway README](../../gateway/README.md) for current setup and +[upgrading from 0.7](../migrating.md#go-gateway) for the full migration table. -Simplify gateway configuration and remove duplicate entry points and repository-only harness -exports. Preserve authorization, projection, bounded HTTP delivery, shared-watch lifecycle and v1 -wire behavior. Deliver independently from [browser -separation](0009-stream-and-editor-separation.md). Follow the [design -rules](../../CONTRIBUTING.md#design-rules) and [release policy](../releasing.md): remove superseded -names before 1.0 and migrate callers together. +## Decision -The source audit establishes repository use, not every consumer's use. Field reports record -krm-foyer adoption but do not establish which exports its current checkout imports. Inventory -repository callers and available host source during implementation; report unavailable checks. -Published APIs can change before 1.0, with migration notes and no forwarding aliases. +Keep one name per API, share stream configuration and remove repository-only runtime exports before +1.0. Preserve authorization, named projections, bounded HTTP delivery, shared-watch lifecycle and +v1 wire behavior. The gateway remains the projected watch layer and current SSE delivery path; +[native browser transport](../field-reports/third-our-identity.md#native-watch-connector) is separate +requested work. The gateway remains the projected source over SSE; a native viewer does not require +changes to its APIs or framing. -## Selected changes - -| Area | Current problem | Result | -|---|---|---| -| Stream methods | Default-view methods forward to projection-aware variants | One HTTP method and one transport-neutral method, each taking a requested projection | -| Shared backend | Default constructor forwards to an options constructor | One constructor with explicit zero-value-capable options | -| Configuration | Options and Gateway repeat fields with manual copying and different authorization names | Shared stream configuration with one name per setting | -| Projection | Projection is ignored when Projections is supplied | One policy field; static selection uses StaticProjection | -| SSE details | Public Comment is used only by a test; heartbeat default docs are ambiguous | Private comment helper and default constant; public configurable interval | -| Repository harness | Production package exports fixtures, loaders and scripted replay | Internal harness used by replay and fixture-driven tests | -| Scope formatting | Scope.Query is used only by a conformance test | Test-owned query construction; public scope parsing retained | - -## Stream methods and shared constructor - -Keep these names with projection-aware signatures: - -```go -func (g *Gateway) ServeStream( - w http.ResponseWriter, r *http.Request, - principal Principal, scope Scope, requested Projection, -) - -func (g *Gateway) Stream( - ctx context.Context, principal Principal, scope Scope, - requested Projection, sink Sink, -) error - -func NewSharedBackend(upstream Backend, options SharedOptions) *SharedBackend -``` - -Empty requested projection asks the configured policy for its default; it never bypasses policy. -Rename existing projection-aware implementations to these names. Remove `ServeStreamProjection`, -`StreamProjection` and `NewSharedBackendWithOptions`. Default callers pass an empty projection or -`SharedOptions{}` respectively. Migrate Handler, replay, tests and kube examples without wrappers. - -## Shared stream configuration - -Extract shared fields into one exported `StreamConfig`. Options and Gateway embed it: - -```go -type Options struct { - StreamConfig - Principal func(*http.Request) (Principal, error) - Scopes ScopePolicy -} - -type Gateway struct { - StreamConfig - // Existing private runtime/test fields stay here. -} -``` - -`StreamConfig` contains `Authorizer`, `Clients`, `Projections`, `Ordering`, `Observer`, -`Diagnostics`, `HeartbeatInterval`, `WriteTimeout`, `ReauthorizationInterval` and -`ReauthorizationTimeout`. Use Authorizer consistently, removing Gateway.Auth. Handler copies the -single config value rather than assigning every shared field individually. Preserve host-callback -validation. - -Go composite literals must explicitly name the embedded configuration: - -```go -gateway.Options{ - Principal: userFromSession, - Scopes: allowedScopes, - StreamConfig: gateway.StreamConfig{ - Authorizer: authorizeScope, - Clients: clientsForUser, - Projections: gateway.StaticProjection(gateway.ProjectionSpec), - WriteTimeout: 5 * time.Second, - ReauthorizationInterval: 30 * time.Second, - ReauthorizationTimeout: 5 * time.Second, - }, -} -``` - -The cost is one level of nesting in a host's literals. Assignments such as `options.Authorizer = …` -keep working through field promotion, and the [upgrade guide](../migrating.md) maps each old literal -to the new one. - -Remove the Projection configuration field. Nil Projections retains the safe full policy through -`StaticProjection(ProjectionFull)`. A host with one view supplies StaticProjection; a host selecting -by principal/scope supplies ProjectionPolicy. Raw Secret disclosure continues to require explicit -host policy. Preserve projection authorization during opening, cycles and timed checks. - -Preserve validation, including negative values and positive HTTP write timeout with timed checks. -The shared struct must not impose HTTP-only restrictions on transport-neutral sinks. Direct HTTP -validation still occurs before response I/O or backend opening. Update comments and error messages -to identify the shared Ordering setting without assuming Handler users configure a Gateway literal. -The strict-ordering refusal is the one message on the wire this touches: its golden transcript -changes in message prose only, which both suites compare loosely. - -## Repository-only exports - -Move [conformance.go](../../gateway/conformance.go) and [scripted.go](../../gateway/scripted.go) -into `gateway/internal/conformance` or similarly narrow internal packages. Move Fixture, WatchOp, -FixtureEvent, Corpus, loaders and ScriptedBackend together as needed. The harness may import gateway -domain types; production gateway must not import it. Use explicit corpus paths from commands/tests. -Remove the production relative-path convenience loader; a test helper may supply the repository -path. - -A `package gateway` test importing a harness that imports gateway creates a cycle. Move -fixture-driven conformance and golden tests to `package gateway_test`, including fixture-dependent -cases currently in `stream_test.go`; adapt their recording/replay helpers. Leave private -implementation tests in `package gateway`. Do not convert the entire suite or export private helpers -to avoid the cycle. Inspect shared helper dependencies before editing. - -Keep replay and tests on the same fixture interpreter. Preserve use of the production SSE encoder in -golden generation, update flags, protocol.json generation, fixture paths and task names. Do not -replace executable wire evidence with manually encoded transcripts. Core gateway must no longer -include repository corpus filesystem loading in its runtime module graph. - -Unexport SSESink.Comment and retain its short-write/failure tests. Rename the heartbeat default to -private `defaultHeartbeatInterval` and document the 20-second default on public config. Move -Scope.Query's only current use into scope conformance test support. Retain ScopeFromQuery and -canonical parser/TypeScript URL checks against the existing corpus. - -## Capabilities retained deliberately - -| Reviewed removal | Decision and reason | +| Area | Implemented result | |---|---| -| SLOW_CONSUMER | Keep Go constant, TypeScript type, schema and spec. Built-in sharing resyncs on overflow, but v1 defines this code for other producers/backends. A teardown test also uses it. No built-in emission does not establish no protocol use. | -| Sink, Stream and SSE encoding APIs | Keep the transport-neutral seam and current framing APIs. They provide distinct capabilities rather than forwarding aliases. Removing them is a separate scope decision. | -| OrderingLenient | Keep explicit support for unorderable aggregated backends. Both strict and lenient behavior have unit tests; the recorded cluster run did not exercise lenient ordering. Removing it changes supported backend behavior. | -| Event.Target on reset | Keep v1 fields and generated output. Redundancy with scope.target is a protocol design question, not justification for a silent removal. | - -For generic sinks, clarify the existing requirement that the host bounds delivery and honors -cancellation. Proposal 0008 repaired library-owned HTTP configuration; it did not promise to bound -arbitrary host I/O. Keep generic bounded-sink tests separate from HTTP blocked-reader/deadline -tests. Deleting the protocol seam is not needed to preserve the HTTP guarantee. - -Protocol removals require a separately reviewed compatibility/version decision. This cleanup removes -duplicate configuration and forwarding paths while preserving the protocol and backend contract. - -## Implementation order and acceptance - -Deliver two changes, each buildable and fully migrated: - -| Change | Deliver | Evidence | -|---|---|---| -| 1 | Shared config, projection-policy field, consolidated stream methods and constructor, corrected comments/errors | Handler/direct HTTP and generic sinks preserve policy, validation, deadlines and observations; examples build | -| 2 | Internal corpus/replay harness, test relocation, private Comment/default, test-owned scope query | Fixture/golden output unchanged; replay/browser checks pass; public gateway no longer owns corpus loading | - -Update READMEs, adoption, auth and operations guides, root examples and kube examples. Historical -proposals and dated field reports remain historical: update current-guidance links or add a short -migration note when useful, rather than rewriting the APIs they originally reviewed. [Proposal -0006](0006-stream-and-save-implementation-plan.md) keeps its acceptance criteria and uses the final -method/configuration names in later work. - -Run `task fixtures-check`, `task test`, `task lint`, `task e2e-wire`, `task e2e-browser` and -`task pack-client`. Run race checks in both Go modules for changed lifecycle paths. No new cluster -campaign or capacity claim is required for these API/harness changes. - -Focused acceptance must establish: - -- Nil projection policy redacts Secrets; static policy refuses a different requested view; dynamic - policy runs at opening, cycles and timed checks. -- Handler/direct HTTP preserve fail-before-I/O validation, unsupported-writer refusal, - blocked-client deadlines and healthy-peer isolation. Generic bounded sinks remain usable. -- Shared-backend zero/explicit options preserve cancellable openings, sharing, overflow recovery, - backoff, last-subscriber cleanup and balanced observations. -- Both config entry points use the same fields/names without per-field mapping. Current code and - guidance no longer reference removed public APIs. -- Fixture bytes, protocol.json and canonical query expectations remain unchanged. External fixture - tests and private tests compile without cycles; replay uses the internal harness. - -Use `feat!:` and `BREAKING CHANGE:` notes listing removals, new signatures, nested config literals, -Authorizer naming and StaticProjection migration. Keep lockstep releases and generated changelogs. -Report final tested commit, outcomes, available host checks and unavailable integrations. Complete -this proposal only after those changes land with evidence, independently of proposal 0009. +| Stream methods | `ServeStream` and transport-neutral `Stream`, each taking the requested projection | +| Shared backend | `NewSharedBackend(upstream, options)` with zero-value-capable options | +| Configuration | `Options` and `Gateway` embed the same `StreamConfig`, using `Authorizer` consistently | +| Projection policy | One `Projections` field; static selection uses `StaticProjection`; nil permits full view only | +| Repository harness | Corpus/fixture/scripted-watch machinery moved into `gateway/internal/conformance` | +| SSE/query details | Private comment helper and heartbeat default; test-owned scope query construction | + +Composite literals explicitly name `StreamConfig`; promoted assignments remain available. Empty +requested projection asks the host policy for its default and never bypasses it. Raw Secret disclosure +still requires explicit permission. HTTP validation fails before I/O or backend opening; arbitrary +transport-neutral sinks remain the host's to bound and cancel. + +## Capabilities retained + +- `Sink`, `Stream` and SSE encoding support distinct host integrations. +- `OrderingLenient` supports known upstreams with unorderable versions, with reduced monotonicity. +- `SLOW_CONSUMER` and reset target fields remain in v1 even when the built-in path does not need them. +- Replay and tests share the fixture interpreter and production encoder; internalizing the harness + does not replace executable wire evidence with manually written transcripts. + +No protocol field or error code was removed. Compatibility changes to the wire require their own +version decision. Browser separation is owned by [proposal 0009](0009-stream-and-editor-separation.md), +and behavioral work by [proposal 0006](0006-stream-and-save-implementation-plan.md). + +## Evidence and limits + +Focused checks cover policy defaults/refusal, validation, writer capabilities/deadlines, healthy-peer +isolation, shared opening/overflow/backoff/cleanup and internal replay/fixture imports. Runtime change +validation includes fixtures, package/lint, real wire, browser, packed client and Go race checks. +These criteria do not by themselves prove final-commit CI, consumer upgrades or cluster capacity. +See the PR for actual run evidence. Release versions and changelogs remain generated. diff --git a/docs/saving.md b/docs/saving.md index e964125..e278d9d 100644 --- a/docs/saving.md +++ b/docs/saving.md @@ -1,12 +1,27 @@ # Saving edits safely -krm-stream is a read library. Your application owns its HTTP save endpoint, audit policy and -Kubernetes client. The recommended sequence is: +The optional editor preserves drafts as the [live resource view](why-a-gateway.md) changes and +captures a narrow save intent. Your application owns its HTTP save endpoint, write authorization, +audit policy and Kubernetes client. Read the [editor state model](client-state-model.md) for edit +policy, three-way reconciliation and explicit conflict resolution. The supported save sequence is: 1. the browser captures a save intent; 2. the host validates it and sends a conditional Kubernetes merge PATCH; -3. the host answers 204, and the watch echoes the write back into the store; -4. on 409, the browser reconciles a guarded, projected GET, and the person saves again deliberately. +3. the host answers 204 or a receipt; the watch normally reflects a visible change, with a guarded + projected read when an echo cannot confirm the result; +4. on 409, the browser reconciles a guarded, projected GET, and the person reviews and saves again + deliberately. + +Serialize saves per editor, resolve field conflicts before submission and enable writes while the +connection is live. Capture intent before any await and preserve later typing. A dirty draft, an +accepted write and application progress are separate states; neither a clean form nor a watch echo +proves a rollout or Git workflow completed. A definite 409 rejection differs from a lost response +whose write outcome is unknown; do not blindly replay an ambiguous write. + +This guide describes projected editing through the current gateway. The requested native connector +starts with read-only viewing. Native editing is a later slice requiring an explicit read/write and +edit-policy contract before sharing these recipes. A raw response must never be fed into a +projected editor as recovery or save confirmation. The [conditional-save example](../examples/conditional-save/README.md) implements all of it: a compilable host endpoint, client reconciliation, race tests and a real-cluster 409 test. @@ -131,6 +146,9 @@ guarantee. Sustained invisible churn can prevent save progress. An accepted projected GET advances the base without a snapshot. When no field conflicts exist, explain the refreshed base and offer a new save. +Bounded recovery of the originally submitted intent and optional coalesced version delivery are +[evaluation work](proposals/0006-stream-and-save-implementation-plan.md#5-save-progress-under-suppressed-churn), +not supported automatic retry policies. The default spec view stays quiet for status-only updates. ## What the person editing sees @@ -141,7 +159,8 @@ and any receipt or Git workflow; connection state is not a save guarantee. |---|---|---| | `version-stale`, no field conflicts | “Configuration refreshed. Your edits are intact; review and save again.” | Capture a new intent on the next deliberate Save. Do not show an empty conflict panel. | | `draft-conflict` | Show local and current values at each conflicting field. | Offer explicit resolution: `revert` takes the server's value, the [keep-local recipe](../examples/editor-recipes/README.md#keep-the-local-value-in-a-conflict) keeps the person's. Keep the rest of the form visible. | -| Connection retrying or `recovering` | “Reconnecting. Your unsaved changes are still here.” | Disable writes until live; after a refused GET, require a later accepted guarded read before another write. | +| Connection `retrying` / `syncing` | “Reconnecting. Your unsaved changes are still here.” | Disable writes until live. | +| Example editor outcome `recovering` | “Refreshing configuration. Your edits are still here.” | A usable base is not established; require an accepted guarded read before another write. This is not a connection status. | | `saved`, watch confirmation pending | “Saved to Kubernetes; waiting for live confirmation.” | Preserve later typing. Track any receipt separately from draft state. | | Session expiry or access denial | Explain sign-in or access outcome. | Handle identity-scoped recovery; do not retry terminal auth failures indefinitely or label them field conflicts. | | `unavailable`, deleted/recreated UID | “This configuration was removed. A replacement must be opened separately.” | Offer copy-out from a retained recovery copy; never apply the old draft to the replacement. | diff --git a/docs/vue.md b/docs/vue.md deleted file mode 100644 index e73e826..0000000 --- a/docs/vue.md +++ /dev/null @@ -1,35 +0,0 @@ -# Vue integration - -A copyable Vue 3 composable covers the adapter: reactive snapshots for drafts, changes, conflicts, -redactions and connection state, plus automatic subscription cleanup. Vue remains a host dependency; -the core package and its browser bundle have no framework dependencies. - -The [composable source](../examples/vue/useLiveResource.ts) and -[tests](../examples/vue/useLiveResource.test.ts) are typechecked and executed by `task test-vue` and CI. -Copy the source into your host and change its library import to `@configbutler/krm-stream`. - -```ts -const { resource, state } = useLiveResource(store, uid, connection); -``` - -Use `resource.value?.draft` in script and `resource?.draft` in a template. A missing/deleted UID gives -`null`. For deletion recovery, capture the initial draft and subscribe to store notifications, retaining -`store.draft(uid)` while the fixed UID exists and keeping the last copy once it is absent. Capturing -only when Save is clicked loses subsequent edits. The null notification is too late to read the -removed draft. Use the tested -[recovery-copy recipe](../examples/editor-recipes/README.md#recover-work-after-a-deletion) beside -`useLiveResource` for the same UID, dispose it with the editor, and never apply a recovery copy -automatically to a replacement UID. - -For a changing selection, remount a keyed editor or create a new effect scope. Use store -methods such as `setValue`, `removeKey`, `revert` and the -[keep-local recipe](../examples/editor-recipes/README.md#keep-the-local-value-in-a-conflict) for -edits: do not bind `v-model` directly to the draft snapshot. Store reads are detached copies, and -editing them bypasses policy checks and change notifications. - -The connection belongs to its creator. A parent sharing one connection across many editors closes -it when the parent unmounts; an editor that creates its own connection can register -`onScopeDispose(() => connection.close())`. Removing one editor's subscriptions must not close a -connection used by other editors. Capture and submit saves with the -[conditional-save example](../examples/conditional-save/README.md); expose request errors and saving -state from that host-owned controller. Enable Save while `state.status === "live"`. diff --git a/docs/why-a-gateway.md b/docs/why-a-gateway.md index 78defea..1e69d52 100644 --- a/docs/why-a-gateway.md +++ b/docs/why-a-gateway.md @@ -1,33 +1,108 @@ -# Why a gateway +# Watching resources -The gateway embeds Kubernetes reads in a browser application while keeping credentials and policy -on the host. It also gives the browser a snapshot boundary and a stable event vocabulary. +krm-stream starts with Kubernetes watches and delivers a defined live resource view to a browser. +Use it for lists and status pages, then add the [editor](client-state-model.md) when a page needs +local drafts. The host owns authentication, authorization policy, credentials and writes. -## From Kubernetes watch to browser stream +## Choose a scope and a view -A Kubernetes watch is a chunked HTTP response containing newline-delimited JSON. It carries object -updates, bookmarks and history-expiry errors. Native `EventSource` expects SSE framing and cannot set -an authorization header, so it cannot consume that response directly. +Scope selects the target, resource kind, namespace, optional name and allowed label selector. +Projection selects what is delivered. Suppression decides whether a change to that view needs a +new object event. These are separate decisions. -Direct browser access would also require cluster credentials and appropriate API-server CORS -configuration. Raw watch objects include fields such as Secret values and managed fields. The host -must decide which scopes and fields it can disclose before sending them to the browser. +The host maps target names to configured clusters and authorizes the scope and projection. The +browser can request a named view; it cannot supply arbitrary field rules or an API-server URL. +There is no general gateway field selector or built-in status-only projection. -The gateway uses a host-supplied Kubernetes client, applies the selected projection and emits -`reset` … `synced` snapshots followed by live updates. The browser connector, `connectResourceStream`, -handles bounded reconnection; a new connection receives a fresh snapshot. See the -[architecture diagram](../README.md#how-it-fits) and [protocol](../spec/v1.md). +| Projection | Fields delivered | Secret values | Updates suppressed | +|---|---|---|---| +| `krm-full/v1` | Includes status | Withheld | Bookkeeping-only | +| `krm-spec/v1` | Omits status | Withheld | Bookkeeping-only and status-only | +| `krm-raw/v1` | Includes status | Included when host policy permits | Bookkeeping-only | + +Every view strips `metadata.managedFields` and the last-applied-configuration annotation. Raw is a +named projection, not native passthrough. Full/spec redaction covers core Kubernetes Secret values, +not arbitrary sensitive content in ConfigMaps or custom resources. Withheld key paths and change +revisions are disclosed in `redacted`; the object contains no mask to accidentally save back. + +For a host that permits these requests: + +```text +/resource-stream/v1?target=production&version=v1&resource=configmaps&namespace=app&projection=krm-full/v1 +/resource-stream/v1?target=production&group=apps&version=v1&resource=deployments&namespace=app&projection=krm-spec/v1 +/resource-stream/v1?target=production&version=v1&resource=pods&namespace=app&labelSelector=app%3Dhello&projection=krm-full/v1 +``` + +Enable `ScopePolicy.AllowLabelSelector` deliberately before accepting the last request. Use +`resourceStreamURL` to encode queries in application code; see [adoption](adopting.md). + +## What suppression saves + +The gateway compares projected content excluding `metadata.resourceVersion`, plus redaction records, +with what it last delivered for that UID. Unchanged views produce no object event. Secret rotations +still produce updates because their redaction revisions change. Every complete snapshot includes all +members, including unchanged objects. + +This reduces downstream bytes and event processing. Fewer store notifications and renders depend +on how the application consumes events; suppression does not eliminate the gateway's upstream work. +The stream represents current state and may coalesce intermediate changes; use an audit system when +every transition matters. + +A quiet view can hold an older write version even while its displayed fields are correct. That +tradeoff matters when adding editing: see [quiet streams and saving](saving.md#why-a-quiet-stream-can-still-reject-a-save). + +## Snapshot and recovery + +Each connection begins with `reset`, zero or more resource upserts, then `synced`. Live upserts replace +complete projected objects. `reset` retains old state while the snapshot arrives; only `synced` prunes +members that were not resent. An interrupted snapshot cannot prove absence. UID distinguishes a +recreated resource from an earlier object with the same name. + +The fetch connector exposes `connecting`, `syncing`, `live`, `retrying`, `closed`, `terminal` and +`exhausted`. Apply state events synchronously before rendering completion. Retryable failures recover +within the connector's bounded budget. For the current SSE connector, every HTTP 4xx except 408/429, +terminal stream errors and a mismatched protocol header stop the connection. Native history-expiry +410 needs different recovery; it must be classified by native transport, not this SSE rule. +Dispose subscriptions and close the handle when their owner leaves. See the +[client lifecycle reference](../packages/krm-stream/README.md#connections). + +A new browser connection receives a fresh snapshot. SSE sequence numbers and object versions are not +browser resume tokens. Proposed gateway upstream continuation can reduce resnapshots without changing +that browser contract; see [proposal 0006](proposals/0006-stream-and-save-implementation-plan.md#4-measured-upstream-continuation). ## Optional watch sharing -Without sharing, each stream uses its own backend watch. [`SharedBackend`](../gateway/shared.go) can -instead keep one upstream watch per scope and serve each subscriber from its cache. A joining -subscriber still receives a complete projected snapshot, so sharing saves upstream work without -eliminating browser transfer or reconciliation costs. - -A shared watch uses one service identity, so the host's `Authorizer` becomes the only access check -between each subscriber and cached objects; a permissive check can disclose the service identity's -data. Sharing is opt-in for this reason. Pair it with -[`kube.SubjectAccessReviewAuthorizer`](../gateway/kube/authz.go) to check each subscriber's -Kubernetes permissions, and configure timed checks when quiet-stream revocation must be bounded. See -[authorization](auth.md) and [operations](operations.md). +Without sharing, each stream uses its own backend watch. `SharedBackend` can keep one upstream watch +per matching scope in that backend and serve subscribers from its cache. Each subscriber still needs +its own snapshot, delivery and browser reconciliation. Different scopes or host processes do not +become one watch automatically. + +A shared watch uses one service identity. The host authorizes each subscriber before serving cached +objects, checks its disclosure policy, and configures timed checks when quiet-stream revocation must +be bounded. Pair sharing with `kube.SubjectAccessReviewAuthorizer` for Kubernetes permission checks. +See [shared-watch authorization](auth.md#shared-watch-authorization) and [operations](operations.md). +Sharing reduces duplicate resource consumption; it does not supply a rate limiter or admission policy. + +## Native and gateway sources + +Native access is the straightforward starting point for a host that already proxies Kubernetes. +The requested native fetch connector adds lifecycle and state-store reuse while retaining original +resource fields. The gateway adds defined views, redaction, suppression and watch sharing, delivered +over SSE. Choose the guarantees the page needs. Native access alone does not provide those gateway +capabilities, and `krm-raw/v1` still differs from native data. + +Both sources use fetch; native Kubernetes JSON and gateway SSE are different framing formats, and +both can carry errors. The existing SSE connector already handles refusal, expiry, cancellation and +bounded retries. Browser `EventSource` is a different client mechanism with header/retry limitations. + +The [first native slice](field-reports/third-our-identity.md#slice-1-a-native-viewer) is a read-only +viewer: ordinary LIST, then WATCH from the collection version, with a fresh LIST on every reconnect. +It reuses the existing lifecycle and `LiveResourceStore(readOnlyPolicy)`. HTTP/in-stream 410 recovers +within the bounded policy; terminal auth refusal never selects another source. An unexpected paginated +response must be refused rather than marked complete. Native editing, resume and streaming lists follow +separately; neither editor cleanup nor a comparative benchmark blocks the viewer. + +Today use the [gateway wiring](adopting.md); native code snippets will be added with the implementation. +A same-origin, cookie-authenticated host proxy keeps Kubernetes credentials server-side. Keep stores +separate across sources, scopes and identities and never fall back from a refused projected source +into native access. The native connector is not implemented by this documentation change. diff --git a/examples/README.md b/examples/README.md index 8d62f9b..c2ba10a 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,7 +1,11 @@ # Examples -The checked browser example in [`vanilla-browser/`](vanilla-browser/) runs against the replay gateway. -For host integration patterns, use these small recipes: +Start with [watching resources](../docs/why-a-gateway.md) and supported host wiring, then add the +optional editor. The checked browser example in [`vanilla-browser/`](vanilla-browser/) runs against +the current gateway source over SSE. A minimal native viewer is +[next work](../docs/field-reports/third-our-identity.md#slice-1-a-native-viewer); it reuses the current +read-only store and lifecycle. Native editing and the larger view/sharing comparison follow separately. +For current host integration patterns, use these small recipes: - [Same-origin cookie application](../docs/adopting.md#1-mount-the-stream-endpoint): fetch connector, session cookie, dynamic client acting as the user. diff --git a/examples/conditional-save/README.md b/examples/conditional-save/README.md index d79f5b6..da6731e 100644 --- a/examples/conditional-save/README.md +++ b/examples/conditional-save/README.md @@ -1,7 +1,10 @@ # Conditional save with a live draft -This example composes the existing store, connector and host-owned writes. It adds no merge -algorithm or shared watch implementation. +Add this controller when a live resource view needs editing. It composes the existing store, +fetch-based gateway connector and host-owned writes under the [editor model](../../docs/client-state-model.md) +and [saving contract](../../docs/saving.md). Draft dirtiness, write acceptance and application progress +remain separate; typing after Save stays in the draft. The example uses the current projected/SSE +path and performs no automatic write retry. - [editor.ts](editor.ts) captures a save intent synchronously, handles HTTP 409 by reconciling a projected GET, retains in-flight edits and rejects late responses superseded by the watch. @@ -48,7 +51,8 @@ resourceVersion into the Kubernetes patch. A stale version produces a safe 409, bookkeeping-only changes were suppressed by the full projection. Spec projection also suppresses status-only updates. This is safe rejection, but sustained churn can hinder save progress. An accepted GET advances the base without requiring a snapshot. Reconcile first; never transplant an old -patch onto the latest version. No automatic write retry is performed. +patch onto the latest version. [Save-progress evaluation](../../docs/proposals/0006-stream-and-save-implementation-plan.md#5-save-progress-under-suppressed-churn) +compares possible improvements without changing this supported baseline. This endpoint returns 204 for successful writes. A host may instead return a receipt-only HTTP 200 under the [saving guide’s receipt contract](../../docs/saving.md#answer-204-or-a-receipt-and-let-the-watch-echo-it); diff --git a/examples/vanilla-browser/README.md b/examples/vanilla-browser/README.md index bb52f45..25b4de6 100644 --- a/examples/vanilla-browser/README.md +++ b/examples/vanilla-browser/README.md @@ -26,3 +26,10 @@ Secret values. Useful fixtures include: | `named-object-absent` | An empty snapshot is a valid resource state. | Add `pace=0ms` for the fastest replay or use a positive value to inspect each event. + +The example uses today's projected SSE connector over fetch. For a read-only page, render server +state with `LiveResourceStore(readOnlyPolicy)` as shown in the [README](../../README.md#watch-a-resource-view-today). +Editing is an optional layer; the [editor model](../../docs/client-state-model.md) and +[saving guide](../../docs/saving.md) describe its intended use. A small native viewer is the next +slice; native editing and a larger source comparison follow separately. They are not features of +this gateway replay demo. diff --git a/examples/vue/README.md b/examples/vue/README.md index 5fcaddc..bd721db 100644 --- a/examples/vue/README.md +++ b/examples/vue/README.md @@ -1,9 +1,37 @@ # Vue adapter example -Copy `useLiveResource.ts` into a Vue 3 host and change its library import to -`@configbutler/krm-stream`. Call it in `setup()` or an active effect scope. See -[Vue integration](../../docs/vue.md) for ownership and editing guidance. +A copyable Vue 3 composable covers the adapter: reactive snapshots for drafts, changes, conflicts, +redactions and connection state, plus automatic subscription cleanup. Vue remains a host dependency; +the core package and its browser bundle have no framework dependencies. -Run `task test-vue` from the repository root. This typechecks the source and tests its reactive -resource/connection state and subscription disposal. CI runs the same task. Vue is a dependency of -this private example only; the core package and bundle remain framework-independent. +The [composable source](useLiveResource.ts) and +[tests](useLiveResource.test.ts) are typechecked and executed by `task test-vue` and CI. +Copy the source into your host and change its library import to `@configbutler/krm-stream`. + +```ts +const { resource, state } = useLiveResource(store, uid, connection); +``` + +Use `resource.value?.draft` in script and `resource?.draft` in a template. A missing/deleted UID gives +`null`. For deletion recovery, capture the initial draft and subscribe to store notifications, retaining +`store.draft(uid)` while the fixed UID exists and keeping the last copy once it is absent. Capturing +only when Save is clicked loses subsequent edits. The null notification is too late to read the +removed draft. Use the tested +[recovery-copy recipe](../editor-recipes/README.md#recover-work-after-a-deletion) beside +`useLiveResource` for the same UID, dispose it with the editor, and never apply a recovery copy +automatically to a replacement UID. + +For a changing selection, remount a keyed editor or create a new effect scope. Use store +methods such as `setValue`, `removeKey`, `revert` and the +[keep-local recipe](../editor-recipes/README.md#keep-the-local-value-in-a-conflict) for +edits: do not bind `v-model` directly to the draft snapshot. Store reads are detached copies, and +editing them bypasses policy checks and change notifications. + +The connection belongs to its creator. A parent sharing one connection across many editors closes +it when the parent unmounts; an editor that creates its own connection can register +`onScopeDispose(() => connection.close())`. Removing one editor's subscriptions must not close a +connection used by other editors. Capture and submit saves with the +[conditional-save example](../conditional-save/README.md); expose request errors and saving +state from that host-owned controller. Enable Save while `state.status === "live"`. + +Vue is a dependency of this private example only; the core package and bundle remain framework-independent. diff --git a/gateway/README.md b/gateway/README.md index 6a62530..f0f3368 100644 --- a/gateway/README.md +++ b/gateway/README.md @@ -1,9 +1,14 @@ # Go gateway -The `gateway` module converts a Kubernetes-style `Backend` watch into the KRM resource-stream -protocol. It is transport-neutral until `Handler` or `ServeStream` adds SSE framing. It has no -Kubernetes client dependency; [`gateway/kube`](kube/) provides the `client-go` backend and the -SubjectAccessReview authorizer. +The `gateway` module adds authorized scopes, named views, redaction, suppression and optional sharing +to Kubernetes-style `Backend` watches. It delivers live state for viewers and optional browser editors. +It is transport-neutral until `Handler` or `ServeStream` adds the current SSE framing. Core has no +Kubernetes client dependency; [`gateway/kube`](kube/) provides the client-go backend and authorizer. + +A fetch-based native browser connector is [requested work](../docs/field-reports/third-our-identity.md#native-watch-connector). +Native is the original-resource source; this gateway delivers projected views over SSE. Native +access alone does not provide these projected-view guarantees. A small native viewer can ship +independently of native editing or changes to this gateway. See [watching resources](../docs/why-a-gateway.md) for the overall contract. ```go handler := gateway.Handler(gateway.Options{ @@ -51,10 +56,15 @@ upstream whose versions cannot be ordered. The normative details are in [`spec/v | Projection | Purpose | |---|---| -| `krm-raw/v1` | Full upstream object for a host that has already made its own disclosure decision. | +| `krm-raw/v1` | Includes Secret values with host permission, but still strips managed fields and last-applied configuration; not native passthrough. | | `krm-full/v1` | Removes metadata noise and redacts Secret values. | | `krm-spec/v1` | The full view without status-driven browser churn. | +All views strip managed fields and last-applied configuration. Full/spec withhold core Secret values; +key paths and change revisions remain visible. Sensitive fields in other resource kinds are not +automatically redacted. Suppression compares projected content excluding RV plus redaction records; +a hidden Secret rotation still emits an update. It reduces downstream work, not incoming watch changes. + The gateway never writes. `ValidateMergePatch` checks a browser's patch against the projection before the host sends it; see [saving](../docs/saving.md). @@ -70,7 +80,8 @@ resnapshotted after `SharedOptions.QueueDepth` live events rather than buffered | Guide | For | |---|---| -| [Adopting](../docs/adopting.md) | The recommended host and browser wiring, start to finish. | +| [Watching](../docs/why-a-gateway.md) | Scope, views, suppression, sharing and transport direction. | +| [Adopting](../docs/adopting.md) | Supported host and browser wiring, start to finish. | | [Authorization](../docs/auth.md) | Backend choice, session validity, timed checks and the revocation budget. | | [Saving](../docs/saving.md) | The host-owned conditional write path. | | [Operations](../docs/operations.md) | Runtime controls, observations, alerts and bounded HTTP delivery. | diff --git a/packages/krm-stream/README.md b/packages/krm-stream/README.md index 47ae970..4e4bcbb 100644 --- a/packages/krm-stream/README.md +++ b/packages/krm-stream/README.md @@ -9,17 +9,25 @@ stream in a browser or JavaScript application. It provides: `applyStreamEvent` to apply a state event to it. - `resourceStreamURL` for the v1 scope query format. -The package is headless and does not choose a UI framework. It works with the Go gateway in this -repository or any conforming producer. +Start with a watch-backed list or viewer; add draft reconciliation only when the page needs editing. +The package is headless and does not choose a UI framework. Today it consumes gateway SSE through +fetch, from the Go gateway or another v1 producer. A fetch-based native Kubernetes watch connector +is [next work](../../docs/field-reports/third-our-identity.md#slice-1-a-native-viewer): a read-only +LIST/WATCH source with re-list recovery and a small viewer, reusing the same lifecycle and store. +Native retains original authorized resources; gateway SSE delivers projected views with redaction, +suppression and optional sharing. The native connector is not implemented yet; editing and resume +are later slices. Keep current `applyStreamEvent` unchanged for the viewer. ```ts -import { LiveResourceStore, applyStreamEvent, connectResourceStream, resourceStreamURL } from "@configbutler/krm-stream"; +import { LiveResourceStore, readOnlyPolicy, applyStreamEvent, connectResourceStream, resourceStreamURL } from "@configbutler/krm-stream"; -const store = new LiveResourceStore(); -connectResourceStream( +const store = new LiveResourceStore(readOnlyPolicy); +const connection = connectResourceStream( resourceStreamURL("/resource-stream/v1", { version: "v1", resource: "configmaps", namespace: "app" }), (event) => applyStreamEvent(store, event), ); +connection.closed.catch(reportApplicationError); +// Close connection and dispose rendering subscriptions on view teardown. ``` ## Vendoring it without a bundler @@ -51,7 +59,7 @@ before 1.0. Renamed APIs are removed rather than retained as compatibility alias [upgrading from 0.7](../../docs/migrating.md). See the repository [README](../../README.md), [client state model](../../docs/client-state-model.md), and [release guide](../../docs/releasing.md). -## Connections and conditional saves +## Connections ```ts const connection = connectResourceStream(url, event => renderChange(applyStreamEvent(store, event)), { @@ -67,7 +75,7 @@ unsubscribe(); connection.close(); ``` -`connectResourceStream` is the package's one connector. It uses fetch for same-origin cookies or +`connectResourceStream` is the current gateway connector. It uses fetch for same-origin cookies or bearer headers; `credentials: "include"` opts into cross-origin cookies. It decodes and sequence-checks the stream, and calls your callback synchronously, exactly once per state event (`reset`, `added`, `modified`, `deleted`, `synced`), in stream order, without the wire `seq`. @@ -76,7 +84,8 @@ it type-checks, but `live` could then be published before `synced` is applied, a would never reach `closed`. Errors go to `onError`, never to the callback; unknown event types are ignored. It requests a fresh snapshot on sequence gaps, network failures, HTTP 408/429/5xx, EOF, and a connection the gateway -closes after a retryable error such as `UPSTREAM_UNAVAILABLE`. Existing drafts survive recovery. +closes after a retryable error such as `UPSTREAM_UNAVAILABLE`. Drafts for surviving UIDs survive +recovery; deletion and completed snapshot pruning remove them. Read `connection.state` for the initial presentation and `subscribe` for every later state. States are `connecting`, `syncing`, `live`, `retrying`, `closed`, `terminal`, and `exhausted`. A reset makes the @@ -106,9 +115,19 @@ The server's retry hint sets the least the next reconnect waits, within `maxRetr a refusal's body is a Kubernetes `Status`, as a host proxying `/k8s` might send, `onError` receives its `message` instead of `stream: HTTP 403`. -There is no native `EventSource` connector: `EventSource` cannot send headers, leaves reconnect timing -to the browser, and cannot request a fresh snapshot after a sequence gap. Gateways still serve native -`EventSource` clients with session cookies (spec §7). +There is no official `EventSource` helper. It cannot send explicit authorization headers and leaves +reconnect timing to the browser; a host-owned helper must handle protocol sequence gaps and terminal +errors itself. Gateway framing still supports browser `EventSource` with session cookies (spec §7). + +## Optional editing + +Use `new LiveResourceStore()` for an editor, or provide a policy that narrows the form's editable +regions. The store keeps delivered state and the draft separate: untouched fields follow incoming +updates, local changes stay, and differing concurrent edits become explicit conflicts. Use store +methods rather than mutating detached reads. See the [editor state model](../../docs/client-state-model.md). + +A dirty draft, an accepted write and observed workload progress are separate states. A save captures +only the submitted intent; typing after Save must survive the result. The host performs all writes. A quiet stream can still hold an older write version. See the [visual explanation](../../docs/saving.md#why-a-quiet-stream-can-still-reject-a-save) and @@ -119,5 +138,5 @@ A quiet stream can still hold an older write version. See the See the [complete conditional-save example](../../examples/conditional-save/README.md) and [save guide](../../docs/saving.md) for host preconditions, real Kubernetes 409s, and draft preservation. -For Vue 3, use the [copyable composable](../../docs/vue.md) for reactive resource and connection state +For Vue 3, use the [copyable composable](../../examples/vue/README.md) for reactive resource and connection state with automatic subscription cleanup. Vue stays in the host application. diff --git a/spec/v1.md b/spec/v1.md index 690fe7f..660cd5e 100644 --- a/spec/v1.md +++ b/spec/v1.md @@ -1,15 +1,13 @@ # KRM resource stream protocol — the wire contract -> **Status:** specification, **v1 candidate**. The thin, language-neutral contract shared by two -> independently useful components: the **[resource-stream-gateway](../gateway/README.md)** (Go — -> *produces* this stream from a Kubernetes watch) and the **[live KRM reconcile engine](../docs/client-state-model.md)** -> (JS — *consumes* it to merge edits and render live status). Specified once, here, so the two sides -> cannot drift. +> **Status:** specification, **v1 candidate**. This is the current gateway SSE contract shared by +> [the Go gateway](../gateway/README.md) and [the browser client](../packages/krm-stream/README.md). +> It defines live projected resource state; draft editing is an optional consumer behavior. > -> **It streams KRM, and says so.** The payload is a Kubernetes object — `apiVersion`, `kind`, -> `metadata`, and whatever else that kind really has — not an abstracted "document". That fidelity is -> the point: the whole value of this family is being a faithful, live window onto the Kubernetes -> Resource Model. **Live `status` in a browser, with real RBAC and real attribution, is the product.** +> A read-only native fetch connector is [requested work](../docs/field-reports/third-our-identity.md#slice-1-a-native-viewer). +> This specification describes the gateway's projected source over SSE. Native Kubernetes JSON is a +> different source format and does not use the SSE sequence or protocol-header rules. No wire behavior +> or conformance requirement changes in this documentation revision. A resource stream is a one-way feed, over Server-Sent Events, of the current-and-then-live state of a **scope** (one named object, or a collection of a kind in a namespace). The server absorbs every