Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ updates:
- dependency-name: k8s.io/api
- dependency-name: k8s.io/apimachinery
- dependency-name: k8s.io/client-go
# The adapter's requirement on its own core module is Release Please's, not Dependabot's:
# every release PR moves it to the version being released (see gateway/kube/go.mod).
- dependency-name: github.com/ConfigButler/krm-stream/gateway
commit-message:
prefix: "chore(deps)"

Expand Down
67 changes: 48 additions & 19 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,17 @@ jobs:

- run: hadolint .devcontainer/Dockerfile

# Release Please writes every changelog from the conventional commits it releases. It inserts
# each version below whatever heading it finds and never removes a hand-written section, so an
# "Unreleased" section ships labelled unreleased forever. Detail belongs in the commit or PR.
- name: changelogs are Release Please's
run: |
set -euo pipefail
if git ls-files -z '*CHANGELOG.md' | xargs -0 grep -nE '^## ' | grep -vE ':## (\[[0-9]+\.[0-9]+\.[0-9]+\]\(|[0-9]+\.[0-9]+\.[0-9]+ \()'; then
echo "::error::a CHANGELOG.md has a hand-written section. Release Please generates them from commit messages; put the detail in the commit or PR description instead."
exit 1
fi

- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "24"
Expand Down Expand Up @@ -216,32 +227,49 @@ jobs:
run: |
set -euo pipefail

# gateway/kube requires the core module by version. On an ordinary commit that version is
# one that has already been released, so the resolution below is exactly what an adopter
# gets. On a RELEASE commit it is the version being released — whose tag does not exist
# until the release PR merges — and `go get` would fail here for a reason that is not a
# bug. Skip that case in ordinary CI; release CI requires the tags to exist.
core="$(sed -n 's|^[[:space:]]*github.com/ConfigButler/krm-stream/gateway v\([^ ]*\).*|\1|p' gateway/kube/go.mod | head -1)"
# gateway/kube requires the core module at the release it ships with, and Release Please
# rewrites that line in every release PR (x-release-please-version). Nobody edits it by
# hand, so it must keep its marker, and it must never be a pseudo-version: the release
# updater would mangle one (v0.6.0-e723903f0c9e), and a released adapter must not depend
# on a commit that may live only on a merged branch.
line="$(grep -E '^[[:space:]]*github.com/ConfigButler/krm-stream/gateway v' gateway/kube/go.mod | head -1)"
core="$(printf '%s' "$line" | sed -n 's|^[[:space:]]*github.com/ConfigButler/krm-stream/gateway v\([^ ]*\).*|\1|p')"
if [ -z "$core" ]; then
echo "::error::gateway/kube/go.mod does not require the core module at all" && exit 1
echo "::error file=gateway/kube/go.mod::gateway/kube/go.mod does not require the core module at all" && exit 1
fi
if ! printf '%s' "$line" | grep -q '// x-release-please-version'; then
echo "::error file=gateway/kube/go.mod::the core requirement lost its x-release-please-version marker, so releases would stop updating it" && exit 1
fi
if printf '%s' "$core" | grep -Eq '[-.][0-9]{14}-[0-9a-f]{12}$'; then
echo "::error file=gateway/kube/go.mod::the core requirement is a pseudo-version (v${core}). Release Please owns this line; restore the released version. CI already builds an unreleased adapter with the core from the same commit."
exit 1
fi
# A release's adapter must require the core it is released with. Release Please writes that
# line; if it ever did not, the adapter would resolve an older core, and nothing else here
# would notice, because the older tag exists.
if [ -n "$RELEASE_VERSION" ] && [ "$core" != "$RELEASE_VERSION" ]; then
echo "::error file=gateway/kube/go.mod::the released adapter requires core v${core}, not the v${RELEASE_VERSION} it is released with"
exit 1
fi
# A pseudo-version names a commit, and always resolves. It ends in a 14-digit timestamp and a
# 12-character revision, whatever its base: v0.0.0-…, or v0.4.1-0.… once a tag precedes it,
# which is the form Go chooses when an adapter change needs unreleased core API.
if ! printf '%s' "$core" | grep -Eq '[-.][0-9]{14}-[0-9a-f]{12}$'; then
if ! git ls-remote --exit-code --tags origin "refs/tags/gateway/v${core}" >/dev/null 2>&1; then
if [ -n "$RELEASE_VERSION" ]; then
echo "::error::released adapter requires missing core tag gateway/v${core}"
exit 1
fi
echo "::notice::core v${core} is not tagged yet; release CI verifies the public tag after tagging."
exit 0
# On a release PR the line names the version being released, whose tag does not exist until
# the PR merges. Skip that case in ordinary CI; release CI requires the tags to exist.
if ! git ls-remote --exit-code --tags origin "refs/tags/gateway/v${core}" >/dev/null 2>&1; then
if [ -n "$RELEASE_VERSION" ]; then
echo "::error::released adapter requires missing core tag gateway/v${core}"
exit 1
Comment thread
coderabbitai[bot] marked this conversation as resolved.
fi
echo "::notice::core v${core} is not tagged yet; release CI verifies the public tag after tagging."
exit 0
fi

revision="${{ inputs.ref || github.event.pull_request.head.sha || github.sha }}"
# A release is fetched by its tag ALONE: the adapter must resolve its own core. An ordinary
# commit may use core API that is not released yet, so it is fetched the way a consumer of
# an unreleased commit must fetch it: both modules at that same commit.
modules=("github.com/ConfigButler/krm-stream/gateway/kube@${revision}" "github.com/ConfigButler/krm-stream/gateway@${revision}")
if [ -n "$RELEASE_VERSION" ]; then
revision="v${RELEASE_VERSION}"
modules=("github.com/ConfigButler/krm-stream/gateway/kube@${revision}")
fi
tmp="$(mktemp -d)"
cd "$tmp"
Expand All @@ -261,8 +289,9 @@ jobs:

func main() {}
GO
go get "github.com/ConfigButler/krm-stream/gateway/kube@${revision}"
go get "${modules[@]}"
go build ./...
go list -m github.com/ConfigButler/krm-stream/gateway github.com/ConfigButler/krm-stream/gateway/kube
echo "a stranger can go get and build the Kubernetes adapter at ${revision}."

- name: warm the released module versions in the Go proxy
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ mux.Handle("/resource-stream/v1", gateway.Handler(gateway.Options{
Principal: func(r *http.Request) (gateway.Principal, error) { return userFromSession(r) },
Authorizer: authorizeScope,
Clients: func(_ context.Context, _ string, p gateway.Principal) (gateway.Backend, error) {
return kube.NewBackend(dynamicClientFor(p.(*User))), nil
return kube.NewBackendForConfig(restConfigFor(p.(*User)))
},
Scopes: gateway.ScopePolicy{
Targets: []string{"production"},
Expand Down
13 changes: 9 additions & 4 deletions docs/adopting.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,19 +75,24 @@ A `rest.Config.Host` may include a path prefix, such as a kcp workspace URL
(`https://kcp.example/clusters/root:org:ws`). The dynamic client preserves it when constructing API
requests. Use `kube.NewBackendForConfig(cfg)` to include the endpoint in upstream errors, which reach
`Options.Diagnostics` and never the browser;
`kube.NewBackend(dynamicClient)` also works when a dynamic client already exists.
`kube.NewBackend(dynamicClient)` also works when a dynamic client already exists. Build that client on
`kube.HTTPClientFor(cfg)` (`dynamic.NewForConfigAndClient(cfg, httpClient)`): client-go follows
redirects and carries the configured credential along, so a client holding a user's token must refuse
them, as `NewBackendForConfig`'s does.

## 2. Mount the same-origin cookie endpoint

```go
import (
"net/http"

"k8s.io/client-go/rest"

"github.com/ConfigButler/krm-stream/gateway"
"github.com/ConfigButler/krm-stream/gateway/kube"
)

func mount(mux *http.ServeMux, dynamicClientFor func(*User) dynamic.Interface) {
func mount(mux *http.ServeMux, restConfigFor func(*User) *rest.Config) {
mux.Handle("/resource-stream/v1", gateway.Handler(gateway.Options{
// Your session cookie -> your application user. krm-stream never sees a token.
Principal: func(r *http.Request) (gateway.Principal, error) {
Expand All @@ -101,9 +106,9 @@ func mount(mux *http.ServeMux, dynamicClientFor func(*User) dynamic.Interface) {
}
return nil
}),
// Build a dynamic client acting as this user. Kubernetes RBAC remains the boundary.
// Build a backend acting as this user. Kubernetes RBAC remains the boundary.
Clients: func(_ context.Context, _ string, p gateway.Principal) (gateway.Backend, error) {
return kube.NewBackend(dynamicClientFor(p.(*User))), nil
return kube.NewBackendForConfig(restConfigFor(p.(*User)))
},
Scopes: gateway.ScopePolicy{
Targets: []string{"production"},
Expand Down
6 changes: 5 additions & 1 deletion docs/auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ gateway.Handler(gateway.Options{
Principal: sessionUser,
Authorizer: authorizeScope,
Clients: func(_ context.Context, target string, p gateway.Principal) (gateway.Backend, error) {
return kube.NewBackend(dynamicClientFor(target, p.(*User))), nil
return kube.NewBackendForConfig(restConfigFor(target, p.(*User)))
},
Scopes: scopePolicy,
})
Expand All @@ -58,6 +58,10 @@ gateway.Handler(gateway.Options{
`UNAUTHENTICATED` without its text.
- `Authorizer` denies unauthorized scopes before a watch opens and on subsequent checks.
- `Clients` is a `ClientFor` callback supplying the backend for that identity and target.
`kube.NewBackendForConfig` builds a client that refuses redirects, since client-go would
otherwise carry the caller's token to wherever a redirect points. A host that builds its own
dynamic client for `kube.NewBackend` must build it on `kube.HTTPClientFor(cfg)` for the same
reason.
- `Scopes` allowlists targets and resources, or with `AnyResource` leaves resource admission to the
API server's RBAC for a backend that acts as the caller. A browser cannot supply a raw API-server
URL.
Expand Down
52 changes: 52 additions & 0 deletions docs/field-reports/first-fieldreport.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,18 @@ Everything was checked against krm-stream v0.4.0 (`gateway`, `gateway/kube` and
`@configbutler/krm-stream`) on 2026-10-02, by reading the source and, for the error
output below, by running the gateway.

## Status (2026-10-02): asks 1 to 5 are in krm-stream 0.5.0

krm-stream 0.5.0 took all five asks below (krm-stream #40), and krm-foyer moved to it:
the gateway maps the API server's answers, keeps their text off the wire, sends it to a
`Diagnostics` hook, leaves retries to the browser's client with the server's hint, admits
any resource with `ScopePolicy.AnyResource`, and lets `Principal` choose its refusal.
krm-foyer's own mapping, status capture and backoff are gone.

Reviewing krm-foyer's integration found three more, below the original asks as
[asks 6 to 8](#ask-6-refuse-redirects-in-the-kubernetes-backend). krm-foyer works around
each meanwhile.

## The asks at a glance

| # | Ask | Priority | What krm-foyer does meanwhile |
Expand Down Expand Up @@ -208,3 +220,43 @@ Smaller, and related: on a non-200 answer the connector reports only `stream: HT
and discards the body. When the body is a Kubernetes `Status`, as krm-foyer's refusals
are, its `message` would be more useful to show. A bounded read of a JSON `Status`, or a
decoder the host supplies, with today's text as the fallback, would cover it.

## Ask 6: refuse redirects in the Kubernetes backend

**This one is about credentials.** client-go follows redirects, and its transport adds the
bearer token to the redirected request too. A backend built with
`kube.NewBackendForConfig(cfg)` from a `rest.Config` holding the caller's token therefore
sends that token wherever a redirect points: another host, or plain `http`. Pinning the
host and leaving `Proxy` unset does not prevent it. krm-foyer reproduced it against a
test API server that redirects to a recording server, which received the token. An
aggregated API's answer passes back through the API server, so a redirect is not only a
hostile-server case.

krm-foyer builds its client with `rest.HTTPClientFor`, sets `CheckRedirect` to refuse,
and ends the stream with a terminal error. We suggest `NewBackendForConfig` do the same,
and that the gateway map a refused redirect to a terminal error, since a redirect will
not go away on retry. The docs that show `kube.NewBackend(dynamicClientFor(user))`
should say a host-built client must refuse redirects too.

## Ask 7: treat a watch that ends before it was of use as a failure

The stream loop reopens at once, on the same connection, after `ErrWatchClosed`, and
after a `RESYNC_REQUIRED`. That is right for the API server's routine timeout, which
comes after half an hour. But an upstream that ends every watch before its snapshot is
complete, or answers 410 right after it, gets a tight loop: krm-foyer measured about 100
reopenings a second against a test server that does so, on one browser connection.

krm-foyer's backend now answers such an end with a non-terminal `UPSTREAM_UNAVAILABLE`,
so the connection closes and the client waits: an end before the `synced` boundary, or
within a second of it. We suggest the gateway do this itself, for every backend. One
detail cost us a bug: `StreamError.Unwrap` returns `Cause`, so an `UPSTREAM_UNAVAILABLE`
whose cause is `ErrWatchClosed` is still read by the loop as a routine close.

## Ask 8: say that client-go retries a 429 with Retry-After itself

client-go asks again on its own, up to ten times, when a 429 or 5xx answer carries a
`Retry-After` header, before the backend sees the error. The API server sets the pace, so
this is not a tight loop, but the browser hears nothing meanwhile, and the
`retryAfterMs` mapping of 0.5.0 only applies once client-go has given up. We suggest
saying so in the operations guide, and considering a client with those retries off,
so the browser's client, which shows the wait, owns them as 0.5.0 intends.
19 changes: 19 additions & 0 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,25 @@ as `INTERNAL` "internal error". Decide what to log there, and redact it.
| stream count / snapshot duration | connection pressure or oversized scopes | narrow namespaces/selectors; avoid accidental all-namespaces watches |
| unorderable `resourceVersion` terminal errors | an unsupported or aggregated API does not meet strict ordering | use `OrderingLenient` only after accepting the reduced monotonicity guarantee |

## When the API server pushes back

client-go retries on its own before the gateway sees an error. A 429 from Priority and Fairness, or
any 5xx, that carries a `Retry-After` header is retried up to ten times at the pace that header sets.
Only when client-go gives up does the gateway map the answer, to `UPSTREAM_UNAVAILABLE` with
`retryAfterMs`, and close the connection for the browser's client to retry.

While client-go waits, the browser holds an open connection with heartbeats and no snapshot yet: the
managed connector reports `syncing`, not an error. krm-stream keeps client-go's default on purpose.
The API server sets the pace, so it is not a tight loop, and the wait happens once per stream at the
API server's request, where a reconnect would cost every browser a new connection and a fresh
snapshot. The cost is that the browser is not told why it is waiting. A rising `syncing` time,
without errors, is the signal to look at API-server load.

A watch that ends before it is of use (before its snapshot completes, or within a second of it) is
recovered once on the same connection. A second such end in a row closes the connection with
`UPSTREAM_UNAVAILABLE`, so an upstream that ends every watch early meets the client's backoff and not
a reconnect loop. It shows as `retryable_error`.

## Runtime controls

| Control | Default | Use |
Expand Down
22 changes: 13 additions & 9 deletions docs/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

Releases are generated from conventional commits on `main`. Release Please opens a release pull
request with version bumps and changelog entries; merging that pull request creates the release.
Nothing in it needs editing by hand.

The changelogs are written entirely from commit subjects and `BREAKING CHANGE:` footers. Do not add
sections to a `CHANGELOG.md` yourself: Release Please never removes them, so an "Unreleased" section
would ship labelled unreleased. CI refuses one. Put the detail in the commit body or the PR
description, which the changelog links to.

| Commit | Release effect |
|---|---|
Expand All @@ -24,19 +30,17 @@ produces these tags and packages:

Only `@configbutler/krm-stream` is maintained on npm. Do not publish to the unscoped `krm-stream` name.

`gateway/kube/go.mod` requires the core module by version, and `go.work` hides that version in this
checkout. When an adapter change needs core API or behaviour that is not released yet, push the core
change first, then point the adapter at that commit from outside the workspace:
`gateway/kube/go.mod` requires the core module at the release it ships with. Every release pull
request rewrites that line to the version being released (its `x-release-please-version` marker),
so a released adapter always resolves at least its own core. Do not edit it, and do not pin a
pseudo-version when an adapter change needs unreleased core API: `go.work` wires the two modules
together in this checkout, and CI's `a stranger can go get this` job builds an unreleased adapter
with the core from the same commit. Someone using an unreleased commit does the same:

```sh
cd gateway/kube
GOWORK=off GOPROXY=direct go get github.com/ConfigButler/krm-stream/gateway@<commit>
GOWORK=off go mod tidy
go get github.com/ConfigButler/krm-stream/gateway/kube@<commit> github.com/ConfigButler/krm-stream/gateway@<commit>
```

Check it from a clean module outside this workspace, as CI's `a stranger can go get this` job does.
After the next release, move the requirement to the released core tag.

Before 1.0, remove superseded API names and forwarding packages instead of maintaining compatibility
shims. Record each removal and its replacement in release notes, and update repository callers,
examples and documentation together. Changes to wire semantics still require explicit spec and
Expand Down
Loading
Loading