diff --git a/CLAUDE.md b/CLAUDE.md index a1621c6a..d88e4bf6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -227,7 +227,7 @@ obol sell delete ollama-gated -n llm ## RPC Gateway -`obol network add|remove|status` manages remote RPCs via eRPC ConfigMap. Default: read-only (blocks `eth_sendRawTransaction`, `eth_sendTransaction`). `--allow-writes` flips readOnly → route `eth_sendRawTransaction` to the user-chosen upstream via eRPC per-method selection policy. `--endpoint` adds a custom RPC directly (skips ChainList). Key functions in `internal/network/rpc.go`: `AddPublicRPCs()` (ChainList), `AddCustomRPC()`, `ListRPCNetworks()`. Record-on-write: add/remove update `$CONFIG_DIR/rpc/recorded-upstreams.yaml` (`internal/network/record.go`); `obol stack up` replays via `ReconcileRecordedRPCs()` so remote RPCs survive cluster recreation. Local-node upstreams are NOT recorded (`network sync` re-registers them). +`obol network add|remove|status` manages remote RPCs via eRPC ConfigMap. Default: read-only (blocks `eth_sendRawTransaction`, `eth_sendTransaction`). `--allow-writes` flips readOnly → route `eth_sendRawTransaction` to the user-chosen upstream via eRPC per-method selection policy. `--endpoint` adds a custom RPC directly (skips ChainList). Key functions in `internal/network/rpc.go`: `AddPublicRPCs()` (ChainList), `AddCustomRPC()`, `ListRPCNetworks()`. Record-on-write: add/remove update `$CONFIG_DIR/rpc/recorded-upstreams.yaml` (`internal/network/record.go`); `obol stack up` replays via `ReconcileRecordedRPCs()` so remote RPCs survive cluster recreation. Local-node upstreams are NOT recorded (`network sync` re-registers them). **Durable eRPC operator config** (so local baskets are not lost on stack up): `obol network erpc set|status|reset` persists `$CONFIG_DIR/rpc/erpc-overlay.yaml` (merge-by-id into the live ConfigMap; not a full replace) and re-applies after recorded remotes on stack up via `ReconcileERPCOverlay()` (`internal/network/overlay.go`, #763). Example: `examples/erpc-overlay-hyperevm.yaml`. ## Network Management @@ -242,7 +242,7 @@ Two-stage templating: `values.yaml.gotmpl` annotated with `@enum`/`@default`/`@d | Command | Action | |---------|--------| | `obol stack init` | Generate cluster ID (petname), resolve absolute paths, write k3d.yaml, copy infrastructure | -| `obol stack up` | `k3d cluster create`, export kubeconfig, k3s auto-applies manifests, auto-configures LiteLLM with Ollama models (preserves Ollama modified-time order; `:cloud` aliases demoted behind local chat models; embedding-only models last; warns + suggests `ollama pull qwen3.5:4b` when empty or all-`:cloud`), re-imposes recorded model config (`model.ReconcileRecorded` — operator's `obol model` choices win over auto-detect), deploys default Hermes agent, applies agent capabilities, starts persistent Cloudflare tunnel, then replays recorded state: RPC upstreams (`network.ReconcileRecordedRPCs`) → Agent CRs (`agentcrd.ResumeAll`) → sell offers (`resumeSellOffers`; agent-type offers ride the sell-http store). Guard: `cmd/obol/stackup_resume_guard_test.go` | +| `obol stack up` | `k3d cluster create`, export kubeconfig, k3s auto-applies manifests, auto-configures LiteLLM with Ollama models (preserves Ollama modified-time order; `:cloud` aliases demoted behind local chat models; embedding-only models last; warns + suggests `ollama pull qwen3.5:4b` when empty or all-`:cloud`), re-imposes recorded model config (`model.ReconcileRecorded` — operator's `obol model` choices win over auto-detect), deploys default Hermes agent, applies agent capabilities, starts persistent Cloudflare tunnel, then replays recorded state: RPC upstreams (`network.ReconcileRecordedRPCs`) → eRPC operator overlay (`network.ReconcileERPCOverlay`) → Agent CRs (`agentcrd.ResumeAll`) → sell offers (`resumeSellOffers`; agent-type offers ride the sell-http store). Guard: `cmd/obol/stackup_resume_guard_test.go` | | `obol stack down` | `k3d cluster stop` (delete fallback; preserves config + data) | | `obol stack purge [-f]` | Delete config; `-f` also deletes root-owned PVCs; `-f` offers a full `stack export` first (fallback: OpenClaw wallet prompt) | | `obol stack export` | Full backup archive: config dir (minus kubeconfig/defaults), agent data dirs (brains + keystores, deployments quiesced for consistency), encrypted wallet backups, etcd-drift resources (Agent CRs, ServiceOffers, LiteLLM/eRPC CMs). `internal/stackbackup/` | diff --git a/README.md b/README.md index f4b2eb01..10c4a9b2 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ Built on [Kubernetes](https://kubernetes.io) with [Helm](https://helm.sh/) for p - **Linux**: [Docker Engine installation guide](https://docs.docker.com/engine/install/) - **macOS/Windows**: [Docker Desktop](https://docs.docker.com/desktop/) -For local models, install [Ollama](https://ollama.com) and pull at least one chat-capable model (e.g. `ollama pull qwen3:8b`). Skip this if you'll use a cloud provider (see [Models](#models)). +For local models, install [Ollama](https://ollama.com) and pull at least one chat-capable model (e.g. `ollama pull qwen3.5:4b`). The installer may offer to install Ollama; **if you decline and do not configure a cloud model**, `obol stack up` **skips the default Hermes agent** until you run `obol model setup` and then `obol agent init`. Prefer a cloud provider anytime with `obol model setup` (see [Models](#models)). ### Install @@ -42,7 +42,25 @@ For local models, install [Ollama](https://ollama.com) and pull at least one cha bash <(curl -fsSL https://stack.obol.org) ``` -The installer sets up the `obol` CLI and all dependencies (`kubectl`, `helm`, `k3d`, `helmfile`, `k9s`) into `~/.local/bin/`, verifies release checksums, configures your PATH, and offers to start the cluster. +Pin a release (use the current tag from the [releases page](https://github.com/ObolNetwork/obol-stack/releases), not an ancient example): + +```bash +OBOL_RELEASE=v0.13.0 bash <(curl -fsSL https://stack.obol.org) +``` + +The installer sets up the `obol` CLI and all dependencies (`kubectl`, `helm`, `k3d`, `helmfile`, `k9s`) into `~/.local/bin/`, verifies release checksums, configures your PATH, tries to add `obol.stack` to `/etc/hosts`, and offers to start the cluster. + +If `/etc/hosts` cannot be updated (no sudo / you cancel), the install still finishes — add the host, then start the stack yourself: + +```bash +echo "127.0.0.1 obol.stack" | sudo tee -a /etc/hosts +# After stack up, agent hostnames are also managed (obol-agent.obol.stack, …) +obol stack init +obol stack up +obol agent init # if the default agent was skipped (no model yet) +``` + +Non-interactive / CI: set `OBOL_NONINTERACTIVE=true` so hosts updates do not prompt for a sudo password (they fail fast if credentials are not already cached). Combine with a pre-written `/etc/hosts` entry or run `sudo -v` first. Verify: @@ -65,7 +83,33 @@ obol agent list obol agent auth obol-agent ``` -`obol stack up` provisions the cluster, auto-detects your local Ollama models into the LiteLLM gateway, deploys the default Hermes agent (with its own wallet behind a remote signer), and starts a Cloudflare quick-tunnel. From here you can chat with your agent locally at `http://obol.stack:8080` — or go straight to selling. +`obol stack up` provisions the cluster, auto-detects host Ollama models into the **LiteLLM** gateway (when present), and deploys the default **Hermes** agent with its own wallet behind a remote signer. The Cloudflare tunnel stays **dormant** until the first sell workflow (or `obol tunnel restart` / `obol tunnel setup`). + +### Local UI URL (Host header) + +Open the frontend at: + +```text +http://obol.stack:8080 +``` + +Use **`obol.stack`**, not `localhost`. Traefik routes the frontend (and eRPC) only for `Host: obol.stack`. `http://localhost:8080` returns **404** even when the stack is healthy. + +- Prefer `:8080` on macOS when port 80 is unavailable (or after editing `k3d.yaml` to drop privileged 80/443 binds). +- If port 80 is mapped, `http://obol.stack/` works too. + +Hermes dashboard (root edge-redirects to the password form): + +```text +http://obol-agent.obol.stack +``` + +(or with `:8080` when that is your ingress) + +- **Username:** `obol` +- **Password:** the agent API token from `obol agent auth obol-agent` + +Details: [Hermes dashboard login](docs/guides/hermes-dashboard-login.md). ## Sell: Your First Paid Service @@ -406,7 +450,19 @@ Edit `~/.config/obol/k3d.yaml`, remove the `80:80` and `443:443` port entries (k obol stack down && obol stack up ``` -Access at http://obol.stack:8080 instead. +Access at http://obol.stack:8080 instead (still with the `obol.stack` host — not `localhost`). + +#### `/etc/hosts` / sudo prompts + +`obol stack up` also tries to refresh managed hostnames (`obol.stack`, `obol-agent.obol.stack`, …). A failed hosts write is a **warning**, not a hard stop — the cluster still comes up. Fix hosts, then re-run `obol stack up` or `obol agent sync` so agent hostnames are registered. + +```bash +# Minimal manual entry +echo "127.0.0.1 obol.stack" | sudo tee -a /etc/hosts + +# Skip interactive sudo during automation (must already have NOPASSWD or a cached timestamp) +OBOL_NONINTERACTIVE=true obol stack up +``` #### Monetize Flow Preflight diff --git a/cmd/obol/bootstrap.go b/cmd/obol/bootstrap.go index 502df83b..1db87933 100644 --- a/cmd/obol/bootstrap.go +++ b/cmd/obol/bootstrap.go @@ -31,7 +31,7 @@ func bootstrapCommand(cfg *config.Config) *cli.Command { // Step 1: Initialize stack backendName := stack.DetectExistingBackend(cfg) - if err := stack.Init(cfg, u, false, backendName); err != nil { + if err := stack.Init(cfg, u, false, backendName, false); err != nil { if !strings.Contains(err.Error(), "already exists") { return fmt.Errorf("bootstrap init failed: %w", err) } diff --git a/cmd/obol/main.go b/cmd/obol/main.go index d82d2f69..395913fd 100644 --- a/cmd/obol/main.go +++ b/cmd/obol/main.go @@ -190,9 +190,14 @@ GLOBAL OPTIONS:{{template "visibleFlagTemplate" .}}{{end}} Usage: "Cluster backend: k3d (Docker-based) or k3s (bare-metal)", Sources: cli.EnvVars("OBOL_BACKEND"), }, + &cli.BoolFlag{ + Name: "yes", + Aliases: []string{"y"}, + Usage: "Skip the live-services confirmation prompt when --force switches backends (required in non-interactive shells when offers are running)", + }, }, Action: func(ctx context.Context, cmd *cli.Command) error { - return stack.Init(cfg, getUI(cmd), cmd.Bool("force"), cmd.String("backend")) + return stack.Init(cfg, getUI(cmd), cmd.Bool("force"), cmd.String("backend"), cmd.Bool("yes")) }, }, { @@ -212,6 +217,11 @@ GLOBAL OPTIONS:{{template "visibleFlagTemplate" .}}{{end}} // Replay recorded remote RPC upstreams into the // (possibly fresh) eRPC ConfigMap. Best-effort. network.ReconcileRecordedRPCs(cfg, u) + // Re-apply durable multi-upstream eRPC operator + // overlays (baskets/scoring/rate-limits) AFTER + // simple recorded remotes. Best-effort. + // See ObolNetwork/obol-stack#763. + network.ReconcileERPCOverlay(cfg, u) // Re-apply recorded Agent CRs BEFORE sell offers: // agent-backed ServiceOffers resolve agent.ref and // would dangle without their Agent. Best-effort. diff --git a/cmd/obol/network.go b/cmd/obol/network.go index 701b2226..69845170 100644 --- a/cmd/obol/network.go +++ b/cmd/obol/network.go @@ -76,6 +76,7 @@ func networkCommand(cfg *config.Config) *cli.Command { networkAddCommand(cfg), networkRemoveCommand(cfg), networkStatusCommand(cfg), + networkERPCCommand(cfg), }, } } @@ -534,6 +535,91 @@ func networkStatusCommand(cfg *config.Config) *cli.Command { } } +// network erpc — durable operator eRPC config that survives stack up (#763). +// Verbs match sell-info set/reset (host-side intent) + status. +// --------------------------------------------------------------------------- + +func networkERPCCommand(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "erpc", + Usage: "Manage durable operator eRPC config (host-side; re-applied on stack up so local baskets are not lost)", + Commands: []*cli.Command{ + { + Name: "set", + Usage: "Set durable eRPC config from a YAML file and merge it into the live ConfigMap", + Flags: []cli.Flag{ + &cli.StringFlag{ + Name: "file", + Aliases: []string{"f"}, + Usage: "YAML fragment: networks, upstreams, rateLimiters, cachePoliciesAdd (saved to $CONFIG_DIR/rpc/erpc-overlay.yaml)", + Required: true, + }, + }, + Action: func(_ context.Context, cmd *cli.Command) error { + return network.SetERPC(cfg, getUI(cmd), cmd.String("file")) + }, + }, + { + Name: "status", + Usage: "Show the durable eRPC config on disk", + Action: func(_ context.Context, cmd *cli.Command) error { + u := getUI(cmd) + st, err := network.StatusERPC(cfg) + if err != nil { + return err + } + u.Printf("eRPC operator config\n") + u.Printf("====================\n\n") + u.Printf("Path: %s\n", st.Path) + if !st.Present { + u.Info("Status: not set") + u.Info("Set with: obol network erpc set -f ") + return nil + } + u.Printf("Status: set (v%d, hash %s)\n", st.Version, st.ContentHash) + switch st.ClusterSync { + case "": + u.Info("Cluster: unknown (could not reach cluster to check for drift)") + case network.ERPCSyncInSync: + u.Info("Cluster: in-sync") + case network.ERPCSyncNotApplied: + u.Warn("Cluster: not-applied (overlay is on disk but not on the live ConfigMap — run `obol network erpc set -f ` or `obol stack up`)") + default: + u.Warnf("Cluster: %s (live ConfigMap does not match the on-disk overlay — re-run `obol network erpc set -f `)", st.ClusterSync) + } + u.Printf("Networks (%d):\n", st.NetworkCount) + for _, k := range st.NetworkKeys { + u.Printf(" - %s\n", k) + } + if st.NetworkCount == 0 { + u.Info(" (none)") + } + u.Printf("Upstreams (%d):\n", st.UpstreamCount) + for _, id := range st.UpstreamIDs { + u.Printf(" - %s\n", id) + } + if st.UpstreamCount == 0 { + u.Info(" (none)") + } + u.Printf("Rate-limit budgets: %d\n", st.BudgetCount) + u.Printf("Cache policies to add: %d\n", st.CachePolicyAdd) + return nil + }, + }, + { + Name: "reset", + Usage: "Reset durable eRPC config (strip from live ConfigMap and delete host file)", + Action: func(_ context.Context, cmd *cli.Command) error { + return network.ResetERPC(cfg, getUI(cmd)) + }, + }, + }, + Action: func(ctx context.Context, cmd *cli.Command) error { + return cli.ShowSubcommandHelp(cmd) + }, + } +} + // chainIDToName returns a human-readable name for a chain ID. func chainIDToName(chainID int) string { names := map[int]string{ @@ -550,6 +636,7 @@ func chainIDToName(chainID int) string { 43114: "Avalanche", 59144: "Linea", 84532: "Base Sepolia", + 999: "HyperEVM", 534352: "Scroll", 560048: "Hoodi", 11155111: "Sepolia", diff --git a/cmd/obol/sell.go b/cmd/obol/sell.go index 7e278377..2df0f18a 100644 --- a/cmd/obol/sell.go +++ b/cmd/obol/sell.go @@ -11,6 +11,7 @@ import ( "math/big" "net" "net/http" + "net/url" "os" "os/exec" "os/signal" @@ -746,8 +747,10 @@ Examples: Value: 8080, }, &cli.StringFlag{ - Name: "health-path", - Usage: "Upstream health check path", + Name: "health-path", + Usage: "Upstream health check path. When left at the default, --upstream litellm " + + "defaults to /health/readiness and --upstream ollama defaults to / — " + + "neither serves a usable /health.", Value: "/health", }, &cli.StringFlag{ @@ -966,13 +969,19 @@ Examples: } wallet = primaryPayTo + upstreamSvc := cmd.String("upstream") + healthPath := cmd.String("health-path") + if defaulted := defaultHealthPathForUpstream(upstreamSvc, healthPath); defaulted != healthPath { + healthPath = defaulted + u.Dim(fmt.Sprintf(" --upstream %s: no usable /health, defaulting --health-path to %q", upstreamSvc, healthPath)) + } spec := map[string]any{ "type": "http", "upstream": map[string]any{ - "service": cmd.String("upstream"), + "service": upstreamSvc, "namespace": ns, "port": cmd.Int("port"), - "healthPath": cmd.String("health-path"), + "healthPath": healthPath, }, "payment": paymentBlock, } @@ -1125,6 +1134,16 @@ Examples: u.Infof("The agent will reconcile: health-check → payment gate → route") u.Infof("Check status: obol sell status %s -n %s", name, ns) + if strings.TrimSpace(cmd.String("hostname")) != "" { + // The offer's spec.hostname binding just landed on the + // cluster; narrow (or tear down) the storefront's catch-all + // route for it now rather than waiting for some later, + // unrelated tunnel/sell invocation to notice. + if err := tunnel.RefreshStorefront(cfg); err != nil { + u.Warnf("could not refresh storefront: %v", err) + } + } + if !registerEnabled { // Best-effort tunnel for --no-register: not fatal if it doesn't come up. u.Blank() @@ -1403,19 +1422,27 @@ func autoRegisterServiceOffer(ctx context.Context, cfg *config.Config, u *ui.UI, u.Info(note) } - agentURI := strings.TrimRight(opts.Endpoint, "/") + "/.well-known/agent-registration.json" + origin, err := normalizeAgentOrigin(opts.Endpoint) + if err != nil { + return err + } + agentURI := origin + "/.well-known/agent-registration.json" u.Printf(" Agent URI: %s", agentURI) - var successes int + var successes, metadataFailures int for _, net := range networks { u.Blank() u.Printf(" [%s] (chain ID %d)", net.Name, net.ChainID) u.Printf(" Registry: %s", net.RegistryAddress) - if err := registerDirectViaSigner(ctx, cfg, u, net, agentURI, signerNS); err != nil { + metadataOK, err := registerDirectViaSigner(ctx, cfg, u, net, agentURI, signerNS) + if err != nil { u.Warnf("direct registration failed: %v", err) continue } + if !metadataOK { + metadataFailures++ + } u.Printf(" Name: %s", opts.AgentName) u.Printf(" Summary: %s", opts.AgentDesc) @@ -1429,6 +1456,9 @@ func autoRegisterServiceOffer(ctx context.Context, cfg *config.Config, u *ui.UI, u.Blank() u.Successf("Seller agent registered on %d/%d networks.", successes, len(networks)) + if metadataFailures > 0 { + u.Warnf("x402 metadata missing on %d/%d network(s); mint + agentURI are registered — see warnings above, re-run 'obol sell register' to retry the metadata write.", metadataFailures, successes) + } return nil } @@ -1989,12 +2019,16 @@ func autoRegisterDemo(ctx context.Context, cfg *config.Config, u *ui.UI, chain, } agentURI := strings.TrimRight(tunnelURL, "/") + "/.well-known/agent-registration.json" - if registerAgentOnNetworks(ctx, cfg, u, agentURI, signerNS, []erc8004.NetworkConfig{net}) == 0 { + succeeded, metadataFailures := registerAgentOnNetworks(ctx, cfg, u, agentURI, signerNS, []erc8004.NetworkConfig{net}) + if len(succeeded) == 0 { u.Warn("Auto-register did not succeed.") u.Dim(" Retry with: obol sell register --network " + chain) return } u.Successf("Agent registered on %s.", net.Name) + if metadataFailures > 0 { + u.Warnf("x402 metadata missing; mint + agentURI are registered — re-run 'obol sell register --network %s' to retry the metadata write.", chain) + } } // waitForOfferReady polls a ServiceOffer's Ready condition for up to `timeout`. @@ -3221,8 +3255,9 @@ Examples: Value: "mainnet", }, &cli.StringFlag{ - Name: "endpoint", - Usage: "Service endpoint URL (auto-detected from tunnel if not set)", + Name: "origin", + Aliases: []string{"endpoint"}, + Usage: "Storefront origin (scheme://host[:port]) — agent-registration.json is always served at its root, never a service path (auto-detected from tunnel if not set)", }, &cli.StringFlag{ Name: "name", @@ -3292,23 +3327,27 @@ Examples: } } - // Resolve endpoint. - endpoint := cmd.String("endpoint") - if endpoint == "" { + // Resolve origin. + origin := cmd.String("origin") + if origin == "" { tunnelURL, err := tunnel.GetTunnelURL(cfg) if err != nil { if u.IsTTY() { - endpoint, _ = u.Input("Service endpoint URL", "") + origin, _ = u.Input("Storefront origin", "") } - if endpoint == "" { - return fmt.Errorf("--endpoint required (tunnel auto-detect failed: %v)", err) + if origin == "" { + return fmt.Errorf("--origin required (tunnel auto-detect failed: %v)", err) } } else { - endpoint = tunnelURL - u.Infof("Auto-detected endpoint from tunnel: %s", endpoint) + origin = tunnelURL + u.Infof("Auto-detected origin from tunnel: %s", origin) } } - agentURI := endpoint + "/.well-known/agent-registration.json" + origin, err = normalizeAgentOrigin(origin) + if err != nil { + return err + } + agentURI := origin + "/.well-known/agent-registration.json" // All signing happens via the agent's remote-signer; the CLI never // sees raw key material. If no Hermes default agent is configured, @@ -3327,39 +3366,159 @@ Examples: u.Printf(" Agent URI: %s", agentURI) u.Printf(" Networks: %s", chainCSV) - successes := registerAgentOnNetworks(ctx, cfg, u, agentURI, signerNS, networks) - if successes == 0 { + succeeded, metadataFailures := registerAgentOnNetworks(ctx, cfg, u, agentURI, signerNS, networks) + if len(succeeded) == 0 { return fmt.Errorf("registration failed on all networks") } + // Offers created with --no-register stay registration.enabled=false + // and never flip to Registered/Active even after a successful CLI + // register. Enable them so the controller attaches them to the + // shared AgentIdentity and discovery surfaces services. Scoped to + // the networks that just succeeded so a register on one chain + // doesn't flip on offers pinned to a different, unregistered chain. + if n, err := enableRegistrationOnOffers(cfg, u, succeeded); err != nil { + u.Warnf("could not enable registration on existing ServiceOffers: %v", err) + u.Dim(" Manually set registration.enabled=true on offers created with --no-register.") + } else if n > 0 { + u.Infof("Enabled registration on %d ServiceOffer(s) previously created with --no-register", n) + } + u.Blank() - u.Successf("Agent registered on %d/%d networks.", successes, len(networks)) + u.Successf("Agent registered on %d/%d networks.", len(succeeded), len(networks)) + if metadataFailures > 0 { + u.Warnf("x402 metadata missing on %d/%d network(s); mint + agentURI are registered — see warnings above, re-run 'obol sell register' to retry the metadata write.", metadataFailures, len(succeeded)) + } return nil }, } } +// defaultHealthPathForUpstream fills in a working health-check path for +// upstreams whose generic "/health" default (the --health-path flag's zero +// value) never returns 2xx, which would permanently fail UpstreamHealthy's +// 2xx-only probe gate. Returns healthPath unchanged for any explicit, +// non-default value or an upstream with no special case. +func defaultHealthPathForUpstream(upstreamSvc, healthPath string) string { + if healthPath != "" && healthPath != "/health" { + return healthPath + } + switch { + case strings.EqualFold(upstreamSvc, "litellm"): + // LiteLLM's /health requires the master key (401 unauthenticated). + // Its readiness/liveliness probes are public and return 2xx. + return "/health/readiness" + case strings.EqualFold(upstreamSvc, "ollama"): + // Ollama serves neither /health nor /health/readiness — only "/" + // returns 2xx. + return "/" + default: + return healthPath + } +} + +// normalizeAgentOrigin validates that raw is a bare storefront origin +// (scheme://host[:port], no path) and returns it with any trailing slash +// stripped. The controller only ever publishes agent-registration.json at +// the storefront origin root (never a per-offer service path — see +// serviceoffercontroller's PublishedURL/HTTPRoute construction), so a +// path-bearing value would mint an on-chain agentURI that resolves to a +// route the controller never serves. Reject rather than silently truncate: +// silent truncation is exactly what produces a broken, payment-gated +// metadata URI in the field. +func normalizeAgentOrigin(raw string) (string, error) { + parsed, err := url.Parse(raw) + if err != nil || parsed.Host == "" { + return "", fmt.Errorf("--origin must be a bare host with no path (got %q): expected scheme://host[:port]", raw) + } + if parsed.Scheme != "http" && parsed.Scheme != "https" { + return "", fmt.Errorf("--origin must use http or https (got %q): expected scheme://host[:port]", raw) + } + if parsed.Path != "" && parsed.Path != "/" { + return "", fmt.Errorf("--origin must be a bare host with no path (got %q); agent-registration.json is served at the storefront root, not a service path", raw) + } + return strings.TrimRight(parsed.Scheme+"://"+parsed.Host, "/"), nil +} + +// autoRegisterOptOutAnnotation lets an offer opt out of +// enableRegistrationOnOffers' cluster-wide sweep permanently — e.g. an +// operator who deliberately created an offer with --no-register and wants it +// to stay unlisted even after a later `obol sell register` succeeds on its +// network. Not set by --no-register itself (that would defeat the sweep for +// every ordinary --no-register offer); it's an explicit escape hatch. +const autoRegisterOptOutAnnotation = "obol.stack/no-auto-register" + +// offerEligibleForAutoEnable reports whether an offer should have +// registration.enabled flipped on: it must still be disabled, pinned to one +// of the networks the triggering `obol sell register` call actually +// succeeded on (registeredNetworks, keyed by erc8004.NetworkConfig.Name), +// and not carrying the autoRegisterOptOutAnnotation opt-out. +func offerEligibleForAutoEnable(o monetizeapi.ServiceOffer, registeredNetworks map[string]bool) bool { + if o.Annotations[autoRegisterOptOutAnnotation] == "true" { + return false + } + return !o.Spec.Registration.Enabled && registeredNetworks[o.Spec.Payment.Network] +} + +// enableRegistrationOnOffers patches every ServiceOffer on one of the given +// networks that still has registration.enabled=false to true, so post-hoc +// `obol sell register` activates existing offers instead of leaving them +// Disabled while the AgentIdentity is already on-chain. Scoped to +// networks — a register call for one chain must not enable offers pinned to +// a different chain that was never registered. +func enableRegistrationOnOffers(cfg *config.Config, u *ui.UI, networks []erc8004.NetworkConfig) (int, error) { + offers, err := listServiceOffers(cfg) + if err != nil { + return 0, err + } + registered := make(map[string]bool, len(networks)) + for _, n := range networks { + registered[n.Name] = true + } + enabled := 0 + for _, o := range offers { + if !offerEligibleForAutoEnable(o, registered) { + continue + } + // Merge-patch only the enabled flag; leave name/description/skills alone. + patch := `{"spec":{"registration":{"enabled":true}}}` + if _, err := kubectlOutput(cfg, "patch", "serviceoffer.obol.org", o.Name, + "-n", o.Namespace, "--type", "merge", "-p", patch); err != nil { + u.Warnf(" %s/%s: %v", o.Namespace, o.Name, err) + continue + } + u.Dim(fmt.Sprintf(" %s/%s: registration.enabled → true", o.Namespace, o.Name)) + enabled++ + } + return enabled, nil +} + // registerAgentOnNetworks runs the per-network ERC-8004 registration loop used // by both `obol sell register` and the auto-register step of `obol sell demo`. // Each registration is signed by the Hermes remote-signer and pays gas from -// the agent's wallet. Returns the number of networks that registered -// successfully. -func registerAgentOnNetworks(ctx context.Context, cfg *config.Config, u *ui.UI, agentURI, signerNS string, networks []erc8004.NetworkConfig) int { - var successes int +// the agent's wallet. Returns the networks that registered successfully (so +// callers can scope follow-up actions to exactly those networks) and a count +// of networks where the mint/URI stage succeeded but the trailing x402 +// metadata write failed. +func registerAgentOnNetworks(ctx context.Context, cfg *config.Config, u *ui.UI, agentURI, signerNS string, networks []erc8004.NetworkConfig) (succeeded []erc8004.NetworkConfig, metadataFailures int) { for _, net := range networks { u.Blank() u.Printf(" [%s] (chain ID %d)", net.Name, net.ChainID) u.Printf(" Registry: %s", net.RegistryAddress) - if err := registerDirectViaSigner(ctx, cfg, u, net, agentURI, signerNS); err != nil { + metadataOK, err := registerDirectViaSigner(ctx, cfg, u, net, agentURI, signerNS) + if err != nil { u.Warnf("registration failed: %v", err) continue } + if !metadataOK { + metadataFailures++ + } u.Printf(" CAIP-10: %s", net.CAIP10Registry()) - successes++ + succeeded = append(succeeded, net) } - return successes + return succeeded, metadataFailures } // registerDirectViaSigner performs an idempotent ERC-8004 registration via @@ -3371,34 +3530,52 @@ func registerAgentOnNetworks(ctx context.Context, cfg *config.Config, u *ui.UI, // // The AgentIdentity CR persists only durable ERC-8004 identity keys: // status.registrations[] entries keyed by chain. -func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, net erc8004.NetworkConfig, agentURI, namespace string) error { +// +// Returns metadataOK=false (with a nil error) when the mint/URI-update stage +// succeeded but the trailing x402 metadata write failed — that's reported as +// a distinct, non-fatal stage by the caller rather than swallowed. +func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, net erc8004.NetworkConfig, agentURI, namespace string) (metadataOK bool, err error) { u.Printf(" Using direct on-chain registration via remote-signer...") pf, err := startSignerPortForward(cfg, namespace) if err != nil { - return fmt.Errorf("port-forward to remote-signer: %w", err) + return true, fmt.Errorf("port-forward to remote-signer: %w", err) } defer pf.Stop() signer := erc8004.NewRemoteSigner(fmt.Sprintf("http://localhost:%d", pf.localPort)) addr, err := signer.GetAddress(ctx) if err != nil { - return err + return true, err } u.Printf(" Wallet: %s", addr.Hex()) rpcBaseURL := stack.LocalIngressURL(cfg) + "/rpc" client, err := erc8004.NewClientForNetwork(ctx, rpcBaseURL, net) if err != nil { - return fmt.Errorf("connect to %s via eRPC: %w", net.Name, err) + return true, fmt.Errorf("connect to %s via eRPC: %w", net.Name, err) } defer client.Close() opts := signer.RemoteTransactOpts(ctx, addr, client.ChainID()) + // Pin the nonce locally for the whole call instead of letting each + // Transact re-query PendingNonceAt: this function sends up to two + // sequential txs (e.g. setAgentURI then setMetadata), and the eRPC + // gateway fans requests out to independent upstreams per call, so a + // nonce read immediately following a just-broadcast tx can hit a + // lagging upstream and return a stale (too-low) nonce. bumpNonce below + // advances the pinned value after each successful send. + nonce, err := client.PendingNonceAt(ctx, addr) + if err != nil { + return true, fmt.Errorf("read pending nonce for %s on %s: %w", addr.Hex(), net.Name, err) + } + opts.Nonce = new(big.Int).SetUint64(nonce) + bumpNonce := func() { opts.Nonce = new(big.Int).Add(opts.Nonce, big.NewInt(1)) } + identity, err := ensureAgentIdentity(cfg, monetizeapi.AgentIdentityDefaultNamespace, monetizeapi.AgentIdentityDefaultName, monetizeapi.AgentIdentitySpec{}) if err != nil { - return fmt.Errorf("load AgentIdentity: %w", err) + return true, fmt.Errorf("load AgentIdentity: %w", err) } existingAgentID := monetizeapi.AgentIdentityAgentIDForChain(identity.Status, net.Name) @@ -3406,21 +3583,21 @@ func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, if existingAgentID != "" { agentID, ok := new(big.Int).SetString(existingAgentID, 10) if !ok { - return fmt.Errorf("AgentIdentity %s/%s has malformed agentId %q for chain %s", + return true, fmt.Errorf("AgentIdentity %s/%s has malformed agentId %q for chain %s", identity.Metadata.Namespace, identity.Metadata.Name, existingAgentID, net.Name) } owner, walletErr := client.AgentWallet(ctx, agentID) if walletErr != nil { - return fmt.Errorf("verify agent %s on %s: %w", agentID, net.Name, walletErr) + return true, fmt.Errorf("verify agent %s on %s: %w", agentID, net.Name, walletErr) } if owner != addr { - return fmt.Errorf("signer %s does not control AgentIdentity agent %s (on-chain owner: %s)", addr.Hex(), agentID, owner.Hex()) + return true, fmt.Errorf("signer %s does not control AgentIdentity agent %s (on-chain owner: %s)", addr.Hex(), agentID, owner.Hex()) } u.Printf(" Agent ID: %s (existing)", agentID.String()) currentURI, err := client.TokenURI(ctx, agentID) if err != nil { - return fmt.Errorf("read tokenURI(%s): %w", agentID, err) + return true, fmt.Errorf("read tokenURI(%s): %w", agentID, err) } if currentURI == agentURI { u.Printf(" URI: unchanged, skipping setAgentURI") @@ -3428,15 +3605,21 @@ func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, u.Printf(" Updating agentURI via setAgentURI...") uriTx, err := client.SetAgentURIWithOpts(ctx, opts, agentID, agentURI) if err != nil { - return fmt.Errorf("setAgentURI: %w", err) + return true, fmt.Errorf("setAgentURI: %w", err) } u.Printf(" Tx: %s", uriTx) + bumpNonce() + // No WaitForAgent here: the agent already existed before this + // branch (AgentWallet above already confirmed the reader sees + // it), so there's no token-visibility race to close — only the + // nonce race, which the pinning above already handles. } // Refresh x402 metadata to keep parity with first-mint behavior. if err := client.SetMetadataWithOpts(ctx, opts, agentID, "x402", []byte(`{"x402":true}`)); err != nil { u.Warnf("failed to set x402 metadata: %v", err) + return false, nil } - return nil + return true, nil } // No recorded agentId: try owner+URI recovery from genesis before @@ -3446,13 +3629,14 @@ func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, u.Printf(" Agent ID: %s (recovered via owner+URI)", recoveredID.String()) identity.Status = monetizeapi.UpsertAgentIdentityRegistration(identity.Status, net.Name, recoveredID.String()) if err := applyAgentIdentity(cfg, identity); err != nil { - return fmt.Errorf("persist recovered AgentIdentity registration %s on %s: %w", recoveredID, net.Name, err) + return true, fmt.Errorf("persist recovered AgentIdentity registration %s on %s: %w", recoveredID, net.Name, err) } _, _ = client.WaitForAgent(ctx, recoveredID, 30*time.Second) if err := client.SetMetadataWithOpts(ctx, opts, recoveredID, "x402", []byte(`{"x402":true}`)); err != nil { u.Warnf("failed to set x402 metadata: %v", err) + return false, nil } - return nil + return true, nil } // Fresh mint. @@ -3461,8 +3645,9 @@ func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, return client.RegisterWithOptsDetailed(ctx, opts, agentURI) }) if err != nil { - return err + return true, err } + bumpNonce() u.Printf(" Agent ID: %s", agentID.String()) u.Printf(" Owner: %s", addr.Hex()) @@ -3476,13 +3661,16 @@ func registerDirectViaSigner(ctx context.Context, cfg *config.Config, u *ui.UI, if err := client.SetMetadataWithOpts(ctx, opts, agentID, "x402", []byte(`{"x402":true}`)); err != nil { u.Warnf("failed to set x402 metadata: %v", err) + metadataOK = false + } else { + metadataOK = true } identity.Status = monetizeapi.UpsertAgentIdentityRegistration(identity.Status, net.Name, agentID.String()) if err := applyAgentIdentity(cfg, identity); err != nil { - return fmt.Errorf("persist AgentIdentity registration %s on %s: %w\n\n The on-chain registration succeeded; recover with `obol sell identity import --network %s --agent-id %s`.", agentID, net.Name, err, net.Name, agentID) + return metadataOK, fmt.Errorf("persist AgentIdentity registration %s on %s: %w\n\n The on-chain registration succeeded; recover with `obol sell identity import --network %s --agent-id %s`.", agentID, net.Name, err, net.Name, agentID) } - return nil + return metadataOK, nil } func registrationRecoveryStartBlock(ctx context.Context, client *erc8004.Client, u *ui.UI) uint64 { @@ -3557,6 +3745,15 @@ func isTransientRegistrationError(err error) bool { "connection reset", "eof", "too many requests", + // registerWithRecovery pins the nonce for the whole call and reuses + // it across retries, so a broadcast-timeout retry can resubmit the + // exact tx the node already has: these three responses mean no new + // tx was accepted (no double-mint risk), and retrying gives + // recoverRegistrationByOwnerAndURI more time to see the original + // broadcast land instead of reporting a landed mint as failed. + "already known", + "nonce too low", + "replacement transaction underpriced", } { if strings.Contains(msg, needle) { return true @@ -3817,6 +4014,10 @@ func resolvePriceTable(cmd *cli.Command, allowPerHour bool) (schemas.PriceTable, switch { case perRequest != "": + if err := validate.Price(perRequest); err != nil { + return schemas.PriceTable{}, fmt.Errorf("invalid --price/--per-request value %q: %w", perRequest, err) + } + return schemas.PriceTable{PerRequest: perRequest}, nil case perMTok != "": if _, err := schemas.ApproximateRequestPriceFromPerMTok(perMTok); err != nil { diff --git a/cmd/obol/sell_agent.go b/cmd/obol/sell_agent.go index c7ee9038..783d81bb 100644 --- a/cmd/obol/sell_agent.go +++ b/cmd/obol/sell_agent.go @@ -82,7 +82,7 @@ Examples: &cli.StringFlag{ Name: "description", Aliases: []string{"register-description"}, - Usage: "Human-readable description of the service. Surfaced on the 402 payment page, in the storefront catalog, and (when registration is enabled) on the ERC-8004 registration document. Defaults to the agent's objective.", + Usage: "Human-readable description of the service. Surfaced on the 402 payment page, in the storefront catalog, and (when registration is enabled) on the ERC-8004 registration document. Defaults to a generic service description.", }, }, acceptFlags()...), Action: func(ctx context.Context, cmd *cli.Command) error { @@ -228,9 +228,9 @@ Examples: if regName == "" { regName = name } - regDesc := strings.TrimSpace(cmd.String("description")) - if regDesc == "" { - regDesc = agent.Objective + regDesc := resolveAgentOfferDescription(cmd.String("description"), agent.Objective) + if regDesc == "" && register { + u.Warnf("No --description set; the storefront listing will use a generic description. Pass --description to customize it.") } // Build the ServiceOffer manifest. type=agent + agent.ref tells @@ -696,6 +696,18 @@ func agentOfferRegistrationMetadata(agent *agentRefForSale, price, symbol, chain return metadata } +// resolveAgentOfferDescription returns the value to write to +// spec.registration.description given the --description flag and the +// agent's Objective. It deliberately ignores objective: that field is +// operator-authored system-prompt text (tool addresses, wallet details, +// operational instructions) and must never default onto the public +// storefront listing. An operator who wants a description must pass +// --description explicitly; otherwise the controller's own generic +// fallback applies. +func resolveAgentOfferDescription(flagValue, objective string) string { + return strings.TrimSpace(flagValue) +} + // agentOfferBundle wraps an agent-typed ServiceOffer manifest together // with its namespace in a v1 List for the resume ledger. After a full // stack recreation the agent namespace is gone, and a bare offer diff --git a/cmd/obol/sell_agent_test.go b/cmd/obol/sell_agent_test.go index 758e01af..ce9cc74c 100644 --- a/cmd/obol/sell_agent_test.go +++ b/cmd/obol/sell_agent_test.go @@ -179,6 +179,27 @@ func TestAgentOfferRegistrationMetadata_DefaultsRuntimeHermes(t *testing.T) { } } +func TestResolveAgentOfferDescription_NoDescriptionFlagStaysEmpty(t *testing.T) { + // Regression: sell_agent.go used to default an omitted --description to + // agent.Objective — leaking operator-authored system-prompt text (tool + // addresses, wallet details, operational instructions) into the public + // storefront/registration description. resolveAgentOfferDescription must + // never manufacture a non-empty description; it stays empty and the + // controller's own generic fallback applies. + objective := "You are a focused sub-agent. Trade using wallet 0xDEADBEEF via tool http://internal.example/mcp" + got := resolveAgentOfferDescription("", objective) + if got != "" { + t.Fatalf("resolveAgentOfferDescription(\"\", objective) = %q, want empty (must not fall back to the objective)", got) + } +} + +func TestResolveAgentOfferDescription_PassesThroughExplicitFlag(t *testing.T) { + got := resolveAgentOfferDescription(" Quant trading agent ", "unrelated objective text") + if got != "Quant trading agent" { + t.Errorf("resolveAgentOfferDescription = %q, want trimmed explicit value", got) + } +} + func TestSellAgentCommand_FlagShape(t *testing.T) { cfg := newTestConfig(t) cmd := sellCommand(cfg) @@ -187,6 +208,12 @@ func TestSellAgentCommand_FlagShape(t *testing.T) { requireFlags(t, flags, "pay-to", "wallet", "chain", "token", "price", "per-request", "path", "max-timeout", "no-register") + // The --description Usage text used to advertise the leaky + // objective-fallback default; it must not document that anymore. + if desc, ok := flags["description"].(*cli.StringFlag); ok && strings.Contains(desc.Usage, "objective") { + t.Errorf("description usage still documents the objective fallback: %q", desc.Usage) + } + if chain, ok := flags["chain"].(*cli.StringFlag); ok && chain.Value != "base" { t.Errorf("chain default = %q, want base", chain.Value) } diff --git a/cmd/obol/sell_identity.go b/cmd/obol/sell_identity.go index 81812c92..c8f4b9e1 100644 --- a/cmd/obol/sell_identity.go +++ b/cmd/obol/sell_identity.go @@ -291,6 +291,70 @@ func sellIdentityCommand(cfg *config.Config) *cli.Command { Usage: "Manage the durable ERC-8004 AgentIdentity record", Commands: []*cli.Command{ sellIdentityImportCommand(cfg), + sellIdentityForgetCommand(cfg), + }, + } +} + +func sellIdentityForgetCommand(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "forget", + Usage: "Remove a chain's registration from the AgentIdentity record", + ArgsUsage: "", + Description: `Removes the recorded agentId for from the AgentIdentity CR. + +Use this to correct a registration that was recorded under the wrong +chain (e.g. a wrong-chain agentId written by a bug, or an offer that +switched networks before an on-chain id was verified). This does not +touch the on-chain NFT itself; it only clears the local record so the +controller re-derives the id from scratch on the next reconcile.`, + Action: func(ctx context.Context, cmd *cli.Command) error { + if cmd.NArg() != 1 { + return fmt.Errorf("chain required: obol sell identity forget ") + } + chain := strings.TrimSpace(cmd.Args().First()) + + ns := monetizeapi.AgentIdentityDefaultNamespace + name := monetizeapi.AgentIdentityDefaultName + rec, err := loadAgentIdentity(cfg, ns, name) + if err != nil { + return err + } + if rec == nil { + return fmt.Errorf("AgentIdentity %s/%s not found", ns, name) + } + existing := monetizeapi.AgentIdentityAgentIDForChain(rec.Status, chain) + if existing == "" { + getUI(cmd).Printf("AgentIdentity %s/%s has no registration on %s; nothing to do.", ns, name, chain) + return nil + } + rec.Status = monetizeapi.RemoveAgentIdentityRegistration(rec.Status, chain) + // Patch registrations explicitly rather than via + // patchAgentIdentityStatus: that helper marshals with + // `omitempty`, so dropping the last entry (the common case — + // the poisoned record is usually the only one) would omit + // the field from the JSON merge patch and leave the stale + // array untouched server-side instead of clearing it. + registrations := rec.Status.Registrations + if registrations == nil { + registrations = []monetizeapi.AgentIdentityRegistration{} + } + patch, err := json.Marshal(map[string]any{"status": map[string]any{"registrations": registrations}}) + if err != nil { + return err + } + if err := kubectlRun( + cfg, + "patch", "agentidentities.obol.org", name, + "-n", ns, + "--subresource=status", + "--type=merge", + "-p", string(patch), + ); err != nil { + return err + } + getUI(cmd).Successf("Removed agent %s on %s from AgentIdentity %s/%s.", existing, chain, ns, name) + return nil }, } } diff --git a/cmd/obol/sell_register_x402scan.go b/cmd/obol/sell_register_x402scan.go index 7a823056..e2261725 100644 --- a/cmd/obol/sell_register_x402scan.go +++ b/cmd/obol/sell_register_x402scan.go @@ -7,6 +7,7 @@ import ( "fmt" "net/http" "net/url" + "sort" "strings" "time" @@ -15,6 +16,7 @@ import ( "github.com/ObolNetwork/obol-stack/internal/hermes" "github.com/ObolNetwork/obol-stack/internal/kubectl" "github.com/ObolNetwork/obol-stack/internal/ui" + x402verifier "github.com/ObolNetwork/obol-stack/internal/x402" "github.com/ObolNetwork/obol-stack/internal/x402scan" "github.com/urfave/cli/v3" ) @@ -72,8 +74,10 @@ Examples: // Non-fatal preflight: x402scan discovers services from the // origin's /openapi.json, so a missing/empty doc means the // registration will come back "no discovery" or "no valid - // resources". Warn early with the local fix. - preflightOpenAPI(ctx, u, origin) + // resources". Warn early with the local fix. allTestnet feeds + // the ErrNoValidResources message below with the accurate cause + // when every advertised offer prices on a testnet. + allTestnet := preflightOpenAPI(ctx, u, origin) // Same signer posture as ERC-8004 registration: all signing via // the agent's remote-signer, no key material in the CLI. @@ -114,8 +118,13 @@ Examples: u.Dim(fmt.Sprintf(" Check that %s/openapi.json is publicly reachable and the tunnel is up (obol tunnel status).", origin)) case errors.Is(err, x402scan.ErrNoValidResources): u.Error(err.Error()) - u.Dim(" A discovery document was found, but no endpoint answered with a live x402 402 challenge.") - u.Dim(" Check offers are Ready (obol sell status) and publicly reachable through the tunnel.") + if allTestnet { + u.Dim(" This origin's offers price on a testnet — x402scan indexes mainnet resources only.") + u.Dim(" Switch the payment network to base (mainnet) and re-register.") + } else { + u.Dim(" A discovery document was found, but no endpoint answered with a live x402 402 challenge.") + u.Dim(" Check offers are Ready (obol sell status) and publicly reachable through the tunnel.") + } } return err } @@ -181,33 +190,36 @@ func resolveX402scanOrigin(cfg *config.Config, explicit string) (string, error) // preflightOpenAPI warns (never fails) when the origin's /openapi.json is // unreachable or advertises no operations — the two states that make the -// remote registration a guaranteed no-op. -func preflightOpenAPI(ctx context.Context, u *ui.UI, origin string) { +// remote registration a guaranteed no-op. It returns true when every paid +// operation it found prices exclusively on testnet networks — x402scan +// indexes mainnet resources only, so that's a third guaranteed no-op the +// caller can use to explain an ErrNoValidResources accurately. +func preflightOpenAPI(ctx context.Context, u *ui.UI, origin string) bool { ctx, cancel := context.WithTimeout(ctx, 15*time.Second) defer cancel() req, err := http.NewRequestWithContext(ctx, http.MethodGet, origin+"/openapi.json", nil) if err != nil { - return + return false } resp, err := http.DefaultClient.Do(req) if err != nil { u.Warnf("could not fetch %s/openapi.json (%v) — x402scan discovers services from it; check the tunnel is up", origin, err) - return + return false } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { u.Warnf("%s/openapi.json returned HTTP %d — x402scan discovers services from it; check the tunnel is up", origin, resp.StatusCode) - return + return false } var doc struct { Paths map[string]json.RawMessage `json:"paths"` } if err := json.NewDecoder(resp.Body).Decode(&doc); err != nil { - return + return false } if len(doc.Paths) == 0 { u.Warn("the published /openapi.json advertises no operations — put at least one offer on sale (obol sell inference|http|agent) before registering") - return + return false } // x402scan indexes per ORIGIN — it crawls this one /openapi.json and // lists everything it finds under the origin we submit. On the shared @@ -224,6 +236,60 @@ func preflightOpenAPI(ctx context.Context, u *ui.UI, origin string) { if len(offers) > 1 { u.Warnf("this origin advertises %d offers in one /openapi.json — x402scan indexes per origin, so all of them will be listed together under %s rather than as distinct products. For clean discovery, give each offer its own subdomain (obol tunnel hostname add ) and register that origin.", len(offers), origin) } + + networks := paidNetworksFromOpenAPI(doc.Paths) + if len(networks) == 0 { + return false + } + allTestnet := true + for net := range networks { + if !x402verifier.IsTestnetCAIP2(net) { + allTestnet = false + break + } + } + if allTestnet { + list := make([]string, 0, len(networks)) + for net := range networks { + list = append(list, net) + } + sort.Strings(list) + u.Warnf("this origin's offers price on %s (testnet) — x402scan indexes mainnet resources only; switch payment network to base (mainnet) and re-register", strings.Join(list, ", ")) + } + return allTestnet +} + +// paidNetworksFromOpenAPI collects the distinct CAIP-2 network ids +// advertised across every paid operation's `x-payment-info.accepts[]` +// (published by internal/serviceoffercontroller/openapi.go's +// paymentInfoAccept). Operations without x-payment-info (free/auth-gated +// routes) contribute nothing. +func paidNetworksFromOpenAPI(paths map[string]json.RawMessage) map[string]struct{} { + networks := map[string]struct{}{} + for _, pathRaw := range paths { + var methods map[string]json.RawMessage + if err := json.Unmarshal(pathRaw, &methods); err != nil { + continue + } + for _, opRaw := range methods { + var op struct { + PaymentInfo *struct { + Accepts []struct { + Network string `json:"network"` + } `json:"accepts"` + } `json:"x-payment-info"` + } + if err := json.Unmarshal(opRaw, &op); err != nil || op.PaymentInfo == nil { + continue + } + for _, accept := range op.PaymentInfo.Accepts { + if accept.Network != "" { + networks[accept.Network] = struct{}{} + } + } + } + } + return networks } // servicesOfferName extracts the offer name from a shared-origin OpenAPI path diff --git a/cmd/obol/sell_register_x402scan_preflight_test.go b/cmd/obol/sell_register_x402scan_preflight_test.go index 44a2b576..a9aff0b6 100644 --- a/cmd/obol/sell_register_x402scan_preflight_test.go +++ b/cmd/obol/sell_register_x402scan_preflight_test.go @@ -51,6 +51,55 @@ func TestPreflightOpenAPI_MultiOfferSharedOrigin(t *testing.T) { } } +func TestPreflightOpenAPI_AllTestnet(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"paths":{ + "/services/foo/v1/chat/completions": { + "post": {"x-payment-info": {"accepts": [{"network": "eip155:84532"}]}} + } + }}`)) + })) + defer srv.Close() + + var stderr bytes.Buffer + u := ui.NewForTest(&bytes.Buffer{}, &stderr) + allTestnet := preflightOpenAPI(context.Background(), u, srv.URL) + + if !allTestnet { + t.Fatal("expected allTestnet=true for an origin whose only accepted network is base-sepolia") + } + if got := stderr.String(); !strings.Contains(got, "eip155:84532 (testnet)") { + t.Fatalf("expected testnet warning, got: %q", got) + } +} + +func TestPreflightOpenAPI_MixedMainnetAndTestnetIsNotAllTestnet(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"paths":{ + "/services/foo/v1/chat/completions": { + "post": {"x-payment-info": {"accepts": [ + {"network": "eip155:84532"}, + {"network": "eip155:8453"} + ]}} + } + }}`)) + })) + defer srv.Close() + + var stderr bytes.Buffer + u := ui.NewForTest(&bytes.Buffer{}, &stderr) + allTestnet := preflightOpenAPI(context.Background(), u, srv.URL) + + if allTestnet { + t.Fatal("expected allTestnet=false when a mainnet network is also accepted") + } + if got := stderr.String(); strings.Contains(got, "testnet") { + t.Fatalf("did not expect a testnet warning when base mainnet is accepted, got: %q", got) + } +} + func TestPreflightOpenAPI_UnreachableOrNon200(t *testing.T) { // Non-200: server up but returns an error status. srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { diff --git a/cmd/obol/sell_test.go b/cmd/obol/sell_test.go index af522b20..40e5efeb 100644 --- a/cmd/obol/sell_test.go +++ b/cmd/obol/sell_test.go @@ -777,7 +777,9 @@ func TestSellRegister_Flags(t *testing.T) { requireFlags(t, flags, "chain", - "endpoint", "name", "description", "image", + // "endpoint" is kept as a deprecated alias of "origin" so existing + // scripts (e.g. flows/flow-14-live-obol-base-sepolia.sh) keep working. + "origin", "endpoint", "name", "description", "image", // --sponsored is intentionally retained as a deprecated flag that // errors with a clear message; users with old muscle memory or stale // docs need a louder signal than "unknown flag". @@ -789,6 +791,109 @@ func TestSellRegister_Flags(t *testing.T) { assertStringDefault(t, flags, "description", "Obol Stack AI agent with x402 payment-gated services") } +func TestNormalizeAgentOrigin(t *testing.T) { + for _, tc := range []struct { + in string + want string + errPart string + }{ + {in: "https://store.example.com", want: "https://store.example.com"}, + {in: "https://store.example.com/", want: "https://store.example.com"}, + {in: "https://abc-def.trycloudflare.com", want: "https://abc-def.trycloudflare.com"}, + {in: "https://store.example.com/services/myagent", errPart: "no path"}, + {in: "https://store.example.com/.well-known/agent-registration.json", errPart: "no path"}, + {in: "not a url", errPart: "no path"}, + {in: "", errPart: "no path"}, + // Scheme-less and non-HTTP origins must be rejected, not silently + // accepted — a bare "//host" or "ftp://host" would mint a malformed, + // unfetchable on-chain agentURI. + {in: "//store.example.com", errPart: "http or https"}, + {in: "ftp://store.example.com", errPart: "http or https"}, + } { + got, err := normalizeAgentOrigin(tc.in) + if tc.errPart != "" { + if err == nil || !strings.Contains(err.Error(), tc.errPart) { + t.Fatalf("%q: expected error containing %q, got %v", tc.in, tc.errPart, err) + } + continue + } + if err != nil { + t.Fatalf("%q: %v", tc.in, err) + } + if got != tc.want { + t.Fatalf("%q: got %q, want %q", tc.in, got, tc.want) + } + } +} + +// TestOfferEligibleForAutoEnable guards enableRegistrationOnOffers' chain +// filter: `obol sell register --network base` must not flip on an offer +// pinned to a different, never-registered network (84dcbf83 originally +// enabled every disabled offer cluster-wide with no chain check at all). +func TestOfferEligibleForAutoEnable(t *testing.T) { + registered := map[string]bool{"base": true, "base-sepolia": true} + + offer := func(network string, enabled bool) monetizeapi.ServiceOffer { + return monetizeapi.ServiceOffer{ + Spec: monetizeapi.ServiceOfferSpec{ + Payment: monetizeapi.ServiceOfferPayment{Network: network}, + Registration: monetizeapi.ServiceOfferRegistration{Enabled: enabled}, + }, + } + } + + optedOut := offer("base", false) + optedOut.Annotations = map[string]string{autoRegisterOptOutAnnotation: "true"} + + tests := []struct { + name string + o monetizeapi.ServiceOffer + want bool + }{ + {"disabled offer on a just-registered network", offer("base", false), true}, + {"disabled offer on a different, unregistered network", offer("ethereum", false), false}, + {"already-enabled offer on a just-registered network", offer("base", true), false}, + {"disabled offer on an unregistered network stays disabled", offer("polygon", false), false}, + {"opt-out annotation exempts an otherwise-eligible offer", optedOut, false}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + if got := offerEligibleForAutoEnable(tc.o, registered); got != tc.want { + t.Errorf("offerEligibleForAutoEnable(network=%s, enabled=%v) = %v, want %v", + tc.o.Spec.Payment.Network, tc.o.Spec.Registration.Enabled, got, tc.want) + } + }) + } +} + +// TestDefaultHealthPathForUpstream pins the --upstream-specific health-path +// defaults: ollama serves 2xx only at "/" (neither /health nor +// /health/readiness), so under the 2xx-only UpstreamHealthy gate the plain +// "/health" default would leave the offer permanently un-Ready. +func TestDefaultHealthPathForUpstream(t *testing.T) { + tests := []struct { + name string + upstream string + healthPath string + want string + }{ + {"ollama gets / when health-path unset", "ollama", "", "/"}, + {"ollama gets / when health-path left at the flag default", "ollama", "/health", "/"}, + {"ollama case-insensitive", "Ollama", "/health", "/"}, + {"ollama explicit override is respected", "ollama", "/custom-health", "/custom-health"}, + {"litellm still defaults to /health/readiness", "litellm", "/health", "/health/readiness"}, + {"other upstream keeps the flag default", "my-svc", "/health", "/health"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + if got := defaultHealthPathForUpstream(tc.upstream, tc.healthPath); got != tc.want { + t.Errorf("defaultHealthPathForUpstream(%q, %q) = %q, want %q", + tc.upstream, tc.healthPath, got, tc.want) + } + }) + } +} + func TestSellPricing_Flags(t *testing.T) { cfg := newTestConfig(t) cmd := sellCommand(cfg) @@ -865,6 +970,16 @@ func TestIsTransientRegistrationError(t *testing.T) { {name: "rpc 500", err: errors.New("erc8004: register tx: 500 Internal Server Error"), want: true}, {name: "timeout", err: errors.New("context deadline exceeded while waiting for headers"), want: true}, {name: "revert", err: errors.New("erc8004: register tx: execution reverted"), want: false}, + // registerWithRecovery pins and reuses the same nonce across a + // broadcast-timeout retry, so these three responses mean no new tx + // was accepted (no double-mint risk) — they must be transient so the + // retry loop keeps giving recoverRegistrationByOwnerAndURI time to + // see the original broadcast land, instead of reporting an + // already-landed mint as failed. + {name: "already known", err: errors.New("already known"), want: true}, + {name: "nonce too low", err: errors.New("nonce too low"), want: true}, + {name: "replacement underpriced", err: errors.New("replacement transaction underpriced"), want: true}, + {name: "nonce error uppercase variant", err: errors.New("Nonce Too Low"), want: true}, } for _, tt := range tests { @@ -2623,3 +2738,66 @@ func TestOfferPathCollisionInList_Hostname(t *testing.T) { t.Fatalf("self-update must pass: %v", err) } } + +// TestResolvePriceTable_RejectsMalformedPrice is the regression guard for +// the Canary402 finding: an EU-style comma decimal ("0,01") or other +// malformed --price value used to sail through resolvePriceTable with zero +// validation, silently mispricing the offer at $0 (decimalToAtomic parses +// "0" and drops the rest) or panicking the verifier hot path on every +// request (nil *big.Float from a discarded Parse error). --per-mtok/--per-hour +// already validate via ApproximateRequestPriceFrom*; --price/--per-request +// must too. +func TestResolvePriceTable_RejectsMalformedPrice(t *testing.T) { + badPrices := []string{"0,01", "abc", "$0.01", "-1", ""} + + for _, price := range badPrices { + t.Run(price, func(t *testing.T) { + var resolveErr error + cmd := &cli.Command{ + Name: "x", + Flags: []cli.Flag{ + &cli.StringFlag{Name: "price"}, + &cli.StringFlag{Name: "per-request"}, + &cli.StringFlag{Name: "per-mtok"}, + &cli.StringFlag{Name: "per-hour"}, + }, + Action: func(_ context.Context, c *cli.Command) error { + _, resolveErr = resolvePriceTable(c, true) + return nil + }, + } + if err := cmd.Run(context.Background(), []string{"x", "--price", price}); err != nil { + t.Fatalf("cmd.Run: %v", err) + } + if resolveErr == nil { + t.Fatalf("resolvePriceTable(--price=%q) expected error, got nil", price) + } + }) + } + + // A well-formed price must still pass. + var resolveErr error + var got schemas.PriceTable + cmd := &cli.Command{ + Name: "x", + Flags: []cli.Flag{ + &cli.StringFlag{Name: "price"}, + &cli.StringFlag{Name: "per-request"}, + &cli.StringFlag{Name: "per-mtok"}, + &cli.StringFlag{Name: "per-hour"}, + }, + Action: func(_ context.Context, c *cli.Command) error { + got, resolveErr = resolvePriceTable(c, true) + return nil + }, + } + if err := cmd.Run(context.Background(), []string{"x", "--price", "0.01"}); err != nil { + t.Fatalf("cmd.Run: %v", err) + } + if resolveErr != nil { + t.Fatalf("resolvePriceTable(--price=0.01): unexpected error %v", resolveErr) + } + if got.PerRequest != "0.01" { + t.Fatalf("PerRequest = %q, want 0.01", got.PerRequest) + } +} diff --git a/cmd/obol/stackup_resume_guard_test.go b/cmd/obol/stackup_resume_guard_test.go index e37d3f9a..c089f947 100644 --- a/cmd/obol/stackup_resume_guard_test.go +++ b/cmd/obol/stackup_resume_guard_test.go @@ -21,6 +21,7 @@ func TestStackUpAction_ReplaysRecordedState(t *testing.T) { upIdx := strings.Index(body, "stack.Up(cfg") rpcIdx := strings.Index(body, "network.ReconcileRecordedRPCs(") + overlayIdx := strings.Index(body, "network.ReconcileERPCOverlay(") agentsIdx := strings.Index(body, "agentcrd.ResumeAll(") appsIdx := strings.Index(body, "app.ResumeAll(") offersIdx := strings.Index(body, "resumeSellOffers(") @@ -28,6 +29,9 @@ func TestStackUpAction_ReplaysRecordedState(t *testing.T) { if rpcIdx < 0 { t.Fatal("cmd/obol/main.go must call network.ReconcileRecordedRPCs — without it recorded remote RPCs never reach a freshly-recreated cluster") } + if overlayIdx < 0 { + t.Fatal("cmd/obol/main.go must call network.ReconcileERPCOverlay — without it durable eRPC baskets (e.g. HyperEVM) never re-apply after stack up (#763)") + } if agentsIdx < 0 { t.Fatal("cmd/obol/main.go must call agentcrd.ResumeAll — without it recorded Agent CRs never reach a freshly-recreated cluster") } @@ -37,9 +41,12 @@ func TestStackUpAction_ReplaysRecordedState(t *testing.T) { if upIdx < 0 || offersIdx < 0 { t.Fatalf("expected stack.Up and resumeSellOffers in main.go; upIdx=%d offersIdx=%d", upIdx, offersIdx) } - if rpcIdx < upIdx || agentsIdx < upIdx || appsIdx < upIdx { + if rpcIdx < upIdx || overlayIdx < upIdx || agentsIdx < upIdx || appsIdx < upIdx { t.Error("recorded-state replay must run AFTER stack.Up — before it there is no kubeconfig/cluster") } + if overlayIdx < rpcIdx { + t.Error("ReconcileERPCOverlay must run AFTER ReconcileRecordedRPCs — overlay merges onto base+recorded remotes") + } if agentsIdx > offersIdx { t.Error("agentcrd.ResumeAll must run BEFORE resumeSellOffers — agent-backed ServiceOffers need their Agent CR first") } diff --git a/docs/getting-started.md b/docs/getting-started.md index 8823f3eb..2fde75ae 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -11,10 +11,19 @@ This guide walks you through installing the Obol Stack, starting a local Kuberne - **Docker** -- The stack runs a local Kubernetes cluster via [k3d](https://k3d.io), which requires Docker. - Linux: [Docker Engine](https://docs.docker.com/engine/install/) - macOS / Windows: [Docker Desktop](https://docs.docker.com/desktop/) -- **Ollama** -- For LLM inference. Install from [ollama.com](https://ollama.com) and start with `ollama serve`. +- **LLM access** -- Either: + - **Ollama** (local) — install from [ollama.com](https://ollama.com), start `ollama serve`, pull a chat model (e.g. `ollama pull qwen3.5:4b`), **or** + - **Cloud / custom** — after `stack up`, run `obol model setup` (OpenRouter, Anthropic, OpenAI, Venice, custom vLLM, …). - **Foundry** (optional) -- For on-chain payment testing. Install from [getfoundry.sh](https://getfoundry.sh). - **Go 1.25+** (development mode only) -- For building from source. +> [!IMPORTANT] +> Declining the installer’s Ollama prompt **without** later configuring a model means **no LiteLLM models**, so `stack up` **skips the default Hermes agent**. Fix with: +> ```bash +> obol model setup +> obol agent init +> ``` + ## Install Run the bootstrap installer: @@ -23,7 +32,26 @@ Run the bootstrap installer: bash <(curl -fsSL https://stack.obol.org) ``` -This installs the `obol` CLI and all required tools (kubectl, helm, k3d, helmfile, k9s) to `~/.local/bin/`. +Pin a current release (see [GitHub releases](https://github.com/ObolNetwork/obol-stack/releases)): + +```bash +OBOL_RELEASE=v0.13.0 bash <(curl -fsSL https://stack.obol.org) +``` + +This installs the `obol` CLI and all required tools (kubectl, helm, k3d, helmfile, k9s) to `~/.local/bin/`, and tries to add `127.0.0.1 obol.stack` to `/etc/hosts`. + +### If `/etc/hosts` cannot be updated + +The installer prints the manual hosts line and still completes. Resume: + +```bash +echo "127.0.0.1 obol.stack" | sudo tee -a /etc/hosts +obol stack init +obol stack up +obol agent init # if Hermes was skipped (no model configured yet) +``` + +`obol stack up` will attempt hosts again (including agent hostnames). A failed write is a **warning** — the cluster still starts. For automation without a sudo prompt, set `OBOL_NONINTERACTIVE=true` (hosts update fails fast unless sudo is already cached / NOPASSWD). > [!TIP] > **Development mode** -- Contributors working from source can use: @@ -42,15 +70,15 @@ obol stack up `stack init` generates a unique stack ID (e.g., `vast-flounder`) and writes cluster configuration to `~/.config/obol/`. -`stack up` creates a local k3d cluster, deploys all infrastructure, and sets up a default AI agent with an Ethereum wallet. +`stack up` creates a local k3d cluster, deploys infrastructure, configures **LiteLLM**, and (when a model is available) deploys the default **Hermes** agent with an Ethereum wallet. The **Cloudflare tunnel stays dormant** until you sell something (`obol sell …`) or run `obol tunnel restart` / `obol tunnel setup`. On first run, `stack up` will: 1. Create the k3d cluster -2. Deploy infrastructure (Traefik, monitoring, LLM gateway, etc.) -3. Build and import the x402-verifier image (development mode only) -4. Deploy a default Hermes agent instance with embedded Obol skills +2. Deploy infrastructure (Traefik, monitoring, LiteLLM gateway, etc.) +3. Build and import local-dev images (development mode only) +4. Deploy a default Hermes agent with embedded Obol skills **if** LiteLLM has a model 5. Generate an Ethereum signing wallet for the agent -6. Import runtime state for the stack-managed agent +6. Leave the public tunnel dormant (or restore a previously configured permanent tunnel) ## Step 2 -- Verify the Cluster @@ -63,21 +91,42 @@ All pods should show `Running` or `Completed` within ~2 minutes: | Component | Namespace | Description | |-----------|-----------|-------------| | **Traefik** | `traefik` | Gateway API ingress controller | -| **Cloudflared** | `traefik` | Quick tunnel for public access | -| **LiteLLM** | `llm` | OpenAI-compatible LLM gateway (proxies to host Ollama) | +| **Cloudflared** | `traefik` | Tunnel connector chart (often 0 replicas until first sell / `tunnel restart`) | +| **LiteLLM** | `llm` | OpenAI-compatible LLM gateway (Ollama, cloud providers, custom endpoints) | | **eRPC** | `erpc` | Unified RPC load balancer | -| **Frontend** | `obol-frontend` | Web interface at http://obol.stack/ | +| **Frontend** | `obol-frontend` | Web UI — local only, `Host: obol.stack` | | **Monitoring** | `monitoring` | Prometheus + kube-prometheus-stack | | **Reloader** | `reloader` | Auto-restarts workloads on config changes | | **x402 Gateway** | `x402` | Shared seller-owned payment gateway for priced HTTP routes | -| **Hermes** | `hermes-obol-agent` | Default AI agent with Ethereum wallet | +| **Hermes** | `hermes-obol-agent` | Default AI agent with Ethereum wallet (if a model was configured) | | **Remote Signer** | `hermes-obol-agent` | Ethereum transaction signing service | -Open the frontend: http://obol.stack/ +### Local URLs (important) + +Open the frontend at: + +```text +http://obol.stack:8080 +``` + +Use the **`obol.stack` hostname**, not `localhost`. Traefik matches `Host: obol.stack` for the frontend and eRPC. **`http://localhost:8080` returns 404** even when the stack is healthy. + +- On many Mac setups port **8080** is the reliable mapping; port 80 may need root or be disabled in `k3d.yaml`. +- Hermes dashboard (separate host): `http://obol-agent.obol.stack` (add `:8080` if that is your ingress). Login username `obol`, password from `obol agent auth obol-agent`. + +The Hermes agent dashboard is separate from the frontend. Open the pretty host +(edge-redirects to the password form — see +[Hermes dashboard login](guides/hermes-dashboard-login.md)): + +``` +http://obol-agent.obol.stack +``` + +Username `obol`, password from `obol agent auth obol-agent`. ## Step 3 -- Test LLM Inference -The stack routes all LLM requests through LiteLLM, an OpenAI-compatible gateway that forwards to your host Ollama. +The stack routes all LLM requests through LiteLLM, an OpenAI-compatible gateway (host Ollama when present, or cloud / custom endpoints from `obol model setup`). ### 3a. Verify Ollama has models diff --git a/docs/guides/hermes-dashboard-login.md b/docs/guides/hermes-dashboard-login.md new file mode 100644 index 00000000..c788e65b --- /dev/null +++ b/docs/guides/hermes-dashboard-login.md @@ -0,0 +1,56 @@ +# Hermes dashboard login (basic-auth) + +Local Hermes dashboards are gated with password basic-auth after hermes-agent +v2026.7.x. + +## Working URL (pretty host) + +``` +http://obol-agent.obol.stack +``` + +Named instances use `http://hermes--ui.obol.stack`. + +Obol’s dashboard `HTTPRoute` **edge-redirects** Exact `/` → `/auth/password-login` +(302) so you never have to type the form path. The agent API host +(`hermes-obol-agent.obol.stack`) is unchanged. + +| Field | Value | +|-------|--------| +| Username | `obol` | +| Password | Agent API token from `obol agent auth ` | + +The password is the same secret Obol wires as `HERMES_DASHBOARD_BASIC_AUTH_PASSWORD` +(and `API_SERVER_KEY`). It never leaves your machine (`obol.stack` → loopback). + +CLI alternative (no browser): `obol hermes chat`. + +## Why the redirect exists + +Hermes’s own root handling is broken for password-only basic-auth: + +``` +GET / (if proxied straight to Hermes) + → Hermes redirects to /auth/login?provider=basic + → NotImplementedError: BasicAuthProvider is password-only + → HTTP 500 +``` + +The real form lives at `/auth/password-login`. Obol intercepts Exact `/` on the +**dashboard hostname only** and issues: + +``` +302 Location: /auth/password-login +``` + +So operators open the pretty host; Traefik never forwards bare `/` into Hermes’s +broken auto-login path. + +| Layer | Ownership | +|-------|-----------| +| Upstream Hermes (`nousresearch/hermes-agent`) | Ideally fix `/` (and login CTAs) to land on `/auth/password-login` when basic-auth is password-only | +| Obol Stack | Edge-redirect dashboard `/` → password form; print pretty URL + credentials after install/sync; smoke-test root + password-login non-500 in flow-04 | + +Fallback if the edge rule is missing (old install before re-sync): open +`http://obol-agent.obol.stack/auth/password-login` directly, then +`obol agent sync` / re-install to pick up the redirect. diff --git a/docs/guides/monetize-inference.md b/docs/guides/monetize-inference.md index 70b794bf..7cc6a85d 100644 --- a/docs/guides/monetize-inference.md +++ b/docs/guides/monetize-inference.md @@ -156,6 +156,12 @@ obol sell http my-qwen \ --port 11434 ``` +Ollama doesn't serve `/health` or `/health/readiness` — only `/` returns +2xx — so `obol sell http` automatically defaults `--health-path` to `/` when +`--upstream ollama` is left at its default `/health`. No extra flag is +needed for the example above; pass `--health-path` yourself only if you're +fronting a different health endpoint. + By default this also registers the seller agent on ERC-8004. Use `--no-register` only for local or private-only testing where on-chain discovery is intentionally skipped. diff --git a/examples/erpc-overlay-hyperevm.yaml b/examples/erpc-overlay-hyperevm.yaml new file mode 100644 index 00000000..9e7c9789 --- /dev/null +++ b/examples/erpc-overlay-hyperevm.yaml @@ -0,0 +1,103 @@ +# Example eRPC operator overlay — HyperEVM (chain 999) basket. +# +# Durable operator eRPC config so local baskets survive `obol stack up`. Set with: +# +# obol network erpc set -f examples/erpc-overlay-hyperevm.yaml +# obol network erpc status +# +# Host copy lives at $OBOL_CONFIG_DIR/rpc/erpc-overlay.yaml (0600) and is +# re-merged after recorded remotes on every stack up (see #763). +# +# Edit the local endpoint for your co-located hl-node before applying. + +version: 1 + +networks: + - alias: hyperevm + architecture: evm + evm: + chainId: 999 + failsafe: + timeout: + duration: 10s + retry: + maxAttempts: 3 + delay: 100ms + hedge: + delay: 150ms + maxCount: 1 + +upstreams: + # Prefer a co-located non-validator HyperEVM RPC when present. + - id: local-hl-node + endpoint: http://192.168.50.21:3001/evm + evm: + chainId: 999 + routing: + scoreMultipliers: + - network: "*" + method: "*" + overall: 5.0 + + - id: hyperevm-official + endpoint: https://rpc.hyperliquid.xyz/evm + evm: + chainId: 999 + tags: ["tier:fallback"] + rateLimitBudget: hyperevm-official + routing: + scoreMultipliers: + - network: "*" + method: "*" + overall: 0.3 + + - id: hyperevm-drpc + endpoint: https://hyperliquid.drpc.org + evm: + chainId: 999 + tags: ["tier:fallback"] + rateLimitBudget: hyperevm-drpc + routing: + scoreMultipliers: + - network: "*" + method: "*" + overall: 0.3 + + - id: hyperevm-onfinality + endpoint: https://hyperliquid.api.onfinality.io/evm/public + evm: + chainId: 999 + tags: ["tier:fallback"] + rateLimitBudget: hyperevm-onfinality + routing: + scoreMultipliers: + - network: "*" + method: "*" + overall: 0.3 + +rateLimiters: + budgets: + - id: hyperevm-official + rules: + - method: "*" + maxCount: 100 + period: second + - id: hyperevm-drpc + rules: + - method: "*" + maxCount: 30 + period: second + - id: hyperevm-onfinality + rules: + - method: "*" + maxCount: 25 + period: second + +# eth_call @ latest (HyperCore precompiles) needs realtime caching; the chart +# default finality:unfinalized can serve stale precompile reads under burst. +cachePoliciesAdd: + - network: "*" + method: eth_call + finality: realtime + connector: memory-cache + ttl: 2s diff --git a/flows/flow-04-agent.sh b/flows/flow-04-agent.sh index aa72067b..70b0406d 100755 --- a/flows/flow-04-agent.sh +++ b/flows/flow-04-agent.sh @@ -231,14 +231,28 @@ fi # stack configures basic-auth. Assert the dashboard container is UP (public # /api/status → 200) and ENFORCES auth (protected /api/sessions → 401 for an # unauthenticated caller; /api routes gate on the session token, not basic-auth). +# +# Also probe /auth/password-login (working form) and Exact "/" on the +# dashboard host. Obol edge-redirects "/" → /auth/password-login so operators +# can open the pretty host; Hermes's own /auth/login?provider=basic still 500s. +# See docs/guides/hermes-dashboard-login.md. dash_code() { curl --resolve "${HERMES_DASHBOARD_HOST}:${ingress_port}:127.0.0.1" \ -s -o /dev/null -w "%{http_code}" --max-time 10 "$@" 2>/dev/null; } -dash_status="" dash_protected="" +# -L follows redirect once so we can assert the pretty root lands on a non-500 page. +dash_code_follow() { curl --resolve "${HERMES_DASHBOARD_HOST}:${ingress_port}:127.0.0.1" \ + -s -o /dev/null -w "%{http_code}" --max-time 10 -L --max-redirs 3 "$@" 2>/dev/null; } +dash_status="" dash_protected="" dash_login="" dash_root="" dash_root_final="" for i in $(seq 1 15); do dash_status=$(dash_code "$HERMES_DASHBOARD_URL/api/status") dash_protected=$(dash_code "$HERMES_DASHBOARD_URL/api/sessions") - if [ "$dash_status" = "200" ] && [ "$dash_protected" = "401" ]; then - pass "Hermes dashboard up + auth-gated (status=$dash_status protected=$dash_protected)" + dash_login=$(dash_code "$HERMES_DASHBOARD_URL/auth/password-login") + dash_root=$(dash_code "$HERMES_DASHBOARD_URL/") + dash_root_final=$(dash_code_follow "$HERMES_DASHBOARD_URL/") + if [ "$dash_status" = "200" ] && [ "$dash_protected" = "401" ] && \ + { [ "$dash_login" = "200" ] || [ "$dash_login" = "401" ]; } && \ + [ "$dash_root" = "302" ] && \ + [ "$dash_root_final" != "500" ] && [ -n "$dash_root_final" ] && [ "$dash_root_final" != "000" ]; then + pass "Hermes dashboard up + auth-gated + root/login ok (status=$dash_status protected=$dash_protected login=$dash_login root=$dash_root final=$dash_root_final)" break fi sleep 2 @@ -246,6 +260,15 @@ done if [ "$dash_status" != "200" ] || [ "$dash_protected" != "401" ]; then fail "Hermes dashboard check failed — status=$dash_status protected=$dash_protected" fi +if [ "$dash_login" != "200" ] && [ "$dash_login" != "401" ]; then + fail "Hermes dashboard password-login path failed — expected HTTP 200 or 401, got HTTP $dash_login" +fi +if [ "$dash_root" != "302" ]; then + fail "Hermes dashboard root failed — expected edge 302 redirect to password-login, got HTTP $dash_root" +fi +if [ -z "$dash_root_final" ] || [ "$dash_root_final" = "000" ] || [ "$dash_root_final" = "500" ]; then + fail "Hermes dashboard root follow failed — expected non-500 after redirect, got HTTP $dash_root_final" +fi # §4: Verify Hermes config still has the expected model/provider wiring. oc_config=$("$OBOL" kubectl get cm hermes-config -n hermes-obol-agent \ diff --git a/flows/flow-06-sell-setup.sh b/flows/flow-06-sell-setup.sh index e9ab454d..5e662c6b 100755 --- a/flows/flow-06-sell-setup.sh +++ b/flows/flow-06-sell-setup.sh @@ -15,6 +15,16 @@ else SELL_MODEL_RUNTIME="${SELL_MODEL_RUNTIME:-ollama}" fi +# LiteLLM: unauthenticated readiness probe. /health requires the master key +# (401) and fails UpstreamHealthy under the 2xx-only probe gate. Ollama +# serves neither /health nor /health/readiness — only "/" returns 2xx — +# matching sell.go's --upstream ollama default. +if [ "$SELL_UPSTREAM_SERVICE" = "ollama" ]; then + SELL_HEALTH_PATH="${SELL_HEALTH_PATH:-/}" +else + SELL_HEALTH_PATH="${SELL_HEALTH_PATH:-/health/readiness}" +fi + apply_flow_qwen_inference_offer() { "$OBOL" sell http flow-qwen --namespace llm --from-json - <-", and for +// Hermes the "-ui" DashboardHostname suffix) within the 63-character DNS +// label limit. validate.Name alone allows ids up to 63 chars, which is too +// permissive here since Onboard prepends a runtime prefix before the id +// ever reaches a Kubernetes object. +func MaxIDLength(runtime Runtime) int { + reserved := len(string(runtime)) + 1 // "-" + if runtime == Hermes { + reserved += len("-ui") // DashboardHostname's "hermes--ui" label + } + return dnsLabelMaxLen - reserved +} + func Hostnames(runtime Runtime, id string) []string { if strings.TrimSpace(id) == "" { return nil diff --git a/internal/agentruntime/runtime_test.go b/internal/agentruntime/runtime_test.go index 2bb53778..61a8f140 100644 --- a/internal/agentruntime/runtime_test.go +++ b/internal/agentruntime/runtime_test.go @@ -3,11 +3,42 @@ package agentruntime import ( "os" "path/filepath" + "strings" "testing" "github.com/ObolNetwork/obol-stack/internal/config" ) +// TestMaxIDLengthKeepsDerivedLabelsWithinDNSLimit guards the DNS-label +// overflow finding: an id at MaxIDLength must keep every label this package +// derives (Namespace/Hostname, and for Hermes DashboardHostname's "-ui" +// suffix) at or under the 63-character RFC 1123 limit. +func TestMaxIDLengthKeepsDerivedLabelsWithinDNSLimit(t *testing.T) { + for _, rt := range []Runtime{OpenClaw, Hermes} { + id := strings.Repeat("a", MaxIDLength(rt)) + + if n := len(Namespace(rt, id)); n > 63 { + t.Errorf("%s: Namespace(%q) label is %d chars, want <=63", rt, id, n) + } + if n := len(strings.SplitN(Hostname(rt, id), ".", 2)[0]); n > 63 { + t.Errorf("%s: Hostname(%q) label is %d chars, want <=63", rt, id, n) + } + if n := len(strings.SplitN(DashboardHostname(rt, id), ".", 2)[0]); n > 63 { + t.Errorf("%s: DashboardHostname(%q) label is %d chars, want <=63", rt, id, n) + } + + // One character longer must overflow at least one derived label — + // otherwise MaxIDLength is too conservative, not just safe. + tooLong := id + "a" + overflow := len(Namespace(rt, tooLong)) > 63 || + len(strings.SplitN(Hostname(rt, tooLong), ".", 2)[0]) > 63 || + len(strings.SplitN(DashboardHostname(rt, tooLong), ".", 2)[0]) > 63 + if !overflow { + t.Errorf("%s: MaxIDLength+1 (%d chars) did not overflow any derived DNS label", rt, len(tooLong)) + } + } +} + func TestHermesPaths(t *testing.T) { cfg := &config.Config{ ConfigDir: "/tmp/obol-config", diff --git a/internal/dns/resolver.go b/internal/dns/resolver.go index e13a3aa6..f57acc3a 100644 --- a/internal/dns/resolver.go +++ b/internal/dns/resolver.go @@ -18,6 +18,7 @@ import ( "os" "os/exec" "path/filepath" + "regexp" "runtime" "strings" "time" @@ -120,6 +121,19 @@ const ( hostsFile = "/etc/hosts" ) +// validHostnameRegex matches DNS-safe hostnames: labels of lowercase/uppercase +// alphanumerics and hyphens, joined by dots. No spaces, newlines, or other +// characters that could inject extra lines into /etc/hosts. +var validHostnameRegex = regexp.MustCompile(`^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*$`) + +// isValidHostname reports whether h is safe to write as an /etc/hosts entry. +// Belt-and-suspenders check: callers are expected to validate ids before +// they ever become a hostname, but this guards the file write regardless of +// how a malformed hostname got here. +func isValidHostname(h string) bool { + return validHostnameRegex.MatchString(h) +} + // EnsureHostsEntries adds /etc/hosts entries for the given hostnames. // Always includes "obol.stack" plus any additional hostnames (e.g. openclaw subdomains). // Entries are idempotent — existing managed block is replaced. @@ -129,7 +143,7 @@ func EnsureHostsEntries(hostnames []string) error { seen := map[string]bool{domain: true} for _, h := range hostnames { - if h != "" && !seen[h] { + if h != "" && !seen[h] && isValidHostname(h) { all = append(all, h) seen[h] = true } diff --git a/internal/dns/resolver_test.go b/internal/dns/resolver_test.go index 7c0024b2..16377d08 100644 --- a/internal/dns/resolver_test.go +++ b/internal/dns/resolver_test.go @@ -40,6 +40,30 @@ func TestGetNMDNSMode(t *testing.T) { _ = mode } +func TestIsValidHostname(t *testing.T) { + valid := []string{"obol.stack", "hermes-abc.obol.stack", "openclaw-my-agent.obol.stack", "a"} + for _, h := range valid { + if !isValidHostname(h) { + t.Errorf("isValidHostname(%q) = false, want true", h) + } + } + + // Belt-and-suspenders guard for the Canary402 audit finding: a hostname + // carrying a newline (e.g. from an unsanitized agent --id) must never be + // written to /etc/hosts, even if it slipped past upstream validation. + invalid := []string{ + "", + "evil\n127.0.0.1 attacker.com", + "has space", + "has/slash", + } + for _, h := range invalid { + if isValidHostname(h) { + t.Errorf("isValidHostname(%q) = true, want false", h) + } + } +} + func TestHasNMDnsmasqConfig(t *testing.T) { // On a clean system without obol config, this should return false // unless the test system has it installed diff --git a/internal/erc8004/client.go b/internal/erc8004/client.go index 5c99c55e..455b47fb 100644 --- a/internal/erc8004/client.go +++ b/internal/erc8004/client.go @@ -75,6 +75,16 @@ func (c *Client) ChainID() *big.Int { return new(big.Int).Set(c.chainID) } +// PendingNonceAt returns the account's next usable nonce. Callers issuing a +// sequence of transactions in one run should fetch this once and pin it into +// TransactOpts.Nonce, bumping it locally after each successful send — the +// eRPC gateway fans requests out to independent upstreams per call, so +// re-querying the pending nonce between sequential sends can hit a lagging +// upstream and return a stale (too-low) value. +func (c *Client) PendingNonceAt(ctx context.Context, addr common.Address) (uint64, error) { + return c.eth.PendingNonceAt(ctx, addr) +} + // RegisterWithOptsDetailed mints a new agent NFT using the provided // TransactOpts and returns both the minted agentId and transaction hash. func (c *Client) RegisterWithOptsDetailed(ctx context.Context, opts *bind.TransactOpts, agentURI string) (*big.Int, string, error) { diff --git a/internal/erc8004/client_test.go b/internal/erc8004/client_test.go index 44908bdf..f76c436c 100644 --- a/internal/erc8004/client_test.go +++ b/internal/erc8004/client_test.go @@ -17,6 +17,7 @@ import ( "github.com/ethereum/go-ethereum/accounts/abi" "github.com/ethereum/go-ethereum/accounts/abi/bind" "github.com/ethereum/go-ethereum/common" + "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/crypto" ) @@ -848,3 +849,133 @@ func TestWaitForAgent_TimeoutReturnsError(t *testing.T) { t.Errorf("expected 'timed out' in error, got: %v", err) } } + +// decodeSentNonce extracts the nonce from a raw eth_sendRawTransaction +// param, mirroring the RLP/EIP-2718 decode RemoteSigner.RemoteTransactOpts +// does on the way back from the remote-signer. +func decodeSentNonce(t *testing.T, params []json.RawMessage) uint64 { + t.Helper() + + var rawHex string + if err := json.Unmarshal(params[0], &rawHex); err != nil { + t.Fatalf("decode sendRawTransaction param: %v", err) + } + raw, err := hex.DecodeString(strings.TrimPrefix(rawHex, "0x")) + if err != nil { + t.Fatalf("hex decode raw tx: %v", err) + } + var tx types.Transaction + if err := tx.UnmarshalBinary(raw); err != nil { + t.Fatalf("unmarshal raw tx: %v", err) + } + return tx.Nonce() +} + +// TestPinnedNonce_SequentialWritesDoNotCollide guards the fix for +// registerDirectViaSigner's "nonce too low" failure: pinning the nonce +// locally (PendingNonceAt once, bump after each successful send) must +// produce increasing on-chain nonces across sequential writes even when +// eth_getTransactionCount keeps reporting the same value — exactly what a +// lagging eRPC upstream does immediately after a just-broadcast tx. +func TestPinnedNonce_SequentialWritesDoNotCollide(t *testing.T) { + key, err := crypto.GenerateKey() + if err != nil { + t.Fatal(err) + } + + handlers := txMockHandlers(common.HexToHash("0x4444")) + handlers["eth_getTransactionCount"] = func(_ []json.RawMessage) (json.RawMessage, error) { + return json.RawMessage(`"0x5"`), nil // stale: never advances on its own + } + var sentNonces []uint64 + handlers["eth_sendRawTransaction"] = func(params []json.RawMessage) (json.RawMessage, error) { + sentNonces = append(sentNonces, decodeSentNonce(t, params)) + return json.RawMessage(`"0x4444"`), nil + } + + srv := mockRPC(t, handlers) + defer srv.Close() + + ctx := context.Background() + client, err := newClient(ctx, srv.URL, IdentityRegistryBaseSepolia) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + defer client.Close() + + opts, err := bind.NewKeyedTransactorWithChainID(key, client.chainID) + if err != nil { + t.Fatalf("transactor: %v", err) + } + opts.Context = ctx + + // Pin, exactly as registerDirectViaSigner does, instead of leaving + // opts.Nonce nil (which re-queries eth_getTransactionCount every call). + nonce, err := client.PendingNonceAt(ctx, crypto.PubkeyToAddress(key.PublicKey)) + if err != nil { + t.Fatalf("PendingNonceAt: %v", err) + } + opts.Nonce = new(big.Int).SetUint64(nonce) + + if _, err := client.SetAgentURIWithOpts(ctx, opts, big.NewInt(1), "https://example.com/a"); err != nil { + t.Fatalf("SetAgentURIWithOpts: %v", err) + } + opts.Nonce = new(big.Int).Add(opts.Nonce, big.NewInt(1)) + + if err := client.SetMetadataWithOpts(ctx, opts, big.NewInt(1), "x402", []byte(`{"x402":true}`)); err != nil { + t.Fatalf("SetMetadataWithOpts: %v", err) + } + + if want := []uint64{5, 6}; len(sentNonces) != 2 || sentNonces[0] != want[0] || sentNonces[1] != want[1] { + t.Errorf("nonces = %v, want %v — sequential writes must not collide even when the RPC reports a stale pending nonce", sentNonces, want) + } +} + +// TestUnpinnedNonce_StaleRPCCollides documents the bug the pinning fix +// closes: without it, two sequential Transact calls against a stale +// eth_getTransactionCount submit the same colliding nonce. +func TestUnpinnedNonce_StaleRPCCollides(t *testing.T) { + key, err := crypto.GenerateKey() + if err != nil { + t.Fatal(err) + } + + handlers := txMockHandlers(common.HexToHash("0x5555")) + handlers["eth_getTransactionCount"] = func(_ []json.RawMessage) (json.RawMessage, error) { + return json.RawMessage(`"0x5"`), nil + } + var sentNonces []uint64 + handlers["eth_sendRawTransaction"] = func(params []json.RawMessage) (json.RawMessage, error) { + sentNonces = append(sentNonces, decodeSentNonce(t, params)) + return json.RawMessage(`"0x5555"`), nil + } + + srv := mockRPC(t, handlers) + defer srv.Close() + + ctx := context.Background() + client, err := newClient(ctx, srv.URL, IdentityRegistryBaseSepolia) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + defer client.Close() + + opts, err := bind.NewKeyedTransactorWithChainID(key, client.chainID) + if err != nil { + t.Fatalf("transactor: %v", err) + } + opts.Context = ctx + // opts.Nonce left nil on purpose: bind re-queries eth_getTransactionCount + // on every call, which here always returns the same stale value. + + if _, err := client.SetAgentURIWithOpts(ctx, opts, big.NewInt(1), "https://example.com/a"); err != nil { + t.Fatalf("SetAgentURIWithOpts: %v", err) + } + if err := client.SetMetadataWithOpts(ctx, opts, big.NewInt(1), "x402", []byte(`{"x402":true}`)); err != nil { + t.Fatalf("SetMetadataWithOpts: %v", err) + } + + if len(sentNonces) != 2 || sentNonces[0] != sentNonces[1] { + t.Fatalf("nonces = %v, want both == the stale RPC nonce (5) to demonstrate the collision this fix avoids", sentNonces) + } +} diff --git a/internal/hermes/hermes.go b/internal/hermes/hermes.go index 9216b31a..98eb52ad 100644 --- a/internal/hermes/hermes.go +++ b/internal/hermes/hermes.go @@ -22,6 +22,7 @@ import ( "github.com/ObolNetwork/obol-stack/internal/model" "github.com/ObolNetwork/obol-stack/internal/tunnel" "github.com/ObolNetwork/obol-stack/internal/ui" + "github.com/ObolNetwork/obol-stack/internal/validate" petname "github.com/dustinkirkland/golang-petname" "gopkg.in/yaml.v3" ) @@ -45,6 +46,18 @@ const ( containerUID = 1000 containerGID = 1000 dashboardPort = 9119 + + // DashboardPasswordLoginPath is the browser entry that works with Hermes + // basic-auth. Hermes root "/" auto-redirects to + // /auth/login?provider=basic, which raises + // NotImplementedError: BasicAuthProvider is password-only (HTTP 500). + // See docs/guides/hermes-dashboard-login.md. + DashboardPasswordLoginPath = "/auth/password-login" + + // DashboardBasicAuthUsername is the fixed username Obol wires into + // HERMES_DASHBOARD_BASIC_AUTH_USERNAME. The password is the agent API + // token (same value as HERMES_DASHBOARD_BASIC_AUTH_PASSWORD / API_SERVER_KEY). + DashboardBasicAuthUsername = "obol" ) type OnboardOptions struct { @@ -94,6 +107,20 @@ func Onboard(cfg *config.Config, opts OnboardOptions, u *ui.UI) error { u.Infof("Using deployment ID: %s", id) } + // id becomes a DNS label (hostname, namespace) below, so it must be + // restricted to a safe charset — an unsanitized id (e.g. containing a + // newline) could otherwise inject arbitrary /etc/hosts entries. + if err := validate.Name(id); err != nil { + return fmt.Errorf("invalid agent id: %w", err) + } + // validate.Name alone allows ids up to 63 chars, but Onboard derives the + // "hermes-" namespace/hostname (and DashboardHostname's + // "hermes--ui" label) below — bound id here so those stay ≤63 + // instead of failing later with an opaque Kubernetes error. + if max := agentruntime.MaxIDLength(agentruntime.Hermes); len(id) > max { + return fmt.Errorf("agent id %q is too long (%d chars): must be at most %d chars so hermes- fits the 63-character DNS label limit", id, len(id), max) + } + deploymentDir := DeploymentPath(cfg, id) namespace := agentruntime.Namespace(agentruntime.Hermes, id) hostname := agentruntime.Hostname(agentruntime.Hermes, id) @@ -266,10 +293,7 @@ func Sync(cfg *config.Config, id string, u *ui.UI) error { u.Success("Hermes installed successfully!") u.Detail("Namespace", agentruntime.Namespace(agentruntime.Hermes, id)) u.Detail("URL", "http://"+agentruntime.Hostname(agentruntime.Hermes, id)) - u.Detail("Dashboard", "http://"+dashboardHostname(id)) - u.Blank() - u.Dim("[Optional] Retrieve an API server token:") - u.Printf(" obol agent auth %s", id) + printDashboardAccessGuidance(u, id) u.Blank() u.Dim("[Optional] Port-forward fallback:") u.Printf(" obol kubectl -n %s port-forward svc/%s %d:%d", @@ -718,6 +742,48 @@ func dashboardHostname(id string) string { return agentruntime.DashboardHostname(agentruntime.Hermes, id) } +// dashboardURL is the pretty operator URL for the Hermes dashboard host. +// An HTTPRoute RequestRedirect on Exact "/" rewrites to the password form +// so operators never need to type /auth/password-login. +func dashboardURL(id string) string { + return "http://" + dashboardHostname(id) +} + +// dashboardPasswordLoginURL is the form path Hermes actually serves (and the +// target of the edge redirect from "/"). +func dashboardPasswordLoginURL(id string) string { + return dashboardURL(id) + DashboardPasswordLoginPath +} + +// dashboardAccessGuidance returns operator-facing lines that name the pretty +// dashboard URL and how credentials map to the agent API token. +// Pure for unit tests; printDashboardAccessGuidance is the Sync success path. +func dashboardAccessGuidance(id string) []string { + return []string{ + "Dashboard: " + dashboardURL(id), + " Username: " + DashboardBasicAuthUsername, + " Password: agent API token from `obol agent auth " + id + "`", + " (root edge-redirects to " + DashboardPasswordLoginPath + ")", + } +} + +func printDashboardAccessGuidance(u *ui.UI, id string) { + // Drive UI from the pure guidance lines so unit tests cover the shipped path. + for i, line := range dashboardAccessGuidance(id) { + if i == 0 { + const prefix = "Dashboard: " + if strings.HasPrefix(line, prefix) { + u.Detail("Dashboard", strings.TrimPrefix(line, prefix)) + continue + } + } + u.Dim(line) + } + u.Blank() + u.Dim("[Optional] Retrieve the dashboard password (API token):") + u.Printf(" obol agent auth %s", id) +} + func generateValues(namespace, hostname, dashboardHostname, agentBaseURL, token, primary string, configData []byte) string { desc := agentruntime.Describe(agentruntime.Hermes) configChecksum := sha256.Sum256(configData) @@ -963,7 +1029,7 @@ func generateValues(namespace, hostname, dashboardHostname, agentBaseURL, token, # existing API token as the password (already surfaced to the # operator). Probe path /api/status stays auth-exempt. - name: HERMES_DASHBOARD_BASIC_AUTH_USERNAME - value: obol + value: %s - name: HERMES_DASHBOARD_BASIC_AUTH_PASSWORD value: %s readinessProbe: @@ -1041,15 +1107,33 @@ func generateValues(namespace, hostname, dashboardHostname, agentBaseURL, token, namespace: traefik sectionName: web rules: + # Hermes root "/" auto-redirects to /auth/login?provider=basic which + # 500s for password-only basic-auth. Edge-redirect Exact "/" to the + # working password form so operators can open the pretty dashboard + # host. Dashboard hostname only — agent API host is untouched. + - matches: + - path: + type: Exact + value: / + filters: + - type: RequestRedirect + requestRedirect: + path: + type: ReplaceFullPath + replaceFullPath: %s + statusCode: 302 - backendRefs: - name: %s port: %d `, desc.DefaultPort, desc.DefaultPort, desc.DefaultPort, - quoteYAML(image()), quoteYAML(hermesBinary), dashboardPort, dashboardPort, desc.DefaultPort, quoteYAML(token), dashboardPort, dashboardPort, dashboardPort, + quoteYAML(image()), quoteYAML(hermesBinary), dashboardPort, dashboardPort, desc.DefaultPort, + quoteYAML(DashboardBasicAuthUsername), quoteYAML(token), + dashboardPort, dashboardPort, dashboardPort, desc.DataPVCName, desc.ServiceName, namespace, desc.ServiceName, desc.ServiceName, desc.DefaultPort, dashboardPort, desc.ServiceName, namespace, quoteYAML(hostname), desc.ServiceName, desc.DefaultPort, - desc.ServiceName, namespace, quoteYAML(dashboardHostname), desc.ServiceName, dashboardPort) + desc.ServiceName, namespace, quoteYAML(dashboardHostname), + DashboardPasswordLoginPath, desc.ServiceName, dashboardPort) return strings.ReplaceAll(b.String(), "\t", "") } diff --git a/internal/hermes/hermes_test.go b/internal/hermes/hermes_test.go index 60d1e469..b97cf318 100644 --- a/internal/hermes/hermes_test.go +++ b/internal/hermes/hermes_test.go @@ -117,6 +117,47 @@ func TestOnboardDefaultAlreadyInstalledIsIdempotent(t *testing.T) { } } +// TestOnboardRejectsUnsafeID guards against the Canary402 full-surface audit +// finding: an unsanitized --id becomes a DNS label (hostname, namespace) and +// was written verbatim into /etc/hosts, so a newline in --id could inject +// arbitrary /etc/hosts lines via "sudo tee". Onboard must reject any id that +// isn't a safe DNS label before it reaches deployment/hostname construction. +func TestOnboardRejectsUnsafeID(t *testing.T) { + invalid := []string{ + "evil\n127.0.0.1 attacker.com", // /etc/hosts line injection + "has space", + "has/slash", + "-leading-dash", + } + for _, id := range invalid { + cfg := testConfig(t) + err := Onboard(cfg, OnboardOptions{ID: id}, newTestUI()) + if err == nil { + t.Errorf("Onboard(ID=%q) = nil, want error", id) + } + } +} + +// TestOnboardRejectsIDTooLongForDNSLabel guards against the "hermes--ui" +// DashboardHostname DNS label overflowing 63 characters: validate.Name alone +// allows a 63-char id, but Onboard derives "hermes-" (and, for the +// dashboard, "hermes--ui") from it, so an id at (or near) that limit +// must be rejected here with a clear error instead of failing later with an +// opaque Kubernetes error. +func TestOnboardRejectsIDTooLongForDNSLabel(t *testing.T) { + max := agentruntime.MaxIDLength(agentruntime.Hermes) + + tooLong := "a" + strings.Repeat("b", max) // max+1 chars, still a valid DNS label on its own + cfg := testConfig(t) + err := Onboard(cfg, OnboardOptions{ID: tooLong}, newTestUI()) + if err == nil { + t.Fatalf("Onboard(ID=%d chars) = nil, want error", len(tooLong)) + } + if !strings.Contains(err.Error(), "too long") { + t.Errorf("error should explain the id is too long, got: %v", err) + } +} + // TestGenerateConfig_PrimaryIsRoundTrippable guards the LiteLLM model_name // contract end-to-end: whatever string the agent's `model.default` is set to // MUST match a `model_name` entry in the LiteLLM ConfigMap byte-for-byte, @@ -303,6 +344,9 @@ func TestGenerateValues_UsesHermesNativeNames(t *testing.T) { "name: GATEWAY_HEALTH_URL", "HERMES_DASHBOARD_BASIC_AUTH_USERNAME", "HERMES_DASHBOARD_BASIC_AUTH_PASSWORD", + // Username is fixed; password is the agent API token (API_SERVER_KEY). + `value: "obol"`, + `value: "secret-token"`, // Must track HERMES_HOME: v2026.7.x images bake // HERMES_WRITE_SAFE_ROOT=/opt/data and deny all file-tool // writes outside the safe root. @@ -314,6 +358,58 @@ func TestGenerateValues_UsesHermesNativeNames(t *testing.T) { } } + // Basic-auth must stay wired: non-loopback dashboard bind requires it, + // and the password must equal the agent API token already in the Secret. + if !strings.Contains(values, "HERMES_DASHBOARD_BASIC_AUTH_USERNAME") || + !strings.Contains(values, "HERMES_DASHBOARD_BASIC_AUTH_PASSWORD") { + t.Fatalf("generateValues() missing dashboard basic-auth env wiring:\n%s", values) + } + // Password env must carry the same token as API_SERVER_KEY (secret-token in this fixture). + if idx := strings.Index(values, "HERMES_DASHBOARD_BASIC_AUTH_PASSWORD"); idx < 0 { + t.Fatal("HERMES_DASHBOARD_BASIC_AUTH_PASSWORD missing") + } else { + window := values[idx:] + if len(window) > 120 { + window = window[:120] + } + if !strings.Contains(window, `value: "secret-token"`) { + t.Fatalf("HERMES_DASHBOARD_BASIC_AUTH_PASSWORD must use agent API token; nearby:\n%s", window) + } + } + + // Dashboard HTTPRoute must edge-redirect Exact "/" → password-login so + // operators can open the pretty host without hitting Hermes's broken + // /auth/login?provider=basic path. + for _, needle := range []string{ + "type: RequestRedirect", + "replaceFullPath: " + DashboardPasswordLoginPath, + "type: Exact", + "value: /", + } { + if !strings.Contains(values, needle) { + t.Fatalf("generateValues() dashboard root redirect missing %q:\n%s", needle, values) + } + } + // Redirect must only live on the dashboard HTTPRoute, not the agent API route. + firstHTTPRoute := strings.Index(values, "kind: HTTPRoute") + if firstHTTPRoute < 0 { + t.Fatal("expected HTTPRoute resources in generateValues()") + } + secondHTTPRouteRel := strings.Index(values[firstHTTPRoute+1:], "kind: HTTPRoute") + if secondHTTPRouteRel < 0 { + t.Fatal("expected two HTTPRoutes (agent + dashboard)") + } + secondHTTPRoute := firstHTTPRoute + 1 + secondHTTPRouteRel + agentRouteYAML := values[firstHTTPRoute:secondHTTPRoute] + if strings.Contains(agentRouteYAML, "RequestRedirect") { + t.Fatalf("agent API HTTPRoute must not edge-redirect root:\n%s", agentRouteYAML) + } + dashRouteYAML := values[secondHTTPRoute:] + if !strings.Contains(dashRouteYAML, "RequestRedirect") || + !strings.Contains(dashRouteYAML, "replaceFullPath: "+DashboardPasswordLoginPath) { + t.Fatalf("dashboard HTTPRoute missing root→password-login redirect:\n%s", dashRouteYAML) + } + for _, banned := range []string{ "bootstrap-hermes-install", "git clone", @@ -392,6 +488,46 @@ func TestDashboardHostname_UsesDefaultAgentHostAndHermesUIHostForNamedInstances( } } +func TestDashboardPasswordLoginURL_UsesPasswordLoginPath(t *testing.T) { + got := dashboardPasswordLoginURL("obol-agent") + want := "http://obol-agent.obol.stack" + DashboardPasswordLoginPath + if got != want { + t.Fatalf("dashboardPasswordLoginURL(obol-agent) = %q, want %q", got, want) + } + if !strings.HasSuffix(got, "/auth/password-login") { + t.Fatalf("login URL must end with /auth/password-login, got %q", got) + } + if gotRoot := dashboardURL("obol-agent"); gotRoot != "http://obol-agent.obol.stack" { + t.Fatalf("dashboardURL(obol-agent) = %q, want pretty host root", gotRoot) + } +} + +func TestDashboardAccessGuidance_NamesPrettyURLAndTokenPassword(t *testing.T) { + id := "obol-agent" + lines := dashboardAccessGuidance(id) + joined := strings.Join(lines, "\n") + + for _, needle := range []string{ + "Dashboard: http://obol-agent.obol.stack", + DashboardPasswordLoginPath, + "Username: " + DashboardBasicAuthUsername, + "obol agent auth " + id, + "Password:", + } { + if !strings.Contains(joined, needle) { + t.Fatalf("dashboardAccessGuidance missing %q:\n%s", needle, joined) + } + } + + // Primary line is the pretty host (edge-redirected), not only the form path. + if !strings.HasPrefix(lines[0], "Dashboard: http://obol-agent.obol.stack") { + t.Fatalf("first guidance line should be pretty Dashboard URL, got %q", lines[0]) + } + if strings.HasPrefix(lines[0], "Dashboard: http://obol-agent.obol.stack/auth/") { + t.Fatalf("primary Dashboard URL should be host root, not form path: %q", lines[0]) + } +} + func extractChecksumAnnotation(t *testing.T, values, key string) string { t.Helper() var doc struct { diff --git a/internal/inference/gateway.go b/internal/inference/gateway.go index 8a917b7f..fcbe574b 100644 --- a/internal/inference/gateway.go +++ b/internal/inference/gateway.go @@ -177,7 +177,10 @@ func (g *Gateway) buildHandler(upstreamURL string) (http.Handler, error) { // Create x402 payment requirement. The standalone gateway has no // operator surface for MaxTimeoutSeconds yet, so pass 0 to fall back // to DefaultMaxTimeoutSeconds. - requirement := x402pkg.BuildV2Requirement(g.config.Chain, g.config.PricePerRequest, g.config.WalletAddress, 0) + requirement, err := x402pkg.BuildV2Requirement(g.config.Chain, g.config.PricePerRequest, g.config.WalletAddress, 0) + if err != nil { + return nil, fmt.Errorf("invalid price %q: %w", g.config.PricePerRequest, err) + } // Configure x402 ForwardAuth middleware. The bazaar discovery extension // advertises the chat-completions invocation shape on every 402 so diff --git a/internal/monetizeapi/agentidentity_test.go b/internal/monetizeapi/agentidentity_test.go index a619d9cf..aad2042d 100644 --- a/internal/monetizeapi/agentidentity_test.go +++ b/internal/monetizeapi/agentidentity_test.go @@ -33,3 +33,55 @@ func TestUpsertAgentIdentityRegistration_DedupesChain(t *testing.T) { t.Fatalf("registrations = %+v, want deduped base + base-sepolia", status.Registrations) } } + +func TestRemoveAgentIdentityRegistration(t *testing.T) { + status := AgentIdentityStatus{} + status = UpsertAgentIdentityRegistration(status, "base-sepolia", "99") + status = UpsertAgentIdentityRegistration(status, "base", "42") + + status = RemoveAgentIdentityRegistration(status, "base") + + if got := AgentIdentityAgentIDForChain(status, "base"); got != "" { + t.Errorf("base agentId = %q, want empty after remove", got) + } + if got := AgentIdentityAgentIDForChain(status, "base-sepolia"); got != "99" { + t.Errorf("base-sepolia agentId = %q, want 99 (untouched)", got) + } + if len(status.Registrations) != 1 { + t.Fatalf("registrations = %+v, want only base-sepolia left", status.Registrations) + } +} + +func TestRemoveAgentIdentityRegistration_UnknownChainAndEmptyChainAreNoops(t *testing.T) { + status := AgentIdentityStatus{} + status = UpsertAgentIdentityRegistration(status, "base", "42") + + status = RemoveAgentIdentityRegistration(status, "polygon") + if got := AgentIdentityAgentIDForChain(status, "base"); got != "42" { + t.Errorf("base agentId = %q, want 42 unchanged by removing an unrelated chain", got) + } + + status = RemoveAgentIdentityRegistration(status, "") + if got := AgentIdentityAgentIDForChain(status, "base"); got != "42" { + t.Errorf("base agentId = %q, want 42 unchanged by an empty-chain no-op", got) + } +} + +// TestRemoveAgentIdentityRegistration_DoesNotMutateSharedBackingArray guards +// against RemoveAgentIdentityRegistration filtering status.Registrations in +// place: a caller (e.g. an informer-cached object) may hold another +// reference to the same backing array, which an in-place compaction would +// silently corrupt. +func TestRemoveAgentIdentityRegistration_DoesNotMutateSharedBackingArray(t *testing.T) { + backing := []AgentIdentityRegistration{ + {Chain: "base", AgentID: "1"}, + {Chain: "base-sepolia", AgentID: "2"}, + } + shared := AgentIdentityStatus{Registrations: backing} + + _ = RemoveAgentIdentityRegistration(shared, "base") + + if backing[0].Chain != "base" || backing[0].AgentID != "1" { + t.Errorf("shared backing array was mutated: backing[0] = %+v, want unchanged {base 1}", backing[0]) + } +} diff --git a/internal/monetizeapi/types.go b/internal/monetizeapi/types.go index d9bdd874..91f851e5 100644 --- a/internal/monetizeapi/types.go +++ b/internal/monetizeapi/types.go @@ -1151,6 +1151,31 @@ func UpsertAgentIdentityRegistration(status AgentIdentityStatus, chain, agentID return status } +// RemoveAgentIdentityRegistration drops chain's registration, if any. It is +// the counterpart to UpsertAgentIdentityRegistration and exists as the +// operator remediation path for a registration that was written under the +// wrong chain (e.g. `obol sell identity forget `) — there is no +// automatic removal anywhere else, since the durable-identity/tombstone +// design otherwise treats a recorded registration as permanent. +func RemoveAgentIdentityRegistration(status AgentIdentityStatus, chain string) AgentIdentityStatus { + chain = strings.TrimSpace(chain) + if chain == "" { + return status + } + // Allocate rather than filter status.Registrations in place: this is an + // exported helper and a future caller could pass in an informer-cached + // object whose backing array must not be mutated out from under it. + out := make([]AgentIdentityRegistration, 0, len(status.Registrations)) + for _, registration := range status.Registrations { + if strings.EqualFold(strings.TrimSpace(registration.Chain), chain) { + continue + } + out = append(out, registration) + } + status.Registrations = out + return status +} + func HasAgentIdentityRegistrations(status AgentIdentityStatus) bool { for _, registration := range status.Registrations { if strings.TrimSpace(registration.Chain) != "" && strings.TrimSpace(registration.AgentID) != "" { diff --git a/internal/network/overlay.go b/internal/network/overlay.go new file mode 100644 index 00000000..44b2642f --- /dev/null +++ b/internal/network/overlay.go @@ -0,0 +1,901 @@ +package network + +import ( + "crypto/sha256" + "encoding/hex" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/ObolNetwork/obol-stack/internal/config" + "github.com/ObolNetwork/obol-stack/internal/kubectl" + "github.com/ObolNetwork/obol-stack/internal/ui" + "gopkg.in/yaml.v3" +) + +// Host-side eRPC operator overlay. Complements recorded-upstreams.yaml +// (simple remote remotes) with durable multi-upstream baskets, scoring, +// rate-limit budgets, and cache policy fragments that survive +// `obol stack up` / helmfile re-renders of the eRPC chart. +// +// Lifecycle (mirrors Hermes mergePreservedHermesConfigKeys + recorded RPCs): +// 1. Operator writes overlay via `obol network overlay apply -f` +// 2. Overlay is stored at $CONFIG_DIR/rpc/erpc-overlay.yaml (0600) +// 3. applyOverlayToCluster deep-merges into the live eRPC ConfigMap +// 4. ReconcileERPCOverlay re-applies after stack up (after ReconcileRecordedRPCs) +// +// See ObolNetwork/obol-stack#763. + +const ( + erpcOverlayVersion = 1 + erpcOverlayAnnotKey = "obol.stack/erpc-overlay-hash" + erpcOverlayAnnotSource = "obol.stack/erpc-overlay-source" +) + +// ERPCOverlay is the durable operator overlay document. +// Fragments are free-form maps so operators can pass full eRPC YAML objects +// (scoreMultipliers, failsafe, rateLimitBudget, …) without a rigid schema. +type ERPCOverlay struct { + Version int `yaml:"version"` + + // Networks are merged into projects[0].networks by chainId (preferred) + // or alias. Matching entries are replaced; others are appended. + Networks []map[string]any `yaml:"networks,omitempty"` + + // Upstreams are merged into projects[0].upstreams by id. Matching + // entries are replaced; others are appended (after existing). + Upstreams []map[string]any `yaml:"upstreams,omitempty"` + + // RateLimiters budgets are merged by id; other top-level keys are + // shallow-replaced (see mergeRateLimiters). + RateLimiters map[string]any `yaml:"rateLimiters,omitempty"` + + // CachePoliciesAdd are appended to database.evmJsonRpcCache.policies + // when no existing policy has the same network+method+finality triple. + CachePoliciesAdd []map[string]any `yaml:"cachePoliciesAdd,omitempty"` +} + +// ERPCOverlayStatus is a read-only summary for CLI status. +type ERPCOverlayStatus struct { + Path string + Present bool + Version int + NetworkCount int + UpstreamCount int + BudgetCount int + CachePolicyAdd int + NetworkKeys []string + UpstreamIDs []string + ContentHash string + + // ClusterSync is a best-effort comparison of ContentHash against the + // live ConfigMap's erpcOverlayAnnotKey annotation: + // "in-sync" | "drifted" | "not-applied" | "" (cluster unreachable/unknown). + ClusterSync string +} + +// ERPCSyncInSync / ERPCSyncDrifted / ERPCSyncNotApplied are the ClusterSync +// values StatusERPC can report; "" means the cluster couldn't be checked. +const ( + ERPCSyncInSync = "in-sync" + ERPCSyncDrifted = "drifted" + ERPCSyncNotApplied = "not-applied" +) + +func erpcOverlayPath(cfg *config.Config) string { + return filepath.Join(cfg.ConfigDir, "rpc", "erpc-overlay.yaml") +} + +// erpcProvenancePath is the host-side record of what mergeERPCOverlay +// replaced the FIRST time each overlay-owned key was applied, so ResetERPC +// can restore chart-base/recorded entries instead of deleting them. +func erpcProvenancePath(cfg *config.Config) string { + return filepath.Join(cfg.ConfigDir, "rpc", "erpc-overlay-provenance.yaml") +} + +// erpcProvenance maps each overlay-owned merge key to the entry it replaced. +// A key present with a nil *map value means the overlay ADDED that entry +// (no prior base entry to restore); a key absent means "not yet recorded". +// (A pointer, not a bare map, because a nil map[string]any round-trips +// through a YAML marshal/unmarshal as an empty map, not nil — see +// captureERPCProvenance.) +type erpcProvenance struct { + Networks map[string]*map[string]any `yaml:"networks,omitempty"` + Upstreams map[string]*map[string]any `yaml:"upstreams,omitempty"` +} + +func readERPCProvenance(cfg *config.Config) (*erpcProvenance, error) { + data, err := os.ReadFile(erpcProvenancePath(cfg)) + if os.IsNotExist(err) { + return &erpcProvenance{}, nil + } + if err != nil { + return nil, err + } + var p erpcProvenance + if err := yaml.Unmarshal(data, &p); err != nil { + return nil, fmt.Errorf("parse eRPC overlay provenance: %w", err) + } + return &p, nil +} + +func writeERPCProvenance(cfg *config.Config, p *erpcProvenance) error { + path := erpcProvenancePath(cfg) + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + return err + } + data, err := yaml.Marshal(p) + if err != nil { + return err + } + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, 0o600); err != nil { + return err + } + return os.Rename(tmp, path) +} + +// captureERPCProvenance records, for every overlay-owned key not already +// tracked, whatever entry currently occupies that key in erpcConfig — the +// chart-base or recorded-RPC entry mergeERPCOverlay is about to replace (or +// nil if the key doesn't exist yet, meaning the overlay is adding it). +// Must be called BEFORE mergeERPCOverlay mutates erpcConfig. +func captureERPCProvenance(cfg *config.Config, erpcConfig map[string]any, ov *ERPCOverlay) error { + if len(ov.Networks) == 0 && len(ov.Upstreams) == 0 { + return nil + } + prov, err := readERPCProvenance(cfg) + if err != nil { + return err + } + project := erpcConfigProject(erpcConfig) + if project == nil { + return nil + } + changed := false + + if len(ov.Networks) > 0 { + if prov.Networks == nil { + prov.Networks = map[string]*map[string]any{} + } + byKey := map[string]map[string]any{} + for _, n := range asMapSlice(project["networks"]) { + if k := networkMergeKey(n); k != "" { + byKey[k] = n + } + } + for _, n := range ov.Networks { + k := networkMergeKey(n) + if k == "" { + continue + } + if _, tracked := prov.Networks[k]; tracked { + continue + } + if orig, hit := byKey[k]; hit { + clone := cloneMap(orig) + prov.Networks[k] = &clone + } else { + prov.Networks[k] = nil // marker: overlay added this key + } + changed = true + } + } + + if len(ov.Upstreams) > 0 { + if prov.Upstreams == nil { + prov.Upstreams = map[string]*map[string]any{} + } + byID := map[string]map[string]any{} + for _, u := range asMapSlice(project["upstreams"]) { + if id, _ := u["id"].(string); id != "" { + byID[id] = u + } + } + for _, u := range ov.Upstreams { + id, _ := u["id"].(string) + if id == "" { + continue + } + if _, tracked := prov.Upstreams[id]; tracked { + continue + } + if orig, hit := byID[id]; hit { + clone := cloneMap(orig) + prov.Upstreams[id] = &clone + } else { + prov.Upstreams[id] = nil // marker: overlay added this key + } + changed = true + } + } + + if !changed { + return nil + } + return writeERPCProvenance(cfg, prov) +} + +// erpcConfigProject returns projects[0] as a map, or nil if the shape is +// unexpected (callers treat that as "nothing to capture/strip"). +func erpcConfigProject(erpcConfig map[string]any) map[string]any { + projects, ok := erpcConfig["projects"].([]any) + if !ok || len(projects) == 0 { + return nil + } + project, ok := projects[0].(map[string]any) + if !ok { + return nil + } + return project +} + +// asMapSlice filters a []any (as decoded from YAML) down to its map[string]any +// entries, skipping anything malformed. +func asMapSlice(v any) []map[string]any { + items, _ := v.([]any) + out := make([]map[string]any, 0, len(items)) + for _, item := range items { + if m, ok := item.(map[string]any); ok { + out = append(out, m) + } + } + return out +} + +// ReconcileERPCOverlay re-applies the host-side overlay into the live eRPC +// ConfigMap. Called after ReconcileRecordedRPCs on stack up. Best-effort: +// missing overlay is a silent no-op; apply errors are warned, not fatal. +func ReconcileERPCOverlay(cfg *config.Config, u *ui.UI) { + ov, err := readERPCOverlay(cfg) + if err != nil { + u.Warnf("Could not read durable eRPC config: %v", err) + return + } + if ov == nil { + return + } + if err := applyOverlayToCluster(cfg, ov, "reconcile"); err != nil { + u.Warnf("Could not restore durable eRPC operator config: %v", err) + return + } + u.Successf("Restored durable eRPC operator config (%d network(s), %d upstream(s))", + len(ov.Networks), len(ov.Upstreams)) +} + +// ApplyERPCOverlayFile loads an overlay YAML from path, persists it under +// ConfigDir, and merges it into the live eRPC ConfigMap. +func SetERPC(cfg *config.Config, u *ui.UI, path string) error { + data, err := os.ReadFile(path) + if err != nil { + return fmt.Errorf("read overlay file: %w", err) + } + ov, err := parseERPCOverlay(data) + if err != nil { + return err + } + if err := writeERPCOverlay(cfg, ov); err != nil { + return err + } + if err := applyOverlayToCluster(cfg, ov, filepath.Base(path)); err != nil { + return err + } + u.Successf("eRPC config set and saved to %s", erpcOverlayPath(cfg)) + u.Infof(" networks=%d upstreams=%d cachePoliciesAdd=%d", + len(ov.Networks), len(ov.Upstreams), len(ov.CachePoliciesAdd)) + return nil +} + +// ClearERPCOverlay removes overlay-owned fragments from the live ConfigMap +// (best-effort), then deletes the host-side overlay file. +func ResetERPC(cfg *config.Config, u *ui.UI) error { + ov, err := readERPCOverlay(cfg) + if err != nil { + return err + } + if ov == nil { + u.Info("No durable eRPC config on disk") + return nil + } + + if err := removeOverlayFromCluster(cfg, ov); err != nil { + u.Warnf("Could not strip eRPC config from live ConfigMap (will still reset host file): %v", err) + } else { + u.Success("Removed overlay-added networks/upstreams and restored any chart-base/recorded entries they replaced") + } + + path := erpcOverlayPath(cfg) + if err := os.Remove(path); err != nil && !os.IsNotExist(err) { + return fmt.Errorf("remove overlay file: %w", err) + } + if err := os.Remove(erpcProvenancePath(cfg)); err != nil && !os.IsNotExist(err) { + return fmt.Errorf("remove overlay provenance file: %w", err) + } + // Best-effort annotation clear + _ = annotateERPCOverlay(cfg, "", "") + u.Successf("Reset eRPC config at %s", path) + u.Info("Chart base + recorded remotes remain; re-run `obol stack up` if you need a full eRPC re-render") + return nil +} + +// StatusERPCOverlay returns a summary of the on-disk overlay (no cluster I/O). +func StatusERPC(cfg *config.Config) (*ERPCOverlayStatus, error) { + path := erpcOverlayPath(cfg) + st := &ERPCOverlayStatus{Path: path} + ov, err := readERPCOverlay(cfg) + if err != nil { + return nil, err + } + if ov == nil { + return st, nil + } + st.Present = true + st.Version = ov.Version + st.NetworkCount = len(ov.Networks) + st.UpstreamCount = len(ov.Upstreams) + st.CachePolicyAdd = len(ov.CachePoliciesAdd) + st.BudgetCount = countRateLimitBudgets(ov.RateLimiters) + for _, n := range ov.Networks { + st.NetworkKeys = append(st.NetworkKeys, describeNetwork(n)) + } + for _, u := range ov.Upstreams { + if id, _ := u["id"].(string); id != "" { + st.UpstreamIDs = append(st.UpstreamIDs, id) + } + } + if data, err := os.ReadFile(path); err == nil { + sum := sha256.Sum256(data) + st.ContentHash = hex.EncodeToString(sum[:8]) + } + clusterHash, clusterErr := readERPCOverlayAnnotation(cfg) + st.ClusterSync = erpcOverlayDriftStatus(st.ContentHash, clusterHash, clusterErr) + return st, nil +} + +// erpcOverlayDriftStatus compares the host-side overlay's content hash +// against the erpcOverlayAnnotKey annotation read from the live ConfigMap. +// Pure/no I/O so it's directly testable; clusterErr != nil (cluster +// unreachable, no permissions, etc.) yields "" — best-effort, not fatal. +func erpcOverlayDriftStatus(localHash, clusterHash string, clusterErr error) string { + if clusterErr != nil { + return "" + } + if clusterHash == "" { + return ERPCSyncNotApplied + } + if clusterHash == localHash { + return ERPCSyncInSync + } + return ERPCSyncDrifted +} + +// readERPCOverlayAnnotation best-effort reads the erpcOverlayAnnotKey +// annotation applyOverlayToCluster stamps on the eRPC ConfigMap, so status +// can detect drift between the on-disk overlay and what's actually live. +func readERPCOverlayAnnotation(cfg *config.Config) (string, error) { + if err := kubectl.EnsureCluster(cfg); err != nil { + return "", err + } + kubectlBin, kubeconfigPath := kubectl.Paths(cfg) + out, err := kubectl.Output(kubectlBin, kubeconfigPath, + "get", "configmap", erpcConfigMapName, "-n", erpcNamespace, + "-o", fmt.Sprintf("jsonpath={.metadata.annotations.%s}", strings.ReplaceAll(erpcOverlayAnnotKey, ".", "\\."))) + if err != nil { + return "", err + } + return strings.TrimSpace(out), nil +} + +// --- persistence --- + +func readERPCOverlay(cfg *config.Config) (*ERPCOverlay, error) { + data, err := os.ReadFile(erpcOverlayPath(cfg)) + if os.IsNotExist(err) { + return nil, nil + } + if err != nil { + return nil, err + } + return parseERPCOverlay(data) +} + +func parseERPCOverlay(data []byte) (*ERPCOverlay, error) { + var ov ERPCOverlay + if err := yaml.Unmarshal(data, &ov); err != nil { + return nil, fmt.Errorf("parse eRPC overlay: %w", err) + } + if ov.Version == 0 { + ov.Version = erpcOverlayVersion + } + if ov.Version != erpcOverlayVersion { + return nil, fmt.Errorf("unsupported erpc-overlay version %d (want %d)", ov.Version, erpcOverlayVersion) + } + if len(ov.Networks) == 0 && len(ov.Upstreams) == 0 && + len(ov.RateLimiters) == 0 && len(ov.CachePoliciesAdd) == 0 { + return nil, fmt.Errorf("eRPC overlay is empty (need networks, upstreams, rateLimiters, and/or cachePoliciesAdd)") + } + // Upstreams must have ids for merge keys + for i, u := range ov.Upstreams { + id, _ := u["id"].(string) + if strings.TrimSpace(id) == "" { + return nil, fmt.Errorf("upstreams[%d]: missing id", i) + } + } + // Networks must have a usable merge key (evm.chainId or alias) — an + // entry with neither gets networkMergeKey "" and would be blindly + // appended as a duplicate on every merge instead of being matched. + for i, n := range ov.Networks { + if err := validateNetworkEntry(n); err != nil { + return nil, fmt.Errorf("networks[%d]: %w", i, err) + } + } + return &ov, nil +} + +// validateNetworkEntry checks the cheap-to-catch type errors and requires a +// usable merge key, so a bad entry fails `erpc set` instead of silently +// duplicating (or crash-looping eRPC) on every merge. +func validateNetworkEntry(n map[string]any) error { + if raw, ok := n["evm"]; ok { + evm, ok := raw.(map[string]any) + if !ok { + return fmt.Errorf("evm must be a mapping, got %T", raw) + } + if cid, ok := evm["chainId"]; ok { + switch cid.(type) { + case int, int64, float64: + default: + return fmt.Errorf("evm.chainId must be a number, got %T", cid) + } + } + } + if raw, ok := n["alias"]; ok { + if _, ok := raw.(string); !ok { + return fmt.Errorf("alias must be a string, got %T", raw) + } + } + if networkMergeKey(n) == "" { + return fmt.Errorf("missing usable merge key (need evm.chainId or alias)") + } + return nil +} + +func writeERPCOverlay(cfg *config.Config, ov *ERPCOverlay) error { + if ov.Version == 0 { + ov.Version = erpcOverlayVersion + } + path := erpcOverlayPath(cfg) + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + return err + } + data, err := yaml.Marshal(ov) + if err != nil { + return err + } + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, 0o600); err != nil { + return err + } + return os.Rename(tmp, path) +} + +// --- cluster apply / merge --- + +func applyOverlayToCluster(cfg *config.Config, ov *ERPCOverlay, source string) error { + erpcConfig, err := readERPCConfig(cfg) + if err != nil { + return err + } + // Snapshot whatever mergeERPCOverlay is about to replace, BEFORE it + // mutates erpcConfig, so ResetERPC can restore it later instead of + // deleting it outright. + if err := captureERPCProvenance(cfg, erpcConfig, ov); err != nil { + return fmt.Errorf("capture eRPC overlay provenance: %w", err) + } + if err := mergeERPCOverlay(erpcConfig, ov); err != nil { + return err + } + if err := writeERPCConfig(cfg, erpcConfig); err != nil { + return err + } + hash := "" + if data, err := os.ReadFile(erpcOverlayPath(cfg)); err == nil { + sum := sha256.Sum256(data) + hash = hex.EncodeToString(sum[:8]) + } + _ = annotateERPCOverlay(cfg, hash, source) + return nil +} + +func removeOverlayFromCluster(cfg *config.Config, ov *ERPCOverlay) error { + erpcConfig, err := readERPCConfig(cfg) + if err != nil { + return err + } + prov, err := readERPCProvenance(cfg) + if err != nil { + return err + } + if err := stripERPCOverlay(erpcConfig, ov, prov); err != nil { + return err + } + return writeERPCConfig(cfg, erpcConfig) +} + +// mergeERPCOverlay mutates erpcConfig in place. Exported for tests via +// the unexported name used from overlay_test.go (same package). +func mergeERPCOverlay(erpcConfig map[string]any, ov *ERPCOverlay) error { + if ov == nil { + return nil + } + projects, ok := erpcConfig["projects"].([]any) + if !ok || len(projects) == 0 { + return fmt.Errorf("eRPC config has no projects") + } + project, ok := projects[0].(map[string]any) + if !ok { + return fmt.Errorf("eRPC config project[0] is not a map") + } + + if len(ov.Networks) > 0 { + networks, _ := project["networks"].([]any) + project["networks"] = mergeNetworksByKey(networks, ov.Networks) + } + if len(ov.Upstreams) > 0 { + upstreams, _ := project["upstreams"].([]any) + project["upstreams"] = mergeUpstreamsByID(upstreams, ov.Upstreams) + } + if len(ov.RateLimiters) > 0 { + existing, _ := erpcConfig["rateLimiters"].(map[string]any) + erpcConfig["rateLimiters"] = mergeRateLimiters(existing, ov.RateLimiters) + } + if len(ov.CachePoliciesAdd) > 0 { + if err := mergeCachePolicies(erpcConfig, ov.CachePoliciesAdd); err != nil { + return err + } + } + return nil +} + +// stripERPCOverlay removes overlay-owned entries from erpcConfig. Per key, +// prov says whether the overlay ADDED it (no prior entry — drop it) or +// REPLACED a chart-base/recorded entry (restore that entry instead of +// deleting it, so the chain doesn't go unroutable). A key with no +// provenance record at all (nil prov, or overlay applied before provenance +// tracking existed) falls back to the old drop-only behavior. +func stripERPCOverlay(erpcConfig map[string]any, ov *ERPCOverlay, prov *erpcProvenance) error { + project := erpcConfigProject(erpcConfig) + if project == nil { + return fmt.Errorf("eRPC config has no projects") + } + if prov == nil { + prov = &erpcProvenance{} + } + + if len(ov.Upstreams) > 0 { + drop := map[string]struct{}{} + restore := map[string]map[string]any{} + for _, u := range ov.Upstreams { + id, _ := u["id"].(string) + if id == "" { + continue + } + if orig, tracked := prov.Upstreams[id]; tracked && orig != nil { + restore[id] = *orig + } else { + drop[id] = struct{}{} + } + } + upstreams, _ := project["upstreams"].([]any) + kept := make([]any, 0, len(upstreams)) + for _, u := range upstreams { + um, ok := u.(map[string]any) + if !ok { + kept = append(kept, u) + continue + } + id, _ := um["id"].(string) + if orig, hit := restore[id]; hit { + kept = append(kept, orig) + continue + } + if _, hit := drop[id]; hit { + continue + } + kept = append(kept, u) + } + project["upstreams"] = kept + } + + if len(ov.Networks) > 0 { + drop := map[string]struct{}{} + restore := map[string]map[string]any{} + for _, n := range ov.Networks { + k := networkMergeKey(n) + if k == "" { + continue + } + if orig, tracked := prov.Networks[k]; tracked && orig != nil { + restore[k] = *orig + } else { + drop[k] = struct{}{} + } + } + networks, _ := project["networks"].([]any) + kept := make([]any, 0, len(networks)) + for _, n := range networks { + nm, ok := n.(map[string]any) + if !ok { + kept = append(kept, n) + continue + } + k := networkMergeKey(nm) + if orig, hit := restore[k]; hit { + kept = append(kept, orig) + continue + } + if _, hit := drop[k]; hit { + continue + } + kept = append(kept, n) + } + project["networks"] = kept + } + // Leave rateLimiters / cache policies in place — removing budgets that + // other upstreams still reference is riskier than leaving unused ones. + return nil +} + +func mergeNetworksByKey(existing []any, overlay []map[string]any) []any { + out := make([]any, 0) // no capacity hint: len(a)+len(b) trips CodeQL allocation-size-overflow + index := map[string]int{} // mergeKey → index in out + for _, n := range existing { + nm, ok := n.(map[string]any) + if !ok { + out = append(out, n) + continue + } + k := networkMergeKey(nm) + if k != "" { + index[k] = len(out) + } + out = append(out, cloneMap(nm)) + } + for _, n := range overlay { + // Deep-ish copy so later mutations don't touch the overlay struct + entry := cloneMap(n) + k := networkMergeKey(entry) + if k != "" { + if i, ok := index[k]; ok { + out[i] = entry + continue + } + index[k] = len(out) + } + out = append(out, entry) + } + return out +} + +func mergeUpstreamsByID(existing []any, overlay []map[string]any) []any { + out := make([]any, 0) // no capacity hint: len(a)+len(b) trips CodeQL allocation-size-overflow + index := map[string]int{} + for _, u := range existing { + um, ok := u.(map[string]any) + if !ok { + out = append(out, u) + continue + } + id, _ := um["id"].(string) + if id != "" { + index[id] = len(out) + } + out = append(out, cloneMap(um)) + } + for _, u := range overlay { + entry := cloneMap(u) + id, _ := entry["id"].(string) + if id != "" { + if i, ok := index[id]; ok { + out[i] = entry + continue + } + index[id] = len(out) + } + out = append(out, entry) + } + return out +} + +func mergeRateLimiters(existing, overlay map[string]any) map[string]any { + if existing == nil { + existing = map[string]any{} + } + out := cloneMap(existing) + // budgets: merge by id + if ob, ok := overlay["budgets"]; ok { + out["budgets"] = mergeBudgetList(out["budgets"], ob) + } + for k, v := range overlay { + if k == "budgets" { + continue + } + out[k] = v + } + return out +} + +func mergeBudgetList(existing any, overlay any) []any { + var out []any + index := map[string]int{} + appendBudget := func(b any) { + bm, ok := b.(map[string]any) + if !ok { + out = append(out, b) + return + } + id, _ := bm["id"].(string) + entry := cloneMap(bm) + if id != "" { + if i, ok := index[id]; ok { + out[i] = entry + return + } + index[id] = len(out) + } + out = append(out, entry) + } + switch e := existing.(type) { + case []any: + for _, b := range e { + appendBudget(b) + } + } + switch o := overlay.(type) { + case []any: + for _, b := range o { + appendBudget(b) + } + case []map[string]any: + for _, b := range o { + appendBudget(b) + } + } + return out +} + +func mergeCachePolicies(erpcConfig map[string]any, add []map[string]any) error { + db, _ := erpcConfig["database"].(map[string]any) + if db == nil { + db = map[string]any{} + erpcConfig["database"] = db + } + cache, _ := db["evmJsonRpcCache"].(map[string]any) + if cache == nil { + cache = map[string]any{} + db["evmJsonRpcCache"] = cache + } + policies, _ := cache["policies"].([]any) + seen := map[string]struct{}{} + for _, p := range policies { + pm, ok := p.(map[string]any) + if !ok { + continue + } + seen[cachePolicyKey(pm)] = struct{}{} + } + for _, p := range add { + entry := cloneMap(p) + k := cachePolicyKey(entry) + if _, ok := seen[k]; ok { + // Replace existing policy with same key + for i, cur := range policies { + cm, ok := cur.(map[string]any) + if ok && cachePolicyKey(cm) == k { + policies[i] = entry + break + } + } + continue + } + policies = append(policies, entry) + seen[k] = struct{}{} + } + cache["policies"] = policies + return nil +} + +func cachePolicyKey(p map[string]any) string { + return fmt.Sprintf("%v|%v|%v", p["network"], p["method"], p["finality"]) +} + +func networkMergeKey(n map[string]any) string { + if evm, ok := n["evm"].(map[string]any); ok { + if cid := yamlInt(evm["chainId"]); cid != 0 { + return fmt.Sprintf("chain:%d", cid) + } + } + if alias, ok := n["alias"].(string); ok && alias != "" { + return "alias:" + alias + } + return "" +} + +func describeNetwork(n map[string]any) string { + alias, _ := n["alias"].(string) + cid := 0 + if evm, ok := n["evm"].(map[string]any); ok { + cid = yamlInt(evm["chainId"]) + } + switch { + case alias != "" && cid != 0: + return fmt.Sprintf("%s (chainId %d)", alias, cid) + case alias != "": + return alias + case cid != 0: + return fmt.Sprintf("chainId %d", cid) + default: + return "(unnamed network)" + } +} + +func countRateLimitBudgets(rl map[string]any) int { + if rl == nil { + return 0 + } + switch b := rl["budgets"].(type) { + case []any: + return len(b) + case []map[string]any: + return len(b) + default: + return 0 + } +} + +func cloneMap(m map[string]any) map[string]any { + // YAML round-trip keeps nested maps as map[string]any for our purposes + // without needing a full deep-copy library. + b, err := yaml.Marshal(m) + if err != nil { + out := make(map[string]any, len(m)) + for k, v := range m { + out[k] = v + } + return out + } + var out map[string]any + if err := yaml.Unmarshal(b, &out); err != nil { + out = make(map[string]any, len(m)) + for k, v := range m { + out[k] = v + } + } + return out +} + +func annotateERPCOverlay(cfg *config.Config, hash, source string) error { + if err := kubectl.EnsureCluster(cfg); err != nil { + return err + } + kubectlBin, kubeconfigPath := kubectl.Paths(cfg) + // Clear when both empty + if hash == "" && source == "" { + _ = kubectl.RunSilent(kubectlBin, kubeconfigPath, + "annotate", "configmap", erpcConfigMapName, "-n", erpcNamespace, + erpcOverlayAnnotKey+"-", erpcOverlayAnnotSource+"-", "--overwrite") + return nil + } + args := []string{ + "annotate", "configmap", erpcConfigMapName, "-n", erpcNamespace, + "--overwrite", + } + if hash != "" { + args = append(args, erpcOverlayAnnotKey+"="+hash) + } + if source != "" { + args = append(args, erpcOverlayAnnotSource+"="+source) + } + return kubectl.RunSilent(kubectlBin, kubeconfigPath, args...) +} diff --git a/internal/network/overlay_test.go b/internal/network/overlay_test.go new file mode 100644 index 00000000..ca28d669 --- /dev/null +++ b/internal/network/overlay_test.go @@ -0,0 +1,507 @@ +package network + +import ( + "errors" + "os" + "path/filepath" + "testing" + + "github.com/ObolNetwork/obol-stack/internal/config" + "gopkg.in/yaml.v3" +) + +func TestParseERPCOverlay_RequiresContent(t *testing.T) { + _, err := parseERPCOverlay([]byte("version: 1\n")) + if err == nil { + t.Fatal("expected empty overlay to fail") + } +} + +func TestParseERPCOverlay_RequiresUpstreamID(t *testing.T) { + _, err := parseERPCOverlay([]byte(` +version: 1 +upstreams: + - endpoint: https://example.com +`)) + if err == nil { + t.Fatal("expected missing id to fail") + } +} + +func TestParseERPCOverlay_RejectsNetworkWithoutMergeKey(t *testing.T) { + _, err := parseERPCOverlay([]byte(` +version: 1 +networks: + - architecture: evm + failsafe: + timeout: + duration: 10s +`)) + if err == nil { + t.Fatal("expected network without evm.chainId or alias to fail") + } +} + +func TestParseERPCOverlay_RejectsBadChainIDType(t *testing.T) { + _, err := parseERPCOverlay([]byte(` +version: 1 +networks: + - alias: hyperevm + evm: + chainId: "not-a-number" +`)) + if err == nil { + t.Fatal("expected non-numeric chainId to fail") + } +} + +func TestERPCOverlayRoundTrip(t *testing.T) { + cfg := &config.Config{ConfigDir: t.TempDir()} + ov := &ERPCOverlay{ + Version: 1, + Networks: []map[string]any{ + { + "alias": "hyperevm", + "architecture": "evm", + "evm": map[string]any{"chainId": 999}, + }, + }, + Upstreams: []map[string]any{ + { + "id": "local-hl-node", + "endpoint": "http://192.168.50.21:3001/evm", + "evm": map[string]any{"chainId": 999}, + }, + { + "id": "hyperevm-official", + "endpoint": "https://rpc.hyperliquid.xyz/evm", + "evm": map[string]any{"chainId": 999}, + }, + }, + RateLimiters: map[string]any{ + "budgets": []any{ + map[string]any{ + "id": "hyperevm-official", + "rules": []any{ + map[string]any{"method": "*", "maxCount": 100, "period": "second"}, + }, + }, + }, + }, + CachePoliciesAdd: []map[string]any{ + {"network": "*", "method": "eth_call", "finality": "realtime", "connector": "memory-cache", "ttl": "2s"}, + }, + } + if err := writeERPCOverlay(cfg, ov); err != nil { + t.Fatal(err) + } + info, err := os.Stat(erpcOverlayPath(cfg)) + if err != nil { + t.Fatal(err) + } + if info.Mode().Perm() != 0o600 { + t.Errorf("mode %v, want 0600", info.Mode().Perm()) + } + + got, err := readERPCOverlay(cfg) + if err != nil || got == nil { + t.Fatalf("read: %v %v", got, err) + } + if len(got.Networks) != 1 || len(got.Upstreams) != 2 { + t.Fatalf("round-trip sizes: nets=%d ups=%d", len(got.Networks), len(got.Upstreams)) + } + if got.Upstreams[0]["id"] != "local-hl-node" { + t.Errorf("upstream id = %v", got.Upstreams[0]["id"]) + } + + st, err := StatusERPC(cfg) + if err != nil || !st.Present { + t.Fatalf("status: %+v err=%v", st, err) + } + if st.UpstreamCount != 2 || st.NetworkCount != 1 || st.ContentHash == "" { + t.Errorf("status incomplete: %+v", st) + } +} + +func TestMergeERPCOverlay_Idempotent(t *testing.T) { + baseYAML := ` +logLevel: debug +database: + evmJsonRpcCache: + connectors: + - id: memory-cache + driver: memory + policies: + - network: "*" + method: "*" + finality: unfinalized + connector: memory-cache + ttl: 10s +projects: + - id: rpc + networks: + - alias: base + architecture: evm + evm: + chainId: 8453 + upstreams: + - id: obol-rpc-base + endpoint: https://example.com/base + evm: + chainId: 8453 +` + var erpcConfig map[string]any + if err := yaml.Unmarshal([]byte(baseYAML), &erpcConfig); err != nil { + t.Fatal(err) + } + + ov := &ERPCOverlay{ + Version: 1, + Networks: []map[string]any{ + { + "alias": "hyperevm", + "architecture": "evm", + "evm": map[string]any{"chainId": 999}, + "failsafe": map[string]any{"timeout": map[string]any{"duration": "10s"}}, + }, + }, + Upstreams: []map[string]any{ + { + "id": "local-hl-node", + "endpoint": "http://192.168.50.21:3001/evm", + "evm": map[string]any{"chainId": 999}, + }, + { + "id": "hyperevm-official", + "endpoint": "https://rpc.hyperliquid.xyz/evm", + "evm": map[string]any{"chainId": 999}, + "rateLimitBudget": "hyperevm-official", + }, + }, + RateLimiters: map[string]any{ + "budgets": []any{ + map[string]any{"id": "hyperevm-official", "rules": []any{}}, + }, + }, + CachePoliciesAdd: []map[string]any{ + {"network": "*", "method": "eth_call", "finality": "realtime", "ttl": "2s"}, + }, + } + + if err := mergeERPCOverlay(erpcConfig, ov); err != nil { + t.Fatal(err) + } + // Second apply must be idempotent (no duplicate ids) + if err := mergeERPCOverlay(erpcConfig, ov); err != nil { + t.Fatal(err) + } + + project := erpcConfig["projects"].([]any)[0].(map[string]any) + networks := project["networks"].([]any) + upstreams := project["upstreams"].([]any) + + if len(networks) != 2 { + t.Fatalf("networks = %d, want 2 (base + hyperevm)", len(networks)) + } + if len(upstreams) != 3 { + t.Fatalf("upstreams = %d, want 3 (base + 2 overlay)", len(upstreams)) + } + + // base preserved + ids := map[string]bool{} + for _, u := range upstreams { + ids[u.(map[string]any)["id"].(string)] = true + } + for _, want := range []string{"obol-rpc-base", "local-hl-node", "hyperevm-official"} { + if !ids[want] { + t.Errorf("missing upstream %s", want) + } + } + + // hyperevm network present + foundHyper := false + for _, n := range networks { + nm := n.(map[string]any) + if nm["alias"] == "hyperevm" && yamlInt(nm["evm"].(map[string]any)["chainId"]) == 999 { + foundHyper = true + } + } + if !foundHyper { + t.Error("hyperevm network missing") + } + + // rate limiters + rl := erpcConfig["rateLimiters"].(map[string]any) + budgets := rl["budgets"].([]any) + if len(budgets) != 1 { + t.Fatalf("budgets = %d, want 1", len(budgets)) + } + + // cache policy added once + policies := erpcConfig["database"].(map[string]any)["evmJsonRpcCache"].(map[string]any)["policies"].([]any) + if len(policies) != 2 { + t.Fatalf("cache policies = %d, want 2 (base unfinalized + realtime eth_call)", len(policies)) + } +} + +func TestMergeERPCOverlay_ReplacesByChainIDAndUpstreamID(t *testing.T) { + erpcConfig := map[string]any{ + "projects": []any{ + map[string]any{ + "id": "rpc", + "networks": []any{ + map[string]any{ + "alias": "hyperevm", + "evm": map[string]any{"chainId": 999}, + "failsafe": map[string]any{ + "timeout": map[string]any{"duration": "30s"}, + }, + }, + }, + "upstreams": []any{ + map[string]any{ + "id": "local-hl-node", + "endpoint": "http://old:3001/evm", + "evm": map[string]any{"chainId": 999}, + }, + }, + }, + }, + } + ov := &ERPCOverlay{ + Version: 1, + Networks: []map[string]any{ + { + "alias": "hyperevm", + "evm": map[string]any{"chainId": 999}, + "failsafe": map[string]any{ + "timeout": map[string]any{"duration": "10s"}, + }, + }, + }, + Upstreams: []map[string]any{ + { + "id": "local-hl-node", + "endpoint": "http://192.168.50.21:3001/evm", + "evm": map[string]any{"chainId": 999}, + }, + }, + } + if err := mergeERPCOverlay(erpcConfig, ov); err != nil { + t.Fatal(err) + } + project := erpcConfig["projects"].([]any)[0].(map[string]any) + networks := project["networks"].([]any) + upstreams := project["upstreams"].([]any) + if len(networks) != 1 || len(upstreams) != 1 { + t.Fatalf("replace must not duplicate: nets=%d ups=%d", len(networks), len(upstreams)) + } + ep := upstreams[0].(map[string]any)["endpoint"] + if ep != "http://192.168.50.21:3001/evm" { + t.Errorf("endpoint not replaced: %v", ep) + } + to := networks[0].(map[string]any)["failsafe"].(map[string]any)["timeout"].(map[string]any)["duration"] + if to != "10s" { + t.Errorf("network failsafe not replaced: %v", to) + } +} + +func TestStripERPCOverlay(t *testing.T) { + erpcConfig := map[string]any{ + "projects": []any{ + map[string]any{ + "id": "rpc", + "networks": []any{ + map[string]any{"alias": "base", "evm": map[string]any{"chainId": 8453}}, + map[string]any{"alias": "hyperevm", "evm": map[string]any{"chainId": 999}}, + }, + "upstreams": []any{ + map[string]any{"id": "obol-rpc-base", "evm": map[string]any{"chainId": 8453}}, + map[string]any{"id": "local-hl-node", "evm": map[string]any{"chainId": 999}}, + }, + }, + }, + } + ov := &ERPCOverlay{ + Version: 1, + Networks: []map[string]any{ + {"alias": "hyperevm", "evm": map[string]any{"chainId": 999}}, + }, + Upstreams: []map[string]any{ + {"id": "local-hl-node", "endpoint": "http://x", "evm": map[string]any{"chainId": 999}}, + }, + } + if err := stripERPCOverlay(erpcConfig, ov, nil); err != nil { + t.Fatal(err) + } + project := erpcConfig["projects"].([]any)[0].(map[string]any) + if len(project["networks"].([]any)) != 1 { + t.Fatalf("networks after strip: %v", project["networks"]) + } + if len(project["upstreams"].([]any)) != 1 { + t.Fatalf("upstreams after strip: %v", project["upstreams"]) + } + if project["upstreams"].([]any)[0].(map[string]any)["id"] != "obol-rpc-base" { + t.Error("base upstream must remain") + } +} + +// TestResetERPC_RestoresBaseEntriesOverlayReplaced covers the reset +// provenance fix: an overlay entry whose key collides with a chart-base +// entry must be RESTORED (not deleted) on reset, while an entry the +// overlay purely added is still dropped. +func TestResetERPC_RestoresBaseEntriesOverlayReplaced(t *testing.T) { + cfg := &config.Config{ConfigDir: t.TempDir()} + + // Chart-base cluster state: one network/upstream at the SAME key the + // operator overlay is about to replace. + erpcConfig := map[string]any{ + "projects": []any{ + map[string]any{ + "id": "rpc", + "networks": []any{ + map[string]any{ + "alias": "hyperevm", + "evm": map[string]any{"chainId": 999}, + "failsafe": map[string]any{ + "timeout": map[string]any{"duration": "30s"}, + }, + }, + }, + "upstreams": []any{ + map[string]any{ + "id": "hyperevm-official", + "endpoint": "https://base-rpc.example.com/evm", + "evm": map[string]any{"chainId": 999}, + }, + }, + }, + }, + } + + ov := &ERPCOverlay{ + Version: 1, + Networks: []map[string]any{ + { // replaces the base "hyperevm" network + "alias": "hyperevm", + "evm": map[string]any{"chainId": 999}, + "failsafe": map[string]any{ + "timeout": map[string]any{"duration": "5s"}, + }, + }, + { // purely additive — no base collision + "alias": "brandnew", + "evm": map[string]any{"chainId": 111}, + }, + }, + Upstreams: []map[string]any{ + { // replaces the base "hyperevm-official" upstream + "id": "hyperevm-official", + "endpoint": "https://overlay-rpc.example.com/evm", + "evm": map[string]any{"chainId": 999}, + }, + { // purely additive + "id": "brand-new-id", + "endpoint": "https://new.example.com", + }, + }, + } + + // Mirrors applyOverlayToCluster: snapshot provenance BEFORE merging. + if err := captureERPCProvenance(cfg, erpcConfig, ov); err != nil { + t.Fatal(err) + } + if err := mergeERPCOverlay(erpcConfig, ov); err != nil { + t.Fatal(err) + } + + project := erpcConfig["projects"].([]any)[0].(map[string]any) + if len(project["networks"].([]any)) != 2 || len(project["upstreams"].([]any)) != 2 { + t.Fatalf("merge shape unexpected: networks=%v upstreams=%v", project["networks"], project["upstreams"]) + } + + // Mirrors removeOverlayFromCluster: read provenance back and strip. + prov, err := readERPCProvenance(cfg) + if err != nil { + t.Fatal(err) + } + if err := stripERPCOverlay(erpcConfig, ov, prov); err != nil { + t.Fatal(err) + } + + project = erpcConfig["projects"].([]any)[0].(map[string]any) + upstreams := project["upstreams"].([]any) + if len(upstreams) != 1 { + t.Fatalf("upstreams after reset = %d, want 1 (base restored, addition dropped)", len(upstreams)) + } + um := upstreams[0].(map[string]any) + if um["id"] != "hyperevm-official" || um["endpoint"] != "https://base-rpc.example.com/evm" { + t.Errorf("base upstream not restored, got %+v", um) + } + + networks := project["networks"].([]any) + if len(networks) != 1 { + t.Fatalf("networks after reset = %d, want 1 (base restored, addition dropped)", len(networks)) + } + nm := networks[0].(map[string]any) + to := nm["failsafe"].(map[string]any)["timeout"].(map[string]any)["duration"] + if to != "30s" { + t.Errorf("base network failsafe not restored, got duration=%v", to) + } +} + +func TestERPCOverlayDriftStatus(t *testing.T) { + cases := []struct { + name string + localHash string + clusterHash string + clusterErr error + want string + }{ + {"in sync", "abc123", "abc123", nil, ERPCSyncInSync}, + {"drifted", "abc123", "def456", nil, ERPCSyncDrifted}, + {"not applied", "abc123", "", nil, ERPCSyncNotApplied}, + {"cluster unreachable", "abc123", "", errors.New("no cluster"), ""}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := erpcOverlayDriftStatus(tc.localHash, tc.clusterHash, tc.clusterErr) + if got != tc.want { + t.Errorf("got %q, want %q", got, tc.want) + } + }) + } +} + +func TestApplyERPCOverlayFile_PersistsThenReadable(t *testing.T) { + cfg := &config.Config{ConfigDir: t.TempDir()} + src := filepath.Join(t.TempDir(), "basket.yaml") + body := []byte(` +version: 1 +networks: + - alias: hyperevm + architecture: evm + evm: + chainId: 999 +upstreams: + - id: hyperevm-official + endpoint: https://rpc.hyperliquid.xyz/evm + evm: + chainId: 999 +`) + if err := os.WriteFile(src, body, 0o600); err != nil { + t.Fatal(err) + } + // applyOverlayToCluster needs a cluster — only test parse+persist via write path + ov, err := parseERPCOverlay(body) + if err != nil { + t.Fatal(err) + } + if err := writeERPCOverlay(cfg, ov); err != nil { + t.Fatal(err) + } + got, err := readERPCOverlay(cfg) + if err != nil || got == nil || len(got.Upstreams) != 1 { + t.Fatalf("persist failed: %+v %v", got, err) + } +} diff --git a/internal/openclaw/import.go b/internal/openclaw/import.go index f08e9966..e7a93d8c 100644 --- a/internal/openclaw/import.go +++ b/internal/openclaw/import.go @@ -8,6 +8,7 @@ import ( "strings" "github.com/ObolNetwork/obol-stack/internal/model" + "gopkg.in/yaml.v3" ) // API key environment variable names for known providers. @@ -217,14 +218,14 @@ func TranslateToOverlayYAML(result *ImportResult) string { var b strings.Builder if result.AgentModel != "" { - fmt.Fprintf(&b, "openclaw:\n agentModel: %s\n\n", result.AgentModel) + fmt.Fprintf(&b, "openclaw:\n agentModel: %s\n\n", yamlScalar(result.AgentModel)) } if len(result.Providers) > 0 { b.WriteString("models:\n") for _, p := range result.Providers { - fmt.Fprintf(&b, " %s:\n", p.Name) + fmt.Fprintf(&b, " %s:\n", yamlScalar(p.Name)) if p.Disabled { b.WriteString(" enabled: false\n") @@ -234,7 +235,7 @@ func TranslateToOverlayYAML(result *ImportResult) string { b.WriteString(" enabled: true\n") if p.BaseURL != "" { - fmt.Fprintf(&b, " baseUrl: %s\n", p.BaseURL) + fmt.Fprintf(&b, " baseUrl: %s\n", yamlScalar(p.BaseURL)) } // Always emit api to override any stale base chart value. // Empty string makes the Helm template omit it from JSON, @@ -246,17 +247,17 @@ func TranslateToOverlayYAML(result *ImportResult) string { } if p.APIKeyEnvVar != "" { - fmt.Fprintf(&b, " apiKeyEnvVar: %s\n", p.APIKeyEnvVar) + fmt.Fprintf(&b, " apiKeyEnvVar: %s\n", yamlScalar(p.APIKeyEnvVar)) } if len(p.Models) > 0 { b.WriteString(" models:\n") for _, m := range p.Models { - fmt.Fprintf(&b, " - id: %s\n", m.ID) + fmt.Fprintf(&b, " - id: %s\n", yamlScalar(m.ID)) if m.Name != "" { - fmt.Fprintf(&b, " name: %s\n", m.Name) + fmt.Fprintf(&b, " name: %s\n", yamlScalar(m.Name)) } } } @@ -457,3 +458,23 @@ func extractEnvVarName(s string) (string, bool) { func isEnvVarRef(s string) bool { return strings.Contains(s, "${") } + +// yamlScalar renders s as a double-quoted YAML scalar so it can be safely +// interpolated into the hand-built overlay YAML. Values sourced from an +// imported ~/.openclaw/openclaw.json are untrusted: without this, an +// embedded newline (e.g. "x\nimage:\n repository: evil") would let the +// string break out of its field and inject new keys into values-obol.yaml, +// which is later applied to the cluster via `helmfile sync`. +func yamlScalar(s string) string { + node := yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: s, Style: yaml.DoubleQuotedStyle} + + out, err := yaml.Marshal(&node) + if err != nil { + // Unreachable for a plain string scalar; fall back to a manual + // double-quote/escape rather than emit the untrusted value raw. + r := strings.NewReplacer(`\`, `\\`, `"`, `\"`, "\n", `\n`, "\r", `\r`, "\t", `\t`) + return `"` + r.Replace(s) + `"` + } + + return strings.TrimSuffix(string(out), "\n") +} diff --git a/internal/openclaw/import_test.go b/internal/openclaw/import_test.go index 78fbb897..dedc4d27 100644 --- a/internal/openclaw/import_test.go +++ b/internal/openclaw/import_test.go @@ -6,6 +6,8 @@ import ( "path/filepath" "strings" "testing" + + "gopkg.in/yaml.v3" ) func TestIsEnvVarRef(t *testing.T) { @@ -226,7 +228,7 @@ func TestTranslateToOverlayYAML_AgentModelOnly(t *testing.T) { } got := TranslateToOverlayYAML(result) - if !strings.Contains(got, "agentModel: claude-sonnet-4-6") { + if !strings.Contains(got, `agentModel: "claude-sonnet-4-6"`) { t.Errorf("YAML missing agentModel, got:\n%s", got) } @@ -252,11 +254,11 @@ func TestTranslateToOverlayYAML_ProviderWithModels(t *testing.T) { got := TranslateToOverlayYAML(result) checks := []string{ - "anthropic:\n enabled: true", - "baseUrl: https://api.anthropic.com", + "\"anthropic\":\n enabled: true", + `baseUrl: "https://api.anthropic.com"`, "api: anthropic-messages", - "- id: claude-sonnet-4-6", - "name: Claude Sonnet 4.6", + `- id: "claude-sonnet-4-6"`, + `name: "Claude Sonnet 4.6"`, } for _, check := range checks { if !strings.Contains(got, check) { @@ -273,7 +275,7 @@ func TestTranslateToOverlayYAML_DisabledProvider(t *testing.T) { } got := TranslateToOverlayYAML(result) - if !strings.Contains(got, "openai:\n enabled: false") { + if !strings.Contains(got, "\"openai\":\n enabled: false") { t.Errorf("YAML missing disabled openai, got:\n%s", got) } @@ -346,15 +348,15 @@ func TestTranslateToOverlayYAML_FullConfig(t *testing.T) { } got := TranslateToOverlayYAML(result) - if !strings.Contains(got, "agentModel: claude-sonnet-4-6") { + if !strings.Contains(got, `agentModel: "claude-sonnet-4-6"`) { t.Errorf("YAML missing agentModel, got:\n%s", got) } - if !strings.Contains(got, "anthropic:\n enabled: true") { + if !strings.Contains(got, "\"anthropic\":\n enabled: true") { t.Errorf("YAML missing enabled anthropic, got:\n%s", got) } - if !strings.Contains(got, "openai:\n enabled: false") { + if !strings.Contains(got, "\"openai\":\n enabled: false") { t.Errorf("YAML missing disabled openai, got:\n%s", got) } @@ -617,3 +619,79 @@ func TestDetectExistingConfigAt_EmptyConfig(t *testing.T) { t.Errorf("AgentModel = %q, want empty", result.AgentModel) } } + +// TestYAMLScalar_EscapesInjection is a Canary402 full-surface audit +// regression test: values imported from ~/.openclaw/openclaw.json must never +// be able to inject additional YAML keys (e.g. overriding `image:` to +// achieve RCE via `helmfile sync`) when hand-formatted into the overlay +// values file. +func TestYAMLScalar_EscapesInjection(t *testing.T) { + tests := []struct { + name string + in string + }{ + {"plain", "openai/claude-sonnet-4-6"}, + {"newline_key_injection", "x\nimage:\n repository: evil"}, + {"colon", "has: colon"}, + {"quotes_and_backslash", `back\slash"quote`}, + {"empty", ""}, + {"trailing_newline_injection", "gpt-5.2\nrbac:\n create: false"}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + line := "key: " + yamlScalar(tt.in) + "\n" + + var doc map[string]any + if err := yaml.Unmarshal([]byte(line), &doc); err != nil { + t.Fatalf("yamlScalar(%q) produced invalid YAML %q: %v", tt.in, line, err) + } + + if got, ok := doc["key"].(string); !ok || got != tt.in { + t.Errorf("round-trip = %#v, want single scalar %q (no injected keys); rendered: %q", doc["key"], tt.in, line) + } + + if len(doc) != 1 { + t.Errorf("doc has %d top-level keys, want 1 (injection leaked extra keys): %#v", len(doc), doc) + } + }) + } +} + +// TestGenerateOverlayValues_RejectsYAMLInjection feeds generateOverlayValues +// a malicious imported agentModel and provider name (newline + "image:") +// and asserts the rendered values-obol.yaml parses to the SAME single +// scalar values, with no injected top-level "image" key — i.e. the fix for +// the confirmed Canary402 full-surface audit finding holds end-to-end. +func TestGenerateOverlayValues_RejectsYAMLInjection(t *testing.T) { + maliciousModel := "x\nimage:\n repository: pwned-by-canary402\nrbac:\n create: false" + maliciousProvider := "ollama\nimage:\n repository: pwned-by-canary402" + + imported := &ImportResult{ + AgentModel: maliciousModel, + Providers: []ImportedProvider{ + {Name: maliciousProvider, BaseURL: "http://localhost:11434"}, + }, + } + + rendered := generateOverlayValues(testConfig(t), "openclaw-default.obol.stack", imported, false, nil, "") + + var doc map[string]any + if err := yaml.Unmarshal([]byte(rendered), &doc); err != nil { + t.Fatalf("rendered overlay is not valid YAML: %v\n%s", err, rendered) + } + + // The chart-default image override is legitimate and expected to stay a + // map with only "tag" — injection would add a "repository" key to it. + if img, ok := doc["image"].(map[string]any); ok { + if _, hasRepo := img["repository"]; hasRepo { + t.Errorf("attacker injected an 'image.repository' key: %#v\nrendered:\n%s", img, rendered) + } + } + + openclawSection, _ := doc["openclaw"].(map[string]any) + if openclawSection == nil || openclawSection["agentModel"] != "openai/"+maliciousModel { + t.Errorf("agentModel = %#v, want single scalar %q (rewritten with openai/ prefix); rendered:\n%s", + openclawSection["agentModel"], "openai/"+maliciousModel, rendered) + } +} diff --git a/internal/openclaw/onboard_test.go b/internal/openclaw/onboard_test.go new file mode 100644 index 00000000..52deeef0 --- /dev/null +++ b/internal/openclaw/onboard_test.go @@ -0,0 +1,49 @@ +package openclaw + +import ( + "strings" + "testing" + + "github.com/ObolNetwork/obol-stack/internal/agentruntime" + "github.com/ObolNetwork/obol-stack/internal/ui" +) + +// TestOnboardRejectsUnsafeID guards against the Canary402 full-surface audit +// finding: an unsanitized --id becomes a DNS label (hostname, namespace) and +// was written verbatim into /etc/hosts, so a newline in --id could inject +// arbitrary /etc/hosts lines via "sudo tee". Onboard must reject any id that +// isn't a safe DNS label before it reaches deployment/hostname construction. +func TestOnboardRejectsUnsafeID(t *testing.T) { + invalid := []string{ + "evil\n127.0.0.1 attacker.com", // /etc/hosts line injection + "has space", + "has/slash", + "-leading-dash", + } + for _, id := range invalid { + cfg := testConfig(t) + err := Onboard(cfg, OnboardOptions{ID: id}, ui.New(false)) + if err == nil { + t.Errorf("Onboard(ID=%q) = nil, want error", id) + } + } +} + +// TestOnboardRejectsIDTooLongForDNSLabel guards against the "openclaw-" +// DNS label overflowing 63 characters: validate.Name alone allows a 63-char +// id, but Onboard prepends "openclaw-" to it, so an id at (or near) that +// limit must be rejected here with a clear error instead of failing later +// with an opaque Kubernetes error. +func TestOnboardRejectsIDTooLongForDNSLabel(t *testing.T) { + max := agentruntime.MaxIDLength(agentruntime.OpenClaw) + + tooLong := "a" + strings.Repeat("b", max) // max+1 chars, still a valid DNS label on its own + cfg := testConfig(t) + err := Onboard(cfg, OnboardOptions{ID: tooLong}, ui.New(false)) + if err == nil { + t.Fatalf("Onboard(ID=%d chars) = nil, want error", len(tooLong)) + } + if !strings.Contains(err.Error(), "too long") { + t.Errorf("error should explain the id is too long, got: %v", err) + } +} diff --git a/internal/openclaw/openclaw.go b/internal/openclaw/openclaw.go index 3fba946c..662c0987 100644 --- a/internal/openclaw/openclaw.go +++ b/internal/openclaw/openclaw.go @@ -28,6 +28,7 @@ import ( "github.com/ObolNetwork/obol-stack/internal/model" "github.com/ObolNetwork/obol-stack/internal/tunnel" "github.com/ObolNetwork/obol-stack/internal/ui" + "github.com/ObolNetwork/obol-stack/internal/validate" petname "github.com/dustinkirkland/golang-petname" ) @@ -168,6 +169,19 @@ func Onboard(cfg *config.Config, opts OnboardOptions, u *ui.UI) error { u.Infof("Using deployment ID: %s", id) } + // id becomes a DNS label (hostname, namespace) below, so it must be + // restricted to a safe charset — an unsanitized id (e.g. containing a + // newline) could otherwise inject arbitrary /etc/hosts entries. + if err := validate.Name(id); err != nil { + return fmt.Errorf("invalid agent id: %w", err) + } + // validate.Name alone allows ids up to 63 chars, but Onboard derives the + // "openclaw-" DNS label below — bound id here so that stays ≤63 + // instead of failing later with an opaque Kubernetes error. + if max := agentruntime.MaxIDLength(agentruntime.OpenClaw); len(id) > max { + return fmt.Errorf("agent id %q is too long (%d chars): must be at most %d chars so %q- fits the 63-character DNS label limit", id, len(id), max, appName) + } + deploymentDir := DeploymentPath(cfg, id) // Idempotent re-run for default deployment: just re-sync @@ -1914,8 +1928,8 @@ func patchOverlayModelList(content string, models []string) (string, bool) { } else { result = append(result, indent+"models:") for _, m := range models { - result = append(result, indent+" - id: "+m) - result = append(result, indent+" name: "+ollamaModelDisplayName(m)) + result = append(result, indent+" - id: "+yamlScalar(m)) + result = append(result, indent+" name: "+yamlScalar(ollamaModelDisplayName(m))) } } @@ -2035,7 +2049,7 @@ rbac: b.WriteString("openclaw:\n") if agentModel != "" { - fmt.Fprintf(&b, " agentModel: %s\n", agentModel) + fmt.Fprintf(&b, " agentModel: %s\n", yamlScalar(agentModel)) } b.WriteString(` gateway: @@ -2062,7 +2076,7 @@ models: b.WriteString(" models:\n") for _, m := range ollamaModels { - fmt.Fprintf(&b, " - id: %s\n name: %s\n", m, ollamaModelDisplayName(m)) + fmt.Fprintf(&b, " - id: %s\n name: %s\n", yamlScalar(m), yamlScalar(ollamaModelDisplayName(m))) } } else { b.WriteString(" models: []\n") diff --git a/internal/openclaw/overlay_test.go b/internal/openclaw/overlay_test.go index 18b1ed55..49243083 100644 --- a/internal/openclaw/overlay_test.go +++ b/internal/openclaw/overlay_test.go @@ -8,6 +8,7 @@ import ( "testing" "github.com/ObolNetwork/obol-stack/internal/config" + "gopkg.in/yaml.v3" ) // testConfig creates a temp config dir with a .stack-id file for testing. @@ -111,21 +112,21 @@ func TestOverlayYAML_LiteLLMRouted(t *testing.T) { yaml := TranslateToOverlayYAML(result) // Agent model should have ollama/ prefix - if !strings.Contains(yaml, "agentModel: openai/claude-sonnet-4-5-20250929") { + if !strings.Contains(yaml, `agentModel: "openai/claude-sonnet-4-5-20250929"`) { t.Errorf("YAML missing agentModel, got:\n%s", yaml) } // openai should be enabled with LiteLLM baseUrl - if !strings.Contains(yaml, "openai:\n enabled: true") { + if !strings.Contains(yaml, "\"openai\":\n enabled: true") { t.Errorf("YAML missing enabled openai provider, got:\n%s", yaml) } - if !strings.Contains(yaml, "baseUrl: http://litellm.llm.svc.cluster.local:4000/v1") { + if !strings.Contains(yaml, `baseUrl: "http://litellm.llm.svc.cluster.local:4000/v1"`) { t.Errorf("YAML missing LiteLLM baseUrl, got:\n%s", yaml) } // apiKeyEnvVar should be OPENAI_API_KEY (LiteLLM master key injected via this env var) - if !strings.Contains(yaml, "apiKeyEnvVar: OPENAI_API_KEY") { + if !strings.Contains(yaml, `apiKeyEnvVar: "OPENAI_API_KEY"`) { t.Errorf("YAML missing apiKeyEnvVar, got:\n%s", yaml) } @@ -140,16 +141,16 @@ func TestOverlayYAML_LiteLLMRouted(t *testing.T) { } // Cloud model should appear in ollama's model list - if !strings.Contains(yaml, "- id: claude-sonnet-4-5-20250929") { + if !strings.Contains(yaml, `- id: "claude-sonnet-4-5-20250929"`) { t.Errorf("YAML missing cloud model ID, got:\n%s", yaml) } // anthropic and ollama should be disabled - if !strings.Contains(yaml, "anthropic:\n enabled: false") { + if !strings.Contains(yaml, "\"anthropic\":\n enabled: false") { t.Errorf("YAML missing disabled anthropic, got:\n%s", yaml) } - if !strings.Contains(yaml, "ollama:\n enabled: false") { + if !strings.Contains(yaml, "\"ollama\":\n enabled: false") { t.Errorf("YAML missing disabled ollama, got:\n%s", yaml) } } @@ -159,7 +160,7 @@ func TestGenerateOverlayValues_OllamaDefaultWithModels(t *testing.T) { models := []string{"llama3.2:3b", "mistral:7b"} yaml := generateOverlayValues(testConfig(t), "openclaw-default.obol.stack", nil, false, models, "") - if !strings.Contains(yaml, "agentModel: openai/llama3.2:3b") { + if !strings.Contains(yaml, `agentModel: "openai/llama3.2:3b"`) { t.Errorf("default overlay missing ollama agentModel, got:\n%s", yaml) } @@ -167,15 +168,53 @@ func TestGenerateOverlayValues_OllamaDefaultWithModels(t *testing.T) { t.Errorf("default overlay missing LiteLLM baseUrl, got:\n%s", yaml) } - if !strings.Contains(yaml, "id: llama3.2:3b") { + if !strings.Contains(yaml, `id: "llama3.2:3b"`) { t.Errorf("default overlay missing first model, got:\n%s", yaml) } - if !strings.Contains(yaml, "id: mistral:7b") { + if !strings.Contains(yaml, `id: "mistral:7b"`) { t.Errorf("default overlay missing second model, got:\n%s", yaml) } } +// TestGenerateOverlayValues_OllamaModelNameInjection is a #775-style +// regression test for a listOllamaModels() name (or its derived display +// name) breaking out of the hand-formatted "- id: %s\n name: %s\n" lines +// and injecting new YAML keys into the overlay applied via `helmfile sync`. +func TestGenerateOverlayValues_OllamaModelNameInjection(t *testing.T) { + malicious := "x\nimage:\n repository: pwned-by-ollama\nrbac:\n create: false" + + rendered := generateOverlayValues(testConfig(t), "openclaw-default.obol.stack", nil, false, []string{malicious}, "") + + var doc map[string]any + if err := yaml.Unmarshal([]byte(rendered), &doc); err != nil { + t.Fatalf("rendered overlay is not valid YAML: %v\n%s", err, rendered) + } + + if img, ok := doc["image"].(map[string]any); ok { + if _, hasRepo := img["repository"]; hasRepo { + t.Errorf("attacker injected an 'image.repository' key: %#v\nrendered:\n%s", img, rendered) + } + } + + modelsSection, _ := doc["models"].(map[string]any) + openai, _ := modelsSection["openai"].(map[string]any) + + modelList, _ := openai["models"].([]any) + if len(modelList) != 1 { + t.Fatalf("expected 1 model entry, got %d\nrendered:\n%s", len(modelList), rendered) + } + + entry, _ := modelList[0].(map[string]any) + if entry["id"] != malicious { + t.Errorf("id = %#v, want single scalar %q (no injected keys); rendered:\n%s", entry["id"], malicious, rendered) + } + + if entry["name"] != ollamaModelDisplayName(malicious) { + t.Errorf("name = %#v, want single scalar %q; rendered:\n%s", entry["name"], ollamaModelDisplayName(malicious), rendered) + } +} + func TestGenerateOverlayValues_OllamaDefaultNoModels(t *testing.T) { // When no Ollama models are available, overlay should have empty model list yaml := generateOverlayValues(testConfig(t), "openclaw-default.obol.stack", nil, false, nil, "") @@ -380,11 +419,11 @@ erpc: t.Fatal("expected change") } - if !strings.Contains(updated, "id: claude-sonnet-4-5-20250929") { + if !strings.Contains(updated, `id: "claude-sonnet-4-5-20250929"`) { t.Errorf("missing claude model in updated overlay:\n%s", updated) } - if !strings.Contains(updated, "id: gpt-4o") { + if !strings.Contains(updated, `id: "gpt-4o"`) { t.Errorf("missing gpt model in updated overlay:\n%s", updated) } // eRPC section should still be present @@ -427,10 +466,46 @@ erpc: t.Fatal("expected change") } - if !strings.Contains(updated, "id: llama3.2:3b") { + if !strings.Contains(updated, `id: "llama3.2:3b"`) { t.Errorf("missing model in updated overlay:\n%s", updated) } }) + + // #775-style regression: a malicious/malformed Ollama model name must not + // break out of the " - id: "+m hand-formatted line and inject new keys + // into the overlay applied via `helmfile sync`. + t.Run("model name injection", func(t *testing.T) { + malicious := "x\nimage:\n repository: pwned-by-ollama\nrbac:\n create: false" + + updated, changed := patchOverlayModelList(overlay, []string{malicious}) + if !changed { + t.Fatal("expected change") + } + + var doc map[string]any + if err := yaml.Unmarshal([]byte(updated), &doc); err != nil { + t.Fatalf("patched overlay is not valid YAML: %v\n%s", err, updated) + } + + if img, ok := doc["image"].(map[string]any); ok { + if _, hasRepo := img["repository"]; hasRepo { + t.Errorf("attacker injected an 'image.repository' key: %#v\nupdated:\n%s", img, updated) + } + } + + modelsSection, _ := doc["models"].(map[string]any) + openai, _ := modelsSection["openai"].(map[string]any) + + modelList, _ := openai["models"].([]any) + if len(modelList) != 1 { + t.Fatalf("expected 1 model entry, got %d\nupdated:\n%s", len(modelList), updated) + } + + entry, _ := modelList[0].(map[string]any) + if entry["id"] != malicious { + t.Errorf("id = %#v, want single scalar %q (no injected keys); updated:\n%s", entry["id"], malicious, updated) + } + }) } func TestPatchAgentModelsJSON(t *testing.T) { diff --git a/internal/serviceoffercontroller/agent_confighash_test.go b/internal/serviceoffercontroller/agent_confighash_test.go index 62029243..3966173f 100644 --- a/internal/serviceoffercontroller/agent_confighash_test.go +++ b/internal/serviceoffercontroller/agent_confighash_test.go @@ -30,32 +30,32 @@ func TestHermesConfigDecision(t *testing.T) { wantDrift: false, }, { - name: "annotation matches and data hashes to desired -> skip, no drift", - live: hermesConfigCM(t, "ns", desiredYAML, desiredHash, "1"), + name: "annotation matches and data hashes to desired -> skip, no drift", + live: hermesConfigCM(t, "ns", desiredYAML, desiredHash, "1"), wantSkip: true, wantDrift: false, }, { - name: "annotation matches but data hand-edited -> skip with drift", - live: hermesConfigCM(t, "ns", otherYAML, desiredHash, "1"), + name: "annotation matches but data hand-edited -> skip with drift", + live: hermesConfigCM(t, "ns", otherYAML, desiredHash, "1"), wantSkip: true, wantDrift: true, }, { - name: "annotation does not match desired -> apply", - live: hermesConfigCM(t, "ns", desiredYAML, otherHash, "1"), + name: "annotation does not match desired -> apply", + live: hermesConfigCM(t, "ns", desiredYAML, otherHash, "1"), wantSkip: false, wantDrift: false, }, { - name: "annotation absent (pre-feature migration) -> apply", - live: hermesConfigCM(t, "ns", desiredYAML, "", "1"), + name: "annotation absent (pre-feature migration) -> apply", + live: hermesConfigCM(t, "ns", desiredYAML, "", "1"), wantSkip: false, wantDrift: false, }, { - name: "annotation matches but data key missing -> skip with drift", - live: hermesConfigCMNoData(t, "ns", desiredHash, "1"), + name: "annotation matches but data key missing -> skip with drift", + live: hermesConfigCMNoData(t, "ns", desiredHash, "1"), wantSkip: true, wantDrift: true, }, diff --git a/internal/serviceoffercontroller/agent_resolver.go b/internal/serviceoffercontroller/agent_resolver.go index 6d397b3b..c3c2a41c 100644 --- a/internal/serviceoffercontroller/agent_resolver.go +++ b/internal/serviceoffercontroller/agent_resolver.go @@ -81,8 +81,12 @@ func (c *Controller) resolveAgentOffer(ctx context.Context, offer *monetizeapi.S offer.Spec.Upstream = monetizeapi.ServiceOfferUpstream{ Service: "hermes", Namespace: ref.Namespace, - Port: 8642, - HealthPath: "/api/status", + Port: hermesPort, + // Hermes API (port hermesPort) serves unauthenticated /health. + // /api/status is the dashboard probe path on a different port and + // returns 404 on the API — that used to slip past as "healthy" + // under the pre-2xx UpstreamHealthy gate. + HealthPath: hermesAPIPath, } offer.Spec.Model = monetizeapi.ServiceOfferModel{ Name: model, diff --git a/internal/serviceoffercontroller/catalog.go b/internal/serviceoffercontroller/catalog.go index a33cfad6..97327377 100644 --- a/internal/serviceoffercontroller/catalog.go +++ b/internal/serviceoffercontroller/catalog.go @@ -40,7 +40,7 @@ func (c *Controller) listServiceOffersForCatalog(ctx context.Context, override * return offers, nil } -func skillCatalogContentMatches(cm *unstructured.Unstructured, content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) bool { +func staticSiteContentMatches(cm *unstructured.Unstructured, content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) bool { if cm == nil { return false } @@ -75,22 +75,22 @@ func skillCatalogContentMatches(cm *unstructured.Unstructured, content, services return true } -func (c *Controller) skillCatalogContentUnchanged(ctx context.Context, content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) (bool, error) { - cm, err := c.configMaps.Namespace(skillCatalogNamespace).Get(ctx, skillCatalogConfigMapName, metav1.GetOptions{}) +func (c *Controller) staticSiteContentUnchanged(ctx context.Context, content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) (bool, error) { + cm, err := c.configMaps.Namespace(staticSiteNamespace).Get(ctx, staticSiteConfigMapName, metav1.GetOptions{}) if apierrors.IsNotFound(err) { return false, nil } if err != nil { return false, err } - return skillCatalogContentMatches(cm, content, servicesJSON, openAPIJSON, apiDocsHTML, bundles), nil + return staticSiteContentMatches(cm, content, servicesJSON, openAPIJSON, apiDocsHTML, bundles), nil } -func computeSkillCatalogContentHash(content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) string { +func computeStaticSiteContentHash(content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) string { return fmt.Sprintf("%x", md5Sum(content+servicesJSON+openAPIJSON+apiDocsHTML+bundleDigestInput(bundles)))[:8] } -func skillCatalogDeployedContentHash(deployment *unstructured.Unstructured) string { +func staticSiteDeployedContentHash(deployment *unstructured.Unstructured) string { if deployment == nil { return "" } diff --git a/internal/serviceoffercontroller/catalog_test.go b/internal/serviceoffercontroller/catalog_test.go index e596ae6c..184a5939 100644 --- a/internal/serviceoffercontroller/catalog_test.go +++ b/internal/serviceoffercontroller/catalog_test.go @@ -6,9 +6,9 @@ import ( "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" ) -func TestComputeSkillCatalogContentHashDeterministic(t *testing.T) { - a := computeSkillCatalogContentHash("# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) - b := computeSkillCatalogContentHash("# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) +func TestComputeStaticSiteContentHashDeterministic(t *testing.T) { + a := computeStaticSiteContentHash("# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) + b := computeStaticSiteContentHash("# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) if a != b { t.Fatalf("hash not deterministic: %q vs %q", a, b) } @@ -16,36 +16,36 @@ func TestComputeSkillCatalogContentHashDeterministic(t *testing.T) { t.Fatalf("hash length = %d, want 8", len(a)) } - changed := computeSkillCatalogContentHash("# cat", `{"services":[{"name":"a"}]}`, `{"openapi":"3.1.0"}`, "", nil) + changed := computeStaticSiteContentHash("# cat", `{"services":[{"name":"a"}]}`, `{"openapi":"3.1.0"}`, "", nil) if changed == a { t.Fatal("expected different hash when catalog content changes") } } -func TestSkillCatalogContentMatches(t *testing.T) { - cm := buildSkillCatalogConfigMap("# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) - if !skillCatalogContentMatches(cm, "# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) { +func TestStaticSiteContentMatches(t *testing.T) { + cm := buildStaticSiteConfigMap("# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) + if !staticSiteContentMatches(cm, "# cat", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) { t.Fatal("expected matching catalog content") } - if skillCatalogContentMatches(cm, "# changed", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) { + if staticSiteContentMatches(cm, "# changed", `{"services":[]}`, `{"openapi":"3.1.0"}`, "", nil) { t.Fatal("expected different skill.md to not match") } - if skillCatalogContentMatches(nil, "# cat", `{}`, `{}`, "", nil) { + if staticSiteContentMatches(nil, "# cat", `{}`, `{}`, "", nil) { t.Fatal("nil configmap must not match") } } -func TestSkillCatalogDeployedContentHash(t *testing.T) { - deployment := buildSkillCatalogDeployment("abc12345", nil) - if got := skillCatalogDeployedContentHash(deployment); got != "abc12345" { +func TestStaticSiteDeployedContentHash(t *testing.T) { + deployment := buildStaticSiteDeployment("abc12345", nil) + if got := staticSiteDeployedContentHash(deployment); got != "abc12345" { t.Fatalf("hash = %q, want abc12345", got) } - if got := skillCatalogDeployedContentHash(nil); got != "" { + if got := staticSiteDeployedContentHash(nil); got != "" { t.Fatalf("nil deployment hash = %q, want empty", got) } empty := &unstructured.Unstructured{Object: map[string]any{"spec": map[string]any{}}} - if got := skillCatalogDeployedContentHash(empty); got != "" { + if got := staticSiteDeployedContentHash(empty); got != "" { t.Fatalf("missing annotation hash = %q, want empty", got) } } diff --git a/internal/serviceoffercontroller/controller.go b/internal/serviceoffercontroller/controller.go index 0d45c95b..1c3ebb8e 100644 --- a/internal/serviceoffercontroller/controller.go +++ b/internal/serviceoffercontroller/controller.go @@ -76,7 +76,13 @@ type Controller struct { identityQueue workqueue.TypedRateLimitingInterface[string] purchaseQueue workqueue.TypedRateLimitingInterface[string] agentQueue workqueue.TypedRateLimitingInterface[string] - catalogMu sync.Mutex + staticSiteMu sync.Mutex + + // upstreamOpenAPICache is populated from each offer's own reconcile + // (refresh, outside staticSiteMu) and only ever read by + // reconcileStaticSite's buildOfferBundles call (under staticSiteMu) — + // see upstream_openapi.go. + upstreamOpenAPICache upstreamOpenAPICache pendingAuths sync.Map // key: "ns/name" → []map[string]string @@ -351,25 +357,25 @@ func (c *Controller) enqueueStorefrontProfileRefresh(obj any) { if u.GetNamespace() != storefront.ProfileNamespace || u.GetName() != storefront.ProfileConfigMap { return } - log.Printf("serviceoffer-controller: storefront profile change detected, refreshing skill catalog") - c.enqueueSkillCatalogRefresh() + log.Printf("serviceoffer-controller: storefront profile change detected, refreshing static site") + c.enqueueStaticSiteRefresh() } -func (c *Controller) enqueueSkillCatalogRefresh() { +func (c *Controller) enqueueStaticSiteRefresh() { items := c.offerInformer.GetStore().List() if len(items) > 0 { // Any single offer reconcile rebuilds the full catalog. c.enqueueOffer(items[0]) return } - go c.refreshSkillCatalogAsync() + go c.refreshStaticSiteAsync() } -func (c *Controller) refreshSkillCatalogAsync() { +func (c *Controller) refreshStaticSiteAsync() { ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) defer cancel() - if err := c.reconcileSkillCatalog(ctx, nil); err != nil { - log.Printf("serviceoffer-controller: refresh skill catalog: %v", err) + if err := c.reconcileStaticSite(ctx, nil); err != nil { + log.Printf("serviceoffer-controller: refresh static site: %v", err) } } @@ -467,9 +473,10 @@ func (c *Controller) reconcileOffer(ctx context.Context, key string) error { now := metav1.Now() tombstone.DeletionTimestamp = &now } - if err := c.reconcileSkillCatalog(ctx, &tombstone); err != nil { + if err := c.reconcileStaticSite(ctx, &tombstone); err != nil { return err } + c.upstreamOpenAPICache.forget(offer.UID) return c.removeFinalizer(ctx, raw, serviceOfferFinalizer) } @@ -661,10 +668,17 @@ func (c *Controller) reconcileOffer(ctx context.Context, key string) error { // requiring a spec mutation or unrelated ConfigMap update. c.offerQueue.AddAfter(offer.Namespace+"/"+offer.Name, 5*time.Second) } - // Rebuild the skill catalog on every reconcile so tunnel URL changes and - // offer status updates propagate immediately. reconcileSkillCatalog skips + // Refresh this offer's upstream OpenAPI cache from its own reconcile, + // outside staticSiteMu, at most once per generation — see + // upstreamOpenAPICache and buildOfferBundles for why the rebuild below + // must never fetch live. + if offer.Spec.Hostname != "" { + c.upstreamOpenAPICache.refresh(offer, tryUpstreamOpenAPI) + } + // Rebuild the static site on every reconcile so tunnel URL changes and + // offer status updates propagate immediately. reconcileStaticSite skips // ConfigMap/Deployment writes when the rendered hash is unchanged. - return c.reconcileSkillCatalog(ctx, nil) + return c.reconcileStaticSite(ctx, nil) } func (c *Controller) reconcileDeletingOffer(ctx context.Context, offer *monetizeapi.ServiceOffer) error { @@ -747,8 +761,12 @@ func (c *Controller) reconcileUpstream(ctx context.Context, status *monetizeapi. } defer response.Body.Close() - if response.StatusCode >= 500 { - setCondition(status, "UpstreamHealthy", "False", "Unhealthy", fmt.Sprintf("HTTP %d from upstream", response.StatusCode)) + // Require a 2xx health response. Treating any <500 (including 404) as + // healthy left agents with a wrong healthPath (or a never-started API) + // stuck in UpstreamHealthy=True and then Ready=True while paid traffic + // 404'd end-to-end. + if !upstreamHealthStatusOK(response.StatusCode) { + setCondition(status, "UpstreamHealthy", "False", "Unhealthy", fmt.Sprintf("HTTP %d from upstream health path %s", response.StatusCode, offer.EffectiveHealthPath())) return false, nil } @@ -756,6 +774,12 @@ func (c *Controller) reconcileUpstream(ctx context.Context, status *monetizeapi. return true, nil } +// upstreamHealthStatusOK reports whether an HTTP status from the offer's +// healthPath should count as UpstreamHealthy=True. +func upstreamHealthStatusOK(code int) bool { + return code >= 200 && code < 300 +} + func (c *Controller) reconcilePaymentGate(ctx context.Context, status *monetizeapi.ServiceOfferStatus, offer *monetizeapi.ServiceOffer) error { if err := c.applyObject(ctx, c.referenceGrants.Namespace("x402"), buildReferenceGrant(offer)); err != nil { setCondition(status, "PaymentGateReady", "False", "ApplyFailed", err.Error()) @@ -793,27 +817,75 @@ func (c *Controller) reconcilePaymentGate(ctx context.Context, status *monetizea func (c *Controller) reconcileRoute(ctx context.Context, status *monetizeapi.ServiceOfferStatus, offer *monetizeapi.ServiceOffer) error { // Protection middleware must exist before the routes that reference it - // (Traefik drops routes with a dangling ExtensionRef). - if hasLimits(offer) { - if err := c.applyObject(ctx, c.middlewares.Namespace(offer.Namespace), buildLimitsMiddleware(offer)); err != nil { + // (Traefik drops routes with a dangling ExtensionRef). inFlightReq and + // rateLimit are separate Middleware CRs — a combined CR is rejected by + // Traefik while the ServiceOffer still looked Ready. + for _, mw := range buildLimitsMiddlewares(offer) { + if err := c.applyObject(ctx, c.middlewares.Namespace(offer.Namespace), mw); err != nil { setCondition(status, "RoutePublished", "False", "ApplyFailed", err.Error()) return err } - } else { - err := c.middlewares.Namespace(offer.Namespace).Delete(ctx, limitsMiddlewareName(offer.Name), metav1.DeleteOptions{}) + } + // Tear down unused limit CRs (including the legacy combined -limits name). + for _, name := range []string{ + limitsInFlightMiddlewareName(offer.Name), + limitsRPSMiddlewareName(offer.Name), + legacyLimitsMiddlewareName(offer.Name), + } { + wanted := false + for _, mw := range buildLimitsMiddlewares(offer) { + if mw.GetName() == name { + wanted = true + break + } + } + if wanted { + continue + } + err := c.middlewares.Namespace(offer.Namespace).Delete(ctx, name, metav1.DeleteOptions{}) if err != nil && !apierrors.IsNotFound(err) { return err } } + // Delete both superseded ReferenceGrant names every reconcile so grants + // orphaned by a rename don't linger and collide with another offer: the + // pre-4726dcfe non-namespaced name, and the 4726dcfe dash-joined name that + // the injective hash suffix replaced. + for _, staleGrant := range []string{ + legacyBackendReferenceGrantName(offer.Name), + intermediateBackendReferenceGrantName(offer.Namespace, offer.Name), + } { + if err := c.referenceGrants.Namespace("x402").Delete(ctx, staleGrant, metav1.DeleteOptions{}); err != nil && !apierrors.IsNotFound(err) { + return err + } + } if err := c.applyObject(ctx, c.httpRoutes.Namespace(offer.Namespace), buildHTTPRoute(offer)); err != nil { setCondition(status, "RoutePublished", "False", "ApplyFailed", err.Error()) return err } + route, err := c.httpRoutes.Namespace(offer.Namespace).Get(ctx, childName(offer.Name), metav1.GetOptions{}) + if err != nil { + setCondition(status, "RoutePublished", "False", "ApplyFailed", err.Error()) + return err + } + if !httpRouteAccepted(route) { + setCondition(status, "RoutePublished", "False", "WaitingForTraefikAcceptance", "HTTPRoute applied but not yet accepted by Traefik") + return nil + } if offer.Spec.Hostname != "" { if err := c.applyObject(ctx, c.httpRoutes.Namespace(offer.Namespace), buildHostHTTPRoute(offer)); err != nil { setCondition(status, "RoutePublished", "False", "ApplyFailed", err.Error()) return err } + hostRoute, err := c.httpRoutes.Namespace(offer.Namespace).Get(ctx, hostChildName(offer.Name), metav1.GetOptions{}) + if err != nil { + setCondition(status, "RoutePublished", "False", "ApplyFailed", err.Error()) + return err + } + if !httpRouteAccepted(hostRoute) { + setCondition(status, "RoutePublished", "False", "WaitingForTraefikAcceptance", "Host HTTPRoute applied but not yet accepted by Traefik") + return nil + } } else { // Hostname removed from the spec: tear the host route down so the // origin frees up (for the storefront catch-all or another offer). @@ -836,6 +908,11 @@ func (c *Controller) reconcileRegistrationStatus(ctx context.Context, status *mo if err := c.deleteRegistrationRequest(ctx, offer.Namespace, offer.Name); err != nil { return err } + // Disabling registration must stop reporting whatever agentId was + // last recorded — otherwise a disabled offer keeps showing a + // (possibly stale, wrong-chain) id in status/CLI output. + status.AgentID = "" + status.RegistrationTxHash = "" setCondition(status, "Registered", "True", "Disabled", "Registration disabled") return nil } @@ -877,8 +954,7 @@ func (c *Controller) reconcileRegistrationStatus(ctx context.Context, status *mo if err != nil { return err } - request.Status = registrationRequestStatusWithIdentity(request, identity) - applySharedRegistrationStatus(status, offer, owner, request) + applySharedRegistrationStatus(status, offer, owner, identity, request) return nil } @@ -898,14 +974,7 @@ func (c *Controller) reconcileRegistrationStatus(ctx context.Context, status *mo return err } - status.AgentID = request.Status.AgentID - status.RegistrationTxHash = request.Status.RegistrationTxHash - if agentID := monetizeapi.AgentIdentityAgentIDForChain(identity.Status, offer.Spec.Payment.Network); agentID != "" { - status.AgentID = agentID - request.Status = registrationRequestStatusWithIdentity(request, identity) - } - - applySharedRegistrationStatus(status, offer, owner, request) + applySharedRegistrationStatus(status, offer, owner, identity, request) return nil } @@ -1044,7 +1113,12 @@ func (c *Controller) reconcileRegistrationActive(ctx context.Context, raw *unstr return c.updateRegistrationStatus(ctx, raw, status) } registrationChain := firstNonEmpty(request.Spec.Chain, offer.Spec.Payment.Network) - agentID := firstNonEmpty(monetizeapi.AgentIdentityAgentIDForChain(identity.Status, registrationChain), status.AgentID, offer.Status.AgentID) + // Chain-scoped only: status.AgentID/offer.Status.AgentID are not + // chain-tagged, so falling back to them here would reuse a + // wrong-chain id after a network switch and skip the on-chain + // ownership verification below. AgentIdentityAgentIDForChain + // correctly returns "" until this chain has a verified registration. + agentID := monetizeapi.AgentIdentityAgentIDForChain(identity.Status, registrationChain) txHash := firstNonEmpty(status.RegistrationTxHash, offer.Status.RegistrationTxHash) offers, err := c.registrationOffersForIdentity(defaultAgentIdentityKey(), "", "") @@ -1187,21 +1261,18 @@ func (c *Controller) recoverRegistration(ctx context.Context, client *erc8004.Cl func (c *Controller) reconcileRegistrationTombstone(ctx context.Context, raw *unstructured.Unstructured, request *monetizeapi.RegistrationRequest, offer *monetizeapi.ServiceOffer, baseURL string) error { status := request.Status - identityRaw, identity, err := c.ensureAgentIdentityForOffer(ctx, offer) + _, identity, err := c.ensureAgentIdentityForOffer(ctx, offer) if err != nil { status.Phase = registrationPhaseAwaitingExternal status.Message = truncateMessage(fmt.Sprintf("Waiting for AgentIdentity tombstone: %v", err)) return c.updateRegistrationStatus(ctx, raw, status) } + // Chain-scoped only — see reconcileRegistrationActive for why the + // status.AgentID/offer.Status.AgentID fallbacks are dropped. Since + // agentID is now always exactly what AgentIdentityAgentIDForChain + // already returns, there is nothing left to backfill into identity. registrationChain := firstNonEmpty(request.Spec.Chain, offer.Spec.Payment.Network) - agentID := firstNonEmpty(monetizeapi.AgentIdentityAgentIDForChain(identity.Status, registrationChain), status.AgentID, offer.Status.AgentID) - - if agentID != "" && monetizeapi.AgentIdentityAgentIDForChain(identity.Status, registrationChain) == "" { - identity.Status = agentIdentityStatusFromRegistration(identity, registrationChain, agentID) - if err := c.updateAgentIdentityStatus(ctx, identityRaw, identity.Status); err != nil { - return err - } - } + agentID := monetizeapi.AgentIdentityAgentIDForChain(identity.Status, registrationChain) document := BuildIdentityRegistrationDocument(IdentityRegistrationView{ Identity: identity, @@ -1246,16 +1317,16 @@ func (c *Controller) loadStorefrontProfile(ctx context.Context) (*schemas.Storef return storefront.ParseProfile(raw) } -// reconcileSkillCatalog rebuilds the /skill.md ConfigMap/Deployment/Service/ +// reconcileStaticSite rebuilds the /skill.md ConfigMap/Deployment/Service/ // HTTPRoute from the current set of operationally-ready ServiceOffers. Offers // are listed from the API server so every reconcile sees a consistent snapshot; // override replaces or appends a just-created offer the list has not observed yet. // ConfigMap and Deployment are only applied when the rendered catalog hash differs // from the live obol-skill-md Deployment annotation, so idle reconciles do not -// roll the skill-catalog pod. -func (c *Controller) reconcileSkillCatalog(ctx context.Context, override *monetizeapi.ServiceOffer) error { - c.catalogMu.Lock() - defer c.catalogMu.Unlock() +// roll the static-site pod. +func (c *Controller) reconcileStaticSite(ctx context.Context, override *monetizeapi.ServiceOffer) error { + c.staticSiteMu.Lock() + defer c.staticSiteMu.Unlock() baseURL, err := c.registrationBaseURL(ctx) if err != nil { @@ -1271,7 +1342,7 @@ func (c *Controller) reconcileSkillCatalog(ctx context.Context, override *moneti if err != nil { return err } - content := buildSkillCatalogMarkdown(offers, baseURL, storefrontProfile) + content := buildSkillMarkdown(offers, baseURL, storefrontProfile) servicesJSON := buildServiceCatalogJSON(offers, baseURL, storefrontProfile) resolvedProfile := storefront.ResolvePublished(storefrontProfile, baseURL) // buildOpenAPIDocument prefers the tunnel URL for the public `servers[0]` @@ -1282,10 +1353,10 @@ func (c *Controller) reconcileSkillCatalog(ctx context.Context, override *moneti // when tunnelURL changes — see enqueueDiscoveryRefresh). openAPIJSON := buildOpenAPIDocument(offers, baseURL, resolvedProfile) apiDocsHTML := scalarHTML(resolvedProfile) - bundles := buildOfferBundles(offers, resolvedProfile) - contentHash := computeSkillCatalogContentHash(content, servicesJSON, openAPIJSON, apiDocsHTML, bundles) + bundles := buildOfferBundles(offers, resolvedProfile, c.upstreamOpenAPICache.get) + contentHash := computeStaticSiteContentHash(content, servicesJSON, openAPIJSON, apiDocsHTML, bundles) - unchanged, err := c.skillCatalogContentUnchanged(ctx, content, servicesJSON, openAPIJSON, apiDocsHTML, bundles) + unchanged, err := c.staticSiteContentUnchanged(ctx, content, servicesJSON, openAPIJSON, apiDocsHTML, bundles) if err != nil { return err } @@ -1295,30 +1366,30 @@ func (c *Controller) reconcileSkillCatalog(ctx context.Context, override *moneti return nil } - if err := c.applyObject(ctx, c.configMaps.Namespace(skillCatalogNamespace), buildSkillCatalogConfigMap(content, servicesJSON, openAPIJSON, apiDocsHTML, bundles)); err != nil { + if err := c.applyObject(ctx, c.configMaps.Namespace(staticSiteNamespace), buildStaticSiteConfigMap(content, servicesJSON, openAPIJSON, apiDocsHTML, bundles)); err != nil { return err } - if err := c.applyObject(ctx, c.deployments.Namespace(skillCatalogNamespace), buildSkillCatalogDeployment(contentHash, bundles)); err != nil { + if err := c.applyObject(ctx, c.deployments.Namespace(staticSiteNamespace), buildStaticSiteDeployment(contentHash, bundles)); err != nil { return err } - if err := c.applyObject(ctx, c.services.Namespace(skillCatalogNamespace), buildSkillCatalogService()); err != nil { + if err := c.applyObject(ctx, c.services.Namespace(staticSiteNamespace), buildStaticSiteService()); err != nil { return err } // Headers Middleware must exist before the routes that reference it, or // Traefik drops the routes for a dangling ExtensionRef. - if err := c.applyObject(ctx, c.middlewares.Namespace(skillCatalogNamespace), buildCatalogHeadersMiddleware()); err != nil { + if err := c.applyObject(ctx, c.middlewares.Namespace(staticSiteNamespace), buildCatalogHeadersMiddleware()); err != nil { return err } - if err := c.applyObject(ctx, c.httpRoutes.Namespace(skillCatalogNamespace), buildSkillCatalogHTTPRoute()); err != nil { + if err := c.applyObject(ctx, c.httpRoutes.Namespace(staticSiteNamespace), buildStaticSiteHTTPRoute()); err != nil { return err } - if err := c.applyObject(ctx, c.httpRoutes.Namespace(skillCatalogNamespace), buildServicesJSONHTTPRoute()); err != nil { + if err := c.applyObject(ctx, c.httpRoutes.Namespace(staticSiteNamespace), buildServicesJSONHTTPRoute()); err != nil { return err } - if err := c.applyObject(ctx, c.httpRoutes.Namespace(skillCatalogNamespace), buildOpenAPIHTTPRoute()); err != nil { + if err := c.applyObject(ctx, c.httpRoutes.Namespace(staticSiteNamespace), buildOpenAPIHTTPRoute()); err != nil { return err } - if err := c.applyObject(ctx, c.httpRoutes.Namespace(skillCatalogNamespace), buildAPIDocsHTTPRoute()); err != nil { + if err := c.applyObject(ctx, c.httpRoutes.Namespace(staticSiteNamespace), buildAPIDocsHTTPRoute()); err != nil { return err } readyOffers := countReadyServiceOffers(offers) @@ -1341,10 +1412,14 @@ func (c *Controller) deleteRouteChildren(ctx context.Context, offer *monetizeapi resource dynamic.ResourceInterface name string }{ - {resource: c.referenceGrants.Namespace("x402"), name: backendReferenceGrantName(offer.Name)}, + {resource: c.referenceGrants.Namespace("x402"), name: backendReferenceGrantName(offer.Namespace, offer.Name)}, + {resource: c.referenceGrants.Namespace("x402"), name: legacyBackendReferenceGrantName(offer.Name)}, + {resource: c.referenceGrants.Namespace("x402"), name: intermediateBackendReferenceGrantName(offer.Namespace, offer.Name)}, {resource: c.httpRoutes.Namespace(offer.Namespace), name: childName(offer.Name)}, {resource: c.httpRoutes.Namespace(offer.Namespace), name: hostChildName(offer.Name)}, - {resource: c.middlewares.Namespace(offer.Namespace), name: limitsMiddlewareName(offer.Name)}, + {resource: c.middlewares.Namespace(offer.Namespace), name: limitsInFlightMiddlewareName(offer.Name)}, + {resource: c.middlewares.Namespace(offer.Namespace), name: limitsRPSMiddlewareName(offer.Name)}, + {resource: c.middlewares.Namespace(offer.Namespace), name: legacyLimitsMiddlewareName(offer.Name)}, } { err := deletion.resource.Delete(ctx, deletion.name, metav1.DeleteOptions{}) if err != nil && !apierrors.IsNotFound(err) { @@ -1404,25 +1479,53 @@ func selectRegistrationOwner(offers []*monetizeapi.ServiceOffer) *monetizeapi.Se return offers[0] } -func applySharedRegistrationStatus(status *monetizeapi.ServiceOfferStatus, offer, owner *monetizeapi.ServiceOffer, request *monetizeapi.RegistrationRequest) { - status.AgentID = request.Status.AgentID - status.RegistrationTxHash = request.Status.RegistrationTxHash +// applySharedRegistrationStatus copies a shared RegistrationRequest's phase +// onto offer's status, but the agentId itself is never trusted straight off +// request.Status: that subresource belongs to the registration owner and is +// never chain-tagged, so it can't tell whether it was recorded under +// offer's own Spec.Payment.Network or a different chain the owner (or offer +// itself, before a network switch) used previously. The agentId is instead +// always re-derived from the chain-scoped AgentIdentity record — if the +// identity has no registration for offer's chain, agentId stays empty and +// Registered does not flip True on the strength of a foreign chain's id. +func applySharedRegistrationStatus(status *monetizeapi.ServiceOfferStatus, offer, owner *monetizeapi.ServiceOffer, identity *monetizeapi.AgentIdentity, request *monetizeapi.RegistrationRequest) { + agentID := monetizeapi.AgentIdentityAgentIDForChain(identity.Status, offer.Spec.Payment.Network) + status.AgentID = agentID + // The tx hash lives on the same chain-blind request.Status subresource + // the agentId was retired from, so it is only mirrored once the offer's + // own chain has a recorded registration for it to describe. + status.RegistrationTxHash = "" + if agentID != "" { + status.RegistrationTxHash = request.Status.RegistrationTxHash + } if !isConditionTrue(*status, "RoutePublished") { setCondition(status, "Registered", "False", "WaitingForRoute", "Waiting for route publication before shared registration") return } - if requestPhaseReady(request.Status.Phase) { - message := defaultString(request.Status.Message, "Registration reconciled") + // Registered only ever flips True on the strength of a chain-scoped + // agentId. requestPhaseReady(request.Status.Phase) alone is NOT a + // sufficient condition: Phase is set by whoever last reconciled this + // RegistrationRequest for whatever chain it targeted at that time, and + // (like the AgentID field) is never chain-tagged — it can read stale + // "Registered" from a chain the offer (or, when shared, the owner) has + // since moved off. Every legitimate Phase=Registered transition sets + // agentID for that same chain in the same reconcile pass, so requiring + // agentID != "" here costs no real case while closing the stale-phase + // gap (this is also what tempers the request.Status.AgentID-based + // widening 84dcbf83 added — that source is gone, but a bare Phase + // check has the identical staleness problem). + if agentID != "" { + message := defaultString(request.Status.Message, fmt.Sprintf("Recorded agent %s", agentID)) if owner != nil && (owner.Namespace != offer.Namespace || owner.Name != offer.Name) { - if request.Status.AgentID != "" { - message = fmt.Sprintf("Shared registration via %s/%s recorded agent %s", owner.Namespace, owner.Name, request.Status.AgentID) - } else { - message = fmt.Sprintf("Shared registration via %s/%s is active", owner.Namespace, owner.Name) - } + message = fmt.Sprintf("Shared registration via %s/%s recorded agent %s", owner.Namespace, owner.Name, agentID) + } + reason := defaultString(request.Status.Phase, "Active") + if !requestPhaseReady(request.Status.Phase) { + reason = "Active" } - setCondition(status, "Registered", "True", request.Status.Phase, message) + setCondition(status, "Registered", "True", reason, message) return } @@ -1589,6 +1692,10 @@ func httpRouteAccepted(route *unstructured.Unstructured) bool { if err != nil || !found { return false } + // metadata.generation defaults to 0 when absent (e.g. hand-built test + // fixtures), matching a condition's absent observedGeneration below so + // older/incomplete fixtures still compare equal. + routeGeneration, _, _ := unstructured.NestedInt64(route.Object, "metadata", "generation") for _, parent := range parents { parentMap, ok := parent.(map[string]any) if !ok { @@ -1598,13 +1705,28 @@ func httpRouteAccepted(route *unstructured.Unstructured) bool { if !ok { continue } + // Both default to false: each must be affirmatively reported True at + // the CURRENT generation. Defaulting resolvedRefs to true would be + // fail-open — a stale or not-yet-emitted ResolvedRefs (behind + // metadata.generation, or False from a rejected prior spec) would be + // skipped by the observedGeneration guard below and leave the true + // default intact, publishing a route Traefik hasn't resolved. accepted := false - resolvedRefs := true + resolvedRefs := false for _, condition := range conditions { condMap, ok := condition.(map[string]any) if !ok { continue } + // A condition Traefik hasn't reconciled against the current spec + // yet (observedGeneration behind metadata.generation) still + // reflects the PREVIOUS generation's verdict — don't trust it, + // or an update to an already-accepted route would flip + // RoutePublished True before Traefik has actually checked it. + observedGeneration, _, _ := unstructured.NestedInt64(condMap, "observedGeneration") + if observedGeneration != routeGeneration { + continue + } condType, _ := condMap["type"].(string) condStatus, _ := condMap["status"].(string) switch condType { diff --git a/internal/serviceoffercontroller/controller_test.go b/internal/serviceoffercontroller/controller_test.go index 8d308b0b..2f59be64 100644 --- a/internal/serviceoffercontroller/controller_test.go +++ b/internal/serviceoffercontroller/controller_test.go @@ -8,6 +8,28 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) +func TestUpstreamHealthStatusOK(t *testing.T) { + // 2xx only — 404/3xx/5xx must not flip UpstreamHealthy=True. + cases := map[int]bool{ + 200: true, + 201: true, + 204: true, + 301: false, + 302: false, + 400: false, + 401: false, + 404: false, + 500: false, + 503: false, + 0: false, + } + for code, want := range cases { + if got := upstreamHealthStatusOK(code); got != want { + t.Errorf("upstreamHealthStatusOK(%d) = %v, want %v", code, got, want) + } + } +} + func TestSelectRegistrationOwnerPrefersOldestEnabledOffer(t *testing.T) { now := time.Now().UTC() offers := []*monetizeapi.ServiceOffer{ @@ -108,6 +130,9 @@ func TestApplySharedRegistrationStatus_NonOwnerUsesSharedAgent(t *testing.T) { } owner := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "alpha", Namespace: "demo"}} offer := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "beta", Namespace: "demo"}} + offer.Spec.Payment.Network = "base" + identity := &monetizeapi.AgentIdentity{} + identity.Status = monetizeapi.UpsertAgentIdentityRegistration(identity.Status, "base", "42") request := &monetizeapi.RegistrationRequest{ Status: monetizeapi.RegistrationRequestStatus{ Phase: registrationPhaseRegistered, @@ -116,7 +141,7 @@ func TestApplySharedRegistrationStatus_NonOwnerUsesSharedAgent(t *testing.T) { }, } - applySharedRegistrationStatus(status, offer, owner, request) + applySharedRegistrationStatus(status, offer, owner, identity, request) if status.AgentID != "42" || status.RegistrationTxHash != "0xtx" { t.Fatalf("shared registration identifiers not copied: %+v", status) @@ -129,6 +154,9 @@ func TestApplySharedRegistrationStatus_NonOwnerUsesSharedAgent(t *testing.T) { func TestApplySharedRegistrationStatus_WaitsForRoute(t *testing.T) { status := &monetizeapi.ServiceOfferStatus{} owner := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "alpha", Namespace: "demo"}} + owner.Spec.Payment.Network = "base" + identity := &monetizeapi.AgentIdentity{} + identity.Status = monetizeapi.UpsertAgentIdentityRegistration(identity.Status, "base", "7") request := &monetizeapi.RegistrationRequest{ Status: monetizeapi.RegistrationRequestStatus{ Phase: registrationPhaseRegistered, @@ -136,9 +164,70 @@ func TestApplySharedRegistrationStatus_WaitsForRoute(t *testing.T) { }, } - applySharedRegistrationStatus(status, owner, owner, request) + applySharedRegistrationStatus(status, owner, owner, identity, request) if isConditionTrue(*status, "Registered") { t.Fatalf("registered should remain false until route is published: %+v", status.Conditions) } } + +// CLI `obol sell register` may leave RegistrationRequest phase empty while +// AgentIdentity already has the agentId — non-owner offers must still flip +// Registered=True so status/checkmarks agree. +func TestApplySharedRegistrationStatus_AgentIDWithoutPhase(t *testing.T) { + status := &monetizeapi.ServiceOfferStatus{ + Conditions: []monetizeapi.Condition{{Type: "RoutePublished", Status: "True"}}, + } + owner := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "alpha", Namespace: "demo"}} + offer := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "beta", Namespace: "other"}} + offer.Spec.Payment.Network = "base" + identity := &monetizeapi.AgentIdentity{} + identity.Status = monetizeapi.UpsertAgentIdentityRegistration(identity.Status, "base", "8104") + request := &monetizeapi.RegistrationRequest{ + Status: monetizeapi.RegistrationRequestStatus{ + AgentID: "8104", + // Phase empty / Pending — common after CLI-only registration path + }, + } + + applySharedRegistrationStatus(status, offer, owner, identity, request) + + if status.AgentID != "8104" { + t.Fatalf("AgentID = %q, want 8104", status.AgentID) + } + if !isConditionTrue(*status, "Registered") { + t.Fatalf("Registered should be True when agentId is known: %+v", status.Conditions) + } +} + +// Chain-switch regression: offer B borrows a shared registration from owner +// A, but B's own Spec.Payment.Network has no verified registration in the +// AgentIdentity (e.g. B just switched networks, or was never registered on +// this chain). Even though the owner's RegistrationRequest is Phase=Registered +// with a non-empty (different-chain) AgentID, B's status must not adopt it +// and Registered must not flip True on the strength of a foreign chain's id. +func TestApplySharedRegistrationStatus_ChainMismatchDoesNotAdoptForeignAgentID(t *testing.T) { + status := &monetizeapi.ServiceOfferStatus{ + Conditions: []monetizeapi.Condition{{Type: "RoutePublished", Status: "True"}}, + } + owner := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "alpha", Namespace: "demo"}} + offer := &monetizeapi.ServiceOffer{ObjectMeta: metav1.ObjectMeta{Name: "beta", Namespace: "demo"}} + offer.Spec.Payment.Network = "base" // switched away from base-sepolia + identity := &monetizeapi.AgentIdentity{} + identity.Status = monetizeapi.UpsertAgentIdentityRegistration(identity.Status, "base-sepolia", "42") + request := &monetizeapi.RegistrationRequest{ + Status: monetizeapi.RegistrationRequestStatus{ + Phase: registrationPhaseRegistered, + AgentID: "42", // owner's base-sepolia id — must not leak onto offer's base status + }, + } + + applySharedRegistrationStatus(status, offer, owner, identity, request) + + if status.AgentID != "" { + t.Fatalf("AgentID = %q, want empty (no verified registration on offer's own chain)", status.AgentID) + } + if isConditionTrue(*status, "Registered") { + t.Fatalf("Registered should not flip True from a different chain's agentId: %+v", status.Conditions) + } +} diff --git a/internal/serviceoffercontroller/helpers_test.go b/internal/serviceoffercontroller/helpers_test.go index f05bc842..4c6a171d 100644 --- a/internal/serviceoffercontroller/helpers_test.go +++ b/internal/serviceoffercontroller/helpers_test.go @@ -139,7 +139,7 @@ func TestHTTPRouteAccepted(t *testing.T) { want: false, }, { - name: "only Accepted condition (ResolvedRefs implicitly True by default)", + name: "only Accepted condition, ResolvedRefs absent — not yet accepted (fail closed)", route: &unstructured.Unstructured{Object: map[string]any{ "status": map[string]any{ "parents": []any{ @@ -151,7 +151,7 @@ func TestHTTPRouteAccepted(t *testing.T) { }, }, }}, - want: true, // function defaults resolvedRefs to true when absent + want: false, // ResolvedRefs must be affirmatively True; absent is not trusted }, { name: "multiple parents: first bad, second good", @@ -185,6 +185,81 @@ func TestHTTPRouteAccepted(t *testing.T) { }}, want: false, }, + { + // #767 route-acceptance gate: an UPDATE to an already-accepted + // route must not be trusted until Traefik reconciles the new + // spec. Status still reflects generation 1's verdict while the + // route is now at generation 2. + name: "Accepted=True but observedGeneration behind metadata.generation", + route: &unstructured.Unstructured{Object: map[string]any{ + "metadata": map[string]any{"generation": int64(2)}, + "status": map[string]any{ + "parents": []any{ + map[string]any{ + "conditions": []any{ + map[string]any{"type": "Accepted", "status": "True", "observedGeneration": int64(1)}, + map[string]any{"type": "ResolvedRefs", "status": "True", "observedGeneration": int64(1)}, + }, + }, + }, + }, + }}, + want: false, + }, + { + name: "Accepted=True with observedGeneration matching metadata.generation", + route: &unstructured.Unstructured{Object: map[string]any{ + "metadata": map[string]any{"generation": int64(2)}, + "status": map[string]any{ + "parents": []any{ + map[string]any{ + "conditions": []any{ + map[string]any{"type": "Accepted", "status": "True", "observedGeneration": int64(2)}, + map[string]any{"type": "ResolvedRefs", "status": "True", "observedGeneration": int64(2)}, + }, + }, + }, + }, + }}, + want: true, + }, + { + // Fail-closed: Accepted is current+True but ResolvedRefs is + // stale (behind metadata.generation) and False. The stale + // ResolvedRefs must NOT be ignored-as-true — the route has not + // been resolved against the current spec. + name: "Accepted current True but ResolvedRefs stale False", + route: &unstructured.Unstructured{Object: map[string]any{ + "metadata": map[string]any{"generation": int64(2)}, + "status": map[string]any{ + "parents": []any{ + map[string]any{ + "conditions": []any{ + map[string]any{"type": "Accepted", "status": "True", "observedGeneration": int64(2)}, + map[string]any{"type": "ResolvedRefs", "status": "False", "observedGeneration": int64(1)}, + }, + }, + }, + }, + }}, + want: false, + }, + { + name: "Accepted current True but ResolvedRefs absent", + route: &unstructured.Unstructured{Object: map[string]any{ + "metadata": map[string]any{"generation": int64(2)}, + "status": map[string]any{ + "parents": []any{ + map[string]any{ + "conditions": []any{ + map[string]any{"type": "Accepted", "status": "True", "observedGeneration": int64(2)}, + }, + }, + }, + }, + }}, + want: false, + }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { diff --git a/internal/serviceoffercontroller/hostoffer_test.go b/internal/serviceoffercontroller/hostoffer_test.go index 43795bb8..1f47ff85 100644 --- a/internal/serviceoffercontroller/hostoffer_test.go +++ b/internal/serviceoffercontroller/hostoffer_test.go @@ -18,6 +18,10 @@ func hostnameOffer() *monetizeapi.ServiceOffer { return offer } +// noUpstreamOpenAPI is the buildOfferBundles cache-lookup stub for tests +// that don't exercise the upstream-OpenAPI path. +func noUpstreamOpenAPI(*monetizeapi.ServiceOffer) map[string]any { return nil } + // TestBuildHostHTTPRoute pins the dedicated-origin route topology: Exact // discovery rules rewriting into the offer's bundle files on the catalog // httpd, and a PathPrefix / rule rewriting the public path-world into @@ -35,18 +39,19 @@ func TestBuildHostHTTPRoute(t *testing.T) { } rules, _, _ := unstructured.NestedSlice(route.Object, "spec", "rules") - if len(rules) != 4 { - t.Fatalf("rules = %d, want 4 (/, /openapi.json, /.well-known/x402, catch-all)", len(rules)) + if len(rules) != 5 { + t.Fatalf("rules = %d, want 5 (/, openapi, x402, agent-registration, catch-all)", len(rules)) } // Rule shapes: first three Exact → catalog httpd with full-path // rewrite; last PathPrefix / → verifier with prefix rewrite + headers. wantExact := map[string]string{ - "/": "/offers/sec/audit/index.html", - "/openapi.json": "/offers/sec/audit/openapi.json", - "/.well-known/x402": "/offers/sec/audit/x402.json", + "/": "/offers/sec/audit/index.html", + "/openapi.json": "/offers/sec/audit/openapi.json", + "/.well-known/x402": "/offers/sec/audit/x402.json", + "/.well-known/agent-registration.json": "/offers/sec/audit/agent-registration.json", } - for i := 0; i < 3; i++ { + for i := 0; i < 4; i++ { rule := rules[i].(map[string]any) match := rule["matches"].([]any)[0].(map[string]any)["path"].(map[string]any) if match["type"] != "Exact" { @@ -59,12 +64,12 @@ func TestBuildHostHTTPRoute(t *testing.T) { t.Errorf("rule %d (%s) rewrites to %v, want %s", i, public, got, wantExact[public]) } backend := rule["backendRefs"].([]any)[0].(map[string]any) - if backend["name"] != skillCatalogConfigMapName || backend["namespace"] != skillCatalogNamespace { + if backend["name"] != staticSiteConfigMapName || backend["namespace"] != staticSiteNamespace { t.Errorf("rule %d backend = %v", i, backend) } } - catchall := rules[3].(map[string]any) + catchall := rules[4].(map[string]any) match := catchall["matches"].([]any)[0].(map[string]any)["path"].(map[string]any) if match["type"] != "PathPrefix" || match["value"] != "/" { t.Fatalf("catch-all match = %v", match) @@ -105,13 +110,13 @@ func TestBuildOfferBundles(t *testing.T) { profile := schemas.StorefrontProfile{DisplayName: "Acme", ContactEmail: "ops@acme.example"} offer := hostnameOffer() - if got := buildOfferBundles([]*monetizeapi.ServiceOffer{routeTableOffer()}, profile); len(got) != 0 { + if got := buildOfferBundles([]*monetizeapi.ServiceOffer{routeTableOffer()}, profile, noUpstreamOpenAPI); len(got) != 0 { t.Fatalf("path-only offer produced bundles: %v", got) } - bundles := buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, profile) - if len(bundles) != 3 { - t.Fatalf("len(bundles) = %d, want 3", len(bundles)) + bundles := buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, profile, noUpstreamOpenAPI) + if len(bundles) != 4 { + t.Fatalf("len(bundles) = %d, want 4", len(bundles)) } byPath := map[string]string{} for _, f := range bundles { @@ -195,7 +200,7 @@ func TestBuildOfferBundles_BrandingOverride(t *testing.T) { Description: "**Deep** audits by AuditCo.", } - bundles := buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, profile) + bundles := buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, profile, noUpstreamOpenAPI) byPath := map[string]string{} for _, f := range bundles { byPath[f.Path] = f.Content @@ -262,8 +267,87 @@ func TestCatalogAdvertisesDedicatedOrigin(t *testing.T) { t.Fatalf("catalog endpoint = %+v, want dedicated origin", catalog.Services) } - skill := buildSkillCatalogMarkdown([]*monetizeapi.ServiceOffer{offer}, "https://shared.example", nil) + skill := buildSkillMarkdown([]*monetizeapi.ServiceOffer{offer}, "https://shared.example", nil) if !strings.Contains(skill, "`POST https://audit.v1337.example/submit`") { t.Errorf("skill.md routes not rooted at dedicated origin:\n%.400s", skill) } } + +// TestHostRouteDiscoveryRulesAreGETOnly pins the method scoping: a +// root-priced offer advertises POST / as its paid resource, so the +// Exact "/" discovery rule must only capture GETs — POSTs fall through to +// the PathPrefix payment gate. +func TestHostRouteDiscoveryRulesAreGETOnly(t *testing.T) { + offer := hostnameOffer() + offer.Spec.Type = "agent" + route := buildHostHTTPRoute(offer) + rules, _, _ := unstructured.NestedSlice(route.Object, "spec", "rules") + for i, raw := range rules[:len(rules)-1] { // all but the catch-all + match := raw.(map[string]any)["matches"].([]any)[0].(map[string]any) + if match["method"] != "GET" { + t.Errorf("rule %d (%v) method = %v, want GET", i, match["path"], match["method"]) + } + } + last := rules[len(rules)-1].(map[string]any)["matches"].([]any)[0].(map[string]any) + if _, scoped := last["method"]; scoped { + t.Errorf("catch-all must match every method") + } +} + +func TestBuildOfferBundles_UpstreamOpenAPI(t *testing.T) { + profile := schemas.StorefrontProfile{DisplayName: "Acme", ContactEmail: "ops@acme.example"} + offer := hostnameOffer() + offer.Spec.Registration.Name = "Hyperliquid Trading Intelligence" + offer.Spec.Registration.Description = "Full first-party catalog." + upstream := func(*monetizeapi.ServiceOffer) map[string]any { + return map[string]any{ + "openapi": "3.1.0", + "info": map[string]any{"title": "upstream-title", "version": "1.1.0"}, + "paths": map[string]any{ + "/v1/leaderboard": map[string]any{ + "get": map[string]any{ + "summary": "Leaderboard", "security": []any{map[string]any{"x402": []any{}}}, + "x-payment-info": map[string]any{"price": map[string]any{"amount": "0.001"}}, + "responses": map[string]any{"200": map[string]any{}, "402": map[string]any{}}, + }, + }, + "/v1/markets/overview": map[string]any{ + "get": map[string]any{"summary": "Free overview", "security": []any{}, "responses": map[string]any{"200": map[string]any{}}}, + }, + }, + } + } + bundles := buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, profile, upstream) + byPath := map[string]string{} + for _, f := range bundles { + byPath[f.Path] = f.Content + } + var doc map[string]any + if err := json.Unmarshal([]byte(byPath["offers/sec/audit/openapi.json"]), &doc); err != nil { + t.Fatal(err) + } + if doc["info"].(map[string]any)["title"] != "Hyperliquid Trading Intelligence" { + t.Errorf("title = %v", doc["info"]) + } + if _, ok := doc["paths"].(map[string]any)["/v1/leaderboard"]; !ok { + t.Fatalf("missing leaderboard in paths: %v", doc["paths"]) + } + var wk struct { + Resources []struct { + Resource string `json:"resource"` + } `json:"resources"` + } + if err := json.Unmarshal([]byte(byPath["offers/sec/audit/x402.json"]), &wk); err != nil { + t.Fatal(err) + } + if len(wk.Resources) != 1 || !strings.Contains(wk.Resources[0].Resource, "/v1/leaderboard") { + t.Fatalf("x402 resources = %+v", wk.Resources) + } + var reg map[string]any + if err := json.Unmarshal([]byte(byPath["offers/sec/audit/agent-registration.json"]), ®); err != nil { + t.Fatal(err) + } + if reg["name"] != "Hyperliquid Trading Intelligence" { + t.Errorf("reg name = %v", reg["name"]) + } +} diff --git a/internal/serviceoffercontroller/identity_controller.go b/internal/serviceoffercontroller/identity_controller.go index 92fcd03e..fefbffb6 100644 --- a/internal/serviceoffercontroller/identity_controller.go +++ b/internal/serviceoffercontroller/identity_controller.go @@ -436,11 +436,3 @@ func agentIdentityStatusFromRegistration(identity *monetizeapi.AgentIdentity, ch return status } -func registrationRequestStatusWithIdentity(request *monetizeapi.RegistrationRequest, identity *monetizeapi.AgentIdentity) monetizeapi.RegistrationRequestStatus { - status := request.Status - if identity == nil { - return status - } - status.AgentID = firstNonEmpty(monetizeapi.AgentIdentityAgentIDForChain(identity.Status, request.Spec.Chain), status.AgentID) - return status -} diff --git a/internal/serviceoffercontroller/identity_controller_test.go b/internal/serviceoffercontroller/identity_controller_test.go index cf9bf43e..0b9d0f0c 100644 --- a/internal/serviceoffercontroller/identity_controller_test.go +++ b/internal/serviceoffercontroller/identity_controller_test.go @@ -5,6 +5,7 @@ import ( "encoding/json" "testing" + "github.com/ObolNetwork/obol-stack/internal/erc8004" "github.com/ObolNetwork/obol-stack/internal/monetizeapi" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" @@ -71,17 +72,112 @@ func TestReconcileRegistrationTombstone_PublishesIdentityDocument(t *testing.T) } } -func TestRecreatedServiceOffer_MirrorsAgentIdentityAgentID(t *testing.T) { - identity := defaultIdentity("777") +// Chain-switch regression (Canary402 field report): identity already has a +// verified base-sepolia registration, but the offer (and its +// RegistrationRequest, re-applied with the new spec.chain on every apply) +// has switched to base. request.Status.AgentID / offer.Status.AgentID still +// carry the old base-sepolia numeric id — that subresource is never reset +// by a chain switch. reconcileRegistrationActive must not adopt the stale +// id for base, must not upsert it into the AgentIdentity CR, and must not +// publish it in the registration document either. +func TestReconcileRegistrationActive_ChainSwitchDoesNotLeakWrongChainAgentID(t *testing.T) { + identity := defaultIdentity("777") // base-sepolia -> 777 + identity.APIVersion = monetizeapi.Group + "/" + monetizeapi.Version + identity.Kind = monetizeapi.AgentIdentityKind + identity.UID = types.UID("identity-uid") + request := &monetizeapi.RegistrationRequest{} - request.Spec.Chain = "base-sepolia" - request.Status.RegistrationTxHash = "0xabc" - status := registrationRequestStatusWithIdentity(request, identity) - if status.AgentID != "777" { - t.Fatalf("AgentID = %q, want 777", status.AgentID) + request.Namespace = "demo" + request.Name = registrationRequestName("svc") + request.Spec.ServiceOfferName = "svc" + request.Spec.ServiceOfferNamespace = "demo" + request.Spec.Chain = "base" // switched + request.Status.AgentID = "777" + request.Status.RegistrationTxHash = "0xsepolia" + + offer := readyOffer("svc") + offer.Namespace = "demo" + offer.Spec.Payment.Network = "base" // switched + offer.Spec.Payment.PayTo = "0xNewOwner" + offer.Status.AgentID = "777" // stale, mirrors pre-fix ServiceOffer.status + + rawRequest := registrationRequestToUnstructured(t, request) + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds( + runtime.NewScheme(), + identityListKinds(), + agentIdentityToUnstructured(identity), + rawRequest, + ) + c := controllerForIdentityTest(dynClient) + + if err := c.reconcileRegistrationActive(context.Background(), rawRequest, request, offer, "https://seller.test"); err != nil { + t.Fatalf("reconcileRegistrationActive: %v", err) + } + + identityRaw, err := c.agentIdentities.Namespace(identity.Namespace).Get(context.Background(), identity.Name, metav1.GetOptions{}) + if err != nil { + t.Fatalf("get AgentIdentity: %v", err) + } + persisted, err := decodeAgentIdentity(identityRaw) + if err != nil { + t.Fatalf("decode AgentIdentity: %v", err) + } + if got := monetizeapi.AgentIdentityAgentIDForChain(persisted.Status, "base"); got != "" { + t.Fatalf("AgentIdentity gained a base registration %q sourced from the stale base-sepolia id", got) + } + if got := monetizeapi.AgentIdentityAgentIDForChain(persisted.Status, "base-sepolia"); got != "777" { + t.Fatalf("base-sepolia registration = %q, want untouched 777", got) + } + + cm, err := c.configMaps.Namespace(identity.Namespace).Get(context.Background(), agentIdentityRegistrationName(identity), metav1.GetOptions{}) + if err != nil { + t.Fatalf("get identity ConfigMap: %v", err) + } + data, _, _ := unstructured.NestedStringMap(cm.Object, "data") + var doc map[string]any + if err := json.Unmarshal([]byte(data["agent-registration.json"]), &doc); err != nil { + t.Fatalf("unmarshal document: %v\n%s", err, data["agent-registration.json"]) + } + regs, _ := doc["registrations"].([]any) + if len(regs) != 1 { + t.Fatalf("registrations = %#v, want only the base-sepolia entry", doc["registrations"]) + } + reg0, _ := regs[0].(map[string]any) + if reg0["agentId"] != float64(777) { + t.Fatalf("registrations[0].agentId = %#v, want 777", reg0["agentId"]) + } + if reg0["agentRegistry"] != erc8004.BaseSepolia.CAIP10Registry() { + t.Fatalf("registrations[0].agentRegistry = %#v, want base-sepolia's registry, not base's", reg0["agentRegistry"]) + } +} + +// Disable must stop reporting whatever agentId was last recorded, including +// a stale/wrong-chain one — otherwise a disabled offer keeps showing it in +// ServiceOffer.status (and `obol sell status` output). +func TestReconcileRegistrationStatus_DisableClearsStaleAgentID(t *testing.T) { + status := &monetizeapi.ServiceOfferStatus{ + AgentID: "stale-sepolia-id", + RegistrationTxHash: "0xstale", + } + offer := readyOffer("svc") + offer.Namespace = "demo" + offer.Spec.Registration.Enabled = false + + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), identityListKinds()) + c := controllerForIdentityTest(dynClient) + + if err := c.reconcileRegistrationStatus(context.Background(), status, offer); err != nil { + t.Fatalf("reconcileRegistrationStatus: %v", err) + } + + if status.AgentID != "" { + t.Fatalf("AgentID = %q, want cleared on disable", status.AgentID) + } + if status.RegistrationTxHash != "" { + t.Fatalf("RegistrationTxHash = %q, want cleared on disable", status.RegistrationTxHash) } - if status.RegistrationTxHash != "0xabc" { - t.Fatalf("RegistrationTxHash = %q, want 0xabc", status.RegistrationTxHash) + if !isConditionTrue(*status, "Registered") { + t.Fatalf("Registered condition not set: %+v", status.Conditions) } } diff --git a/internal/serviceoffercontroller/name_injectivity_test.go b/internal/serviceoffercontroller/name_injectivity_test.go new file mode 100644 index 00000000..df72ac3a --- /dev/null +++ b/internal/serviceoffercontroller/name_injectivity_test.go @@ -0,0 +1,88 @@ +package serviceoffercontroller + +import "testing" + +// Invariant oracle: every child resource created in a namespace SHARED across +// offers must have a name that is injective over the full identity of the +// offer that owns it. A collision means two live offers fight over one object +// — the exact HTTP-500 failure mode of the original Canary402 ReferenceGrant +// report (offers Ready, requests 500). +// +// The ReferenceGrant is the only child the controller creates in the shared +// "x402" namespace (every other child — Middleware, HTTPRoute, +// RegistrationRequest — lives in the offer's own namespace, where the offer +// name is already unique). So its name must be injective over the pair +// (offer.Namespace, offer.Name). +// +// This is a property test, not an example test: it enumerates a space of +// identity pairs and asserts no two distinct pairs map to the same name. It is +// written to be reused for any future shared-namespace child by adding its +// builder to sharedNamespaceNameBuilders. + +// sharedNamespaceNameBuilders lists every name builder for a child resource +// that lives in a namespace shared across offers, keyed by (namespace, name). +var sharedNamespaceNameBuilders = map[string]func(namespace, name string) string{ + "backendReferenceGrant": backendReferenceGrantName, +} + +// labelFragments are pieces that appear in real Kubernetes namespaces and +// object names. Both namespaces and object names are DNS subdomains that may +// contain internal dashes, so any pair-join that leans on a dash separator is +// where injectivity breaks. These fragments are deliberately chosen so several +// distinct (namespace, name) pairs share the same dash-joined string. +var labelFragments = []string{"foo", "bar", "baz", "foo-bar", "bar-baz", "a", "a-b", "b", "team-a", "team", "a-team"} + +func TestSharedNamespaceNames_Injective(t *testing.T) { + for builderName, build := range sharedNamespaceNameBuilders { + t.Run(builderName, func(t *testing.T) { + seen := map[string][2]string{} // generated name -> (namespace, name) that first produced it + for _, ns := range labelFragments { + for _, name := range labelFragments { + got := build(ns, name) + if prev, clash := seen[got]; clash && prev != [2]string{ns, name} { + t.Errorf("collision: (ns=%q, name=%q) and (ns=%q, name=%q) both map to %q", + prev[0], prev[1], ns, name, got) + } + seen[got] = [2]string{ns, name} + } + } + }) + } +} + +// TestBackendReferenceGrantName_KnownCollision pins the exact adversarial pair +// from the injectivity property so a regression is unambiguous, not just a +// probabilistic property failure. +func TestBackendReferenceGrantName_KnownCollision(t *testing.T) { + a := backendReferenceGrantName("foo-bar", "baz") + b := backendReferenceGrantName("foo", "bar-baz") + if a == b { + t.Fatalf("(ns=foo-bar, name=baz) and (ns=foo, name=bar-baz) collide on grant name %q — "+ + "two live offers in different namespaces share one ReferenceGrant in x402 (HTTP 500)", a) + } +} + +// TestBackendReferenceGrantName_StableForSameOffer guards the other direction: +// the disambiguation must be deterministic, so the same offer always resolves +// to the same grant (server-side apply is by name — a nondeterministic name +// would orphan the previous grant on every reconcile). +func TestBackendReferenceGrantName_StableForSameOffer(t *testing.T) { + for i := 0; i < 4; i++ { + if got := backendReferenceGrantName("prod", "hyperliquid-analyst"); got != backendReferenceGrantName("prod", "hyperliquid-analyst") { + t.Fatalf("grant name not stable across calls: %q", got) + } + } + // And distinct offers that happen to dash-collide on the readable prefix + // still resolve to distinct, stable names. + seen := map[string]bool{} + for _, p := range [][2]string{{"foo-bar", "baz"}, {"foo", "bar-baz"}, {"foo-bar-baz", ""}} { + if p[1] == "" { + continue // empty name is not a real offer; skip + } + n := backendReferenceGrantName(p[0], p[1]) + if seen[n] { + t.Fatalf("stable-but-colliding name %q for %v", n, p) + } + seen[n] = true + } +} diff --git a/internal/serviceoffercontroller/offerbundle.go b/internal/serviceoffercontroller/offerbundle.go index 51db1778..ba9c34ba 100644 --- a/internal/serviceoffercontroller/offerbundle.go +++ b/internal/serviceoffercontroller/offerbundle.go @@ -1,6 +1,7 @@ package serviceoffercontroller import ( + _ "embed" "encoding/json" "fmt" "html/template" @@ -19,7 +20,7 @@ import ( // resources per origin. An offer with spec.hostname therefore gets its own // discovery documents — an openapi.json scoped to just that offer with // paths rooted at "/", a /.well-known/x402 resource list, and a minimal -// landing page — served by the same catalog httpd via per-offer ConfigMap +// landing page — served by the same static-site httpd via per-offer ConfigMap // keys and Exact-match rewrite routes on the offer's hostname. // offerBundleFile is one generated file: Key is the ConfigMap data key, @@ -42,7 +43,16 @@ func offerBundleKey(offer *monetizeapi.ServiceOffer, file string) string { // buildOfferBundles renders the discovery bundle for every hostname-bound, // operationally-included offer. Deterministic order (bundle keys sorted) so // content hashing is stable. -func buildOfferBundles(offers []*monetizeapi.ServiceOffer, profile schemas.StorefrontProfile) []offerBundleFile { +// +// upstreamOpenAPI supplies each offer's (possibly nil) upstream OpenAPI +// document. It must be a cache read, never a live fetch: this function runs +// under staticSiteMu on every offer's reconcile (see reconcileStaticSite), +// so a live HTTP call here would serialize every reconcile behind the +// slowest upstream and — on a flapping upstream — flip-flop the rebuilt +// content hash and roll the shared discovery pod. The production caller +// passes the Controller's upstreamOpenAPICache.get, refreshed independently +// from each offer's own reconcile. +func buildOfferBundles(offers []*monetizeapi.ServiceOffer, profile schemas.StorefrontProfile, upstreamOpenAPI func(*monetizeapi.ServiceOffer) map[string]any) []offerBundleFile { var bundles []offerBundleFile for _, offer := range offers { if offer == nil || offer.Spec.Hostname == "" { @@ -52,16 +62,32 @@ func buildOfferBundles(offers []*monetizeapi.ServiceOffer, profile schemas.Store // branding block overrides the storefront profile field-wise // (empty fields inherit). originProfile := storefront.MergeProfile(profile, offer.Spec.Branding.ProfilePatch()) + upstreamDoc := upstreamOpenAPI(offer) + openapiContent := buildOfferScopedOpenAPI(offer, originProfile) + x402Content := buildOfferWellKnownX402(offer) + if upstreamDoc != nil { + if rewritten, ok := rewriteUpstreamOpenAPI(upstreamDoc, offer, originProfile); ok { + openapiContent = rewritten + } + if expanded := buildOfferWellKnownX402FromOpenAPI(offer, upstreamDoc); expanded != "" { + x402Content = expanded + } + } bundles = append(bundles, offerBundleFile{ Key: offerBundleKey(offer, "openapi.json"), Path: offerBundleDir(offer) + "/openapi.json", - Content: buildOfferScopedOpenAPI(offer, originProfile), + Content: openapiContent, }, offerBundleFile{ Key: offerBundleKey(offer, "x402.json"), Path: offerBundleDir(offer) + "/x402.json", - Content: buildOfferWellKnownX402(offer), + Content: x402Content, + }, + offerBundleFile{ + Key: offerBundleKey(offer, "agent-registration.json"), + Path: offerBundleDir(offer) + "/agent-registration.json", + Content: buildOfferAgentRegistration(offer, originProfile), }, offerBundleFile{ Key: offerBundleKey(offer, "index.html"), @@ -74,8 +100,6 @@ func buildOfferBundles(offers []*monetizeapi.ServiceOffer, profile schemas.Store return bundles } -// bundleDigestInput folds bundle contents into the catalog content hash so -// bundle-only changes (e.g. a hostname added to one offer) roll the httpd. func bundleDigestInput(bundles []offerBundleFile) string { var b strings.Builder for _, f := range bundles { @@ -265,71 +289,12 @@ func wellKnownAccept(p monetizeapi.ServiceOfferPayment) map[string]any { return entry } -var offerLandingTmpl = template.Must(template.New("offer_landing").Parse(` - - - - - {{.Title}} - - - - - - - {{if .OGImageURL}} - - {{end}} - {{if .FaviconURL}}{{end}} - - {{if .CustomCSS}}{{end}} - - -
- {{if .LogoURL}}
{{.Operator}}{{if .ShowName}}{{.Operator}}{{end}}
{{end}} - {{.Price}} -

{{.Title}}

-
{{.DescriptionHTML}}
- -
-
-

For agents & developers

-

/openapi.json — request shapes + per-route pricing

-

/.well-known/x402 — signable x402 payment requirements

-

Payment is per-request via x402 micropayments: call an endpoint with no payment to receive the 402 challenge, sign one accepts[] entry, retry with the X-PAYMENT header.

-
- {{if .AboutHTML}} -
-

About {{.Operator}}

-
{{.AboutHTML}}
-
- {{end}} -

Sold by {{.Operator}} · Powered by Obol

-
- - -`)) +//go:embed templates/offer_landing.html +var offerLandingHTMLSrc string + +var offerLandingTmpl = template.Must( + template.New("offer_landing").Parse(offerLandingHTMLSrc), +) // buildOfferLandingHTML renders the hostname's "/" landing page: enough for // a human to understand the product and for previews/scrapers to get sane diff --git a/internal/serviceoffercontroller/render.go b/internal/serviceoffercontroller/render.go index c7340b09..f6829e88 100644 --- a/internal/serviceoffercontroller/render.go +++ b/internal/serviceoffercontroller/render.go @@ -22,12 +22,21 @@ import ( ) const ( - skillCatalogNamespace = "x402" - skillCatalogConfigMapName = "obol-skill-md" - skillCatalogRouteName = "obol-skill-md-route" - servicesJSONRouteName = "obol-services-json-route" - openAPIRouteName = "obol-openapi-route" - apiDocsRouteName = "obol-api-docs-route" + // The static site is the busybox httpd that serves every public artifact: + // skill.md, services.json (the storefront's own backend), openapi.json, + // the API docs, and one bundle per hostname-bound offer (landing page, + // scoped openapi.json, .well-known/x402). The "obol-skill-md" object names + // predate all but the first of those and are load-bearing — they are + // referenced by internal/tunnel (SERVICES_URL), cmd/obol/sell_info.go, + // internal/stackbackup, the embedded x402.yaml, and the storefront's + // next.config.ts. Renaming them is a live-cluster migration, so the Go + // identifiers say what this is and the wire names stay put. + staticSiteNamespace = "x402" + staticSiteConfigMapName = "obol-skill-md" + staticSiteRouteName = "obol-skill-md-route" + servicesJSONRouteName = "obol-services-json-route" + openAPIRouteName = "obol-openapi-route" + apiDocsRouteName = "obol-api-docs-route" // catalogHeadersMiddlewareName is the Traefik headers Middleware attached // to the public catalog HTTPRoutes (/skill.md, /openapi.json, /api, @@ -255,13 +264,13 @@ func agentIdentityLabels(identity *monetizeapi.AgentIdentity, appName string) ma } } -func buildSkillCatalogConfigMap(content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) *unstructured.Unstructured { +func buildStaticSiteConfigMap(content, servicesJSON, openAPIJSON, apiDocsHTML string, bundles []offerBundleFile) *unstructured.Unstructured { data := map[string]any{ "skill.md": content, "services.json": servicesJSON, "openapi.json": openAPIJSON, "api.html": apiDocsHTML, - "httpd.conf": ".md:text/markdown\n.json:application/json\n.html:text/html\n", + "httpd.conf": ".md:text/markdown\n.json:application/json\n.html:text/html\n.js:text/javascript\n", } for _, f := range bundles { data[f.Key] = f.Content @@ -271,10 +280,10 @@ func buildSkillCatalogConfigMap(content, servicesJSON, openAPIJSON, apiDocsHTML "apiVersion": "v1", "kind": "ConfigMap", "metadata": map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "labels": map[string]any{ - "app": skillCatalogConfigMapName, + "app": staticSiteConfigMapName, "obol.org/managed-by": "serviceoffer-controller", }, }, @@ -283,10 +292,10 @@ func buildSkillCatalogConfigMap(content, servicesJSON, openAPIJSON, apiDocsHTML } } -// skillCatalogVolumeItems projects the ConfigMap keys into the httpd's /www -// tree: the four aggregate documents plus one file per hostname-offer -// bundle entry (offers///…). -func skillCatalogVolumeItems(bundles []offerBundleFile) []any { +// staticSiteVolumeItems projects the ConfigMap keys into the httpd's /www +// tree: the aggregate documents plus one file per hostname-offer bundle +// entry (offers///…). +func staticSiteVolumeItems(bundles []offerBundleFile) []any { items := []any{ map[string]any{"key": "skill.md", "path": "skill.md"}, map[string]any{"key": "services.json", "path": "api/services.json"}, @@ -303,9 +312,9 @@ func skillCatalogVolumeItems(bundles []offerBundleFile) []any { return items } -func buildSkillCatalogDeployment(contentHash string, bundles []offerBundleFile) *unstructured.Unstructured { +func buildStaticSiteDeployment(contentHash string, bundles []offerBundleFile) *unstructured.Unstructured { labels := map[string]any{ - "app": skillCatalogConfigMapName, + "app": staticSiteConfigMapName, "obol.org/managed-by": "serviceoffer-controller", } return &unstructured.Unstructured{ @@ -313,8 +322,8 @@ func buildSkillCatalogDeployment(contentHash string, bundles []offerBundleFile) "apiVersion": "apps/v1", "kind": "Deployment", "metadata": map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "labels": labels, }, "spec": map[string]any{ @@ -354,14 +363,14 @@ func buildSkillCatalogDeployment(contentHash string, bundles []offerBundleFile) map[string]any{ "name": "content", "configMap": map[string]any{ - "name": skillCatalogConfigMapName, - "items": skillCatalogVolumeItems(bundles), + "name": staticSiteConfigMapName, + "items": staticSiteVolumeItems(bundles), }, }, map[string]any{ "name": "httpdconf", "configMap": map[string]any{ - "name": skillCatalogConfigMapName, + "name": staticSiteConfigMapName, "items": []any{map[string]any{"key": "httpd.conf", "path": "httpd.conf"}}, }, }, @@ -373,9 +382,9 @@ func buildSkillCatalogDeployment(contentHash string, bundles []offerBundleFile) } } -func buildSkillCatalogService() *unstructured.Unstructured { +func buildStaticSiteService() *unstructured.Unstructured { labels := map[string]any{ - "app": skillCatalogConfigMapName, + "app": staticSiteConfigMapName, "obol.org/managed-by": "serviceoffer-controller", } return &unstructured.Unstructured{ @@ -383,8 +392,8 @@ func buildSkillCatalogService() *unstructured.Unstructured { "apiVersion": "v1", "kind": "Service", "metadata": map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "labels": labels, }, "spec": map[string]any{ @@ -417,7 +426,7 @@ func buildCatalogHeadersMiddleware() *unstructured.Unstructured { "kind": "Middleware", "metadata": map[string]any{ "name": catalogHeadersMiddlewareName, - "namespace": skillCatalogNamespace, + "namespace": staticSiteNamespace, "labels": map[string]any{ "obol.org/managed-by": "serviceoffer-controller", }, @@ -451,14 +460,14 @@ func catalogHeadersFilters() []any { } } -func buildSkillCatalogHTTPRoute() *unstructured.Unstructured { +func buildStaticSiteHTTPRoute() *unstructured.Unstructured { return &unstructured.Unstructured{ Object: map[string]any{ "apiVersion": "gateway.networking.k8s.io/v1", "kind": "HTTPRoute", "metadata": map[string]any{ - "name": skillCatalogRouteName, - "namespace": skillCatalogNamespace, + "name": staticSiteRouteName, + "namespace": staticSiteNamespace, "labels": map[string]any{ "obol.org/managed-by": "serviceoffer-controller", }, @@ -484,8 +493,8 @@ func buildSkillCatalogHTTPRoute() *unstructured.Unstructured { "filters": catalogHeadersFilters(), "backendRefs": []any{ map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "port": int64(8080), }, }, @@ -511,7 +520,7 @@ func buildOpenAPIHTTPRoute() *unstructured.Unstructured { "kind": "HTTPRoute", "metadata": map[string]any{ "name": openAPIRouteName, - "namespace": skillCatalogNamespace, + "namespace": staticSiteNamespace, "labels": map[string]any{ "obol.org/managed-by": "serviceoffer-controller", }, @@ -537,8 +546,8 @@ func buildOpenAPIHTTPRoute() *unstructured.Unstructured { "filters": catalogHeadersFilters(), "backendRefs": []any{ map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "port": int64(8080), }, }, @@ -567,7 +576,7 @@ func buildAPIDocsHTTPRoute() *unstructured.Unstructured { "kind": "HTTPRoute", "metadata": map[string]any{ "name": apiDocsRouteName, - "namespace": skillCatalogNamespace, + "namespace": staticSiteNamespace, "labels": map[string]any{ "obol.org/managed-by": "serviceoffer-controller", }, @@ -599,8 +608,8 @@ func buildAPIDocsHTTPRoute() *unstructured.Unstructured { "filters": catalogHeadersFilters(), "backendRefs": []any{ map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "port": int64(8080), }, }, @@ -618,7 +627,7 @@ func buildServicesJSONHTTPRoute() *unstructured.Unstructured { "kind": "HTTPRoute", "metadata": map[string]any{ "name": servicesJSONRouteName, - "namespace": skillCatalogNamespace, + "namespace": staticSiteNamespace, "labels": map[string]any{ "obol.org/managed-by": "serviceoffer-controller", }, @@ -644,8 +653,8 @@ func buildServicesJSONHTTPRoute() *unstructured.Unstructured { "filters": catalogHeadersFilters(), "backendRefs": []any{ map[string]any{ - "name": skillCatalogConfigMapName, - "namespace": skillCatalogNamespace, + "name": staticSiteConfigMapName, + "namespace": staticSiteNamespace, "port": int64(8080), }, }, @@ -661,50 +670,91 @@ func hasLimits(offer *monetizeapi.ServiceOffer) bool { return offer.Spec.Limits.MaxInFlight > 0 || offer.Spec.Limits.RPS > 0 } -// buildLimitsMiddleware renders the Traefik protection middleware for -// spec.limits: inFlightReq (concurrency cap — the unbounded-concurrency -// hole on paid agents is pentest-proven) and/or rateLimit. Lives in the -// offer's namespace because Gateway API ExtensionRef resolves there. -func buildLimitsMiddleware(offer *monetizeapi.ServiceOffer) *unstructured.Unstructured { - spec := map[string]any{} +// buildLimitsMiddlewares renders one Traefik Middleware CR per limit type. +// Traefik rejects a single Middleware CR that declares both inFlightReq and +// rateLimit ("invalid middleware type" / incompatible types). Combining +// --max-in-flight and --rps therefore must produce two CRs and two +// ExtensionRef filters. Lives in the offer's namespace because Gateway API +// ExtensionRef resolves there. +func buildLimitsMiddlewares(offer *monetizeapi.ServiceOffer) []*unstructured.Unstructured { + var out []*unstructured.Unstructured if offer.Spec.Limits.MaxInFlight > 0 { - spec["inFlightReq"] = map[string]any{"amount": offer.Spec.Limits.MaxInFlight} + out = append(out, &unstructured.Unstructured{ + Object: map[string]any{ + "apiVersion": "traefik.io/v1alpha1", + "kind": "Middleware", + "metadata": map[string]any{ + "name": limitsInFlightMiddlewareName(offer.Name), + "namespace": offer.Namespace, + "ownerReferences": []any{ownerRefMap(offer)}, + }, + "spec": map[string]any{ + "inFlightReq": map[string]any{"amount": offer.Spec.Limits.MaxInFlight}, + }, + }, + }) } if offer.Spec.Limits.RPS > 0 { - spec["rateLimit"] = map[string]any{ - "average": offer.Spec.Limits.RPS, - "burst": offer.Spec.Limits.RPS * 2, - } - } - return &unstructured.Unstructured{ - Object: map[string]any{ - "apiVersion": "traefik.io/v1alpha1", - "kind": "Middleware", - "metadata": map[string]any{ - "name": limitsMiddlewareName(offer.Name), - "namespace": offer.Namespace, - "ownerReferences": []any{ownerRefMap(offer)}, + out = append(out, &unstructured.Unstructured{ + Object: map[string]any{ + "apiVersion": "traefik.io/v1alpha1", + "kind": "Middleware", + "metadata": map[string]any{ + "name": limitsRPSMiddlewareName(offer.Name), + "namespace": offer.Namespace, + "ownerReferences": []any{ownerRefMap(offer)}, + }, + "spec": map[string]any{ + "rateLimit": map[string]any{ + "average": offer.Spec.Limits.RPS, + "burst": offer.Spec.Limits.RPS * 2, + }, + }, }, - "spec": spec, - }, + }) } + return out +} + +func limitsInFlightMiddlewareName(offerName string) string { + return safeName("so-", offerName, "-inflight") } -func limitsMiddlewareName(offerName string) string { +func limitsRPSMiddlewareName(offerName string) string { + return safeName("so-", offerName, "-ratelimit") +} + +// legacyLimitsMiddlewareName is the pre-split combined CR name; still deleted +// on reconcile so upgrades tear down the broken object. +func legacyLimitsMiddlewareName(offerName string) string { return safeName("so-", offerName, "-limits") } -// limitsFilter is the ExtensionRef filter attached to gated rules when -// spec.limits is set. -func limitsFilter(offer *monetizeapi.ServiceOffer) map[string]any { - return map[string]any{ - "type": "ExtensionRef", - "extensionRef": map[string]any{ - "group": "traefik.io", - "kind": "Middleware", - "name": limitsMiddlewareName(offer.Name), - }, +// limitsFilters returns ExtensionRef filters for every configured limit +// middleware (zero, one, or both). +func limitsFilters(offer *monetizeapi.ServiceOffer) []any { + var filters []any + if offer.Spec.Limits.MaxInFlight > 0 { + filters = append(filters, map[string]any{ + "type": "ExtensionRef", + "extensionRef": map[string]any{ + "group": "traefik.io", + "kind": "Middleware", + "name": limitsInFlightMiddlewareName(offer.Name), + }, + }) } + if offer.Spec.Limits.RPS > 0 { + filters = append(filters, map[string]any{ + "type": "ExtensionRef", + "extensionRef": map[string]any{ + "group": "traefik.io", + "kind": "Middleware", + "name": limitsRPSMiddlewareName(offer.Name), + }, + }) + } + return filters } func buildHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructured { @@ -738,7 +788,7 @@ func buildHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructured // offer. Topology (proven live before being generalized here — see // docs/proposals/multistore-storefront-routing.md appendix): // -// - Exact /, /openapi.json, /.well-known/x402 → the catalog httpd, with +// - Exact /, /openapi.json, /.well-known/x402 → the static-site httpd, with // full-path rewrites into the offer's generated bundle files. These are // structurally free — they never touch the payment gate. // - PathPrefix / → x402-verifier, with the public path rewritten into the @@ -750,10 +800,31 @@ func buildHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructured // win their paths and everything else reaches the gate. func buildHostHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructured { dir := "/" + offerBundleDir(offer) + // The catalog httpd (busybox) sends no Cache-Control, so browsers + // heuristically cache these documents — a landing page or chat widget + // fix then never reaches returning visitors (an iframe kept replaying a + // stale /chat for hours). no-cache forces revalidation on every use; + // the vendor bundle is immutable behind its ?v= content hash instead. + cacheFilter := func(value string) map[string]any { + return map[string]any{ + "type": "ResponseHeaderModifier", + "responseHeaderModifier": map[string]any{ + "set": []any{map[string]any{"name": "Cache-Control", "value": value}}, + }, + } + } exactTo := func(publicPath, file string) map[string]any { return map[string]any{ + // Method-scoped to GET: discovery documents are read-only, and a + // root-priced offer (route pattern "/") advertises POST / + // as its paid resource — an unscoped Exact "/" match would + // shadow that POST into the static httpd (501) instead of the + // payment gate. "matches": []any{ - map[string]any{"path": map[string]any{"type": "Exact", "value": publicPath}}, + map[string]any{ + "path": map[string]any{"type": "Exact", "value": publicPath}, + "method": "GET", + }, }, "filters": []any{ map[string]any{ @@ -762,9 +833,39 @@ func buildHostHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructu "path": map[string]any{"type": "ReplaceFullPath", "replaceFullPath": dir + "/" + file}, }, }, + cacheFilter("no-cache"), + }, + "backendRefs": []any{ + map[string]any{"name": staticSiteConfigMapName, "namespace": staticSiteNamespace, "port": int64(8080)}, + }, + } + } + + // exactToShared rewrites to a file at the /www root (shared across + // offers) rather than into this offer's bundle directory. + exactToShared := func(publicPath, file string) map[string]any { + cache := "no-cache" + if strings.HasSuffix(file, ".js") { + cache = "public, max-age=31536000, immutable" + } + return map[string]any{ + "matches": []any{ + map[string]any{ + "path": map[string]any{"type": "Exact", "value": publicPath}, + "method": "GET", + }, + }, + "filters": []any{ + map[string]any{ + "type": "URLRewrite", + "urlRewrite": map[string]any{ + "path": map[string]any{"type": "ReplaceFullPath", "replaceFullPath": "/" + file}, + }, + }, + cacheFilter(cache), }, "backendRefs": []any{ - map[string]any{"name": skillCatalogConfigMapName, "namespace": skillCatalogNamespace, "port": int64(8080)}, + map[string]any{"name": staticSiteConfigMapName, "namespace": staticSiteNamespace, "port": int64(8080)}, }, } } @@ -790,7 +891,7 @@ func buildHostHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructu }, } if hasLimits(offer) { - catchallFilters = append(catchallFilters, limitsFilter(offer)) + catchallFilters = append(catchallFilters, limitsFilters(offer)...) } return &unstructured.Unstructured{ @@ -811,25 +912,33 @@ func buildHostHTTPRoute(offer *monetizeapi.ServiceOffer) *unstructured.Unstructu "sectionName": "web", }, }, - "rules": []any{ - exactTo("/", "index.html"), - exactTo("/openapi.json", "openapi.json"), - exactTo("/.well-known/x402", "x402.json"), - map[string]any{ - "matches": []any{ - map[string]any{"path": map[string]any{"type": "PathPrefix", "value": "/"}}, - }, - "filters": catchallFilters, - "backendRefs": []any{ - map[string]any{"name": "x402-verifier", "namespace": "x402", "port": int64(8080)}, - }, - }, - }, + "rules": hostRouteRules(offer, exactTo, exactToShared, catchallFilters), }, }, } } +// hostRouteRules assembles the dedicated-origin rule list: the four +// discovery rules (landing, openapi, x402, agent-registration) and the +// PathPrefix / payment gate last. +func hostRouteRules(offer *monetizeapi.ServiceOffer, exactTo, exactToShared func(string, string) map[string]any, catchallFilters []any) []any { + rules := []any{ + exactTo("/", "index.html"), + exactTo("/openapi.json", "openapi.json"), + exactTo("/.well-known/x402", "x402.json"), + exactTo("/.well-known/agent-registration.json", "agent-registration.json"), + } + return append(rules, map[string]any{ + "matches": []any{ + map[string]any{"path": map[string]any{"type": "PathPrefix", "value": "/"}}, + }, + "filters": catchallFilters, + "backendRefs": []any{ + map[string]any{"name": "x402-verifier", "namespace": "x402", "port": int64(8080)}, + }, + }) +} + // sharedOriginRule is the /services/ PathPrefix rule → verifier, // with the protection middleware attached when spec.limits is set. func sharedOriginRule(offer *monetizeapi.ServiceOffer) map[string]any { @@ -851,7 +960,7 @@ func sharedOriginRule(offer *monetizeapi.ServiceOffer) map[string]any { }, } if hasLimits(offer) { - rule["filters"] = []any{limitsFilter(offer)} + rule["filters"] = limitsFilters(offer) } return rule } @@ -866,7 +975,11 @@ func buildReferenceGrant(offer *monetizeapi.ServiceOffer) *unstructured.Unstruct "apiVersion": "gateway.networking.k8s.io/v1beta1", "kind": "ReferenceGrant", "metadata": map[string]any{ - "name": backendReferenceGrantName(offer.Name), + // Name must include the offer namespace: grants live in x402 and + // two offers with the same name in different namespaces would + // otherwise overwrite each other's ReferenceGrant (HTTP 500 / + // flapping backendRefs for one of the two). + "name": backendReferenceGrantName(offer.Namespace, offer.Name), "namespace": "x402", "labels": map[string]any{ "obol.org/serviceoffer-namespace": offer.Namespace, @@ -890,11 +1003,11 @@ func buildReferenceGrant(offer *monetizeapi.ServiceOffer) *unstructured.Unstruct }, // Hostname-bound offers backend their discovery bundle // (Exact / + /openapi.json + /.well-known/x402 rules) - // to the catalog httpd in this namespace. + // to the static-site httpd in this namespace. map[string]any{ "group": "", "kind": "Service", - "name": skillCatalogConfigMapName, + "name": staticSiteConfigMapName, }, }, }, @@ -925,8 +1038,32 @@ func childName(name string) string { return safeName("so-", name, "") } -func backendReferenceGrantName(name string) string { - return safeName("so-", name, "-backend-grant") +func backendReferenceGrantName(namespace, name string) string { + // All grants share the x402 namespace, so this name must be injective over + // (namespace, name). namespace and name are DNS subdomains that may both + // contain internal dashes, so NO literal separator between them is + // injective — (ns "foo-bar", name "baz") and (ns "foo", name "bar-baz") + // would both dash-join to "foo-bar-baz" and fight over one ReferenceGrant + // (the HTTP-500 collision the namespace-qualification was meant to end). + // Disambiguate with a hash of the exact tuple: "/" is illegal in a DNS + // label, so namespace+"/"+name is a collision-free encoding of the pair. + tuple := md5.Sum([]byte(namespace + "/" + name)) + return safeName("so-", namespace+"-"+name, "-"+fmt.Sprintf("%x", tuple)[:8]+"-backend-grant") +} + +// legacyBackendReferenceGrantName is the pre-4726dcfe non-namespaced grant +// name; still deleted on reconcile so upgrades tear down the orphaned object +// (grants created before the rename are never touched by the new name). +func legacyBackendReferenceGrantName(offerName string) string { + return safeName("so-", offerName, "-backend-grant") +} + +// intermediateBackendReferenceGrantName is the dash-joined 4726dcfe grant name +// (never released, but live on integration deployments) that the hash suffix +// superseded. Swept alongside the pre-4726dcfe name so an upgrade tears down +// both stale forms. +func intermediateBackendReferenceGrantName(namespace, name string) string { + return safeName("so-", namespace+"-"+name, "-backend-grant") } func registrationRequestName(name string) string { @@ -1140,7 +1277,7 @@ func offerPublishedForRegistration(offer *monetizeapi.ServiceOffer) bool { isConditionTrue(offer.Status, "RoutePublished") } -func buildSkillCatalogMarkdown(offers []*monetizeapi.ServiceOffer, baseURL string, explicit *schemas.StorefrontProfile) string { +func buildSkillMarkdown(offers []*monetizeapi.ServiceOffer, baseURL string, explicit *schemas.StorefrontProfile) string { baseURL = strings.TrimRight(baseURL, "/") profile := storefront.ResolvePublished(explicit, baseURL) @@ -1186,7 +1323,7 @@ func buildSkillCatalogMarkdown(offers []*monetizeapi.ServiceOffer, baseURL strin "", } - lines = append(lines, skillCatalogHowToPay(baseURL)...) + lines = append(lines, skillMarkdownHowToPay(baseURL)...) if len(ready) == 0 { lines = append(lines, "## Services", "", "**No services currently available.**", "") @@ -1250,7 +1387,7 @@ func buildSkillCatalogMarkdown(offers []*monetizeapi.ServiceOffer, baseURL strin lines = append(lines, fmt.Sprintf(" %d. %s", i+1, describePaymentDetail(payments[i]))) } } - lines = append(lines, skillCatalogRouteLines(offer, endpoint)...) + lines = append(lines, skillMarkdownRouteLines(offer, endpoint)...) if offer.Spec.Async.Enabled { access := "results are gated to the paying wallet (SIWX sign-in) or the `jobToken` from the 202 body" if offer.Spec.Async.EffectiveResultVisibility() == monetizeapi.ResultVisibilityPublic { @@ -1268,17 +1405,17 @@ func buildSkillCatalogMarkdown(offers []*monetizeapi.ServiceOffer, baseURL strin description = fmt.Sprintf("x402 payment-gated %s service", fallbackOfferType(offer)) } lines = append(lines, fmt.Sprintf("- **Description**: %s", description), "") - lines = append(lines, skillCatalogTryIt(offer, endpoint)...) + lines = append(lines, skillMarkdownTryIt(offer, endpoint)...) } return strings.Join(lines, "\n") } -// skillCatalogHowToPay returns the self-contained "How to pay" section. It +// skillMarkdownHowToPay returns the self-contained "How to pay" section. It // is written so any LLM agent — not just one running on Obol Stack — can // pay these endpoints by following the x402 v2 loop, without first reading // any external doc. baseURL points the reader at the machine-readable specs. -func skillCatalogHowToPay(baseURL string) []string { +func skillMarkdownHowToPay(baseURL string) []string { return []string{ "## How to pay (x402)", "", @@ -1330,12 +1467,12 @@ func catalogModelName(offer *monetizeapi.ServiceOffer) string { return "" } -// skillCatalogTryIt renders the per-offer "Try it" subsection: one curl that +// skillMarkdownTryIt renders the per-offer "Try it" subsection: one curl that // probes the 402 pricing, and one worked paid request. The paid example for // chat-shaped offers is buyprompts.Build's Example — the exact same bytes // /api/services.json publishes in the entry's buy.example — so the two // surfaces cannot drift. Agent buyers convert off copy-paste, not prose. -func skillCatalogTryIt(offer *monetizeapi.ServiceOffer, endpoint string) []string { +func skillMarkdownTryIt(offer *monetizeapi.ServiceOffer, endpoint string) []string { // Route-table offers: probe + pay against the primary paid route, not // the offer root (which may not be served at all when the table has no // catch-all). @@ -1391,7 +1528,7 @@ func skillCatalogTryIt(offer *monetizeapi.ServiceOffer, endpoint string) []strin // active, and upstream healthy serves buyers correctly regardless of // whether the on-chain identity has been minted yet. // -// Used by the storefront catalog (and the skill catalog) so an offer that +// Used by the services.json catalog (and skill.md) so an offer that // is functionally usable doesn't disappear from the operator's own // dashboard just because the agent wallet hasn't been funded with gas // yet. Callers should set ServiceCatalogEntry.RegistrationPending = true @@ -1877,12 +2014,12 @@ func describePaymentDetail(p monetizeapi.ServiceOfferPayment) string { return b.String() } -// skillCatalogRouteLines renders the per-route list for offers with a +// skillMarkdownRouteLines renders the per-route list for offers with a // declared route table (spec.routes). One line per route: methods, full // URL, gate/price, and the route summary. Offers without a route table // contribute nothing — their single implicit catch-all is already fully // described by the Endpoint/Payment lines above. -func skillCatalogRouteLines(offer *monetizeapi.ServiceOffer, endpoint string) []string { +func skillMarkdownRouteLines(offer *monetizeapi.ServiceOffer, endpoint string) []string { if len(offer.Spec.Routes) == 0 { return nil } diff --git a/internal/serviceoffercontroller/render_builders_test.go b/internal/serviceoffercontroller/render_builders_test.go index bac5fa10..a650ffc5 100644 --- a/internal/serviceoffercontroller/render_builders_test.go +++ b/internal/serviceoffercontroller/render_builders_test.go @@ -74,14 +74,14 @@ func assertRestrictedPSS(t *testing.T, deploymentName string, spec map[string]an } } -// TestBuildSkillCatalogDeployment_RestrictedPSS verifies the skill-md +// TestBuildStaticSiteDeployment_RestrictedPSS verifies the skill-md // httpd Deployment ships a Restricted-PSS-compliant securityContext. // Regression test for the cross-PR interaction with #521 surfaced by // the 14-PR integration test (Bug #3). -func TestBuildSkillCatalogDeployment_RestrictedPSS(t *testing.T) { - d := buildSkillCatalogDeployment("hash-x", nil) +func TestBuildStaticSiteDeployment_RestrictedPSS(t *testing.T) { + d := buildStaticSiteDeployment("hash-x", nil) spec, _ := d.Object["spec"].(map[string]any) - assertRestrictedPSS(t, skillCatalogConfigMapName, spec) + assertRestrictedPSS(t, staticSiteConfigMapName, spec) } // TestBuildAgentIdentityRegistrationDeployment_RestrictedPSS verifies the @@ -100,16 +100,16 @@ func TestBuildAgentIdentityRegistrationDeployment_RestrictedPSS(t *testing.T) { assertRestrictedPSS(t, agentIdentityRegistrationName(identity), spec) } -// TestBuildSkillCatalogConfigMap: exposes skill.md + services.json + openapi.json +// TestBuildStaticSiteConfigMap: exposes skill.md + services.json + openapi.json // + api docs HTML + httpd conf. -func TestBuildSkillCatalogConfigMap(t *testing.T) { - cm := buildSkillCatalogConfigMap("# Catalog", `{"displayName":"Acme","tagline":"t","logoUrl":"https://x/logo.png","services":[{"name":"a"}]}`, `{"openapi":"3.1.0"}`, "shell", nil) +func TestBuildStaticSiteConfigMap(t *testing.T) { + cm := buildStaticSiteConfigMap("# Catalog", `{"displayName":"Acme","tagline":"t","logoUrl":"https://x/logo.png","services":[{"name":"a"}]}`, `{"openapi":"3.1.0"}`, "shell", nil) - if cm.GetName() != skillCatalogConfigMapName { - t.Errorf("name = %q, want %q", cm.GetName(), skillCatalogConfigMapName) + if cm.GetName() != staticSiteConfigMapName { + t.Errorf("name = %q, want %q", cm.GetName(), staticSiteConfigMapName) } - if cm.GetNamespace() != skillCatalogNamespace { - t.Errorf("namespace = %q, want %q", cm.GetNamespace(), skillCatalogNamespace) + if cm.GetNamespace() != staticSiteNamespace { + t.Errorf("namespace = %q, want %q", cm.GetNamespace(), staticSiteNamespace) } data, _ := cm.Object["data"].(map[string]any) if data["skill.md"] != "# Catalog" { @@ -134,11 +134,11 @@ func TestBuildSkillCatalogConfigMap(t *testing.T) { } } -// TestBuildSkillCatalogDeployment: content-hash annotation + correct volume wiring +// TestBuildStaticSiteDeployment: content-hash annotation + correct volume wiring // (skill.md and api/services.json paths). -func TestBuildSkillCatalogDeployment(t *testing.T) { - d1 := buildSkillCatalogDeployment("hash-1", nil) - d2 := buildSkillCatalogDeployment("hash-2", nil) +func TestBuildStaticSiteDeployment(t *testing.T) { + d1 := buildStaticSiteDeployment("hash-1", nil) + d2 := buildStaticSiteDeployment("hash-2", nil) spec1, _ := d1.Object["spec"].(map[string]any) template1, _ := spec1["template"].(map[string]any) @@ -187,13 +187,13 @@ func TestBuildSkillCatalogDeployment(t *testing.T) { } } -// TestBuildSkillCatalogService: ClusterIP service on port 8080 with the +// TestBuildStaticSiteService: ClusterIP service on port 8080 with the // managed-by selector. -func TestBuildSkillCatalogService(t *testing.T) { - svc := buildSkillCatalogService() +func TestBuildStaticSiteService(t *testing.T) { + svc := buildStaticSiteService() - if svc.GetName() != skillCatalogConfigMapName { - t.Errorf("name = %q, want %q", svc.GetName(), skillCatalogConfigMapName) + if svc.GetName() != staticSiteConfigMapName { + t.Errorf("name = %q, want %q", svc.GetName(), staticSiteConfigMapName) } spec, _ := svc.Object["spec"].(map[string]any) if spec["type"] != "ClusterIP" { diff --git a/internal/serviceoffercontroller/render_test.go b/internal/serviceoffercontroller/render_test.go index 036ac28c..2f4209b8 100644 --- a/internal/serviceoffercontroller/render_test.go +++ b/internal/serviceoffercontroller/render_test.go @@ -90,6 +90,55 @@ func TestBuildHTTPRoute(t *testing.T) { } } +func TestBuildLimitsMiddlewares_SplitsInFlightAndRPS(t *testing.T) { + offer := &monetizeapi.ServiceOffer{ + ObjectMeta: metav1.ObjectMeta{Name: "gated", Namespace: "llm"}, + Spec: monetizeapi.ServiceOfferSpec{ + Limits: monetizeapi.ServiceOfferLimits{MaxInFlight: 4, RPS: 10}, + }, + } + mws := buildLimitsMiddlewares(offer) + if len(mws) != 2 { + t.Fatalf("len(middlewares) = %d, want 2 (one per Traefik type)", len(mws)) + } + names := map[string]map[string]any{} + for _, mw := range mws { + spec, _ := mw.Object["spec"].(map[string]any) + names[mw.GetName()] = spec + // Each CR must carry exactly one middleware type key. + if len(spec) != 1 { + t.Fatalf("middleware %q has %d spec keys; Traefik requires one type per CR: %v", mw.GetName(), len(spec), spec) + } + } + if _, ok := names[limitsInFlightMiddlewareName("gated")]["inFlightReq"]; !ok { + t.Fatal("missing inFlightReq middleware") + } + if _, ok := names[limitsRPSMiddlewareName("gated")]["rateLimit"]; !ok { + t.Fatal("missing rateLimit middleware") + } + filters := limitsFilters(offer) + if len(filters) != 2 { + t.Fatalf("len(filters) = %d, want 2 ExtensionRefs", len(filters)) + } +} + +func TestBuildLimitsMiddlewares_SingleType(t *testing.T) { + onlyInFlight := buildLimitsMiddlewares(&monetizeapi.ServiceOffer{ + ObjectMeta: metav1.ObjectMeta{Name: "a", Namespace: "llm"}, + Spec: monetizeapi.ServiceOfferSpec{Limits: monetizeapi.ServiceOfferLimits{MaxInFlight: 2}}, + }) + if len(onlyInFlight) != 1 { + t.Fatalf("maxInFlight only: len = %d", len(onlyInFlight)) + } + onlyRPS := buildLimitsMiddlewares(&monetizeapi.ServiceOffer{ + ObjectMeta: metav1.ObjectMeta{Name: "b", Namespace: "llm"}, + Spec: monetizeapi.ServiceOfferSpec{Limits: monetizeapi.ServiceOfferLimits{RPS: 5}}, + }) + if len(onlyRPS) != 1 { + t.Fatalf("rps only: len = %d", len(onlyRPS)) + } +} + func TestBuildReferenceGrant(t *testing.T) { offer := &monetizeapi.ServiceOffer{ ObjectMeta: metav1.ObjectMeta{ @@ -103,6 +152,12 @@ func TestBuildReferenceGrant(t *testing.T) { if grant.GetNamespace() != "x402" { t.Fatalf("grant namespace = %q, want x402", grant.GetNamespace()) } + if grant.GetName() != backendReferenceGrantName("llm", "demo") { + t.Fatalf("grant name = %q, want namespaced unique name", grant.GetName()) + } + if !strings.Contains(grant.GetName(), "llm") || !strings.Contains(grant.GetName(), "demo") { + t.Fatalf("grant name %q must include offer namespace and name", grant.GetName()) + } spec := grant.Object["spec"].(map[string]any) from := spec["from"].([]any)[0].(map[string]any) to := spec["to"].([]any)[0].(map[string]any) @@ -114,6 +169,26 @@ func TestBuildReferenceGrant(t *testing.T) { } } +// Two offers with the same name in different namespaces must not share a +// ReferenceGrant object name in x402 (overwrite / flapping 500s). +func TestBuildReferenceGrant_DisambiguatesByNamespace(t *testing.T) { + a := buildReferenceGrant(&monetizeapi.ServiceOffer{ + ObjectMeta: metav1.ObjectMeta{Name: "canary402", Namespace: "ns-a"}, + }) + b := buildReferenceGrant(&monetizeapi.ServiceOffer{ + ObjectMeta: metav1.ObjectMeta{Name: "canary402", Namespace: "ns-b"}, + }) + if a.GetName() == b.GetName() { + t.Fatalf("same-named offers in different namespaces must get distinct grant names; both = %q", a.GetName()) + } + if a.Object["spec"].(map[string]any)["from"].([]any)[0].(map[string]any)["namespace"] != "ns-a" { + t.Fatal("grant A from.namespace must be ns-a") + } + if b.Object["spec"].(map[string]any)["from"].([]any)[0].(map[string]any)["namespace"] != "ns-b" { + t.Fatal("grant B from.namespace must be ns-b") + } +} + func TestSetConditionUpdatesExistingEntry(t *testing.T) { status := monetizeapi.ServiceOfferStatus{ Conditions: []monetizeapi.Condition{{ @@ -611,7 +686,7 @@ func TestRegistrationDataURL(t *testing.T) { } } -func TestBuildSkillCatalogMarkdown(t *testing.T) { +func TestBuildSkillMarkdown(t *testing.T) { readyOffer := &monetizeapi.ServiceOffer{ ObjectMeta: metav1.ObjectMeta{Name: "flow-qwen", Namespace: "llm"}, Spec: monetizeapi.ServiceOfferSpec{ @@ -639,7 +714,7 @@ func TestBuildSkillCatalogMarkdown(t *testing.T) { }, } - content := buildSkillCatalogMarkdown([]*monetizeapi.ServiceOffer{readyOffer, notReadyOffer}, "https://example.com", nil) + content := buildSkillMarkdown([]*monetizeapi.ServiceOffer{readyOffer, notReadyOffer}, "https://example.com", nil) if !strings.Contains(content, "# Obol Stack Service Catalog") { t.Fatalf("catalog missing title:\n%s", content) @@ -655,13 +730,13 @@ func TestBuildSkillCatalogMarkdown(t *testing.T) { } } -// TestBuildSkillCatalogMarkdown_DrainAdditiveDetail locks in the +// TestBuildSkillMarkdown_DrainAdditiveDetail locks in the // pure-additive markdown surface: active offers must NOT emit a // `- **Available**:` detail bullet (that wire was removed when drain // landed). Draining offers may have a `- **Drain ends at**:` bullet // but never a separate Available bullet, because consumers detect // drain solely via the timestamp's presence. -func TestBuildSkillCatalogMarkdown_DrainAdditiveDetail(t *testing.T) { +func TestBuildSkillMarkdown_DrainAdditiveDetail(t *testing.T) { readyCond := []monetizeapi.Condition{{Type: "Ready", Status: "True"}} activeOffer := &monetizeapi.ServiceOffer{ ObjectMeta: metav1.ObjectMeta{Name: "alpha", Namespace: "llm"}, @@ -693,7 +768,7 @@ func TestBuildSkillCatalogMarkdown_DrainAdditiveDetail(t *testing.T) { Status: monetizeapi.ServiceOfferStatus{Conditions: readyCond}, } - content := buildSkillCatalogMarkdown( + content := buildSkillMarkdown( []*monetizeapi.ServiceOffer{activeOffer, drainingOffer}, "https://example.com", nil, @@ -717,10 +792,10 @@ func TestBuildSkillCatalogMarkdown_DrainAdditiveDetail(t *testing.T) { } } -func TestBuildSkillCatalogHTTPRoute(t *testing.T) { - route := buildSkillCatalogHTTPRoute() - if route.GetName() != skillCatalogRouteName { - t.Fatalf("route name = %q, want %q", route.GetName(), skillCatalogRouteName) +func TestBuildStaticSiteHTTPRoute(t *testing.T) { + route := buildStaticSiteHTTPRoute() + if route.GetName() != staticSiteRouteName { + t.Fatalf("route name = %q, want %q", route.GetName(), staticSiteRouteName) } spec := route.Object["spec"].(map[string]any) rules := spec["rules"].([]any) @@ -732,8 +807,8 @@ func TestBuildSkillCatalogHTTPRoute(t *testing.T) { } backends := firstRule["backendRefs"].([]any) backend := backends[0].(map[string]any) - if backend["name"] != skillCatalogConfigMapName { - t.Fatalf("backend name = %v, want %s", backend["name"], skillCatalogConfigMapName) + if backend["name"] != staticSiteConfigMapName { + t.Fatalf("backend name = %v, want %s", backend["name"], staticSiteConfigMapName) } } @@ -1622,8 +1697,8 @@ func TestBuildCatalogHeadersMiddleware(t *testing.T) { if mw.GetName() != catalogHeadersMiddlewareName { t.Fatalf("name = %q, want %q", mw.GetName(), catalogHeadersMiddlewareName) } - if mw.GetNamespace() != skillCatalogNamespace { - t.Fatalf("namespace = %q, want %q", mw.GetNamespace(), skillCatalogNamespace) + if mw.GetNamespace() != staticSiteNamespace { + t.Fatalf("namespace = %q, want %q", mw.GetNamespace(), staticSiteNamespace) } headers := mw.Object["spec"].(map[string]any)["headers"].(map[string]any) @@ -1648,7 +1723,7 @@ func TestBuildCatalogHeadersMiddleware(t *testing.T) { // (locked separately by TestBuildHTTPRoute's no-filters assertion). func TestCatalogRoutesCarryHeadersFilter(t *testing.T) { routes := map[string]*unstructured.Unstructured{ - "/skill.md": buildSkillCatalogHTTPRoute(), + "/skill.md": buildStaticSiteHTTPRoute(), "/api/services.json": buildServicesJSONHTTPRoute(), "/openapi.json": buildOpenAPIHTTPRoute(), "/api": buildAPIDocsHTTPRoute(), @@ -1699,12 +1774,12 @@ func TestBuildServiceCatalogJSON_SchemaVersion(t *testing.T) { } } -// TestBuildSkillCatalogMarkdown_TryIt asserts every offer detail block ends +// TestBuildSkillMarkdown_TryIt asserts every offer detail block ends // with a copy-paste "Try it" section: a 402 probe curl plus a worked paid // request. Chat-shaped offers must show the REAL model id (including the // AgentResolution fallback for type=agent) — the same id services.json // publishes — and http offers a curl of the gated path. -func TestBuildSkillCatalogMarkdown_TryIt(t *testing.T) { +func TestBuildSkillMarkdown_TryIt(t *testing.T) { readyCond := []monetizeapi.Condition{{Type: "Ready", Status: "True"}} inferenceOffer := &monetizeapi.ServiceOffer{ ObjectMeta: metav1.ObjectMeta{Name: "flow-qwen", Namespace: "llm"}, @@ -1747,7 +1822,7 @@ func TestBuildSkillCatalogMarkdown_TryIt(t *testing.T) { Status: monetizeapi.ServiceOfferStatus{Conditions: readyCond}, } - content := buildSkillCatalogMarkdown( + content := buildSkillMarkdown( []*monetizeapi.ServiceOffer{inferenceOffer, agentOffer, httpOffer}, "https://seller.example", nil, ) diff --git a/internal/serviceoffercontroller/route_accept_test.go b/internal/serviceoffercontroller/route_accept_test.go new file mode 100644 index 00000000..155f11e8 --- /dev/null +++ b/internal/serviceoffercontroller/route_accept_test.go @@ -0,0 +1,262 @@ +package serviceoffercontroller + +import ( + "context" + "encoding/json" + "testing" + + "github.com/ObolNetwork/obol-stack/internal/monetizeapi" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/dynamic/fake" + ktesting "k8s.io/client-go/testing" +) + +// fake.FakeDynamicClient's ObjectTracker doesn't implement server-side-apply +// merge semantics for pure unstructured objects, so applyObject's +// types.ApplyPatchType Patch fails against it. Reactor turns an apply patch +// into create-or-update, leaving any existing .status untouched — matching +// real SSA, which never touches a subresource it doesn't target. Goes +// straight to the Tracker (not back through the client) — routing through +// dynClient itself would re-enter the Fake's non-reentrant action lock and +// deadlock. +func withApplyPatchSupport(dynClient *fake.FakeDynamicClient) *fake.FakeDynamicClient { + tracker := dynClient.Tracker() + dynClient.PrependReactor("patch", "*", func(action ktesting.Action) (bool, runtime.Object, error) { + patchAction := action.(ktesting.PatchAction) + if patchAction.GetPatchType() != types.ApplyPatchType { + return false, nil, nil + } + var patched map[string]any + if err := json.Unmarshal(patchAction.GetPatch(), &patched); err != nil { + return true, nil, err + } + gvr := patchAction.GetResource() + ns := patchAction.GetNamespace() + + existing, err := tracker.Get(gvr, ns, patchAction.GetName()) + if apierrors.IsNotFound(err) { + obj := &unstructured.Unstructured{Object: patched} + if err := tracker.Create(gvr, obj, ns); err != nil { + return true, nil, err + } + return true, obj, nil + } + if err != nil { + return true, nil, err + } + merged := existing.(*unstructured.Unstructured).DeepCopy() + for k, v := range patched { + merged.Object[k] = v + } + if err := tracker.Update(gvr, merged, ns); err != nil { + return true, nil, err + } + return true, merged, nil + }) + return dynClient +} + +func controllerForRouteTest(dynClient *fake.FakeDynamicClient) *Controller { + dynClient = withApplyPatchSupport(dynClient) + return &Controller{ + dynClient: dynClient, + client: dynClient, + middlewares: dynClient.Resource(monetizeapi.MiddlewareGVR), + httpRoutes: dynClient.Resource(monetizeapi.HTTPRouteGVR), + referenceGrants: dynClient.Resource(monetizeapi.ReferenceGrantGVR), + } +} + +func routeListKinds() map[schema.GroupVersionResource]string { + return map[schema.GroupVersionResource]string{ + monetizeapi.MiddlewareGVR: "MiddlewareList", + monetizeapi.HTTPRouteGVR: "HTTPRouteList", + monetizeapi.ReferenceGrantGVR: "ReferenceGrantList", + } +} + +// acceptedHTTPRoute seeds an HTTPRoute that Traefik has already reported as +// programmed (status.parents Accepted=True, ResolvedRefs=True). +func acceptedHTTPRoute(namespace, name string) *unstructured.Unstructured { + return &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "gateway.networking.k8s.io/v1", + "kind": "HTTPRoute", + "metadata": map[string]any{ + "name": name, + "namespace": namespace, + }, + "status": map[string]any{ + "parents": []any{ + map[string]any{ + "conditions": []any{ + map[string]any{"type": "Accepted", "status": "True"}, + map[string]any{"type": "ResolvedRefs", "status": "True"}, + }, + }, + }, + }, + }} +} + +// staleGenerationHTTPRoute seeds an HTTPRoute at metadata.generation 2 whose +// status.parents conditions are Accepted=True / ResolvedRefs=True but still +// carry observedGeneration 1 — Traefik reported on the PREVIOUS spec and +// hasn't reconciled the update yet. +func staleGenerationHTTPRoute(namespace, name string) *unstructured.Unstructured { + return &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "gateway.networking.k8s.io/v1", + "kind": "HTTPRoute", + "metadata": map[string]any{ + "name": name, + "namespace": namespace, + "generation": int64(2), + }, + "status": map[string]any{ + "parents": []any{ + map[string]any{ + "conditions": []any{ + map[string]any{"type": "Accepted", "status": "True", "observedGeneration": int64(1)}, + map[string]any{"type": "ResolvedRefs", "status": "True", "observedGeneration": int64(1)}, + }, + }, + }, + }, + }} +} + +func TestReconcileRoute_WaitsWhenAcceptanceIsStale(t *testing.T) { + offer := readyOffer("svc") + offer.Namespace = "stalegen" + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds( + runtime.NewScheme(), + routeListKinds(), + // applyObject's fake-SSA reactor merges the applied object over this + // fixture without touching .status, so the route Get right after + // apply still returns this stale generation/status pairing. + staleGenerationHTTPRoute(offer.Namespace, childName(offer.Name)), + ) + c := controllerForRouteTest(dynClient) + status := &monetizeapi.ServiceOfferStatus{} + + if err := c.reconcileRoute(context.Background(), status, offer); err != nil { + t.Fatalf("reconcileRoute: %v", err) + } + if isConditionTrue(*status, "RoutePublished") { + t.Fatalf("RoutePublished should stay False when Traefik hasn't reconciled the current generation: %+v", status.Conditions) + } + if reason := conditionReason(status, "RoutePublished"); reason != "WaitingForTraefikAcceptance" { + t.Fatalf("RoutePublished reason = %q, want WaitingForTraefikAcceptance", reason) + } +} + +func conditionReason(status *monetizeapi.ServiceOfferStatus, conditionType string) string { + for _, c := range status.Conditions { + if c.Type == conditionType { + return c.Reason + } + } + return "" +} + +// --- RoutePublished gating on Traefik acceptance ---------------------------- + +func TestReconcileRoute_WaitsForTraefikAcceptance(t *testing.T) { + offer := readyOffer("svc") + offer.Namespace = "pending" + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), routeListKinds()) + c := controllerForRouteTest(dynClient) + status := &monetizeapi.ServiceOfferStatus{} + + // applyObject creates the HTTPRoute with no status yet — Traefik hasn't + // reconciled it, so RoutePublished must stay False, not flip True on + // apply success alone. + if err := c.reconcileRoute(context.Background(), status, offer); err != nil { + t.Fatalf("reconcileRoute: %v", err) + } + if isConditionTrue(*status, "RoutePublished") { + t.Fatalf("RoutePublished should stay False until Traefik accepts the route: %+v", status.Conditions) + } + if reason := conditionReason(status, "RoutePublished"); reason != "WaitingForTraefikAcceptance" { + t.Fatalf("RoutePublished reason = %q, want WaitingForTraefikAcceptance", reason) + } +} + +func TestReconcileRoute_PublishesWhenAccepted(t *testing.T) { + offer := readyOffer("svc") + offer.Namespace = "accepted" + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds( + runtime.NewScheme(), + routeListKinds(), + acceptedHTTPRoute(offer.Namespace, childName(offer.Name)), + ) + c := controllerForRouteTest(dynClient) + status := &monetizeapi.ServiceOfferStatus{} + + if err := c.reconcileRoute(context.Background(), status, offer); err != nil { + t.Fatalf("reconcileRoute: %v", err) + } + if !isConditionTrue(*status, "RoutePublished") { + t.Fatalf("RoutePublished should be True once Traefik has accepted the route: %+v", status.Conditions) + } +} + +func TestReconcileRoute_WaitsForHostRouteAcceptance(t *testing.T) { + offer := readyOffer("svc") + offer.Namespace = "hostpending" + offer.Spec.Hostname = "svc.example.test" + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds( + runtime.NewScheme(), + routeListKinds(), + // Main route is accepted, but the dedicated host route is not. + acceptedHTTPRoute(offer.Namespace, childName(offer.Name)), + ) + c := controllerForRouteTest(dynClient) + status := &monetizeapi.ServiceOfferStatus{} + + if err := c.reconcileRoute(context.Background(), status, offer); err != nil { + t.Fatalf("reconcileRoute: %v", err) + } + if isConditionTrue(*status, "RoutePublished") { + t.Fatalf("RoutePublished should stay False until the host HTTPRoute is accepted too: %+v", status.Conditions) + } + if reason := conditionReason(status, "RoutePublished"); reason != "WaitingForTraefikAcceptance" { + t.Fatalf("RoutePublished reason = %q, want WaitingForTraefikAcceptance", reason) + } +} + +// --- legacy ReferenceGrant sweep -------------------------------------------- + +func TestReconcileRoute_DeletesLegacyReferenceGrant(t *testing.T) { + offer := readyOffer("svc") + offer.Namespace = "legacygrant" + legacyGrant := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "gateway.networking.k8s.io/v1beta1", + "kind": "ReferenceGrant", + "metadata": map[string]any{ + "name": legacyBackendReferenceGrantName(offer.Name), + "namespace": "x402", + }, + }} + dynClient := fake.NewSimpleDynamicClientWithCustomListKinds( + runtime.NewScheme(), + routeListKinds(), + legacyGrant, + acceptedHTTPRoute(offer.Namespace, childName(offer.Name)), + ) + c := controllerForRouteTest(dynClient) + status := &monetizeapi.ServiceOfferStatus{} + + if err := c.reconcileRoute(context.Background(), status, offer); err != nil { + t.Fatalf("reconcileRoute: %v", err) + } + + _, err := c.referenceGrants.Namespace("x402").Get(context.Background(), legacyBackendReferenceGrantName(offer.Name), metav1.GetOptions{}) + if !apierrors.IsNotFound(err) { + t.Fatalf("legacy ReferenceGrant should be deleted on reconcile, got err=%v", err) + } +} diff --git a/internal/serviceoffercontroller/routesurface_golden_test.go b/internal/serviceoffercontroller/routesurface_golden_test.go index 60a7d85e..601cabe1 100644 --- a/internal/serviceoffercontroller/routesurface_golden_test.go +++ b/internal/serviceoffercontroller/routesurface_golden_test.go @@ -137,7 +137,7 @@ func TestRouteSurface_Golden_VerifierAndOpenAPIAgree(t *testing.T) { // appears in skill.md with its method, full URL, and effective price. func TestRouteSurface_Golden_SkillMDListsEveryRoute(t *testing.T) { offer := routeTableOffer() - content := buildSkillCatalogMarkdown([]*monetizeapi.ServiceOffer{offer}, "https://example.com", nil) + content := buildSkillMarkdown([]*monetizeapi.ServiceOffer{offer}, "https://example.com", nil) for _, want := range []string{ "`POST https://example.com/services/audit/submit` — 0.5 USDC/request — Submit source for audit", diff --git a/internal/serviceoffercontroller/templates/offer_landing.html b/internal/serviceoffercontroller/templates/offer_landing.html new file mode 100644 index 00000000..25d5968e --- /dev/null +++ b/internal/serviceoffercontroller/templates/offer_landing.html @@ -0,0 +1,64 @@ + + + + + + {{.Title}} + + + + + + + {{if .OGImageURL}} + + {{end}} + {{if .FaviconURL}}{{end}} + + {{if .CustomCSS}}{{end}} + + +
+ {{if .LogoURL}}
{{.Operator}}{{if .ShowName}}{{.Operator}}{{end}}
{{end}} + {{.Price}} +

{{.Title}}

+
{{.DescriptionHTML}}
+ +
+
+

For agents & developers

+

/openapi.json — request shapes + per-route pricing

+

/.well-known/x402 — signable x402 payment requirements

+

Payment is per-request via x402 micropayments: call an endpoint with no payment to receive the 402 challenge, sign one accepts[] entry, retry with the X-PAYMENT header.

+
+ {{if .AboutHTML}} +
+

About {{.Operator}}

+
{{.AboutHTML}}
+
+ {{end}} +

Sold by {{.Operator}} · Powered by Obol

+
+ + diff --git a/internal/serviceoffercontroller/upstream_openapi.go b/internal/serviceoffercontroller/upstream_openapi.go new file mode 100644 index 00000000..e6266432 --- /dev/null +++ b/internal/serviceoffercontroller/upstream_openapi.go @@ -0,0 +1,457 @@ +package serviceoffercontroller + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "path" + "sort" + "strings" + "sync" + "time" + + "github.com/ObolNetwork/obol-stack/internal/erc8004" + "github.com/ObolNetwork/obol-stack/internal/monetizeapi" + "github.com/ObolNetwork/obol-stack/internal/schemas" + "k8s.io/apimachinery/pkg/types" +) + +// maxUpstreamOpenAPIBytes caps both the fetched body and the re-marshaled +// document. The rewritten doc lands in the shared "obol-skill-md" ConfigMap +// alongside every other offer's bundle, which is subject to Kubernetes' +// ~1MiB ConfigMap size limit; a single oversized upstream doc would brick +// static-site publishing for every offer, not just its own. 200KiB leaves +// ample headroom for many offers to coexist. +const maxUpstreamOpenAPIBytes = 200 * 1024 + +var upstreamOpenAPIClient = &http.Client{ + Timeout: 3 * time.Second, + // Never follow redirects: the target is offer-author-controlled + // (service/namespace/port/path) and the fetched document is republished + // publicly, so a redirect must not be able to silently retarget the + // fetch. + CheckRedirect: func(_ *http.Request, _ []*http.Request) error { + return http.ErrUseLastResponse + }, +} + +// tryUpstreamOpenAPI is the seam tests replace. It performs the actual +// network fetch; callers that need determinism across a slow/flapping +// upstream should go through upstreamOpenAPICache instead of calling this +// directly. +var tryUpstreamOpenAPI = fetchUpstreamOpenAPI + +func fetchUpstreamOpenAPI(offer *monetizeapi.ServiceOffer) map[string]any { + if offer == nil || offer.IsAgent() || offer.IsInference() { + return nil + } + if strings.TrimSpace(offer.Spec.Upstream.Service) == "" { + return nil + } + base := upstreamOpenAPIBase(offer) + for _, path := range upstreamOpenAPIPathCandidates(offer) { + doc, err := getJSONMap(base + path) + if err != nil || doc == nil { + continue + } + paths, _ := doc["paths"].(map[string]any) + if len(paths) == 0 { + continue + } + return doc + } + return nil +} + +// upstreamOpenAPIBase builds the fetch target for one offer's own upstream +// service. Deliberately offer.Namespace, not EffectiveNamespace(): +// Upstream.Namespace is offer-author-controlled, and the Gateway API data +// path only trusts a cross-namespace target once a ReferenceGrant +// authorizes it. This controller-side probe has no such check, so it must +// never leave the offer's own namespace. +func upstreamOpenAPIBase(offer *monetizeapi.ServiceOffer) string { + return fmt.Sprintf("http://%s.%s.svc.cluster.local:%d", + offer.Spec.Upstream.Service, + offer.Namespace, + offer.EffectivePort(), + ) +} + +func upstreamOpenAPIPathCandidates(offer *monetizeapi.ServiceOffer) []string { + var out []string + if offer.Spec.Registration.Metadata != nil { + if p := strings.TrimSpace(offer.Spec.Registration.Metadata["openapiPath"]); p != "" && isSimpleUpstreamPath(p) { + out = append(out, p) + } + } + out = append(out, "/v1/openapi.json", "/openapi.json") + return out +} + +// isSimpleUpstreamPath rejects anything but a plain, '/'-prefixed path: no +// traversal segments and no embedded scheme/authority that could redirect +// the request off the intended upstream once concatenated onto base. +func isSimpleUpstreamPath(p string) bool { + if !strings.HasPrefix(p, "/") || strings.Contains(p, "://") { + return false + } + for _, seg := range strings.Split(p, "/") { + if seg == ".." { + return false + } + } + return true +} + +func getJSONMap(url string) (map[string]any, error) { + resp, err := upstreamOpenAPIClient.Get(url) + if err != nil { + return nil, err + } + defer resp.Body.Close() + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return nil, fmt.Errorf("HTTP %d", resp.StatusCode) + } + body, err := io.ReadAll(io.LimitReader(resp.Body, maxUpstreamOpenAPIBytes+1)) + if err != nil { + return nil, err + } + if len(body) > maxUpstreamOpenAPIBytes { + return nil, fmt.Errorf("upstream openapi document exceeds %d bytes", maxUpstreamOpenAPIBytes) + } + var doc map[string]any + if err := json.Unmarshal(body, &doc); err != nil { + return nil, err + } + return doc, nil +} + +// upstreamOpenAPICacheEntry is one offer's last-fetched result. +type upstreamOpenAPICacheEntry struct { + generation int64 + doc map[string]any +} + +// upstreamOpenAPICache holds the last-good upstream fetch per offer, keyed +// by UID. refresh is called from an offer's own reconcile (outside +// staticSiteMu) at most once per observed generation; get is read-only and +// is all buildOfferBundles ever calls. This keeps a slow or flapping +// upstream from blocking, or flip-flopping the content hash of, the shared +// static-site rebuild that runs for every offer on every offer's reconcile. +type upstreamOpenAPICache struct { + mu sync.Mutex + entries map[types.UID]upstreamOpenAPICacheEntry +} + +// get returns the cached doc, or nil if no fetch has completed yet for this +// offer's current generation. +func (c *upstreamOpenAPICache) get(offer *monetizeapi.ServiceOffer) map[string]any { + if offer == nil { + return nil + } + c.mu.Lock() + defer c.mu.Unlock() + return c.entries[offer.UID].doc +} + +// refresh fetches (via fetch) and caches the result, but only when the +// offer's generation has moved on from what's cached — a requeue with no +// spec change (e.g. the 5s convergence retry, a tunnel URL change) reuses +// the last-good result instead of hitting the upstream again. +func (c *upstreamOpenAPICache) refresh(offer *monetizeapi.ServiceOffer, fetch func(*monetizeapi.ServiceOffer) map[string]any) { + if offer == nil { + return + } + c.mu.Lock() + cached, ok := c.entries[offer.UID] + c.mu.Unlock() + if ok && cached.generation == offer.Generation { + return + } + doc := fetch(offer) + c.mu.Lock() + if c.entries == nil { + c.entries = map[types.UID]upstreamOpenAPICacheEntry{} + } + c.entries[offer.UID] = upstreamOpenAPICacheEntry{generation: offer.Generation, doc: doc} + c.mu.Unlock() +} + +// forget drops a cache entry (offer deleted). +func (c *upstreamOpenAPICache) forget(uid types.UID) { + c.mu.Lock() + delete(c.entries, uid) + c.mu.Unlock() +} + +func rewriteUpstreamOpenAPI(doc map[string]any, offer *monetizeapi.ServiceOffer, profile schemas.StorefrontProfile) (string, bool) { + if doc == nil { + return "", false + } + out := map[string]any{} + for k, v := range doc { + out[k] = v + } + origin := offer.EffectiveOrigin() + out["servers"] = []any{map[string]any{"url": origin}} + info, _ := out["info"].(map[string]any) + if info == nil { + info = map[string]any{} + } else { + copied := map[string]any{} + for k, v := range info { + copied[k] = v + } + info = copied + } + if title := strings.TrimSpace(offer.Spec.Registration.Name); title != "" { + info["title"] = title + } + if desc := strings.TrimSpace(offer.Spec.Registration.Description); desc != "" { + info["description"] = desc + } + if email := strings.TrimSpace(profile.ContactEmail); email != "" { + info["contact"] = map[string]any{"name": strings.TrimSpace(profile.DisplayName), "email": email} + } + out["info"] = info + if _, has := out["x-discovery"]; !has { + out["x-discovery"] = map[string]any{ + "source": "upstream-openapi", + "note": "Full first-party API surface; paid ops carry x-payment-info + 402 responses (x402scan OpenAPI-first).", + } + } + encoded, err := json.MarshalIndent(out, "", " ") + if err != nil || len(encoded) > maxUpstreamOpenAPIBytes { + return "", false + } + return string(encoded), true +} + +func buildOfferWellKnownX402FromOpenAPI(offer *monetizeapi.ServiceOffer, doc map[string]any) string { + if doc == nil { + return "" + } + origin := strings.TrimRight(offer.EffectiveOrigin(), "/") + paths, _ := doc["paths"].(map[string]any) + if len(paths) == 0 { + return "" + } + pathKeys := make([]string, 0, len(paths)) + for p := range paths { + pathKeys = append(pathKeys, p) + } + sort.Strings(pathKeys) + defaultAccepts := []any{} + for _, p := range offer.EffectivePayments() { + defaultAccepts = append(defaultAccepts, wellKnownAccept(p)) + } + if len(defaultAccepts) == 0 { + return "" + } + var resources []any + for _, p := range pathKeys { + item, _ := paths[p].(map[string]any) + if item == nil { + continue + } + for _, m := range []string{"get", "post", "put", "patch", "delete", "head", "options"} { + op, _ := item[m].(map[string]any) + if op == nil || !openAPIOpIsPaid(op) { + continue + } + desc, _ := op["summary"].(string) + if desc == "" { + desc, _ = op["description"].(string) + } + if desc == "" { + desc = offerDescription(offer, "x402 payment-gated service.") + } + accepts := defaultAccepts + if price, ok := routePriceOverride(offer, m, p); ok { + payment := offer.EffectivePayments()[0] + payment.Price = price + accepts = []any{wellKnownAccept(payment)} + } + resources = append(resources, map[string]any{ + "resource": origin + joinOpenAPIPath("/", p), + "type": "http", + "method": strings.ToUpper(m), + "description": desc, + "x402Version": 2, + "accepts": accepts, + }) + } + } + if len(resources) == 0 { + return "" + } + if len(resources) > 200 { + resources = resources[:200] + } + encoded, err := json.MarshalIndent(map[string]any{"x402Version": 2, "resources": resources}, "", " ") + if err != nil { + return "" + } + return string(encoded) +} + +// routePriceOverride returns the price of the most specific spec.routes[] +// entry that both matches the upstream path p and method, and carries a +// price override, mirroring how buildOfferWellKnownX402 (offerbundle.go) +// honors rt.HasPriceOverride()/rt.Price for the non-upstream document. +// Routes without an override are skipped so lookups fall through to the +// offer's default payments instead of pinning to a route that has nothing +// to override. +func routePriceOverride(offer *monetizeapi.ServiceOffer, method, p string) (monetizeapi.ServiceOfferPriceTable, bool) { + routes := append([]monetizeapi.ServiceOfferRoute(nil), offer.EffectiveRoutes()...) + sortRouteTableBySpecificity(routes) + for _, rt := range routes { + if !rt.HasPriceOverride() || !routeMethodMatches(rt.Methods, method) { + continue + } + if openAPIRoutePatternMatches(rt.Path, p) { + return rt.Price, true + } + } + return monetizeapi.ServiceOfferPriceTable{}, false +} + +// routeMethodMatches reports whether method (any case) is among the route's +// declared methods; an empty list means the route applies to every method. +func routeMethodMatches(methods []string, method string) bool { + if len(methods) == 0 { + return true + } + for _, m := range methods { + if strings.EqualFold(m, method) { + return true + } + } + return false +} + +// openAPIRoutePatternMatches mirrors matchPattern in internal/x402/matcher.go +// (unexported there, so duplicated here): exact match, a trailing "/*" +// greedy prefix, or path.Match segment globs. Both packages interpret +// spec.routes[].path against the same offer-rooted path-world, so this must +// agree with what the verifier actually gates. +func openAPIRoutePatternMatches(pattern, p string) bool { + if !strings.Contains(pattern, "*") { + return pattern == p + } + if prefix, ok := strings.CutSuffix(pattern, "/*"); ok && !strings.Contains(prefix, "*") { + return p == prefix || strings.HasPrefix(p, prefix+"/") + } + patParts := strings.Split(strings.TrimPrefix(pattern, "/"), "/") + pParts := strings.Split(strings.TrimPrefix(p, "/"), "/") + if len(pParts) < len(patParts) { + return false + } + for i, pp := range patParts { + if i == len(patParts)-1 && pp == "*" { + return true + } + if i >= len(pParts) { + return false + } + if matched, err := path.Match(pp, pParts[i]); err != nil || !matched { + return false + } + } + return len(pParts) == len(patParts) +} + +func openAPIOpIsPaid(op map[string]any) bool { + if op == nil { + return false + } + if gate, _ := op["x-gate"].(string); gate == "free" { + return false + } + if _, hasPay := op["x-payment-info"]; hasPay { + return true + } + if sec, ok := op["security"].([]any); ok && len(sec) == 0 { + return false + } + if responses, ok := op["responses"].(map[string]any); ok { + if _, has402 := responses["402"]; has402 { + return true + } + } + return false +} + +func buildOfferAgentRegistration(offer *monetizeapi.ServiceOffer, profile schemas.StorefrontProfile) string { + origin := strings.TrimRight(offer.EffectiveOrigin(), "/") + name := strings.TrimSpace(offer.Spec.Registration.Name) + if name == "" { + name = offer.Name + } + desc := strings.TrimSpace(offer.Spec.Registration.Description) + if desc == "" { + desc = offerDescription(offer, "x402 payment-gated service.") + } + image := strings.TrimSpace(offer.Spec.Registration.Image) + if image == "" { + if logo := strings.TrimSpace(profile.LogoURL); logo != "" && (strings.HasPrefix(logo, "http://") || strings.HasPrefix(logo, "https://")) { + image = logo + } else { + image = origin + "/agent-icon.png" + } + } + services := []erc8004.ServiceDef{{Name: "web", Endpoint: origin, Version: "1.0.0"}} + if len(offer.Spec.Registration.Services) > 0 { + var scoped []erc8004.ServiceDef + for _, s := range offer.Spec.Registration.Services { + ep := strings.TrimSpace(s.Endpoint) + if ep == "" { + continue + } + if strings.Contains(ep, "://") && ep != origin && !strings.HasPrefix(ep, origin+"/") { + continue + } + scoped = append(scoped, erc8004.ServiceDef{ + Name: defaultString(s.Name, "web"), + Endpoint: ep, + Version: defaultString(s.Version, "1.0.0"), + }) + } + if len(scoped) > 0 { + services = scoped + } + } + if len(offer.Spec.Registration.Skills) > 0 || len(offer.Spec.Registration.Domains) > 0 { + services = append(services, erc8004.ServiceDef{ + Name: "OASF", Version: "0.8", + Skills: offer.Spec.Registration.Skills, Domains: offer.Spec.Registration.Domains, + }) + } + doc := erc8004.AgentRegistration{ + Type: erc8004.RegistrationType, Name: name, Description: desc, Image: image, + Active: offer.Spec.Registration.Enabled, X402Support: true, Services: services, + SupportedTrust: offer.Spec.Registration.SupportedTrust, + } + // ERC-8004 requires registrations[] to have >=1 entry once the offer is + // on-chain (status.AgentID). Buyers verifying --expected-agent-id + // (internal/buy/discover.go) fail closed without it. + if agentID := strings.TrimSpace(offer.Status.AgentID); agentID != "" { + registry := fmt.Sprintf("eip155:%d:%s", erc8004.BaseSepoliaChainID, erc8004.IdentityRegistryBaseSepolia) + if net, err := erc8004.ResolveNetwork(offer.Spec.Payment.Network); err == nil { + registry = net.CAIP10Registry() + } + doc.Registrations = []erc8004.OnChainReg{{ + AgentID: parseInt64(agentID), + AgentRegistry: registry, + }} + } + if meta := nonEmptyStringMap(offer.Spec.Registration.Metadata); len(meta) > 0 { + doc.Metadata = meta + } + encoded, err := json.MarshalIndent(doc, "", " ") + if err != nil { + return `{"type":"` + erc8004.RegistrationType + `","name":"` + name + `","active":false,"services":[]}` + } + return string(encoded) +} diff --git a/internal/serviceoffercontroller/upstream_openapi_test.go b/internal/serviceoffercontroller/upstream_openapi_test.go new file mode 100644 index 00000000..08bc2507 --- /dev/null +++ b/internal/serviceoffercontroller/upstream_openapi_test.go @@ -0,0 +1,302 @@ +package serviceoffercontroller + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/ObolNetwork/obol-stack/internal/erc8004" + "github.com/ObolNetwork/obol-stack/internal/monetizeapi" + "github.com/ObolNetwork/obol-stack/internal/schemas" +) + +// TestGetJSONMap_SizeCap pins the BLOCKER fix: an upstream body over the +// cap must be rejected outright, not read in full and re-marshaled into the +// shared "obol-skill-md" ConfigMap where it would blow the ~1MiB k8s limit +// for every offer's bundle, not just its own. +func TestGetJSONMap_SizeCap(t *testing.T) { + big := `{"paths":{"/x":{}},"pad":"` + strings.Repeat("a", maxUpstreamOpenAPIBytes) + `"}` + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Write([]byte(big)) + })) + defer srv.Close() + + if _, err := getJSONMap(srv.URL); err == nil { + t.Fatal("getJSONMap accepted an oversized body, want a size-cap error") + } + + small := `{"paths":{"/x":{}}}` + srv2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Write([]byte(small)) + })) + defer srv2.Close() + if _, err := getJSONMap(srv2.URL); err != nil { + t.Fatalf("getJSONMap rejected a small body: %v", err) + } +} + +// TestRewriteUpstreamOpenAPI_SizeCapFallsBack pins the re-marshal side of +// the same cap: even a document that arrived under budget must not be +// republished if rewriting (adding servers/info/x-discovery) pushes it over. +func TestRewriteUpstreamOpenAPI_SizeCapFallsBack(t *testing.T) { + offer := hostnameOffer() + oversized := map[string]any{ + "paths": map[string]any{"/x": map[string]any{}}, + "pad": strings.Repeat("a", maxUpstreamOpenAPIBytes), + } + if _, ok := rewriteUpstreamOpenAPI(oversized, offer, schemas.StorefrontProfile{}); ok { + t.Fatal("rewriteUpstreamOpenAPI accepted a document that re-marshals over the size cap") + } + + // End-to-end: buildOfferBundles must fall back to the offer-scoped + // openapi.json (buildOfferScopedOpenAPI) instead of failing the whole + // static site. + fallback := buildOfferScopedOpenAPI(offer, schemas.StorefrontProfile{}) + upstream := func(*monetizeapi.ServiceOffer) map[string]any { return oversized } + bundles := buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, schemas.StorefrontProfile{}, upstream) + var openapiContent string + for _, f := range bundles { + if f.Path == "offers/sec/audit/openapi.json" { + openapiContent = f.Content + } + } + if openapiContent != fallback { + t.Fatalf("oversized upstream doc leaked into the bundle instead of falling back to buildOfferScopedOpenAPI") + } +} + +// TestUpstreamOpenAPICache_DeterministicAcrossFlappingFetch pins the second +// BLOCKER fix: buildOfferBundles must never re-fetch live, and the cache +// that feeds it must fetch at most once per offer generation — a flapping +// upstream (different content on every call) must not change what the +// cache serves between reconciles of unrelated offers. +func TestUpstreamOpenAPICache_DeterministicAcrossFlappingFetch(t *testing.T) { + offer := hostnameOffer() + offer.UID = "offer-uid-1" + offer.Generation = 1 + + fetchCount := 0 + flapping := func(*monetizeapi.ServiceOffer) map[string]any { + fetchCount++ + return map[string]any{"paths": map[string]any{"/x": map[string]any{}}, "call": fetchCount} + } + + var cache upstreamOpenAPICache + cache.refresh(offer, flapping) + first := cache.get(offer) + if fetchCount != 1 { + t.Fatalf("fetchCount after first refresh = %d, want 1", fetchCount) + } + + // Same generation: refresh again (simulating a requeue with no spec + // change while the upstream is flapping) must not fetch again, and the + // bundle-facing read must be byte-identical to the first fetch. + cache.refresh(offer, flapping) + if fetchCount != 1 { + t.Fatalf("fetchCount after same-generation refresh = %d, want 1 (no re-fetch)", fetchCount) + } + second := cache.get(offer) + firstJSON, _ := json.Marshal(first) + secondJSON, _ := json.Marshal(second) + if string(firstJSON) != string(secondJSON) { + t.Fatalf("cached doc changed across a same-generation refresh: %s vs %s", firstJSON, secondJSON) + } + + // A real generation bump (spec change) does fetch again. + offer.Generation = 2 + cache.refresh(offer, flapping) + if fetchCount != 2 { + t.Fatalf("fetchCount after generation bump = %d, want 2", fetchCount) + } + + // buildOfferBundles only ever reads the cache — never triggers a fetch + // itself, however many times it's called (the static-site rebuild that + // happens on every offer's reconcile). + for i := 0; i < 3; i++ { + buildOfferBundles([]*monetizeapi.ServiceOffer{offer}, schemas.StorefrontProfile{}, cache.get) + } + if fetchCount != 2 { + t.Fatalf("fetchCount after 3 bundle rebuilds = %d, want 2 (buildOfferBundles must not fetch)", fetchCount) + } +} + +// TestBuildOfferWellKnownX402FromOpenAPI_PerRoutePriceOverride pins the +// price-override fix: a paid upstream operation whose path matches a +// spec.routes[] entry with a price override must be priced at that +// override, not the offer's default price for every op. +func TestBuildOfferWellKnownX402FromOpenAPI_PerRoutePriceOverride(t *testing.T) { + offer := hostnameOffer() // routeTableOffer: /submit POST overridden to 0.5, offer default 0.1 + doc := map[string]any{ + "paths": map[string]any{ + "/submit": map[string]any{ + "post": map[string]any{ + "security": []any{map[string]any{"x402": []any{}}}, + "x-payment-info": map[string]any{}, + "responses": map[string]any{"402": map[string]any{}}, + }, + }, + "/v1/other": map[string]any{ + "get": map[string]any{ + "security": []any{map[string]any{"x402": []any{}}}, + "x-payment-info": map[string]any{}, + "responses": map[string]any{"402": map[string]any{}}, + }, + }, + }, + } + var wk struct { + Resources []struct { + Resource string `json:"resource"` + Method string `json:"method"` + Accepts []map[string]any `json:"accepts"` + } `json:"resources"` + } + if err := json.Unmarshal([]byte(buildOfferWellKnownX402FromOpenAPI(offer, doc)), &wk); err != nil { + t.Fatal(err) + } + byResource := map[string]map[string]any{} + for _, r := range wk.Resources { + byResource[r.Resource] = r.Accepts[0] + } + submit := byResource["https://audit.v1337.example/submit"] + if submit["amount"] != "500000" { + t.Errorf("/submit accepts = %v, want the 0.5 route override (500000 atomic units)", submit) + } + other := byResource["https://audit.v1337.example/v1/other"] + if other["amount"] != "100000" { + t.Errorf("/v1/other accepts = %v, want the offer default (0.1 => 100000 atomic units)", other) + } +} + +// TestBuildOfferAgentRegistration_RegistrationsField pins the ERC-8004 +// registrations[] fix: the field must be populated once the offer carries +// an on-chain agentId (status.AgentID), and absent otherwise — buyers using +// --expected-agent-id fail closed on an empty registrations[] +// (internal/buy/discover.go verifyAgentID). +func TestBuildOfferAgentRegistration_RegistrationsField(t *testing.T) { + offer := hostnameOffer() + offer.Spec.Registration.Enabled = true + + var unregistered map[string]any + if err := json.Unmarshal([]byte(buildOfferAgentRegistration(offer, schemas.StorefrontProfile{})), &unregistered); err != nil { + t.Fatal(err) + } + if _, has := unregistered["registrations"]; has { + t.Errorf("unregistered offer already carries registrations[]: %v", unregistered["registrations"]) + } + + offer.Status.AgentID = "42" + var reg map[string]any + if err := json.Unmarshal([]byte(buildOfferAgentRegistration(offer, schemas.StorefrontProfile{})), ®); err != nil { + t.Fatal(err) + } + regs, ok := reg["registrations"].([]any) + if !ok || len(regs) != 1 { + t.Fatalf("registrations = %v, want one entry", reg["registrations"]) + } + entry := regs[0].(map[string]any) + if entry["agentId"] != float64(42) { + t.Errorf("agentId = %v, want 42", entry["agentId"]) + } + if entry["agentRegistry"] != erc8004.BaseSepolia.CAIP10Registry() { + t.Errorf("agentRegistry = %v, want %s (offer's base-sepolia payment network)", entry["agentRegistry"], erc8004.BaseSepolia.CAIP10Registry()) + } +} + +// TestBuildOfferAgentRegistration_OriginScope pins the origin-scope fix: +// an https(s) service endpoint must equal the offer's own origin, or be a +// sub-path of it — a same-prefix different-host origin like +// "https://audit.v1337.example.evil.tld" must not pass HasPrefix. +func TestBuildOfferAgentRegistration_OriginScope(t *testing.T) { + offer := hostnameOffer() + offer.Spec.Registration.Services = []monetizeapi.ServiceOfferService{ + {Name: "web", Endpoint: "https://audit.v1337.example.evil.tld/steal"}, + {Name: "a2a", Endpoint: "https://audit.v1337.example/a2a"}, + {Name: "root", Endpoint: "https://audit.v1337.example"}, + } + var reg map[string]any + if err := json.Unmarshal([]byte(buildOfferAgentRegistration(offer, schemas.StorefrontProfile{})), ®); err != nil { + t.Fatal(err) + } + services, _ := reg["services"].([]any) + var endpoints []string + for _, s := range services { + endpoints = append(endpoints, s.(map[string]any)["endpoint"].(string)) + } + for _, want := range endpoints { + if strings.Contains(want, "evil.tld") { + t.Fatalf("services leaked the evil-suffix origin: %v", endpoints) + } + } + joined := strings.Join(endpoints, ",") + if !strings.Contains(joined, "https://audit.v1337.example/a2a") || !strings.Contains(joined, "https://audit.v1337.example") { + t.Errorf("services missing legitimately-scoped endpoints: %v", endpoints) + } +} + +// TestUpstreamOpenAPIBase_NamespaceConstraint pins the SSRF fix's namespace +// constraint: the openapi probe must target the offer's OWN namespace, not +// an offer-author-controlled spec.upstream.namespace override — unlike the +// Gateway API data path, this controller-side fetch has no ReferenceGrant +// check gating a cross-namespace target. +func TestUpstreamOpenAPIBase_NamespaceConstraint(t *testing.T) { + offer := hostnameOffer() + offer.Namespace = "tenant-a" + offer.Spec.Upstream.Namespace = "tenant-b-secrets" + + base := upstreamOpenAPIBase(offer) + if !strings.Contains(base, "tenant-a.svc.cluster.local") { + t.Errorf("base = %q, want the offer's own namespace (tenant-a)", base) + } + if strings.Contains(base, "tenant-b") { + t.Errorf("base = %q, leaked spec.upstream.namespace override", base) + } +} + +// TestGetJSONMap_NoRedirectFollow pins the SSRF fix's redirect guard: a +// republished document must never come from wherever a 3xx points, since +// the offer author controls the fetch target and the result is served +// publicly. +func TestGetJSONMap_NoRedirectFollow(t *testing.T) { + var hitTarget bool + target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + hitTarget = true + w.Write([]byte(`{"paths":{"/x":{}}}`)) + })) + defer target.Close() + + redirector := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, target.URL, http.StatusFound) + })) + defer redirector.Close() + + if _, err := getJSONMap(redirector.URL); err == nil { + t.Fatal("getJSONMap followed a redirect, want it rejected as a non-2xx response") + } + if hitTarget { + t.Fatal("getJSONMap followed the redirect to a different host") + } +} + +// TestIsSimpleUpstreamPath pins the path-validation half of the SSRF fix: +// spec.registration.metadata["openapiPath"] is offer-author-controlled and +// gets string-concatenated onto the fetch base, so it must stay a plain +// '/'-prefixed path. +func TestIsSimpleUpstreamPath(t *testing.T) { + cases := map[string]bool{ + "/v1/openapi.json": true, + "/openapi.json": true, + "openapi.json": false, // not '/'-prefixed + "/../../etc/passwd": false, // traversal + "/v1/../../../secrets": false, // traversal + "http://evil.tld/x": false, // embedded scheme + "/x?y=http://evil.tld": false, // embedded scheme, even mid-path + } + for path, want := range cases { + if got := isSimpleUpstreamPath(path); got != want { + t.Errorf("isSimpleUpstreamPath(%q) = %v, want %v", path, got, want) + } + } +} diff --git a/internal/stack/safety.go b/internal/stack/safety.go index 643c844d..e76d3823 100644 --- a/internal/stack/safety.go +++ b/internal/stack/safety.go @@ -273,9 +273,11 @@ func pidAlive(pid int) bool { return p.Signal(syscall.Signal(0)) == nil } -// errSafetyAborted lets cmd/obol/stack.go distinguish "user said no" from -// other errors. The CLI maps this to exit code 0 with a brief message -// instead of a noisy error trace. +// errSafetyAborted is a sentinel for "user declined a ConfirmRunningServicesLoss +// prompt". destroyOldBackendIfSwitching returns it so Init stops the whole +// command before switching backends (Down/Purge instead return nil directly +// from their own top-level check); Init maps it to a clean exit-0. Exported via +// ErrSafetyAborted() for callers that want to detect the abort with errors.Is. var errSafetyAborted = errors.New("aborted by operator at safety prompt") // ErrSafetyAborted is exported for callers that want to detect the abort diff --git a/internal/stack/stack.go b/internal/stack/stack.go index 9135aceb..9ef3fcc6 100644 --- a/internal/stack/stack.go +++ b/internal/stack/stack.go @@ -42,8 +42,10 @@ const ( stackIDFile = ".stack-id" ) -// Init initializes the stack configuration -func Init(cfg *config.Config, u *ui.UI, force bool, backendName string) error { +// Init initializes the stack configuration. skipConfirm (--yes) bypasses the +// live-services confirmation prompt when --force switches backends and would +// otherwise destroy a cluster serving live traffic (mirrors Down/Purge). +func Init(cfg *config.Config, u *ui.UI, force bool, backendName string, skipConfirm bool) error { // Check if any stack config already exists (legacy k3d.yaml included). stackIDPath := filepath.Join(cfg.ConfigDir, stackIDFile) hasExistingConfig := false @@ -76,13 +78,13 @@ func Init(cfg *config.Config, u *ui.UI, force bool, backendName string) error { backendName = BackendK3d } - // If switching backends, destroy the old one first to prevent - // orphaned clusters (e.g., k3d containers still running after - // switching to k3s, or k3s process still alive after switching to k3d). - if hasExistingConfig && force { - destroyOldBackendIfSwitching(cfg, u, backendName, stackID) - } - + // Validate the new backend BEFORE destroying anything. Previously the old + // backend was torn down first and NewBackend/Prerequisites only checked + // afterward — an invalid or unavailable target backend left the old + // cluster destroyed with nothing running in its place (confirmed + // Canary402 full-surface audit finding). Resolve and validate the new + // backend first so a failure here aborts Init without having touched the + // old cluster. backend, err := NewBackend(backendName) if err != nil { return err @@ -97,6 +99,23 @@ func Init(cfg *config.Config, u *ui.UI, force bool, backendName string) error { return fmt.Errorf("prerequisites check failed: %w", err) } + // If switching backends, destroy the old one first to prevent + // orphaned clusters (e.g., k3d containers still running after + // switching to k3s, or k3s process still alive after switching to k3d). + // Gated on ConfirmRunningServicesLoss (same safety bar as Down/Purge) so + // this destructive step can't run unconfirmed against a cluster serving + // live traffic. + if hasExistingConfig && force { + if err := destroyOldBackendIfSwitching(cfg, u, backendName, stackID, skipConfirm); err != nil { + // A declined safety prompt stops Init entirely (nothing switched, + // old cluster left intact) and, like Down/Purge, exits cleanly. + if errors.Is(err, errSafetyAborted) { + return nil + } + return err + } + } + // Generate backend-specific config if err := backend.Init(cfg, u, stackID); err != nil { return err @@ -123,15 +142,30 @@ func Init(cfg *config.Config, u *ui.UI, force bool, backendName string) error { } // destroyOldBackendIfSwitching checks if the backend is changing and tears down -// the old one to prevent orphaned clusters running side by side. -func destroyOldBackendIfSwitching(cfg *config.Config, u *ui.UI, newBackend, stackID string) { +// the old one to prevent orphaned clusters running side by side. The destroy +// is gated on ConfirmRunningServicesLoss — the same safety bar `obol stack +// down`/`purge` use — since it's just as capable of destroying a cluster that +// is currently serving paid traffic. +func destroyOldBackendIfSwitching(cfg *config.Config, u *ui.UI, newBackend, stackID string, skipConfirm bool) error { oldBackend, err := LoadBackend(cfg) if err != nil { - return + return nil } if oldBackend.Name() == newBackend { - return // same backend, nothing to clean up + return nil // same backend, nothing to clean up + } + + proceed, err := ConfirmRunningServicesLoss(cfg, u, "stack init --force (backend switch)", skipConfirm) + if err != nil { + return err + } + if !proceed { + // Signal the decline to Init, which stops the whole command before + // touching anything (the old cluster is still serving traffic — it + // must NOT be left undestroyed while the backend config switches). + u.Info("Aborted.") + return errSafetyAborted } u.Warnf("Switching backend from %s to %s — destroying old cluster", oldBackend.Name(), newBackend) @@ -145,6 +179,7 @@ func destroyOldBackendIfSwitching(cfg *config.Config, u *ui.UI, newBackend, stac // Clean up stale config files from the old backend cleanupStaleBackendConfigs(cfg, oldBackend.Name()) + return nil } // cleanupStaleBackendConfigs removes config files belonging to the old backend diff --git a/internal/stack/stack_test.go b/internal/stack/stack_test.go index 4244e16c..2d4866e8 100644 --- a/internal/stack/stack_test.go +++ b/internal/stack/stack_test.go @@ -304,7 +304,9 @@ func TestDestroyOldBackendIfSwitching_CleansStaleConfigs(t *testing.T) { // Switch to k3s — k3d config should be cleaned up // (Destroy will fail because no real cluster, but cleanup should still work) - destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3s, "test-id") + if err := destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3s, "test-id", false); err != nil { + t.Fatalf("destroyOldBackendIfSwitching: %v", err) + } if _, err := os.Stat(k3dPath); !os.IsNotExist(err) { t.Error("k3d.yaml should be removed when switching to k3s") @@ -325,7 +327,9 @@ func TestDestroyOldBackendIfSwitching_NoopSameBackend(t *testing.T) { os.WriteFile(k3dPath, []byte("k3d config"), 0o644) // Same backend — nothing should be cleaned up - destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3d, "test-id") + if err := destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3d, "test-id", false); err != nil { + t.Fatalf("destroyOldBackendIfSwitching: %v", err) + } if _, err := os.Stat(k3dPath); os.IsNotExist(err) { t.Error("k3d.yaml should NOT be removed when re-initing same backend") @@ -347,7 +351,9 @@ func TestDestroyOldBackendIfSwitching_K3sToK3d(t *testing.T) { } // Switch to k3d — k3s files should be cleaned up - destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3d, "test-id") + if err := destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3d, "test-id", false); err != nil { + t.Fatalf("destroyOldBackendIfSwitching: %v", err) + } for _, f := range []string{k3sConfigFile, k3sPidFile, k3sLogFile} { if _, err := os.Stat(filepath.Join(tmpDir, f)); !os.IsNotExist(err) { @@ -367,7 +373,78 @@ func TestDestroyOldBackendIfSwitching_NoBackendFile(t *testing.T) { } // Should not panic or error - destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3d, "test-id") + if err := destroyOldBackendIfSwitching(cfg, ui.New(false), BackendK3d, "test-id", false); err != nil { + t.Fatalf("destroyOldBackendIfSwitching: %v", err) + } +} + +// TestDestroyOldBackendIfSwitching_LiveServicesRefusesNonInteractiveWithoutYes +// is the regression test for the Canary402 `stack init --force --backend ` +// finding: destroying the old backend's cluster is exactly as dangerous as +// `obol stack down`/`purge`, so it must be gated on the same +// ConfirmRunningServicesLoss safety bar instead of running unconditionally. +func TestDestroyOldBackendIfSwitching_LiveServicesRefusesNonInteractiveWithoutYes(t *testing.T) { + tmpDir := t.TempDir() + cfg := &config.Config{ + ConfigDir: tmpDir, + DataDir: filepath.Join(tmpDir, "data"), + BinDir: filepath.Join(tmpDir, "bin"), + StateDir: filepath.Join(tmpDir, "state"), + } + + SaveBackend(cfg, BackendK3d) + k3dPath := filepath.Join(tmpDir, k3dConfigFile) + os.WriteFile(k3dPath, []byte("k3d config"), 0o644) + + // A live sell-inference gateway makes this stack "serving traffic". + writeGatewayPID(t, cfg, "aeon", os.Getpid()) + + var buf bytes.Buffer + u := ui.NewForTest(&buf, &buf) // isTTY defaults false, no --yes + + err := destroyOldBackendIfSwitching(cfg, u, BackendK3s, "test-id", false) + if err == nil { + t.Fatal("expected error when switching backends non-interactively with live services and no --yes") + } + if !strings.Contains(err.Error(), "--yes") { + t.Errorf("error should mention --yes (operator escape hatch): %v", err) + } + + // Destroy (and the cleanup that follows it) must not have run: the old + // backend's stale config file must survive the refused call. + if _, statErr := os.Stat(k3dPath); statErr != nil { + t.Errorf("k3d.yaml should NOT be removed when the safety gate refuses: %v", statErr) + } +} + +// TestDestroyOldBackendIfSwitching_SkipConfirmStillDestroys ensures --yes +// keeps working as the non-interactive escape hatch after the safety gate +// was added, mirroring Down/Purge's --yes behavior. +func TestDestroyOldBackendIfSwitching_SkipConfirmStillDestroys(t *testing.T) { + tmpDir := t.TempDir() + cfg := &config.Config{ + ConfigDir: tmpDir, + DataDir: filepath.Join(tmpDir, "data"), + BinDir: filepath.Join(tmpDir, "bin"), + StateDir: filepath.Join(tmpDir, "state"), + } + + SaveBackend(cfg, BackendK3d) + k3dPath := filepath.Join(tmpDir, k3dConfigFile) + os.WriteFile(k3dPath, []byte("k3d config"), 0o644) + + writeGatewayPID(t, cfg, "aeon", os.Getpid()) + + var buf bytes.Buffer + u := ui.NewForTest(&buf, &buf) + + if err := destroyOldBackendIfSwitching(cfg, u, BackendK3s, "test-id", true); err != nil { + t.Fatalf("destroyOldBackendIfSwitching with skipConfirm: %v", err) + } + + if _, statErr := os.Stat(k3dPath); !os.IsNotExist(statErr) { + t.Error("k3d.yaml should be removed when --yes overrides the confirmation") + } } func TestOllamaHostIPForBackend_K3s(t *testing.T) { diff --git a/internal/tunnel/tunnel.go b/internal/tunnel/tunnel.go index f7335777..d20f6553 100644 --- a/internal/tunnel/tunnel.go +++ b/internal/tunnel/tunnel.go @@ -1042,6 +1042,29 @@ func EnsureTunnelForSell(cfg *config.Config, u *ui.UI) (string, error) { return tunnelURL, nil } +// RefreshStorefront re-applies the storefront's HTTPRoute against the +// tunnel's currently tracked hostnames, narrowing or tearing it down for any +// hostname that is now offer-bound. CreateStorefront only sees ServiceOffer +// bindings that already exist on the cluster at call time, so a hostname +// bound by a manifest applied AFTER the tunnel was last (re)created — e.g. +// `obol sell ... --hostname X` — needs this explicit follow-up to be +// reflected immediately, instead of shadowing the offer's route until some +// later, unrelated tunnel/sell invocation happens to run CreateStorefront +// again (Canary402). +// +// It no-ops quietly when there is no persistent tunnel/hostname state yet +// (e.g. a first `obol sell ... --hostname X --no-register` before any +// tunnel has ever been created) — CreateStorefront has nothing to publish +// in that case, and EnsureTunnelForSell reconciles the storefront once the +// tunnel comes up. +func RefreshStorefront(cfg *config.Config) error { + hosts := storefrontHostnames(cfg, "") + if len(hosts) == 0 { + return nil + } + return CreateStorefront(cfg, hosts...) +} + // Stop scales the cloudflared deployment to 0 replicas. func Stop(cfg *config.Config, u *ui.UI) error { kubectlPath := filepath.Join(cfg.BinDir, "kubectl") @@ -1292,9 +1315,14 @@ func CreateStorefront(cfg *config.Config, hostnames ...string) error { } hosts = kept if len(hosts) == 0 { - // Every tracked hostname is offer-bound; nothing for the - // storefront to serve — a valid configuration. - return nil + // Every tracked hostname is offer-bound: the storefront has + // nothing left to serve at any hostname. Tear it down instead + // of leaving the previously-applied HTTPRoute (with the now- + // stale wider host list) on the cluster — Gateway API breaks + // the resulting PathPrefix-/ tie by route age, so that stale + // route would otherwise keep shadowing the offer's own + // dedicated-origin route. + return DeleteStorefront(cfg) } } diff --git a/internal/tunnel/tunnel_test.go b/internal/tunnel/tunnel_test.go index b00d4df8..2a116dbc 100644 --- a/internal/tunnel/tunnel_test.go +++ b/internal/tunnel/tunnel_test.go @@ -1,11 +1,14 @@ package tunnel import ( + "fmt" "os" "path/filepath" "strings" "testing" "time" + + "github.com/ObolNetwork/obol-stack/internal/config" ) func TestWaitReadyTimeout(t *testing.T) { @@ -155,6 +158,83 @@ func TestBuildLocalManagedConfigYAMLDeduplicates(t *testing.T) { } } +// writeFakeKubectl drops an executable kubectl stub into cfg.BinDir that logs +// every invocation (argv) to logPath and, for a `get serviceoffers.obol.org` +// query, prints boundHostname as the sole offer-bound hostname -- just enough +// to drive offerBoundHostnames()/DeleteStorefront() without a real cluster. +func writeFakeKubectl(t *testing.T, cfg *config.Config, logPath, boundHostname string) { + t.Helper() + if err := os.MkdirAll(cfg.BinDir, 0o755); err != nil { + t.Fatalf("mkdir bin dir: %v", err) + } + script := fmt.Sprintf(`#!/bin/sh +echo "$@" >> %q +case "$*" in + *"get serviceoffers.obol.org"*) echo %q ;; +esac +exit 0 +`, logPath, boundHostname) + if err := os.WriteFile(filepath.Join(cfg.BinDir, "kubectl"), []byte(script), 0o755); err != nil { + t.Fatalf("write fake kubectl: %v", err) + } +} + +// TestCreateStorefront_TearsDownWhenAllHostsOfferBound guards the Canary402 +// fix: when every hostname CreateStorefront was asked to serve turns out to +// be offer-bound, it must tear down the previously-applied tunnel-storefront +// HTTPRoute (via DeleteStorefront) instead of a no-op `return nil` that would +// leave a stale, wider-hostname route on the cluster shadowing the offer's +// own dedicated-origin route (equal PathPrefix-/ specificity, older route +// wins the Gateway API tie). +func TestCreateStorefront_TearsDownWhenAllHostsOfferBound(t *testing.T) { + cfg := newHostnameTestConfig(t) + writeFakeKubeconfig(t, cfg) + + logPath := filepath.Join(cfg.ConfigDir, "kubectl.log") + writeFakeKubectl(t, cfg, logPath, "example.com") + + if err := CreateStorefront(cfg, "example.com"); err != nil { + t.Fatalf("CreateStorefront: %v", err) + } + + logBytes, err := os.ReadFile(logPath) + if err != nil { + t.Fatalf("read kubectl log: %v", err) + } + log := string(logBytes) + + if !strings.Contains(log, "delete httproute/tunnel-storefront") { + t.Fatalf("CreateStorefront must tear down the stale tunnel-storefront HTTPRoute when every requested hostname is offer-bound; kubectl invocations:\n%s", log) + } + if strings.Contains(log, "apply") { + t.Fatalf("CreateStorefront must not re-apply the storefront HTTPRoute when every requested hostname is offer-bound; kubectl invocations:\n%s", log) + } +} + +// TestRefreshStorefront_NoOpWithoutTunnelState guards a first `obol sell +// ... --hostname X --no-register` before any persistent tunnel has ever been +// created: storefrontHostnames("") has nothing to report (no persistent +// tunnel state, no quick-tunnel URL to parse), so RefreshStorefront must +// no-op quietly instead of calling CreateStorefront with zero hostnames and +// surfacing its "requires at least one hostname" error as a confusing +// warning. EnsureTunnelForSell reconciles the storefront later once the +// tunnel exists. +func TestRefreshStorefront_NoOpWithoutTunnelState(t *testing.T) { + cfg := newHostnameTestConfig(t) + writeFakeKubeconfig(t, cfg) + + logPath := filepath.Join(cfg.ConfigDir, "kubectl.log") + writeFakeKubectl(t, cfg, logPath, "") + + if err := RefreshStorefront(cfg); err != nil { + t.Fatalf("RefreshStorefront should no-op without tunnel state, got error: %v", err) + } + + if _, err := os.ReadFile(logPath); err == nil { + t.Fatal("RefreshStorefront must not invoke kubectl when there is no hostname to publish") + } +} + func TestPatchAgentBaseURL_Insert(t *testing.T) { dir := t.TempDir() path := filepath.Join(dir, "values-obol.yaml") diff --git a/internal/validate/validate_test.go b/internal/validate/validate_test.go index 5421206a..744eec41 100644 --- a/internal/validate/validate_test.go +++ b/internal/validate/validate_test.go @@ -18,8 +18,10 @@ func TestName(t *testing.T) { "-leading-hyphen", // starts with hyphen "has spaces", "has_underscore", - "../etc/passwd", // path traversal + "../etc/passwd", // path traversal "a" + string(make([]byte, 63)), // too long (64 chars) + "has\nnewline", // /etc/hosts injection (Canary402 agent --id finding) + "has/slash", } for _, s := range invalid { if err := Name(s); err == nil { diff --git a/internal/x402/authgate.go b/internal/x402/authgate.go index 7a9c0f04..30321a04 100644 --- a/internal/x402/authgate.go +++ b/internal/x402/authgate.go @@ -1,6 +1,7 @@ package x402 import ( + "context" "crypto/hmac" "crypto/rand" "crypto/sha256" @@ -10,6 +11,7 @@ import ( "fmt" "html/template" "log" + "net" "net/http" "net/url" "strings" @@ -81,12 +83,40 @@ func requestHost(r *http.Request) string { // public URLs must not leak the internal prefix; on the shared origin the // prefix is the offer path itself. func publicPrefix(rule *RouteRule, host string) string { - if rule.Hostname != "" && strings.EqualFold(host, rule.Hostname) { + if rule == nil { + return "" + } + h := host + if hh, _, err := net.SplitHostPort(host); err == nil { + h = hh + } + rh := rule.Hostname + if rhH, _, err := net.SplitHostPort(rule.Hostname); err == nil { + rh = rhH + } + if rule.Hostname != "" && strings.EqualFold(h, rh) { return "" } return strings.TrimSuffix(rule.StripPrefix, "/") } +// routeRuleContextKey carries the matched RouteRule on the request context +// so 402 resource.URL builders can recover the public path-world after +// Traefik rewrote a dedicated-origin request into /services//…. +type routeRuleContextKey struct{} + +func withRouteRule(ctx context.Context, rule *RouteRule) context.Context { + if rule == nil { + return ctx + } + return context.WithValue(ctx, routeRuleContextKey{}, rule) +} + +func routeRuleFrom(ctx context.Context) *RouteRule { + rule, _ := ctx.Value(routeRuleContextKey{}).(*RouteRule) + return rule +} + // Broker contract headers (mirrors internal/jobbroker — see the comment on // buildUpstreamProxy for why they aren't imported). const ( diff --git a/internal/x402/chains.go b/internal/x402/chains.go index f12b1bc7..cf910aff 100644 --- a/internal/x402/chains.go +++ b/internal/x402/chains.go @@ -33,6 +33,12 @@ type ChainInfo struct { // EIP3009Version is the EIP-712 domain version. EIP3009Version string + + // Testnet marks a chain as a test network rather than a production + // mainnet. Consumers that only care about "real money" chains (e.g. the + // x402scan registry preflight, which indexes mainnet resources only) + // key off this instead of pattern-matching chain names. + Testnet bool } // AssetInfo describes the token and EIP-712 metadata used for x402 settlement. @@ -71,6 +77,7 @@ var ( CAIP2Network: "eip155:84532", USDCAddress: "0x036CbD53842c5426634e7929541eC2318f3dCF7e", Decimals: 6, + Testnet: true, // Base-Sepolia USDC is FiatTokenV2_2 whose EIP-712 domain name is // "USDC", NOT the mainnet "USD Coin". Advertising "USD Coin" makes a // real facilitator reject otherwise-valid signatures — the recurring @@ -107,6 +114,7 @@ var ( Decimals: 6, EIP3009Name: "USD Coin", EIP3009Version: "2", + Testnet: true, } ChainAvalancheMainnet = ChainInfo{ @@ -127,6 +135,7 @@ var ( Decimals: 6, EIP3009Name: "USD Coin", EIP3009Version: "2", + Testnet: true, } ChainArbitrumOne = ChainInfo{ @@ -147,9 +156,33 @@ var ( Decimals: 6, EIP3009Name: "USD Coin", EIP3009Version: "2", + Testnet: true, + } + + // allChains lists every chain the registry knows about, used to answer + // "is this CAIP-2 id a testnet" without a second hardcoded id list. + allChains = []ChainInfo{ + ChainBaseMainnet, ChainBaseSepolia, + ChainEthereumMainnet, + ChainPolygonMainnet, ChainPolygonAmoy, + ChainAvalancheMainnet, ChainAvalancheFuji, + ChainArbitrumOne, ChainArbitrumSepolia, } ) +// IsTestnetCAIP2 reports whether caip2 (e.g. "eip155:84532") identifies a +// known testnet chain. Unknown ids return false — callers that need to +// distinguish "known mainnet" from "unrecognized" should check membership +// separately. +func IsTestnetCAIP2(caip2 string) bool { + for _, c := range allChains { + if c.CAIP2Network == caip2 { + return c.Testnet + } + } + return false +} + // NormalizeNetworkID maps a human-friendly chain name to its CAIP-2 network // identifier. Already-normalized CAIP-2 values are returned as-is. func NormalizeNetworkID(network string) string { @@ -219,8 +252,22 @@ func ResolveChainInfo(name string) (ChainInfo, error) { // decimalToAtomic converts a decimal token amount (e.g. "0.001") to atomic // units using big.Float with 128-bit precision to avoid floating-point // truncation (e.g. 0.001 * 1e6 must produce 1000, not 999). -func decimalToAtomic(amount string, decimals int) string { - amountFloat, _, _ := new(big.Float).SetPrec(128).Parse(amount, 10) +// +// Returns an error rather than silently mispricing or panicking on +// malformed input (e.g. "0,01" EU-style decimals, "abc", "", "$0.01") — +// amount ultimately comes from operator/CLI input and, via a ServiceOffer +// applied directly to the cluster, can bypass CLI validation entirely. +func decimalToAtomic(amount string, decimals int) (string, error) { + amountFloat, _, err := new(big.Float).SetPrec(128).Parse(amount, 10) + if err != nil { + return "", fmt.Errorf("invalid decimal amount %q: %w", amount, err) + } + if amountFloat == nil { + return "", fmt.Errorf("invalid decimal amount %q", amount) + } + if amountFloat.Sign() < 0 { + return "", fmt.Errorf("amount must be non-negative: %q", amount) + } multiplier := new(big.Float).SetPrec(128).SetInt( new(big.Int).Exp(big.NewInt(10), big.NewInt(int64(decimals)), nil), ) @@ -228,7 +275,7 @@ func decimalToAtomic(amount string, decimals int) string { // Add 0.5 before truncating to int so we round to nearest. atomicFloat.Add(atomicFloat, new(big.Float).SetPrec(128).SetFloat64(0.5)) atomicInt, _ := atomicFloat.Int(nil) - return atomicInt.String() + return atomicInt.String(), nil } // DefaultAsset returns the default settlement asset for a chain. @@ -368,18 +415,31 @@ func ClampMaxTimeoutSeconds(n int64) int64 { // BuildV2Requirement creates a v2 PaymentRequirements for USDC payment on the // given chain. amount is the decimal USDC amount (e.g. "0.001" = $0.001). -func BuildV2Requirement(chain ChainInfo, amount, recipientAddress string, maxTimeoutSeconds int64) x402types.PaymentRequirements { +// Returns an error — rather than a $0 or panicking requirement — if amount +// is not a valid non-negative decimal. +func BuildV2Requirement(chain ChainInfo, amount, recipientAddress string, maxTimeoutSeconds int64) (x402types.PaymentRequirements, error) { return BuildV2RequirementWithAsset(chain, chain.DefaultAsset(), amount, recipientAddress, maxTimeoutSeconds) } // BuildV2RequirementWithAsset creates a v2 PaymentRequirements for the given // chain and settlement asset. Pass maxTimeoutSeconds=0 to fall back to // DefaultMaxTimeoutSeconds; operator-set values are clamped to MaxMaxTimeoutSeconds. -func BuildV2RequirementWithAsset(chain ChainInfo, asset AssetInfo, amount, recipientAddress string, maxTimeoutSeconds int64) x402types.PaymentRequirements { +// Returns an error — rather than a $0 or panicking requirement — if amount +// is not a valid non-negative decimal, or if it rounds to 0 atomic units +// (e.g. "0", or a sub-atomic price like "0.0000001" at 6 decimals); callers +// must fail closed on error, never serve the route for free. +func BuildV2RequirementWithAsset(chain ChainInfo, asset AssetInfo, amount, recipientAddress string, maxTimeoutSeconds int64) (x402types.PaymentRequirements, error) { + atomicAmount, err := decimalToAtomic(amount, asset.Decimals) + if err != nil { + return x402types.PaymentRequirements{}, fmt.Errorf("invalid price %q: %w", amount, err) + } + if atomicAmount == "0" { + return x402types.PaymentRequirements{}, fmt.Errorf("price %q rounds to 0 atomic units at %d decimals: refusing to advertise a $0 paid route", amount, asset.Decimals) + } return x402types.PaymentRequirements{ Scheme: "exact", Network: chain.CAIP2Network, - Amount: decimalToAtomic(amount, asset.Decimals), + Amount: atomicAmount, Asset: asset.Address, PayTo: recipientAddress, MaxTimeoutSeconds: int(ClampMaxTimeoutSeconds(maxTimeoutSeconds)), @@ -388,5 +448,5 @@ func BuildV2RequirementWithAsset(chain ChainInfo, asset AssetInfo, amount, recip "version": asset.EIP712Version, "assetTransferMethod": asset.TransferMethod, }, - } + }, nil } diff --git a/internal/x402/chains_test.go b/internal/x402/chains_test.go index 65835e2a..aa44818d 100644 --- a/internal/x402/chains_test.go +++ b/internal/x402/chains_test.go @@ -41,6 +41,32 @@ func TestResolveChainInfo(t *testing.T) { } } +func TestIsTestnetCAIP2(t *testing.T) { + tests := []struct { + caip2 string + want bool + }{ + {ChainBaseSepolia.CAIP2Network, true}, + {ChainPolygonAmoy.CAIP2Network, true}, + {ChainAvalancheFuji.CAIP2Network, true}, + {ChainArbitrumSepolia.CAIP2Network, true}, + {ChainBaseMainnet.CAIP2Network, false}, + {ChainEthereumMainnet.CAIP2Network, false}, + {ChainPolygonMainnet.CAIP2Network, false}, + {ChainAvalancheMainnet.CAIP2Network, false}, + {ChainArbitrumOne.CAIP2Network, false}, + {"eip155:999999", false}, // unknown chain — not a known testnet + } + + for _, tt := range tests { + t.Run(tt.caip2, func(t *testing.T) { + if got := IsTestnetCAIP2(tt.caip2); got != tt.want { + t.Errorf("IsTestnetCAIP2(%q) = %v, want %v", tt.caip2, got, tt.want) + } + }) + } +} + func TestChainUSDCAddresses(t *testing.T) { // Verify USDC addresses are non-empty and start with 0x. chains := []ChainInfo{ @@ -200,24 +226,33 @@ func TestBuildV2RequirementWithAsset_HonorsMaxTimeoutSeconds(t *testing.T) { EIP712Version: "2", } - got := BuildV2RequirementWithAsset(ChainBaseSepolia, asset, "0.001", "0xRecipient", 0) + got, err := BuildV2RequirementWithAsset(ChainBaseSepolia, asset, "0.001", "0xRecipient", 0) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } if got.MaxTimeoutSeconds != int(DefaultMaxTimeoutSeconds) { t.Errorf("zero spec value should map to default %d, got %d", DefaultMaxTimeoutSeconds, got.MaxTimeoutSeconds) } - got = BuildV2RequirementWithAsset(ChainBaseSepolia, asset, "0.001", "0xRecipient", 1800) + got, err = BuildV2RequirementWithAsset(ChainBaseSepolia, asset, "0.001", "0xRecipient", 1800) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } if got.MaxTimeoutSeconds != 1800 { t.Errorf("operator-set 1800 should reach the 402 verbatim, got %d", got.MaxTimeoutSeconds) } - got = BuildV2RequirementWithAsset(ChainBaseSepolia, asset, "0.001", "0xRecipient", MaxMaxTimeoutSeconds+1000) + got, err = BuildV2RequirementWithAsset(ChainBaseSepolia, asset, "0.001", "0xRecipient", MaxMaxTimeoutSeconds+1000) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } if got.MaxTimeoutSeconds != int(MaxMaxTimeoutSeconds) { t.Errorf("runaway value should clamp to cap %d, got %d", MaxMaxTimeoutSeconds, got.MaxTimeoutSeconds) } } func TestBuildV2RequirementWithAsset(t *testing.T) { - req := BuildV2RequirementWithAsset(ChainEthereumMainnet, AssetInfo{ + req, err := BuildV2RequirementWithAsset(ChainEthereumMainnet, AssetInfo{ Address: "0x0B010000b7624eb9B3DfBC279673C76E9D29D5F7", Symbol: "OBOL", Decimals: 18, @@ -225,6 +260,9 @@ func TestBuildV2RequirementWithAsset(t *testing.T) { EIP712Name: "Obol Network", EIP712Version: "1", }, "0.001", "0xRecipient", 0) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } if req.Amount != "1000000000000000" { t.Fatalf("Amount = %q, want 1000000000000000", req.Amount) @@ -239,3 +277,66 @@ func TestBuildV2RequirementWithAsset(t *testing.T) { t.Fatalf("name/version = %v/%v", req.Extra["name"], req.Extra["version"]) } } + +// TestDecimalToAtomic_RejectsMalformedInput is the regression guard for the +// Canary402 finding: decimalToAtomic used to discard the big.Float Parse +// error, so "0,01" (EU comma decimal) silently parsed as "0" (mispricing +// the route at $0) and "abc"/""/" " left amountFloat nil, panicking the +// next line's Mul(nil, ...) on every request in the verifier hot path. +func TestDecimalToAtomic_RejectsMalformedInput(t *testing.T) { + for _, amount := range []string{"abc", "", " ", "0,01", "$0.01", "-1"} { + t.Run(amount, func(t *testing.T) { + if _, err := decimalToAtomic(amount, 6); err == nil { + t.Fatalf("decimalToAtomic(%q) expected error, got nil", amount) + } + }) + } +} + +// TestBuildV2RequirementWithAsset_RejectsZeroAtomicAmount pins the +// Canary402 zero-price finding: decimalToAtomic only rejected parse errors +// and negative amounts, so a "0" price — or a sub-atomic price like +// "0.0000001" at 6 decimals, which rounds to 0 — sailed through and +// BuildV2RequirementWithAsset returned a payment requirement with +// Amount "0", contradicting its own doc promise to never do that. +// resolvePaidRoute must fail this option closed instead of advertising a +// free accepts[] entry. +func TestBuildV2RequirementWithAsset_RejectsZeroAtomicAmount(t *testing.T) { + asset := AssetInfo{ + Address: ChainBaseSepolia.USDCAddress, + Symbol: "USDC", + Decimals: 6, + TransferMethod: "eip3009", + } + for _, amount := range []string{"0", "0.0000001", "0.00000049"} { + t.Run(amount, func(t *testing.T) { + req, err := BuildV2RequirementWithAsset(ChainBaseSepolia, asset, amount, "0xRecipient", 0) + if err == nil { + t.Fatalf("BuildV2RequirementWithAsset(%q) = %+v, want error (0 atomic units)", amount, req) + } + }) + } +} + +func TestDecimalToAtomic_ValidInput(t *testing.T) { + tests := []struct { + amount string + decimals int + want string + }{ + {"0.01", 6, "10000"}, + {"1.5", 6, "1500000"}, + {"0.001", 18, "1000000000000000"}, + } + for _, tc := range tests { + t.Run(tc.amount, func(t *testing.T) { + got, err := decimalToAtomic(tc.amount, tc.decimals) + if err != nil { + t.Fatalf("decimalToAtomic(%q, %d): unexpected error %v", tc.amount, tc.decimals, err) + } + if got != tc.want { + t.Fatalf("decimalToAtomic(%q, %d) = %q, want %q", tc.amount, tc.decimals, got, tc.want) + } + }) + } +} diff --git a/internal/x402/forwardauth.go b/internal/x402/forwardauth.go index e477a4b4..d1d25b82 100644 --- a/internal/x402/forwardauth.go +++ b/internal/x402/forwardauth.go @@ -598,19 +598,70 @@ func facilitatorSettle(ctx context.Context, client *http.Client, facilitatorURL } func buildResourceURL(r *http.Request) string { - scheme := "http" - if r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https" { - scheme = "https" - } + // Scheme matches resolveSiteURL: default https for public hosts so a + // Cloudflare-terminated tunnel that forwards plaintext (X-Forwarded-Proto + // often "http") still advertises https:// resource URLs that match what + // discovery and x402scan probe. host := r.Host if forwardedHost := r.Header.Get("X-Forwarded-Host"); forwardedHost != "" { host = forwardedHost } - uri := r.RequestURI + scheme := "https" + if r.TLS == nil && r.Header.Get("X-Forwarded-Proto") != "https" && isLocalHost(host) { + scheme = "http" + } + + path := r.URL.Path + if r.URL.RawQuery != "" { + path = path + "?" + r.URL.RawQuery + } if forwardedURI := r.Header.Get("X-Forwarded-Uri"); forwardedURI != "" { - uri = forwardedURI + path = forwardedURI + } + + // Dedicated-origin offers rewrite /public → /services//public before + // the verifier matches. Challenge resource.URL must use the public path + // the buyer/discovery saw, not the internal rewritten path. + if rule := routeRuleFrom(r.Context()); rule != nil && rule.Hostname != "" { + if strings.EqualFold(stripHostPort(host), stripHostPort(rule.Hostname)) { + rawPath := r.URL.Path + if forwardedURI := r.Header.Get("X-Forwarded-Uri"); forwardedURI != "" { + if u, err := parsePathOnly(forwardedURI); err == nil { + rawPath = u + } + } + pub := publicPrefix(rule, host) + stripRoutePrefix(rule.StripPrefix, rawPath) + if r.URL.RawQuery != "" && !strings.Contains(pub, "?") { + pub += "?" + r.URL.RawQuery + } + path = pub + } + } + + return scheme + "://" + host + path +} + +func stripHostPort(host string) string { + if h, _, err := net.SplitHostPort(host); err == nil { + return h + } + return host +} + +// parsePathOnly returns the path (and optional query) from a URI that may be +// path-only (/foo) or absolute (https://h/foo?q=1). +func parsePathOnly(uri string) (string, error) { + if strings.HasPrefix(uri, "/") { + return uri, nil + } + // Absolute form — rare for X-Forwarded-Uri; fall back to as-is path parse. + if i := strings.Index(uri, "://"); i >= 0 { + rest := uri[i+3:] + if j := strings.Index(rest, "/"); j >= 0 { + return rest[j:], nil + } } - return scheme + "://" + host + uri + return uri, nil } // settlementInterceptor wraps a ResponseWriter to intercept the status code. diff --git a/internal/x402/forwardauth_resource_url_test.go b/internal/x402/forwardauth_resource_url_test.go new file mode 100644 index 00000000..739b83e7 --- /dev/null +++ b/internal/x402/forwardauth_resource_url_test.go @@ -0,0 +1,48 @@ +package x402 + +import ( + "net/http/httptest" + "testing" +) + +func TestBuildResourceURL_PublicHostDefaultsHTTPS(t *testing.T) { + r := httptest.NewRequest("GET", "/services/demo/v1", nil) + r.Host = "store.example.com" + // Tunnel edge often forwards X-Forwarded-Proto: http after TLS termination. + r.Header.Set("X-Forwarded-Proto", "http") + got := buildResourceURL(r) + want := "https://store.example.com/services/demo/v1" + if got != want { + t.Fatalf("buildResourceURL = %q, want %q", got, want) + } +} + +func TestBuildResourceURL_HostnameOfferStripsInternalPrefix(t *testing.T) { + rule := &RouteRule{ + Hostname: "audit.example.com", + StripPrefix: "/services/canary402", + OfferName: "canary402", + } + // Traefik rewrote /audit → /services/canary402/audit before the verifier. + r := httptest.NewRequest("GET", "/services/canary402/audit", nil) + r.Host = "audit.example.com" + r.Header.Set("X-Forwarded-Host", "audit.example.com") + r.Header.Set("X-Forwarded-Proto", "https") + r = r.WithContext(withRouteRule(r.Context(), rule)) + + got := buildResourceURL(r) + want := "https://audit.example.com/audit" + if got != want { + t.Fatalf("buildResourceURL = %q, want public path %q", got, want) + } +} + +func TestBuildResourceURL_LocalHostStaysHTTP(t *testing.T) { + r := httptest.NewRequest("GET", "/services/x", nil) + r.Host = "obol.stack:8080" + got := buildResourceURL(r) + want := "http://obol.stack:8080/services/x" + if got != want { + t.Fatalf("buildResourceURL = %q, want %q", got, want) + } +} diff --git a/internal/x402/forwardauth_test.go b/internal/x402/forwardauth_test.go index be68d887..23a786d5 100644 --- a/internal/x402/forwardauth_test.go +++ b/internal/x402/forwardauth_test.go @@ -79,9 +79,11 @@ func validPaymentHeader() string { } func testRequirements() []x402types.PaymentRequirements { - return []x402types.PaymentRequirements{ - BuildV2Requirement(ChainBaseSepolia, "0.001", "0xWallet", 0), + req, err := BuildV2Requirement(ChainBaseSepolia, "0.001", "0xWallet", 0) + if err != nil { + panic(err) // hardcoded valid literal — an error here is a test bug } + return []x402types.PaymentRequirements{req} } func TestForwardAuth_NoPayment_Returns402_AdvertisesExtensions(t *testing.T) { diff --git a/internal/x402/templates/DESIGN.md b/internal/x402/templates/DESIGN.md index 6b000df0..6b4362ed 100644 --- a/internal/x402/templates/DESIGN.md +++ b/internal/x402/templates/DESIGN.md @@ -9,8 +9,32 @@ Canonical design system: [`@obolnetwork/obol-ui`](../../../../obol-packages/packages/obol-ui/DESIGN.md). Sister surface that this page is closest to: [`obol-stack/web/public-storefront`](../../../web/public-storefront/DESIGN.md). -This page is deliberately the **smallest, most constrained** of the four -surfaces. + +## 0. The five public surfaces + +Each is identified by a `data-obol="page-*"` marker on its root element — +the stable hook custom CSS targets. All but the storefront are Go-rendered. + +| Marker | Surface | Rendered by | +| --- | --- | --- | +| `page-storefront` | Catalog storefront — lists every offer | `web/public-storefront` (Next.js) | +| `page-landing` | Per-offer landing, on the offer's own hostname | `internal/serviceoffercontroller/offerbundle.go` | +| `page-402` | **This page** — the payment gate | `internal/x402/paymentrequired.go` | +| `page-signin` | SIWX challenge | `internal/x402/authgate.go` | +| `page-error` | Auth/verifier error | `internal/x402/authgate.go` | + +Two distinctions worth holding onto, because both are easy to get backwards: + +- **`page-402` is not the storefront and not the landing page.** It is served + on a *paid resource path*, and only when `Accept` advertises `text/html` + (`prefersHTML`, `paymentrequired.go`); everything else gets the JSON + challenge. For a root-priced offer the paid resource is `POST /`, so a + browser visiting that origin's `/` gets `page-landing`, never this page. +- **`data-obol="checkout"` is not this page.** It is a reserved mount div that + appears on *both* `page-402` and `page-landing`. Custom CSS targeting it + hits two surfaces at once. + +This page is deliberately the **smallest, most constrained** of the five. --- @@ -37,24 +61,30 @@ buttons. --- -## 2. Brand contract (mirrored) +## 2. Brand contract (server-resolved) Same palette, type, and shape language as [`obol-ui/DESIGN.md § 1`](../../../../obol-packages/packages/obol-ui/DESIGN.md#1-brand-contract). -Token values are duplicated inline at the top of the `