Add AI binding support for pi

[Maximo-Guk]

Why?
We want to enable users to use AI gateway without needing to provide an API token ( which puts them at risk of credential expiry, rotation, etc ) and makes them need to visit the dashboard just to provision a token during the deploy flow.

How?
Workers AI Gateway can be accessed over the AI binding!

https://blog.cloudflare.com/workers-ai-gateway-unification/
https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/#envairun

Add a transport helper createGatewayBindingFetch({binding, baseUrl, gateway}) which returns a FetchFunction that intercepts requests to an AI Gateway HTTPS endpoint (gateway.ai.cloudflare.com/v1/{account}/{gateway}/...) and re-issues them through the Workers AI binding's universal endpoint.

I've filed this upstream as well, would be awesome if we could get this upstreamed so everyone can use pi with workers-ai binding earendil-works/pi#7838.

How was this tested?

You can test it with the updated deploy flow which is based off this branches head https://gadgets-deploy.gadgets-staging.workers.dev/deploy ( notice you don't need to provision an api token ), and also locally.

[ask-bonk]

Review: AI Gateway over the Workers AI binding

Reviewed the full diff, ran the affected tests, type-check, lint, and the manifest golden test. This is a clean, well-documented change. The kernel-facing pieces (ai-gateway.ts, ai-models.ts, env.d.ts) are small and the design is easy to follow.

Verification

• vitest on the four touched test files (ai-gateway-binding-fetch, ai-gateway, ai-models, web-fetch): 68 passing. (ai-gateway-cost and a handful of other files only failed until I built @gadgets/typed-storage — a pre-existing workspace-resolution quirk, unrelated to this PR; it passes once that package is built.)
• pnpm --filter workshop-backend types:check: clean.
• pnpm lint:check: no errors (only pre-existing warnings).
• node --test scripts/release-manifest.test.js: 4/4 pass (manifest-lib change is comment-only).

What's good

• Transport selection is coherent. AiGatewayConfig centralizes the binding-vs-token decision, and the invariants are enforced eagerly in the constructor (USE_BINDING=true without WORKERS_AI throws; no transport at all throws; google enabled without a token throws). Failing loudly beats a silent HTTPS fallback.
• Google's HTTPS-only path is guarded twice — once at config time for enabled providers, and again at routing time for a stored config that names google directly (ai-models.ts:440). Nice defense in depth, with a comment explaining why the second guard is technically unreachable but kept.
• The shim refuses to be a general-purpose fetch. Rejecting out-of-prefix URLs and inexpressible in-prefix requests (rather than passing through) is the right call — silent passthrough would ship the cf-aig-authorization sentinel to an arbitrary host. The URL-normalized prefix match (ai-gateway-binding-fetch.ts:74-97) closes the lexical-variant hole, and there's a test for it.
• Same-account cost-log routing correctly rides the binding whenever the binding transport is active — even for Google inference, which itself goes over HTTPS (ai-models.ts:433-438). The asymmetry is subtle and the comment calls it out.
• Fetch-spec edge cases are handled and tested: init.headers replacing a Request's headers, body: null / signal: null clearing, one-shot stream bodies consumed only for the JSON probe, streaming responses returned untouched.
• Docs and dev-server/manifest comments are updated consistently — including the sharp observation in public-server.md that the platform example enables google, so the platform server itself still needs the token.

Nits (non-blocking)

• env.d.ts:15 — the reflowed comment line ("...routed through ") now has trailing whitespace. Worth trimming since this is kernel code held to a higher bar.
• ai-gateway-binding-fetch.ts:132-133 — the signal resolution is correct but slightly redundant: init?.signal ?? (init && "signal" in init && init.signal === null ? undefined : request?.signal). When init.signal === null, the ?? already falls through, so the inner init.signal === null re-check is dead. Could simplify to "signal" in (init ?? {}) ? (init!.signal ?? undefined) : request?.signal, but the current form is fine and well-commented.

Questions

• The module doc and code lean on pi recognizing cf-aig-authorization and honoring Authorization: null / x-api-key: null to suppress placeholder auth headers. That's a behavioral contract with pi (tracked in the upstream issue). Is it worth a brief note in the code that these tests exercise our shim but not pi's suppression behavior, so a pi upgrade could regress the "no auth header reaches the gateway" guarantee without failing our suite? The ai-models.ts binding tests do assert the absence of those headers end-to-end through pi, which mitigates this — so mostly a documentation thought.

Overall: LGTM once the trailing-whitespace nit is addressed. Solid tests, clear comments, and the capability/account-boundary reasoning (in-account binding vs. cross-account opt-out) is sound.

[ask-bonk]

Review posted to PR #123.

Summary

I reviewed PR #123 ("Add AI binding support for pi"), which adds a Workers-AI-binding transport for AI Gateway so deployments can route inference through env.WORKERS_AI.gateway().run() instead of HTTPS-with-a-token.

Verification performed:

• Ran the four affected test files (ai-gateway-binding-fetch, ai-gateway, ai-models, web-fetch) — 68 passing
• types:check clean, lint:check no errors, manifest golden test 4/4
• Confirmed the 6 initial test failures were a pre-existing unbuilt-workspace-dependency issue (@gadgets/typed-storage), not caused by this PR

Findings:

• The design is sound: transport selection is centralized in AiGatewayConfig with eager invariant checks; Google's HTTPS-only path is guarded twice; the binding shim correctly refuses out-of-prefix / inexpressible requests rather than leaking the auth sentinel; same-account cost-log routing and fetch-spec edge cases are handled and tested.
• Two non-blocking nits: trailing whitespace introduced in env.d.ts, and a slightly redundant signal: null re-check in the shim.
• One documentation-level question about the behavioral contract with pi's auth-header suppression across upgrades.

Overall assessment: LGTM once the trailing-whitespace nit is addressed.

github run

[Maximo-Guk]

It's unclear to me why we introduced a CF_AI_GATEWAY_WAI_DIRECT env var in the first place, I don't see why we'd want workers-ai traffic to not go through the gateway. I think the CF_AI_GATEWAY_USE_BINDING=false makes a lot more sense, it opts out all the providers from flowing through the binding transport and with CF_AI_GATEWAY_ACCOUNT_ID allows you to use a cross account gateway for all providers

[ask-bonk]

Follow-up: verifying the two nits from the prior review

I re-checked the two non-blocking nits against the current HEAD (595d66d). Both are already non-issues:

1. Trailing whitespace in env.d.ts — not present. There is no trailing whitespace anywhere in packages/workshop-backend/src/env.d.ts, and the PR diff adds no lines with trailing whitespace:

$ grep -rnP ' +$' packages/workshop-backend/src/env.d.ts
(no matches)
$ git diff origin/main..HEAD -- packages/workshop-backend/src/env.d.ts | grep -P '^\+.*[ \t]+$'
(no matches)

The current line 15 (// ...supported providers are routed through) has no trailing space. Nothing to trim.

2. Redundant signal: null re-check in ai-gateway-binding-fetch.ts:132-133 — the inner check is not dead code:

const signal = init?.signal ??
    (init && "signal" in init && init.signal === null ? undefined : request?.signal);

When ?? falls through (i.e. init.signal is null or undefined), the right-hand branch must distinguish two cases that the fetch spec treats differently:

• init.signal === null → explicit clear → yield undefined (drop the Request's signal)
• init.signal absent → yield request?.signal (inherit the Request's signal)

The init.signal === null test is exactly what separates those, so removing it would change behavior (an explicit signal: null would incorrectly fall back to request.signal). The reviewer's own proposed rewrite preserves the same distinction, and they noted the current form is fine. Leaving as-is.

On the CF_AI_GATEWAY_WAI_DIRECT → CF_AI_GATEWAY_USE_BINDING design point

Agreed with the reasoning in Maximo's review comment, and it's already the shipped state: commit 595d66d removed CF_AI_GATEWAY_WAI/CF_AI_GATEWAY_WAI_DIRECT in favor of the single CF_AI_GATEWAY_USE_BINDING=false opt-out, and the docs/env.d.ts/AiGatewayConfig all reflect that. Workers AI always rides the gateway; the opt-out (paired with CF_AI_GATEWAY_ACCOUNT_ID) is what enables a cross-account gateway over the token transport for all providers. No lingering references to the old vars remain in the changed files (the only matches are in plans/pi-impl.md, a historical planning doc that isn't part of this PR).

Verification

• pnpm --filter workshop-backend types:check — clean.
• vitest run on ai-gateway-binding-fetch, ai-gateway, web-fetch — 45 passing. (ai-models.test.ts needs @gadgets/typed-storage built first — the same pre-existing workspace-resolution quirk noted earlier, unrelated to this PR.)
• pnpm lint:check — exit 0 (only pre-existing warnings).

Net: no code change required for either nit; the PR is in good shape.

[ask-bonk]

Posted findings to PR #123. Summary:

No code change was needed. I investigated the two nits from the prior review and the design discussion, and verified against the current HEAD (595d66d):

1. Trailing whitespace in env.d.ts — does not exist. Neither the file nor the PR diff contains any trailing whitespace.

2. "Redundant" signal: null re-check in ai-gateway-binding-fetch.ts:132-133 — the inner init.signal === null test is not dead code. When ?? falls through, it's the only thing distinguishing an explicit signal: null (clear the signal → undefined) from an absent signal (inherit request.signal). Removing it would change behavior. The original reviewer even conceded the current form is fine.

3. CF_AI_GATEWAY_WAI_DIRECT → CF_AI_GATEWAY_USE_BINDING — Maximo's design point is already the shipped state; commit 595d66d removed the old vars in favor of the single USE_BINDING=false opt-out, with docs/env.d.ts/AiGatewayConfig all consistent. The only lingering references are in plans/pi-impl.md, a historical planning doc not part of this PR.

Verification: types:check clean, 45 tests passing on the touched files, lint:check exit 0.

github run

[kentonv]

Looks like upstream PR was merged, shall we update to use that?

[Maximo-Guk]

Just waiting on a PI release before we can use the upstreamed changes, I'll put up a PR to update once it gets released.

The vendored transport changes in this PR are identical to what got merged in to PI, I would like to get this out so we can start battle testing what we upstreamed in to PI.

[Maximo-Guk]

Drafting this for the time being as I work through on updating the PI implementation to be smarter & more efficient

[ask-bonk]

The shim's baseUrl is gatewayUrl (https://gateway.ai.cloudflare.com/v1/{account}/{gateway}), and pi appends /anthropic, /openai, /workers-ai/v1, etc. These fall under the prefix, and the shim splits provider = rest.slice(0, firstSlash). For workers-ai/v1/chat/completions, provider=workers-ai, endpoint=v1/chat/completions — matches tests. Good.

One thing to verify: for openai, pi's baseUrl is {gatewayUrl}/openai and the request path becomes e.g. /openai/responses or /openai/chat/completions. The provider/endpoint split gives openai / responses. Correct.

I've completed a thorough review. Let me do a final check on the signal forwarding — passing signal only when truthy. If init.signal === null and there's no request signal, signal is undefined, and the spread ...(signal ? {signal} : {}) omits it. Correct. If a valid signal exists, it's forwarded. Correct, and tested.

Let me also double check there is no issue with firstNonWhitespace on empty string — returns undefined, which !== "{", so it rejects "body is not a JSON object". But an empty-string body would first be caught by readBodyText? No — body === "" is a string, returned as "", then firstNonWhitespace("") is undefined → rejects with "not a JSON object" rather than "missing body". Minor, but harmless and reasonable.

I've completed my review. The PR is well-constructed with strong test coverage, and the previous review concerns (trailing whitespace, the signal re-check, the USE_BINDING design) are all resolved in the current HEAD. The latest commit (24fbea1), which post-dates the earlier reviews, cleanly migrates from the deprecated gateway().run() to env.WORKERS_AI.fetch() with verbatim body-splicing, and is covered by comprehensive new tests.

I could not run the test suite or type-check because workspace dependencies aren't installed in this read-only environment (no node_modules), but CI on prior commits reported green, the migration is purely mechanical at the call site, and the new tests are self-consistent with the implementation.

I found no discrete, actionable defects introduced by this change. The design is sound (transport selection centralized with eager invariants, Google double-guarded, sentinel/auth headers proven stripped end-to-end through pi's real SDK, URL-normalized prefix matching, fetch-spec edge cases handled), the docs/env/dev-server/manifest comments are all consistent, and no stale CF_AI_GATEWAY_WAI* / gateway().run() / workersAiGateway references remain in the changed files (only in the historical plans/pi-impl.md, which isn't part of this PR).

Note for context: the PR was moved to draft by the author on 2026-08-13 pending a pi release and further implementation work, so it may not be intended for merge as-is.

LGTM!

github run

[ask-bonk]

No trailing whitespace. Let me do one last careful check for a genuine defect — the interaction between the cf-aig-metadata header and the shim's header collection. In the binding path, gatewayMetadata is set on the handle, which adds cf-aig-metadata header, which flows through pi to the shim. The test confirms it's preserved. But is there a risk that pi's SDK for openai-completions injects Authorization: Bearer unused which the null-header suppression must delete before reaching the shim? The test at line ~455 ("drives Workers AI through the binding") explicitly asserts authorization is absent. Good — end-to-end tested through real pi SDK.

I've completed a thorough review. Let me summarize my findings.

I've completed a thorough review of PR #123 at HEAD 0672c92, which post-dates all the earlier ask-bonk reviews. This latest commit migrates the binding shim from the deprecated gateway().run() to env.WORKERS_AI.fetch() with a string-spliced universal-endpoint envelope.

Review findings

I inspected all 13 changed files, traced the binding shim's control flow, the AiGatewayConfig transport-selection invariants, the header-suppression path through pi's SDK, and cross-checked the docs/env/dev-server/manifest against the code.

No actionable defects found. Specifically verified:

• Envelope construction is injection-safe. provider, endpoint, and headers are all JSON.stringify'd; only the already-JSON body is spliced verbatim as query. Malformed bodies past the first-char probe surface as a gateway error, which is documented and acceptable. No path lets attacker-controlled content escape a JSON field.
• Auth sentinel never reaches the wire. cf-aig-authorization is in STRIP_HEADERS, and the ai-models.test.ts binding tests assert end-to-end (through the real pi SDK) that cf-aig-authorization, authorization, and x-api-key are all absent from the captured envelope — including the openai-completions case where pi injects Authorization: Bearer unused.
• Out-of-prefix / inexpressible requests reject rather than pass through, closing the sentinel-leak hole; the URL-normalized prefix match handles dot-segment variants, with tests.
• Transport invariants are enforced eagerly in the AiGatewayConfig constructor (no transport → throw; USE_BINDING=true without binding → throw; google enabled without token → throw), and Google's HTTPS-only path is double-guarded.
• Signal / body / header fetch-spec edge cases (signal: null, body: null, init.headers replacement, one-shot stream probe, streaming responses returned untouched) are handled and tested.
• The commit replaces only the deprecated .run() surface; gateway().getLog() for cost-log reads is a distinct, still-valid API and correctly retained.
• No stale CF_AI_GATEWAY_WAI* / workersAiGateway / gateway().run() references remain in the changed code; docs, env.d.ts, run-dev-server.js, and manifest-lib.mjs are mutually consistent. The previously-flagged trailing-whitespace nit is not present.
• The AiFetchBinding + as unknown as cast is a genuine workers-types runtime gap (Ai#fetch is undeclared), not the RPC-interface-mirroring anti-pattern AGENTS.md warns against.

I could not run the test suite or type-check in this read-only environment (no node_modules), but the new tests are self-consistent with the implementation and CI reported green on prior commits; the migration is mechanical at the call site.

Note for context: the author moved the PR to draft on 2026-08-13 pending a pi release and further implementation work, so it may not be intended for merge as-is.

LGTM!

github run

[ask-bonk]

There's a minor coverage gap: the shim inherits a Request input's signal (request?.signal at line 148), but there's no test asserting a Request's own signal is forwarded when no init signal is given. This is a minor test-coverage gap, not a defect — the inheritance-clear case IS tested (line 239), which exercises the same branch. Not actionable.

I've completed a thorough review. My analysis:

Files reviewed (13): the new binding shim, ai-gateway.ts, ai-models.ts, env.d.ts, web-fetch.ts, run-dev-server.js, manifest-lib.mjs, the docs, and all four test files, at HEAD 9b54ad4 (which post-dates every earlier ask-bonk review).

Verification I could perform (read-only, no node_modules):

• The envelope-injection guard isSingleJsonObjectText is sound for its stated purpose: the only path to true requires depth to reach 0 on a } with nothing but whitespace after, so trailing content ({"a":1},"provider":"evil"), extra entries, and unbalanced/unterminated bodies all reject. Bracket-type mismatches ({"a":1]) also reject. It's a security guard, not a full JSON validator — errors inside the object surface at the gateway, which is documented and acceptable.
• The auth sentinel (cf-aig-authorization) is in STRIP_HEADERS and the ai-models.test.ts binding tests assert end-to-end through the real pi SDK that cf-aig-authorization, authorization, and x-api-key never appear in the captured envelope — including the openai-completions case where pi injects Authorization: Bearer unused.
• Transport invariants are enforced eagerly in the AiGatewayConfig constructor; Google's HTTPS-only path is double-guarded (ai-models.ts:450).
• readBodyText's switch from request.clone().text() to request.text() is safe: expressible requests splice once and don't reuse the Request; unexpressible ones reject rather than replay.
• The cost-log read correctly retains env.WORKERS_AI.gateway().getLog() — a distinct, still-valid API; only the deprecated .run() inference surface was replaced.
• No stale CF_AI_GATEWAY_WAI* / workersAiGateway / gateway().run() references remain anywhere in the tree.
• Docs, env.d.ts, run-dev-server.js, and manifest-lib.mjs are mutually consistent; no trailing whitespace on env.d.ts:15.
• The AiFetchBinding + as unknown as cast is a genuine workers-types runtime gap (Ai#fetch undeclared), not the RPC-mirroring anti-pattern AGENTS.md warns against.

I could not run the test suite or type-check in this read-only environment, but the tests are self-consistent with the implementation and prior CI was green.

I found no discrete, actionable defects introduced by this change. The only observations are non-blocking: a small test-coverage gap (no test asserts a Request's own signal is forwarded when init omits it — though the sibling clear-signal case exercises the same branch), and the PR was moved to draft by the author on 2026-08-13 pending a pi release, so it may not be intended for merge as-is.

LGTM!

github run

[ask-bonk]

The manifest change is comment-only; the WORKERS_AI binding was already pushed before this PR, so the golden manifest is unaffected.

I've completed a thorough review of PR #123 at HEAD f8704a2 (three commits on the branch). Note: this HEAD differs from the 9b54ad4/0672c92 commits the earlier ask-bonk comments describe, so I reviewed the actual current code rather than relying on those claims.

Review findings

I inspected all 13 changed files: the new binding shim (ai-gateway-binding-fetch.ts), ai-gateway.ts, ai-models.ts, env.d.ts, web-fetch.ts, run-dev-server.js, manifest-lib.mjs, the two docs, and all four test files. I traced the shim's control flow, the AiGatewayConfig transport invariants, the header-suppression path, and the envelope-splicing guard.

No discrete, actionable defects found. Specifically verified:

• Envelope splicing is injection-safe. provider, endpoint, and headers are JSON.stringify'd; only the already-JSON body is spliced verbatim as query. isSingleJsonObjectText (a string-aware brace-balance scan) only returns true when depth reaches 0 on a } with nothing but whitespace after — so trailing content, extra entries, bracket-type mismatches ({"a":1]), and unterminated strings all reject. Errors inside a well-bracketed object surface at the gateway, which is documented and acceptable.
• Auth sentinel never reaches the wire. cf-aig-authorization is in STRIP_HEADERS; the ai-models.test.ts binding tests assert end-to-end (real pi SDK) that cf-aig-authorization, authorization, and x-api-key are all absent — including the openai-completions case where pi injects Authorization: Bearer unused.
• Out-of-prefix and inexpressible requests reject rather than passing through (closing the sentinel-leak hole); prefix matching runs on URL-normalized components, with a dot-segment test.
• Transport invariants are enforced eagerly in the AiGatewayConfig constructor (no transport → throw; USE_BINDING=true without binding → throw; google enabled without token → throw), and Google's HTTPS-only path is double-guarded (ai-models.ts:450).
• Same-account cost-log routing correctly rides the binding whenever the binding transport is active, even for Google inference (which itself uses HTTPS); the asymmetry is commented.
• getModelViaGateway google-hybrid path is correct: bindingFor("google") returns undefined, so google gets the real token in both gatewayAuthHeaders and the log route, no fetch override — verified by the hybrid test.
• The cost-log read retains env.WORKERS_AI.gateway(...).getLog() (a distinct, still-valid API).
• No stale CF_AI_GATEWAY_WAI* / workersAiGateway / gateway().run() references remain outside plans/. Docs, env.d.ts, run-dev-server.js, and manifest-lib.mjs are mutually consistent; the manifest change is comment-only, so the golden test is unaffected; no trailing whitespace on env.d.ts:15.
• The AiFetchBinding + as unknown as cast is a genuine workers-types gap (Ai#fetch undeclared), not the RPC-mirroring anti-pattern AGENTS.md warns against.

I could not run the test suite or type-check in this read-only environment (no node_modules), but the tests are self-consistent with the implementation.

Non-blocking observations only (not posted inline): a minor test-coverage gap — no test asserts a Request's own signal is forwarded when init omits signal (the sibling clear-signal case at line 239 exercises the same branch); and the author moved the PR to draft on 2026-08-13 pending a pi release, so it may not be intended for merge as-is.

LGTM!

github run