Merge pull request schubergphilis#23 from schubergphilis/fix/gh-proxy-release-asset-probes

fix(gh-proxy): unblock uv on github.com release assets

Author: dtump

README.md

@@ -156,8 +156,8 @@ When a token is found:
 
 - The agent container gets a placeholder `GH_TOKEN=claude-docker-proxy` — enough for `gh` to consider itself authenticated. `gh auth token` inside the container returns this placeholder, not your real token, and it doesn't appear anywhere in the container's environment or filesystem.
 - `github.com`, `api.github.com`, and `uploads.github.com` resolve to the sidecar via `--add-host`. `objects.githubusercontent.com` and `codeload.github.com` (pre-signed release/archive URLs) are **not** intercepted — they resolve normally and never see the token.
-- The sidecar terminates TLS for those three hostnames with a CA generated fresh for the session; the private key never leaves the sidecar and is destroyed with it at teardown. The public root is installed into the agent container's trust store by the entrypoint (`update-ca-certificates`, before privilege drop), and `NODE_EXTRA_CA_CERTS` points at it, so `git`, `gh`, and node-based tooling all trust it natively.
-- The sidecar injects the real `Authorization` header in transit — `Basic base64(x-access-token:<token>)` for `github.com` (git smart-HTTP), `Bearer <token>` for `api.github.com` / `uploads.github.com` — replacing anything the client sent. `/root/.config/gh` stays tmpfs-masked while the sidecar is active: the placeholder token already satisfies `gh`, so persisted in-container login state would just be a second, unneeded secret.
+- The sidecar terminates TLS for those three hostnames with a CA generated fresh for the session; the private key never leaves the sidecar and is destroyed with it at teardown. The public root is installed into the agent container's trust store by the entrypoint (`update-ca-certificates`, before privilege drop), and `NODE_EXTRA_CA_CERTS` points at it, so `git`, `gh`, and node-based tooling all trust it natively. `UV_SYSTEM_CERTS=1` is set for the same reason: `uv`'s rustls client reads neither the OS bundle nor `NODE_EXTRA_CA_CERTS` by default, so without it every `uv` fetch from `github.com` fails `invalid peer certificate: UnknownIssuer` while `git`/`gh`/`curl` work. Verification stays on — `uv` just checks against the same session root.
+- The sidecar injects the real `Authorization` header in transit — `Basic base64(x-access-token:<token>)` for `github.com` (git smart-HTTP), `Bearer <token>` for `api.github.com` / `uploads.github.com` — replacing anything the client sent. One deliberate exception: a **`HEAD` on `github.com/<owner>/<repo>/releases/download/…`** is forwarded with the header *removed*. GitHub routes a release-asset `HEAD` carrying any `Authorization` to a legacy `objects.githubusercontent.com` pre-signed URL that answers `401` to every method, while an anonymous `HEAD` gets the working `release-assets.githubusercontent.com` CDN — so tools that probe with `HEAD` before `GET` (`uv`, `pip`) could not install from a release-asset URL at all. The credential buys nothing there: that endpoint doesn't accept token auth in the first place (see the limitation on private release assets below). `GET` on the same path keeps its credential, as does everything else. `/root/.config/gh` stays tmpfs-masked while the sidecar is active: the placeholder token already satisfies `gh`, so persisted in-container login state would just be a second, unneeded secret.
 
 **Isolation and lifecycle.** Each invocation gets its own network (`claude-gh-<id>`) and sidecar (`claude-gh-proxy-<id>`), so concurrent sessions never share a token copy, a CA, or traffic. Teardown happens in `run.sh`'s existing `EXIT` trap, extended and installed _before_ any sidecar or network is created, so a failure mid-startup can't leak either resource. Startup is **fail-closed**: if the sidecar won't start or its CA can't be retrieved in time, `run.sh` tears everything down and exits with an error — it never falls back to forwarding the real token. `run.sh` prints the sidecar's container name at startup.
 
@@ -173,7 +173,8 @@ When a token is found:
 
 **Limitations:**
 
-- TLS clients that don't read the OS trust store — notably Python's `certifi`-bundled CA set — get certificate errors against the three intercepted hostnames, since they never see the entrypoint-installed session CA. Workarounds: point the tool at the system store explicitly (an `SSL_CERT_FILE`-style override) or use `--gh-direct`.
+- TLS clients that don't read the OS trust store — notably Python's `certifi`-bundled CA set — get certificate errors against the three intercepted hostnames, since they never see the entrypoint-installed session CA. `uv` is handled for you (`UV_SYSTEM_CERTS=1`, above); for others, point the tool at the system store explicitly (`SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt`, or `REQUESTS_CA_BUNDLE` for `requests`/`certifi` consumers) or use `--gh-direct`.
+- Release assets in **private** repos aren't reachable via their `https://github.com/<owner>/<repo>/releases/download/…` URL. That's GitHub's behavior, not the proxy's: the web download path accepts only browser-session auth, so a token-authenticated request 404s with or without the proxy. Use the API asset endpoint instead — `gh release download`, or `GET /repos/{owner}/{repo}/releases/assets/{id}` with `Accept: application/octet-stream` — both of which go through `api.github.com` and work normally. Public release assets work directly.
 - SSH remotes (`git@github.com`) remain unsupported, as before — `--gh` never mounted a key or agent. The failure mode changes from an auth prompt to connection-refused, since `git@github.com` now resolves to the sidecar on a port it doesn't serve.
 - git-LFS is expected to work unchanged: the batch endpoint on `github.com` gets the same `Authorization` injection as any other git smart-HTTP request, and the actual object transfer happens against pre-signed, non-intercepted hosts. It's covered by the [manual checklist](#manual-fallback-checklist-macos) rather than called out as a limitation.
 

openspec/changes/archive/2026-08-06-fix-gh-proxy-release-asset-probes/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-08-06

openspec/changes/archive/2026-08-06-fix-gh-proxy-release-asset-probes/design.md

@@ -0,0 +1,89 @@
+## Context
+
+The `gh-auth-proxy` sidecar injects `Authorization` on **every** request to the three intercepted hostnames. For `github.com` that is `Basic base64(x-access-token:<token>)`, chosen because git smart-HTTP needs it. But `github.com` is not only the git transport — it also serves web endpoints, and the release-download path is one of them.
+
+Two things about that path were unknown when the sidecar was designed, and both were measured for this change rather than assumed:
+
+1. GitHub picks the CDN for a release asset based on **whether an `Authorization` header is present**. Anonymous requests get `release-assets.githubusercontent.com` (current, JWT-signed). Requests carrying a credential get `objects.githubusercontent.com` (legacy S3 pre-signed) — and that legacy URL now answers `401` to every method, including a ranged `GET`, seconds after being issued. It is dead infrastructure that GitHub still hands out.
+
+2. That endpoint **does not accept token auth at all**. The web download path wants a browser session. So the credential the sidecar injects there is inert: it cannot unlock a private asset, and its only observable effect is selecting the dead CDN.
+
+The interaction only bites clients that probe with `HEAD` first. `curl -fsSL` goes straight to `GET`, gets the working CDN, and never notices — which is why the sidecar shipped with this latent for months and why the issue looked like a `uv` bug.
+
+### Evidence
+
+All measured from inside a live `--gh` session on `uv` 0.11.29 / `aarch64`, against real GitHub. The credential-presence tests used a *bogus* token sent directly to `github.com` via `--resolve`, which reproduces the routing exactly — proving the trigger is the header's presence, not its identity, and letting the A/B run without a real token on the wire.
+
+| # | Observation |
+|---|---|
+| 1 | `HEAD` + any `Authorization` → `objects.githubusercontent.com`; `GET` + any `Authorization`, and both methods anonymous → `release-assets.githubusercontent.com`. The conjunction is the trigger. |
+| 2 | The `objects.*` URL returns `401` with an empty body to `GET` and to a ranged `GET`, not just to `HEAD`. Not a "pre-signed for GET only" artifact. |
+| 3 | `X-Forwarded-For`/`-Proto`/`-Host` do **not** trigger it, so Caddy's forwarding headers are not implicated. |
+| 4 | Private repo (two of them), real token injected by the live sidecar: the web `/releases/download/` URL returns `404` for **both** `HEAD` and `GET`. The same token fetches the same asset with `206` via `GET /repos/{o}/{r}/releases/assets/{id}` (`Accept: application/octet-stream`), and `gh release download` retrieves it whole. The credential is valid; the endpoint just doesn't honour it. |
+| 5 | `git ls-remote` against a private repo through the same sidecar succeeds — the injected `Basic` header is well-formed and honoured where it matters, so #4 is not a malformed-header artifact. |
+| 6 | Only `/releases/download/` is auth-sensitive. `/archive/refs/tags/*` (→ `codeload`), `/raw/<ref>/<path>` (→ `raw.githubusercontent.com`) and `/releases/latest/download/*` redirect identically with and without a credential. |
+| 7 | End-to-end, driving real `uvx` through the *generated* Caddyfile from `git HEAD` vs. the working tree, both with a credential injected: pre-fix fails with the exact `401` from the issue, post-fix installs the 12.8 MB wheel. |
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- `uv`/`uvx` install from a public `github.com` release-asset URL inside `--gh`, no flags, no env overrides.
+- Zero change to any request that works today — in particular every request that carries the credential today still carries it.
+- Keep full TLS verification. No `UV_INSECURE_HOST`, no `tls_insecure_skip_verify`.
+- Fix belongs in `run.sh`, so no image rebuild is needed.
+
+**Non-Goals:**
+
+- Making private release-asset *web* URLs work. GitHub doesn't support it; the API asset endpoint is the supported route and already works through the sidecar.
+- A general "inject credentials only on git-transport paths" redesign of the `github.com` block. Tempting, and probably right eventually, but it is a redesign with real LFS/web-path regression surface — not a bug fix.
+- Fixing every private-root-store TLS client. `uv` is the one shipped tool with the blind spot; the general `SSL_CERT_FILE`/`REQUESTS_CA_BUNDLE` escape hatch stays documented for anything the user brings.
+
+## Decisions
+
+### Drop the header on `HEAD`, rather than rewriting the method
+
+Two candidates fix defect 2. Both were implemented and measured.
+
+**A — drop `Authorization` for `HEAD` on the release-download path** (chosen). The probe takes the same anonymous route `curl` already takes successfully.
+
+**B — rewrite `HEAD`→`GET` upstream via Caddy's `method` directive, keeping the credential.** Also works. Verified protocol-correct: two pipelined `HEAD`s over one keep-alive connection each came back a clean `302` with zero body bytes, so Go's server does suppress the body despite the handler having mutated `r.Method`. Real `uvx` installed through it.
+
+B's only advantage would be preserving an authenticated `HEAD` capability — which evidence #4 shows this endpoint does not offer. Against that it costs: the access log records `HEAD` while GitHub receives `GET` (an audit-fidelity gap in a component whose logging is a spec requirement); an upstream `GET` for every probe; observed loss of the upstream `Content-Length` on the synthesized `HEAD` response; and a semantic change a reader of the config would not expect from a transparent proxy. A is the smaller, more legible change and strictly *reduces* the number of paths the host token travels on. Chose A.
+
+### Scope by method **and** path, not by path alone
+
+Dropping the credential for the whole `/releases/download/` prefix (any method) is defensible on the same evidence — it would restore byte-identical un-proxied behavior and is simpler to express. Rejected anyway: authenticated `GET` on that path works today, and evidence #4 covers only OAuth/PAT tokens. If a GitHub App installation token (whose git convention is exactly the `x-access-token:` username the sidecar uses) is ever honoured on that endpoint, a `GET`-inclusive exclusion would silently remove access, while the `HEAD`-only form cannot: it touches only requests that return an unusable `401` for every caller today. That makes "no regressions" a structural property rather than a claim resting on the token types available to test.
+
+### Delete the header, don't just skip injection
+
+`handle` blocks that omit `header_up Authorization` would forward whatever the client sent — and the agent container's `gh`/`git` send the placeholder `GH_TOKEN=claude-docker-proxy`. Since *any* `Authorization` value selects the dead CDN (evidence #1, established with a bogus token), skipping injection would leave the bug in place for exactly the clients that matter. `header_up -Authorization` deletes it. This also keeps the existing spec property that the placeholder never reaches GitHub.
+
+### `UV_SYSTEM_CERTS=1`, sidecar-scoped
+
+Four env-only workarounds clear defect 1. `UV_INSECURE_HOST` is out — it disables verification. `SSL_CERT_FILE` works but is process-wide and would redirect every OpenSSL consumer in the container, which deserves its own decision rather than riding along in a bug fix. That leaves the `uv`-specific pair: `UV_NATIVE_TLS` is deprecated in the pinned `uv` and prints a warning on every invocation, so `UV_SYSTEM_CERTS` it is. Confirmed bound (`[env: UV_SYSTEM_CERTS=]`) and working end-to-end in **0.11.21**, the currently committed pin, as well as 0.11.29 — so the fix does not depend on the pending pin bump.
+
+Set alongside `NODE_EXTRA_CA_CERTS`, i.e. only when the sidecar is active: with no interception there is no session CA and nothing extra to trust. `--gh-direct` and the no-token fallback are untouched.
+
+### Matcher shape
+
+`^/[^/]+/[^/]+/releases/download/.+` on `method HEAD`. The trailing `.+` requires an asset segment, so a bare `/o/r/releases/download/` keeps its credential. `/releases/latest/download/<asset>` is *not* matched directly and does not need to be — its first hop is a `302` to the canonical `/releases/download/<tag>/<asset>` URL (identical with and without a credential), and that second hop comes back through the sidecar and is matched.
+
+Ordering uses two `handle` blocks rather than a bare matcher on a single `reverse_proxy`, because the two branches need different `header_up` treatment. `handle` is mutually exclusive, so the general branch is a plain `handle` fallback.
+
+## Risks / Trade-offs
+
+- **[A tool probes a release asset in a private repo]** → gets `404` instead of today's `401`-after-redirect. Both are failures; neither is reachable. Documented with the `gh release download` alternative.
+- **[The exclusion widens past `HEAD` in a later edit]** → would remove the credential from authenticated `GET`s that work today. Guarded by a paired assertion in the integration harness: `HEAD` must arrive with no `Authorization`, `GET` on the same path must arrive with `Basic`.
+- **[GitHub changes release-asset routing again]** → the fix becomes unnecessary but stays harmless: an anonymous `HEAD` on a public asset is the same request `curl` makes. The failure mode is not a regression, just a dead branch.
+- **[`UV_SYSTEM_CERTS` is only correct if the CA install succeeded]** → if the entrypoint's `update-ca-certificates` didn't run (issue #11's skew scenario), `uv` points at a bundle with no Caddy root and the error changes rather than disappears. The two issues compose; this is not a substitute for #11.
+- **[Anonymous release-asset probes are rate-limited differently]** → not observed; the bytes come from a pre-signed CDN URL either way, and the `GET` that transfers them is unchanged.
+
+## Migration Plan
+
+None. No state, no flags, no image layer. Existing sessions are unaffected; the next `claude-docker --gh` launch regenerates the Caddyfile and the env with the fix in place.
+
+## Open Questions
+
+- Should the `github.com` block eventually inject credentials **only** on git-transport and LFS paths, leaving all web paths anonymous? That is the principled end state and would have prevented this bug by construction. Deferred — it needs its own change with LFS coverage.
+- Does any GitHub App installation token get honoured on the web release-download endpoint? Untestable with the credentials available here. The `HEAD`-only scope means the answer doesn't change this fix's correctness either way.