Declare release credential surfaces once and probe them weekly - #1505
Open
schickling-assistant wants to merge 3 commits into
Conversation
schickling-assistant
added
area:ci
Contributor
PR preview
Report historyPR 1505 · 2026-07-27 22:41 UTC
PR 1505 · 2026-07-27 22:19 UTC
PR 1505 · 2026-07-27 21:18 UTC
PR 1505 · 2026-07-27 21:10 UTC
PR 1505 · 2026-07-27 19:38 UTC
PR 1505 · 2026-07-27 10:39 UTC
|
schickling-assistant
force-pushed
the
schickling-assistant/2026-07-27-release-credential-surfaces
branch
from
July 27, 2026 19:10
0ebf486 to
f63ae8d
Compare
schickling-assistant
force-pushed
the
schickling-assistant/2026-07-27-release-registry-verify
branch
2 times, most recently
from
July 27, 2026 20:40
c3d26f0 to
0d0157b
Compare
schickling-assistant
force-pushed
the
schickling-assistant/2026-07-27-release-credential-surfaces
branch
3 times, most recently
from
July 27, 2026 21:47
336c0ce to
1303c4e
Compare
The production deploy steps depend on three long-lived credentials (Netlify, Cloudflare, Mixedbread). Their names were repeated as string literals across five `env:` blocks in release.yml.genie.ts, which is how #1284 happened: release.yml referenced MXBAI_VECTOR_STORE_ID_PROD, a secret that never existed, and nothing noticed until the publish job reached that step. Declare each surface once in `genie/release-credentials.ts` — its secrets plus a read-only probe that proves they are still accepted. Both consumers derive from it: release.yml's deploy steps derive their `env:` blocks, and a new health-release-credentials.yml derives a weekly liveness probe. The two can no longer disagree about which credentials a release needs, and the #1284 class becomes unrepresentable rather than merely detectable — same idea as `ci.ts` keeping required checks aligned with matrix expansions. The refactor is behavior-preserving: regenerating leaves release.yml byte-identical. The health check is deliberately narrow. validate-release-plan already exercises release code on every PR in the real Nix environment; the only thing it cannot prove is that a token which expires on wall-clock time is still valid. So the check runs weekly, makes only HTTP calls, and sets up no toolchain — an API call has no environment to reproduce, and approximating the release job's shell would just create a second environment that drifts from the first. No tooling probes, no issue filing. Probes fail closed, verified against the live endpoints with an invalid token: Netlify 401, Mixedbread 401, and Cloudflare's /user/tokens/verify returning a non-active status — so a token that still authenticates but was disabled is caught rather than passing. No --location, so a redirect cannot replay the Authorization header to another host. Adds LS.DEL.REL-R10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous wording claimed the shared declaration makes a missing secret "unrepresentable". It does not: a typo in `release-credentials.ts` propagates to both consumers exactly as #1284 did. What the shared declaration removes is divergence *between* release.yml and the probe. The real improvement for #1284 is timing — a secret that does not exist arrives empty and trips the probe's `:?` guard within a week, rather than surfacing when a release reaches that deploy step. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
main #1507 gave each docs surface its own search index, so the prod docs sync takes MXBAI_VECTOR_STORE_ID_PROD rather than MXBAI_VECTOR_STORE_ID. Updating the one declaration moves both the release step's env block and the health probe, which is the point of declaring surfaces once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
schickling-assistant
force-pushed
the
schickling-assistant/2026-07-27-release-registry-verify
branch
from
July 27, 2026 22:04
8897bd7 to
7945351
Compare
schickling-assistant
force-pushed
the
schickling-assistant/2026-07-27-release-credential-surfaces
branch
from
July 27, 2026 22:04
1303c4e to
9b47f72
Compare
schickling-assistant
marked this pull request as ready for review
July 27, 2026 22:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The production deploy steps depend on three long-lived credentials — Netlify, Cloudflare, Mixedbread. Their names were repeated as string literals across five
env:blocks inrelease.yml.genie.ts.That duplication is how #1284 happened:
release.ymlreferencedMXBAI_VECTOR_STORE_ID_PROD, a secret that never existed, and nothing noticed until the publish job reached that step.Separately, nothing checks these credentials until a release needs them.
validate-release-planalready exercises release code on every PR in the real Nix environment — but a dry-run cannot prove that a token which expires on wall-clock time is still accepted. That is the only genuine gap.Solution
Declare each credential surface once in
genie/release-credentials.ts— its secrets, plus a read-only probe that proves they are still accepted. Both consumers derive from that declaration:release.ymlproduction deploy stepsenv:block passed to the devenv taskhealth-release-credentials.yml(new)Same idea as
genie/ci.tskeeping required status checks aligned with matrix expansions: shared generator data keeps two emitted artifacts consistent.The refactor is behavior-preserving — regenerating leaves
release.ymlbyte-identical, so the only new workflow content is the health check itself.What the health check deliberately does not do
release.ymlruns its steps through devenv tasks; awhich pnpmin this job would describe this runner, not that one. (This is exactly why the pre-flight in Add trusted-publisher + secrets pre-flight health check for the release pipeline #1291 could not have caught Run mxbai search sync via devenv task (pnpm not on PATH) #1285 — it ranpnpm/action-setupand then checkedwhich pnpm, installing the very tool whose absence was the bug.)Scope note on #1284
The shared declaration removes divergence between
release.ymland the probe — they cannot disagree about which secrets a surface needs. It does not make a wrong name impossible: a typo inrelease-credentials.tspropagates to both consumers. What changes is timing — a secret that does not exist arrives empty and trips the probe's:?guard on the next weekly run, instead of surfacing when a release reaches that deploy step.Validation
Probes fail closed, verified against the live endpoints with an invalid token:
GET /api/v1/userGET /client/v4/user/tokens/verify.result.statusabsent →!= active→ job failsGET /v1/vector_stores/$IDCloudflare's endpoint reports token status, so a token that still authenticates but has been disabled or expired is caught rather than passing —
gh secret list(as used by #1291) only proves a name exists.actionlintclean on the generated workflow.genie:runregenerates cleanly;release.ymlunchanged.tsc --buildclean;oxlint/oxfmtclean on changed files; intent-layer invariant suite green.--locationon any probe, so a redirect cannot replay theAuthorizationheader to another host.Intent layer
Adds LS.DEL.REL-R10 (declared credential surfaces) and a Credential Surfaces section in the release spec.
Supersedes
Partially closes #1291. That PR's useful half was credential liveness; this keeps it and drops the parts that could not work:
pull_requesttrigger checked outref: main, so a PR changingrelease.ymlwas validated against main's code and the PR's own changes were never examined;gh secret listproves a name exists, not that the credential is valid;id-token: writewas declared but unused.npm is intentionally absent from the surface list: publishing uses OIDC trusted publishing and
release.ymlasserts noNPM_TOKEN/NODE_AUTH_TOKENis set, so there is no long-lived npm credential to expire. (#1291 reached the same conclusion — correct against today'smain.)🤖 Generated with Claude Code
Posted on behalf of @schickling
agent_nameagent_session_idagent_toolagent_tool_versionagent_runtimeagent_modelruntime_profileskills_manifestworktreemachinetooling_profile