[Bug] test.sh and prime-agent.sh --no-env don't unset every key that env-api-keys.ts reads #2030
aarmens702-hub
started this conversation in
Bug reports
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Affected area
AI providers and models
What happened?
I was reading through the repo to borrow the credential-quarantine idea from test.sh for my own project and noticed the unset list doesn't match env-api-keys.ts anymore. env-api-keys.ts reads 42 variables. test.sh unsets 31 of them, and prime-agent.sh --no-env a different 31.
test.sh is missing: AZURE_OPENAI_API_KEY, CLOUDFLARE_API_KEY, DEEPSEEK_API_KEY, GOOGLE_CLOUD_API_KEY, MOONSHOT_API_KEY, PRIME_API_KEY, PRIME_TEAM_ID, and the four
XIAOMI_*keys.prime-agent.sh --no-env is missing: CLOUDFLARE_API_KEY, DEEPSEEK_API_KEY, FIREWORKS_API_KEY, GOOGLE_CLOUD_API_KEY, KIMI_API_KEY, MOONSHOT_API_KEY, PRIME_TEAM_ID, and the same four
XIAOMI_*keys.This matters because several packages/ai suites gate their live tests on exactly those variables:
describe.skipIf(!process.env.DEEPSEEK_API_KEY)in stream.test.ts, the Xiaomi blocks in abort/context-overflow/empty/image-tool-result, hasAzureOpenAICredentials() in azure-utils.ts, the Cloudflare helpers. So if you happen to have DEEPSEEK_API_KEY in your shell, ./test.sh runs the DeepSeek suite against the real API while printing "Running tests without API keys". Minor, but the comment above the list points at stream.ts getEnvApiKey, while the list itself lives in env-api-keys.ts (stream.ts only re-exports it); prime-agent.sh already points there.Steps to reproduce
On a source checkout at d74a75f:
The variable survives the unset block:
prints
x. Same thing with ANTHROPIC_API_KEY printsunset.The gated suite runs instead of skipping:
First one: 198 skipped. Second one: the 5 DeepSeek tests run (3 retries each against the API) and fail.
Expected behavior
./test.sh unsets everything env-api-keys.ts can read, and ideally the two lists can't drift again without CI catching it.
Prime Agent version
d74a75f (from source, v0.9.1-15)
Installation method
From source
Operating system
macOS
Operating system version
macOS 15.5
CPU architecture
arm64 / aarch64
Shell and terminal
zsh 5.9 in Terminal.app
Additional context
I put a fix on a branch: main...aarmens702-hub:prime-agent:fix/env-key-list-drift
It adds scripts/check-env-key-lists.mjs, which pulls the variable names out of env-api-keys.ts and fails if test.sh or prime-agent.sh doesn't unset one of them. I wired it into
npm run checkas check:env-keys next to the installer and browser-smoke checks. It also adds the missing unsets and fixes the comment. Nothing under packages/*/src changed, so no changelog fragment. The check fails on main with the lists above and passes on the branch; npm run check passes on the branch.Looking at the history, these lists have been patched one variable at a time before (59b0efa, 4688553), which is why I went with a check rather than another hand edit. Also noticed #374 adds
unset PRIME_TEAM_IDto prime-agent.sh, so that one line overlaps whichever lands first.Not opening a PR because of the gate. If PRIME_TEAM_ID shouldn't be in there since it's a team id rather than a key, I can drop it.
All reactions