feat(scan): omit model field when AURSCAN_OPENAI_MODEL is unset - #22
Merged
manticore-projects merged 1 commit intoJun 21, 2026
Merged
manticore-projects merged 1 commit into
manticore-projects merged 1 commit into
Conversation
The OpenAI-compatible backend sent the placeholder "default-model" whenever AURSCAN_OPENAI_MODEL was unset, which breaks routing proxies (LiteLLM, vLLM, etc.) that select the model server-side and reject unknown model names. Build the request payload without a "model" key, and add it only when AURSCAN_OPENAI_MODEL is non-empty. Sending "model": "" is avoided since some servers reject an empty string. - internal/scan/llm.go: drop the default-model fallback; add model conditionally on AURSCAN_OPENAI_MODEL. - internal/scan/llm_test.go: new httptest-backed tests asserting the key is omitted when unset and forwarded verbatim when set. - README.md + CHANGELOG.md: document the new default and routing-proxy use case. Fail-closed semantics, timeout handling, fallback-URL loop, and all other backends are unchanged.
Owner
|
Thank you very much! |
Author
|
Thanks for merging! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The OpenAI-compatible backend sent the placeholder "default-model" whenever AURSCAN_OPENAI_MODEL was unset, which breaks routing proxies (LiteLLM, vLLM, etc.) that select the model server-side and reject unknown model names.
Build the request payload without a "model" key, and add it only when AURSCAN_OPENAI_MODEL is non-empty. Sending "model": "" is avoided since some servers reject an empty string.
Fail-closed semantics, timeout handling, fallback-URL loop, and all other backends are unchanged.
Real-world use case. With this change I run aurscan against my own proxy fronting NVIDIA NIM (and other providers), so I can switch the underlying LLM on the fly without editing env vars or restarting aurscan.
Behavioral change to flag. This removes the hardcoded default-model placeholder that was sent when AURSCAN_OPENAI_MODEL was unset. In practice that placeholder was ignored by single-model servers (Ollama, llama.cpp, vLLM) and rejected by strict proxies — so the only users who could regress are on a server that both (a) requires the model field and (b) accepted the literal default-model. Those users should now set AURSCAN_OPENAI_MODEL explicitly. If that trade-off feels too intrusive, happy to revisit — e.g. keep the fallback only as a last resort.