feat: allow excluding models from /v1/models - #2446
Conversation
76b8752 to
83f99bf
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2446 +/- ##
=======================================
Coverage 84.92% 84.92%
=======================================
Files 154 154
Lines 22430 22434 +4
=======================================
+ Hits 19048 19052 +4
Misses 2225 2225
Partials 1157 1157 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
83f99bf to
fbdc8c6
Compare
|
@Killusions can you explain the usecase for exclusion of models? |
@siddharth1036 There are many, the most common one would be backwards-compatible aliases which people and new clients should no longer use. But also generic aliases, multiple definitions (openai + anthropic schema without translation since our backend vLLM supports both), private test models, etc. At Siemens we have all of these and currently need to use a workaround by using regex patterns instead of exact model name matches. |
fbdc8c6 to
f124d9a
Compare
nacx
left a comment
There was a problem hiding this comment.
Thank you! Just a couple minor comments
d8d8dd9 to
c96488f
Compare
401b17d to
181a04c
Compare
|
@nacx Pretty sure it's an unrelated failure (after I fixed the precommit), re-triggered the tests by re-pushing. |
181a04c to
9194a00
Compare
|
Thanks, LGTM! Deferring to @envoyproxy/ai-gateway-maintainers |
Signed-off-by: Linus Schlumberger <linus.schlumberger@siemens.com>
9194a00 to
93ff4ed
Compare
Description
Adds
excludeFromModelsEndpointtoAIGatewayRouterules.Models matched by a rule with this option remain routeable and support
modelNameOverride, but are excluded from the OpenAI-compatible/v1/modelsresponse. This supports backwards-compatible aliases and aliases only intended for selected clients.Related Issues/PRs (if applicable)
Fixes #2445
Special notes for reviewers (if applicable)
Tested manually.
⚒️ with ❤️ by https://github.com/siemens