extproc: scope /v1/models by non-model exact-match headers - #2453
Open
rafaelgaspar wants to merge 2 commits into
Open
extproc: scope /v1/models by non-model exact-match headers#2453rafaelgaspar wants to merge 2 commits into
rafaelgaspar wants to merge 2 commits into
Conversation
When AIGatewayRoute rules match on headers besides x-ai-eg-model, /v1/models now returns only models from rules whose scope headers match the request. Mirrors the hostname scoping added in envoyproxy#2160. Previously the controller indexed only x-ai-eg-model values into the global model list, ignoring tenant/routing headers like x-jwt-sub. Fixes envoyproxy#2452 Signed-off-by: Rafael Gaspar <rafael.gaspar@me.com> Signed-off-by: Rafael Antunes <rafael.gaspar@me.com>
Add an end-to-end fixture that mirrors the issue repro: Envoy Gateway SecurityPolicy projects JWT sub to x-jwt-sub with recomputeRoute, and GET /v1/models returns only models for the authenticated tenant. Fixes envoyproxy#2452 Signed-off-by: Rafael Gaspar <rafael.gaspar@me.com> Signed-off-by: Rafael Antunes <rafael.gaspar@me.com>
📄 Knowledge reviewDosu skipped reviewing this PR because your organization has used its |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When
AIGatewayRouterules match on exact headers other thanx-ai-eg-model(for examplex-jwt-subinjected from JWTsubvia Envoy GatewaySecurityPolicywithclaimToHeadersandrecomputeRoute: true),GET /v1/modelsnow returns only models from rules whose scope headers match the incoming request. This mirrors the hostname scoping behavior added in #2160.Previously the controller indexed only
x-ai-eg-modelvalues into the global model catalog, so tenants on a shared hostname saw the union of all declared models even when chat/completions routing was correctly scoped.Changes:
ModelsByHostAndScope/HeaderScopedModelsto the filter config and wire it through runtime./v1/modelsresponses in extproc using request scope headers.SecurityPolicyJWT claim projection./v1/modelsinsupported-endpoints.md.Related Issues/PRs (if applicable)
Fixes #2452
Related: #2160 (hostname scoping pattern)
Special notes for reviewers (if applicable)
hostnamesand scope headers are indexed under the host bucket and filtered by scope at runtime.claimToHeaders+recomputeRoute, same as chat routing).tests/e2e/testdata/models_header_scope_jwt.yaml+TestModelsHeaderScopeJWT.make precommitand unit tests.Test plan
make precommitgo test ./internal/controller/... -run HeaderScopedgo test ./internal/extproc/... -run selectModelsgo test ./internal/filterapi/... -run HeaderScopeTestModelsHeaderScopeJWTpassed locally viamake test-e2e