Skip to content

extproc: scope /v1/models by non-model exact-match headers - #2453

Open
rafaelgaspar wants to merge 2 commits into
envoyproxy:mainfrom
rafaelgaspar:extproc-scope-models-by-headers
Open

extproc: scope /v1/models by non-model exact-match headers#2453
rafaelgaspar wants to merge 2 commits into
envoyproxy:mainfrom
rafaelgaspar:extproc-scope-models-by-headers

Conversation

@rafaelgaspar

Copy link
Copy Markdown

Description

When AIGatewayRoute rules match on exact headers other than x-ai-eg-model (for example x-jwt-sub injected from JWT sub via Envoy Gateway SecurityPolicy with claimToHeaders and recomputeRoute: true), GET /v1/models now returns only models from rules whose scope headers match the incoming request. This mirrors the hostname scoping behavior added in #2160.

Previously the controller indexed only x-ai-eg-model values into the global model catalog, so tenants on a shared hostname saw the union of all declared models even when chat/completions routing was correctly scoped.

Changes:

  • Add ModelsByHostAndScope / HeaderScopedModels to the filter config and wire it through runtime.
  • Index models by canonical header scope key in the gateway controller.
  • Filter /v1/models responses in extproc using request scope headers.
  • Add unit tests for controller and extproc selection.
  • Add e2e coverage reproducing /v1/models ignores x-jwt-sub when rules require it for routing #2452 with Envoy Gateway SecurityPolicy JWT claim projection.
  • Document header-scoped /v1/models in supported-endpoints.md.

Related Issues/PRs (if applicable)

Fixes #2452

Related: #2160 (hostname scoping pattern)

Special notes for reviewers (if applicable)

  • Header scoping composes with hostname scoping: rules with both hostnames and scope headers are indexed under the host bucket and filtered by scope at runtime.
  • Rules without scope headers remain visible to all requests on the same host (same opt-in pattern as unscoped hostnames).
  • Scope headers must be present on the request when extproc runs (e.g. JWT claimToHeaders + recomputeRoute, same as chat routing).
  • E2e fixture: tests/e2e/testdata/models_header_scope_jwt.yaml + TestModelsHeaderScopeJWT.
  • AI tooling was used to assist with implementation and tests; changes were reviewed and validated locally with make precommit and unit tests.

Test plan

  • make precommit
  • Unit tests: go test ./internal/controller/... -run HeaderScoped
  • Unit tests: go test ./internal/extproc/... -run selectModels
  • Unit tests: go test ./internal/filterapi/... -run HeaderScope
  • TestModelsHeaderScopeJWT passed locally via make test-e2e

When AIGatewayRoute rules match on headers besides x-ai-eg-model,
/v1/models now returns only models from rules whose scope headers
match the request. Mirrors the hostname scoping added in envoyproxy#2160.

Previously the controller indexed only x-ai-eg-model values into the
global model list, ignoring tenant/routing headers like x-jwt-sub.

Fixes envoyproxy#2452

Signed-off-by: Rafael Gaspar <rafael.gaspar@me.com>
Signed-off-by: Rafael Antunes <rafael.gaspar@me.com>
Add an end-to-end fixture that mirrors the issue repro: Envoy Gateway
SecurityPolicy projects JWT sub to x-jwt-sub with recomputeRoute, and
GET /v1/models returns only models for the authenticated tenant.

Fixes envoyproxy#2452

Signed-off-by: Rafael Gaspar <rafael.gaspar@me.com>
Signed-off-by: Rafael Antunes <rafael.gaspar@me.com>
@rafaelgaspar
rafaelgaspar requested a review from a team as a code owner July 29, 2026 10:38
@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 29, 2026
@dosubot

dosubot Bot commented Jul 29, 2026

Copy link
Copy Markdown

📄 Knowledge review

Dosu skipped reviewing this PR because your organization has used its 200 included credits for the month. Your usage will reset on 2026-08-01. To have Dosu review this PR before then, ask your organization admin to upgrade to a pro account.


Leave Feedback Ask Dosu about ai-gateway Add Dosu to your team

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/v1/models ignores x-jwt-sub when rules require it for routing

1 participant