diff --git a/gateway/it/features/llm-provider-wide-ratelimit.feature b/gateway/it/features/llm-provider-global-ratelimit.feature similarity index 77% rename from gateway/it/features/llm-provider-wide-ratelimit.feature rename to gateway/it/features/llm-provider-global-ratelimit.feature index 55c04e275f..be57c5cd6c 100644 --- a/gateway/it/features/llm-provider-wide-ratelimit.feature +++ b/gateway/it/features/llm-provider-global-ratelimit.feature @@ -662,3 +662,195 @@ Feature: Provider-wide rate limiting for LLM providers # /embeddings has its own routename-keyed bucket — completely unaffected When I send a GET request to "http://localhost:8080/adv-op-px/embeddings" Then the response status code should be 200 + + # --------------------------------------------------------------------------- + # Scenarios 9–10: mixed advanced-ratelimit (global) + basic-ratelimit (operation) + # This is the realistic production case: a provider-wide cap enforced by + # advanced-ratelimit (apiname key) combined with a tighter per-path cap + # enforced by basic-ratelimit. The operation policy fires first; but because + # global policies run before operation policies in the chain, the shared global + # counter is still incremented even for requests that the operation policy + # ultimately rejects — so /embeddings eventually hits the global cap even + # though it has no operation policy of its own. + # --------------------------------------------------------------------------- + + # Scenario 9 — Provider: advanced-ratelimit global (5/hr, apiname) + + # basic-ratelimit operation (3/hr for /chat/completions). + Scenario: mixed advanced-ratelimit global and basic-ratelimit operation on provider — global bucket exhausted by rejected operation traffic + Given I authenticate using basic auth as "admin" + When I create this LLM provider template: + """ + apiVersion: gateway.api-platform.wso2.com/v1 + kind: LlmProviderTemplate + metadata: + name: mix-prov-tpl + spec: + displayName: Mix Provider Template + promptTokens: + location: payload + identifier: $.json.usage.prompt_tokens + completionTokens: + location: payload + identifier: $.json.usage.completion_tokens + totalTokens: + location: payload + identifier: $.json.usage.total_tokens + """ + Then the response status code should be 201 + + Given I authenticate using basic auth as "admin" + When I create this LLM provider: + """ + apiVersion: gateway.api-platform.wso2.com/v1 + kind: LlmProvider + metadata: + name: mix-provider + spec: + displayName: Mix RateLimit Provider + version: v1.0 + context: /mix-prov + template: mix-prov-tpl + upstream: + url: http://echo-backend-multi-arch:8080/anything + auth: + type: api-key + header: Authorization + value: test-api-key + accessControl: + mode: deny_all + exceptions: + - path: /chat/completions + methods: [GET] + - path: /embeddings + methods: [GET] + globalPolicies: + - name: advanced-ratelimit + version: v1 + params: + quotas: + - name: request-limit + limits: + - limit: 5 + duration: "1h" + keyExtraction: + - type: apiname + operationPolicies: + - name: basic-ratelimit + version: v1 + paths: + - path: /chat/completions + methods: [GET] + params: + limits: + - requests: 3 + duration: "1h" + """ + Then the response status code should be 201 + And I wait for the endpoint "http://localhost:8080/mix-prov/chat/completions" to be ready + + # Operation policy (3/hr) fires before global (5/hr) for /chat/completions. + # Global still increments on every attempt — including those the operation policy rejects. + When I send 10 GET requests to "http://localhost:8080/mix-prov/chat/completions" + Then the response status code should be 429 + + # KEY ASSERTION: /embeddings has no operation policy but shares the global + # apiname bucket. The global was exhausted by /chat/completions traffic, + # so /embeddings is also blocked. + When I send a GET request to "http://localhost:8080/mix-prov/embeddings" + Then the response status code should be 429 + And the response body should contain "Rate limit exceeded" + + # Scenario 10 — Proxy: advanced-ratelimit global (5/hr, apiname) + + # basic-ratelimit operation (3/hr for /chat/completions). + Scenario: mixed advanced-ratelimit global and basic-ratelimit operation on proxy — global bucket exhausted by rejected operation traffic + Given I authenticate using basic auth as "admin" + When I create this LLM provider template: + """ + apiVersion: gateway.api-platform.wso2.com/v1 + kind: LlmProviderTemplate + metadata: + name: mix-proxy-tpl + spec: + displayName: Mix Proxy Template + promptTokens: + location: payload + identifier: $.json.usage.prompt_tokens + completionTokens: + location: payload + identifier: $.json.usage.completion_tokens + totalTokens: + location: payload + identifier: $.json.usage.total_tokens + """ + Then the response status code should be 201 + + When I create this LLM provider: + """ + apiVersion: gateway.api-platform.wso2.com/v1 + kind: LlmProvider + metadata: + name: mix-proxy-backend + spec: + displayName: Mix Proxy Backend + version: v1.0 + context: /mix-pb + template: mix-proxy-tpl + upstream: + url: http://echo-backend-multi-arch:8080/anything + auth: + type: api-key + header: Authorization + value: test-api-key + accessControl: + mode: allow_all + """ + Then the response status code should be 201 + + When I deploy this LLM proxy configuration: + """ + apiVersion: gateway.api-platform.wso2.com/v1 + kind: LlmProxy + metadata: + name: mix-proxy + spec: + displayName: Mix RateLimit Proxy + version: v1.0 + context: /mix-px + provider: + id: mix-proxy-backend + globalPolicies: + - name: advanced-ratelimit + version: v1 + params: + quotas: + - name: request-limit + limits: + - limit: 5 + duration: "1h" + keyExtraction: + - type: apiname + operationPolicies: + - name: basic-ratelimit + version: v1 + paths: + - path: /chat/completions + methods: [GET] + params: + limits: + - requests: 3 + duration: "1h" + """ + Then the response status code should be 201 + And I wait for the endpoint "http://localhost:8080/mix-px/chat/completions" to be ready + + # Operation policy (3/hr) fires before global (5/hr) for /chat/completions. + # Global still increments on every attempt — including those the operation policy rejects. + When I send 10 GET requests to "http://localhost:8080/mix-px/chat/completions" + Then the response status code should be 429 + + # KEY ASSERTION: /embeddings has no operation policy but shares the global + # apiname bucket. The global was exhausted by /chat/completions traffic, + # so /embeddings is also blocked. + When I send a GET request to "http://localhost:8080/mix-px/embeddings" + Then the response status code should be 429 + And the response body should contain "Rate limit exceeded"