Skip to content

Adding AI gateway 1.1.0 performance test results #2716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Krishanx92 merged 1 commit into from
Jul 17, 2026
Merged

Adding AI gateway 1.1.0 performance test results #2716

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions gateway/perf/ai-gateway-1.1.0-perf-test-results/2-core-results-summary.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Scenario Name,Concurrent Users,Throughput (Requests/sec),Average Response Time (ms),Message Size (Bytes),Response Size (Bytes),Back-end Service Delay (ms),Label,# Samples,Error Count,Error %,Average Users in the System,Standard Deviation of Response Time (ms),Minimum Response Time (ms),Maximum Response Time (ms),75th Percentile of Response Time (ms),90th Percentile of Response Time (ms),95th Percentile of Response Time (ms),98th Percentile of Response Time (ms),99th Percentile of Response Time (ms),99.9th Percentile of Response Time (ms),Received (KB/sec),Sent (KB/sec),Concurrent Users,AI API Auth No Guardrails,AI API PII Masking,AI API Advanced Guardrails
,AI API Auth No Guardrails,100,3760.17,26.38,1024,10240,10,AI Chat Completion Request,2704761,0,0,99,12.6,11,71,37,48,51,54,56,61,38236.95,4932.47,100,3760.17,2005.59,1683.66
,AI API Auth No Guardrails,500,3929.88,126.87,1024,10240,10,AI Chat Completion Request,2827857,0,0,498,21.65,11,242,146,162,168,175,179,188,39962.76,5155.09,500,3929.88,2112.5,1732.18
,AI API Auth No Guardrails,1000,4136.87,241.43,1024,10240,10,AI Chat Completion Request,2977005,0,0,998,33.52,11,460,269,287,297,309,321,369,42067.57,5426.6,1000,4136.87,2053.29,1709.14
,Scenario Name,Concurrent Users,Throughput (Requests/sec),Average Response Time (ms),Message Size (Bytes),Response Size (Bytes),Back-end Service Delay (ms),Label,# Samples,Error Count,Error %,Average Users in the System,Standard Deviation of Response Time (ms),Minimum Response Time (ms),Maximum Response Time (ms),75th Percentile of Response Time (ms),90th Percentile of Response Time (ms),95th Percentile of Response Time (ms),98th Percentile of Response Time (ms),99th Percentile of Response Time (ms),99.9th Percentile of Response Time (ms),Received (KB/sec),Sent (KB/sec),,,,
,AI API PII Masking,100,2005.59,49.65,1024,10240,10,AI Chat Completion Request,1442709,0,0,99,21.44,11,113,70,76,78,82,84,92,20394.76,2636.75,,,,
,AI API PII Masking,500,2112.5,236.49,1024,10240,10,AI Chat Completion Request,1520173,0,0,499,37.17,17,446,267,285,295,307,317,369,21481.88,2777.29,,,,
,AI API PII Masking,1000,2053.29,486.94,1024,10240,10,AI Chat Completion Request,1477640,0,0,999,107.03,13,962,563,631,695,747,783,863,20879.77,2699.45,,,,
,Scenario Name,Concurrent Users,Throughput (Requests/sec),Average Response Time (ms),Message Size (Bytes),Response Size (Bytes),Back-end Service Delay (ms),Label,# Samples,Error Count,Error %,Average Users in the System,Standard Deviation of Response Time (ms),Minimum Response Time (ms),Maximum Response Time (ms),75th Percentile of Response Time (ms),90th Percentile of Response Time (ms),95th Percentile of Response Time (ms),98th Percentile of Response Time (ms),99th Percentile of Response Time (ms),99.9th Percentile of Response Time (ms),Received (KB/sec),Sent (KB/sec),,,,
,AI API Advanced Guardrails,100,1683.66,59.18,1024,10240,10,AI Chat Completion Request,1211251,0,0,99,19.22,14,151,75,81,84,89,92,102,17121.06,2215.15,,,,
,AI API Advanced Guardrails,500,1732.18,288.47,1024,10240,10,AI Chat Completion Request,400829,0,0,499,51.56,57,556,315,359,381,399,411,469,17614.48,2278.99,,,,
,AI API Advanced Guardrails,1000,1709.14,584.9,1024,10240,10,AI Chat Completion Request,1230149,0,0,999,158.77,21,1193,727,819,871,911,947,1047,17380.14,2248.67,,,,

12 changes: 12 additions & 0 deletions gateway/perf/ai-gateway-1.1.0-perf-test-results/4-core-results-summary.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Scenario Name,Concurrent Users,Throughput (Requests/sec),Average Response Time (ms),Message Size (Bytes),Response Size (Bytes),Back-end Service Delay (ms),# Samples,Error Count,Error %,Average Users in the System,Standard Deviation of Response Time (ms),Minimum Response Time (ms),Maximum Response Time (ms),75th Percentile of Response Time (ms),90th Percentile of Response Time (ms),95th Percentile of Response Time (ms),98th Percentile of Response Time (ms),99th Percentile of Response Time (ms),99.9th Percentile of Response Time (ms),Received (KB/sec),Sent (KB/sec),,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API with Auth No Guardrails,100,6512.26,15.2,1024,10240,10,4685643,0,0,98,3.88,10,44,16,19,24,28,31,36,66222.82,8542.57,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API with Auth No Guardrails,500,7507.92,66.38,1024,10240,10,5402204,0,0,498,21.12,11,161,82,89,93,97,99,107,76347.63,9848.65,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API with Auth No Guardrails,1000,7412.08,134.64,1024,10240,10,5333742,0,0,997,27.65,11,308,161,176,183,191,196,209,75373.06,9722.93,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,Scenario Name,Concurrent Users,Throughput (Requests/sec),Average Response Time (ms),Message Size (Bytes),Response Size (Bytes),Back-end Service Delay (ms),# Samples,Error Count,Error %,Average Users in the System,Standard Deviation of Response Time (ms),Minimum Response Time (ms),Maximum Response Time (ms),75th Percentile of Response Time (ms),90th Percentile of Response Time (ms),95th Percentile of Response Time (ms),98th Percentile of Response Time (ms),99th Percentile of Response Time (ms),99.9th Percentile of Response Time (ms),Received (KB/sec),Sent (KB/sec),,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API PII Masking,100,3602.8,27.6,1024,10240,10,2592505,0,0,99,15.42,11,75,46,53,56,58,60,64,36636.71,4736.59,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API PII Masking,500,3749.79,133.13,1024,10240,10,2698479,0,0,499,31.19,11,291,161,177,185,193,199,223,38131.39,4929.83,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API PII Masking,1000,3815.3,262.09,1024,10240,10,2745595,0,0,999,63.13,20,548,305,345,371,389,399,447,38797.55,5015.96,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,Scenario Name,Concurrent Users,Throughput (Requests/sec),Average Response Time (ms),Message Size (Bytes),Response Size (Bytes),Back-end Service Delay (ms),# Samples,Error Count,Error %,Average Users in the System,Standard Deviation of Response Time (ms),Minimum Response Time (ms),Maximum Response Time (ms),75th Percentile of Response Time (ms),90th Percentile of Response Time (ms),95th Percentile of Response Time (ms),98th Percentile of Response Time (ms),99th Percentile of Response Time (ms),99.9th Percentile of Response Time (ms),Received (KB/sec),Sent (KB/sec),,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API Advanced Guardrails,100,3160.78,31.48,1024,10240,10,1705434,0,0,99,15.54,11,80,50,56,58,61,63,68,32141.81,4158.56,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API Advanced Guardrails,500,3271.39,152.62,1024,10240,10,1765233,0,0,499,42.39,12,351,186,203,213,240,254,285,33266.54,4304.08,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,AI API Advanced Guardrails,1000,3272.24,305.61,1024,10240,10,1765854,0,0,1000,82.8,22,605,373,403,423,459,475,519,33275.22,4305.2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Loading