Skip to content

U8: k6 load tests update #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jaturapornchai wants to merge 1 commit into
base: main
Choose a base branch
from
Open

U8: k6 load tests update #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions loadtest/_categories.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// Shared chat prompt categories for load tests.
// Diverse mix of Thai/English and small/medium prompt sizes.
export const categories = [
{ name: 'thai_small', body: { model: 'bcproxy/auto', messages: [{ role: 'user', content: 'สวัสดี ตอบสั้นๆ' }] } },
{ name: 'code_small', body: { model: 'bcproxy/auto', messages: [{ role: 'user', content: 'Python fn reverse string' }] } },
{ name: 'general_medium', body: { model: 'bcproxy/auto', messages: [{ role: 'user', content: 'Explain photosynthesis in 3 sentences for a 10-year-old.' }] } },
{ name: 'thai_medium', body: { model: 'bcproxy/auto', messages: [{ role: 'user', content: 'อธิบายการทำงานของ AI 3 ย่อหน้า สำหรับเด็ก 10 ขวบ' }] } },
];

export function pickCategory() {
return categories[Math.floor(Math.random() * categories.length)];
Comment on lines +10 to +11

Copilot AI Apr 9, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description says categories are “rotated” per VU iteration, but pickCategory() currently selects randomly, which can skew per-category breakdowns and make runs less reproducible. If rotation is required, consider a deterministic selection (e.g., based on __ITER/__VU) to cycle through categories evenly.

Copilot uses AI. Check for mistakes.
}
23 changes: 11 additions & 12 deletions loadtest/chat.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,34 +1,33 @@
import http from 'k6/http';
import { check, sleep } from 'k6';
import { pickCategory } from './_categories.js';

export const options = {
stages: [
{ duration: '30s', target: 10 },
{ duration: '1m', target: 10 },
{ duration: '30s', target: 30 },
{ duration: '1m', target: 30 },
{ duration: '30s', target: 20 },
{ duration: '1m', target: 20 },
{ duration: '30s', target: 0 },
],
thresholds: {
http_req_duration: ['p(95)<10000'],
http_req_failed: ['rate<0.15'],
'http_req_duration{expected_response:true}': ['p(95)<5000', 'p(99)<15000'],
'http_req_failed': ['rate<0.02'],
},
Comment on lines 6 to 14

Copilot AI Apr 9, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given /v1/chat/completions is rate-limited to 100 req/60s per IP, the stage (20 VUs with ~1–3s think time) will exceed the limiter and generate many 429s. That will violate http_req_failed rate<0.02 (since 429 counts as failed by default), so the test is likely to fail in normal local/staging runs even though the script backs off on 429. Consider either (a) lowering the request rate below the limiter, or (b) marking 429 as an expected status in k6 so it doesn’t count toward http_req_failed, depending on what this test is intended to validate.

Copilot uses AI. Check for mistakes.
};

const BASE_URL = __ENV.BASE_URL || 'http://localhost:3333';

const payload = JSON.stringify({
model: 'auto',
messages: [{ role: 'user', content: 'say hi in 1 word' }],
});

const params = {
headers: { 'Content-Type': 'application/json' },
timeout: '20s',
};

export default function () {
const res = http.post(`${BASE_URL}/v1/chat/completions`, payload, params);
const cat = pickCategory();
const res = http.post(
`${BASE_URL}/v1/chat/completions`,
JSON.stringify(cat.body),
{ ...params, tags: { category: cat.name } },
);

if (res.status === 429) {
const retryAfter = parseInt(res.headers['Retry-After'] || '5');
Expand Down
24 changes: 21 additions & 3 deletions loadtest/smoke.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
import http from 'k6/http';
import { check, sleep } from 'k6';
import { reportTo } from './_report.js';
import { pickCategory } from './_categories.js';

export const options = {
vus: 1,
duration: '10s',
vus: 5,
duration: '30s',
thresholds: {
http_req_failed: ['rate<0.01'],
'http_req_duration{expected_response:true}': ['p(95)<5000', 'p(99)<15000'],
'http_req_failed': ['rate<0.02'],
Comment on lines +7 to +11

Copilot AI Apr 9, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the server-side rate limit (100 req/60s per IP on /v1/chat/completions), running 5 VUs for 30s will very likely produce a large number of 429s, causing the new http_req_failed rate<0.02 threshold to fail consistently. Consider lowering VUs/iteration rate to stay under the limiter, or explicitly treating 429 as an expected status in k6 (e.g., via a responseCallback/expectedStatuses) or by varying X-Real-IP/X-Forwarded-For per VU if the goal is to measure backend latency rather than the limiter.

Copilot uses AI. Check for mistakes.
},
};

const BASE_URL = __ENV.BASE_URL || 'http://localhost:3333';

const params = {
headers: { 'Content-Type': 'application/json' },
timeout: '30s',
};

export default function () {
const homepage = http.get(`${BASE_URL}/`);
check(homepage, {
Expand All @@ -23,6 +30,17 @@ export default function () {
'status endpoint 200': (r) => r.status === 200,
});

const cat = pickCategory();
const res = http.post(
`${BASE_URL}/v1/chat/completions`,
JSON.stringify(cat.body),
{ ...params, tags: { category: cat.name } },
);
check(res, {
'chat status 200': (r) => r.status === 200,
'chat body has content': (r) => r.body && r.body.includes('content'),

Copilot AI Apr 9, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chat body has content uses a string substring check on the raw response body, which can produce false positives (e.g., error payloads that include the word "content") and false negatives (format changes). It’s more reliable to parse JSON (e.g., res.json() / selector) and assert the expected shape like choices[0].message.content (with a guard for JSON parse errors).

Suggested change
'chat body has content': (r) => r.body && r.body.includes('content'),
'chat body has content': (r) => {
if (!r.body) {
return false;
}
try {
const body = JSON.parse(r.body);
return Array.isArray(body.choices)
&& body.choices.length > 0
&& body.choices[0]
&& body.choices[0].message
&& typeof body.choices[0].message.content === 'string'
&& body.choices[0].message.content.length > 0;
} catch (_) {
return false;
}
},

Copilot uses AI. Check for mistakes.
});

sleep(1);
}

Expand Down
27 changes: 14 additions & 13 deletions loadtest/stress.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,36 +1,37 @@
import http from 'k6/http';
import { check } from 'k6';
import { Trend } from 'k6/metrics';
import { pickCategory } from './_categories.js';

export const options = {
stages: [
{ duration: '1m', target: 100 },
{ duration: '2m', target: 100 },
{ duration: '1m', target: 300 },
{ duration: '2m', target: 300 },
{ duration: '1m', target: 500 },
{ duration: '1m', target: 500 },
{ duration: '1m', target: 50 },
{ duration: '2m', target: 50 },
{ duration: '30s', target: 100 },
{ duration: '30s', target: 0 },
],
// No strict thresholds — goal is to observe where errors start
thresholds: {
'http_req_duration{expected_response:true}': ['p(95)<5000', 'p(99)<10000'],
'http_req_failed': ['rate<0.02'],
},
Comment on lines +13 to +16

Copilot AI Apr 9, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new http_req_failed rate<0.02 threshold will likely fail quickly under stress conditions because /v1/chat/completions is rate-limited (429) and 429s are counted as failures by default in k6. If the goal is to stress the backend (not the limiter), consider varying X-Real-IP/X-Forwarded-For per VU or adjusting expected statuses/thresholds so rate limiting doesn’t dominate the failure signal.

Copilot uses AI. Check for mistakes.
};

const BASE_URL = __ENV.BASE_URL || 'http://localhost:3333';

const chatDuration = new Trend('chat_req_duration', true);

const payload = JSON.stringify({
model: 'auto',
messages: [{ role: 'user', content: 'say hi in 1 word' }],
});

const params = {
headers: { 'Content-Type': 'application/json' },
timeout: '30s',
};

export default function () {
const res = http.post(`${BASE_URL}/v1/chat/completions`, payload, params);
const cat = pickCategory();
const res = http.post(
`${BASE_URL}/v1/chat/completions`,
JSON.stringify(cat.body),
{ ...params, tags: { category: cat.name } },
);

chatDuration.add(res.timings.duration);

Expand Down