Talking to phi-4 for example causes 400 - roles must alternate

[matthiasg]

025/09/19 08:02:21 Error getting prompt tokens: OpenAI API request failed: error, status code: 400, status: 400 Bad Request, message: Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/..

[matthiasg]

The server might actually have hosted gemma 3 (model id was wrong not phi 4). Sorry for that

[Yoosu-L]

Hi @matthiasg, thanks for your feedback.

That's interesting. I just tested with google/gemma-3-1b running locally via LM Studio and could not reproduce the issue. The benchmark ran successfully.

Test Environment:

• Model: google/gemma-3-1b
• Service: LM Studio (self-hosted)

Test Results:

Input Tokens: 42
Output Tokens: 512
Test Model: google/gemma-3-1b
Latency: 7.40 ms

| Concurrency | Generation Throughput (tokens/s) |  Prompt Throughput (tokens/s) | Min TTFT (s) | Max TTFT (s) |
|-------------|----------------------------------|-------------------------------|--------------|--------------|
|           1 |                           140.79 |                        423.73 |         0.09 |         0.09 |
|           2 |                           140.05 |                         19.01 |         0.06 |         3.69 |
Concurrency 4  44% |█████████████████                       | (902/2048, 139 tokens/s) [6s:8s]

Could you tell me a bit more about your setup? Are you using a self-hosted service (like Ollama, LM Studio, etc.) or a specific API service provider? This information will help me narrow down the cause of the problem.