Working config (OpenAISDK, openai_api_base /v1)
OpenCompass already wraps Chat Completions as OpenAISDK. A host whose public catalog is GET https://api.pzero.studio/v1/models (no key) fits that class.
from opencompass.models import OpenAISDK
models = [
dict(
type=OpenAISDK,
abbr="pzero",
path="deepseek-v4-flash",
key="<Bearer key for that host>",
openai_api_base="https://api.pzero.studio/v1",
query_per_second=1,
max_out_len=1024,
max_seq_len=8192,
batch_size=1,
),
]
openai_api_base is the /v1 root (SDK base_url). path is a catalog id.
Do not use the requests OpenAI class with that /v1 root. That wrapper posts openai_api_base as-is, so /v1 404s; it wants the full .../v1/chat/completions URL. Do not use OpenAISDKResponse (Responses API). any(token in path for token in ('o1', 'o3', 'o4', 'gpt-5')) swaps in max_completion_tokens; pick a catalog id that does not contain those substrings.
Not asking for a first-party provider. OpenAISDK plus openai_api_base is enough.
Working config (OpenAISDK, openai_api_base /v1)
OpenCompass already wraps Chat Completions as
OpenAISDK. A host whose public catalog isGET https://api.pzero.studio/v1/models(no key) fits that class.openai_api_baseis the/v1root (SDKbase_url).pathis a catalog id.Do not use the requests
OpenAIclass with that/v1root. That wrapper postsopenai_api_baseas-is, so/v1404s; it wants the full.../v1/chat/completionsURL. Do not useOpenAISDKResponse(Responses API).any(token in path for token in ('o1', 'o3', 'o4', 'gpt-5'))swaps inmax_completion_tokens; pick a catalog id that does not contain those substrings.Not asking for a first-party provider.
OpenAISDKplusopenai_api_baseis enough.