Hello, I'm retrying to reproduce the Terminal Bench 2.1 score with Terminus 2.
Here is my command to deploy GLM 5.2 (sglang v0.5.13.post1 on H200)
sglang serve \
--model-path ~/.cache/huggingface/hub/models--zai-org--GLM-5.2-FP8/snapshots/3722e206c8461c750e5ab73f5528eba42d0c6a37 \
--served-model-name GLM-5.2-FP8 \
--tp 8 \
--mem-fraction-static 0.9 \
--cuda-graph-max-bs 32 \
--max-running-requests 32 \
--reasoning-parser glm45 \
--tool-call-parser glm47
And here is my Harbor (v0.16.1) config:
job_name: tb21-glm5-2
agents:
- name: terminus-2
model_name: hosted_vllm/glm-5.2
max_timeout_sec: 14400
kwargs:
api_base: http://localhost:8000/v1
parser_name: json
temperature: 1.0
max_turns: 500
interleaved_thinking: true
llm_call_kwargs:
extra_body:
chat_template_kwargs:
clear_thinking: false
reasoning_effort: "max"
model_info:
max_input_tokens: 256000
max_output_tokens: 48000
input_cost_per_token: 0.000003
output_cost_per_token: 0.000015
timeout_multiplier: 1.0
agent_timeout_multiplier: 16.0
verifier_timeout_multiplier: 8
n_concurrent_trials: 32
verifier:
max_timeout_sec: 1800.0
The harbor start command:
uv run harbor run -d terminal-bench/terminal-bench-2-1 -c glm5-2.yaml -n 32
And I can only get 62~64 score.
Could you please check my evaluation configuration?
Hello, I'm retrying to reproduce the Terminal Bench 2.1 score with Terminus 2.
Here is my command to deploy GLM 5.2 (sglang v0.5.13.post1 on H200)
sglang serve \ --model-path ~/.cache/huggingface/hub/models--zai-org--GLM-5.2-FP8/snapshots/3722e206c8461c750e5ab73f5528eba42d0c6a37 \ --served-model-name GLM-5.2-FP8 \ --tp 8 \ --mem-fraction-static 0.9 \ --cuda-graph-max-bs 32 \ --max-running-requests 32 \ --reasoning-parser glm45 \ --tool-call-parser glm47And here is my Harbor (v0.16.1) config:
The harbor start command:
And I can only get 62~64 score.
Could you please check my evaluation configuration?