[Bug]: GGUF files on HuggingFace strip tool call XML tags from output, breaking tool calling

[sobakasu]

Is there an existing issue ? / 是否已有相关的 issue ?

• I have searched, and there is no existing issue. / 我已经搜索过了，没有相关的 issue。

Describe the bug / 描述这个 bug

The official GGUF files hosted at openbmb/MiniCPM5-1B-GGUF produce broken tool calling output. The XML tags (<function>, </function>, <param>, </param>) that the model generates are stripped from decoded text, leaving garbled output like:

 name="get_weather"> name="city">Tokyo

To Reproduce / 如何复现

1. Download any GGUF from openbmb/MiniCPM5-1B-GGUF
2. Load in llama-server or Ollama
3. Send a chat completion with tools:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniCPM5-1B",
    "messages": [{"role": "user", "content": "What is the weather in Tokyo?"}],
    "tools": [{"type":"function","function":{"name":"get_weather","description":"Get weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]
  }'

Result: XML tags are missing from decoded output. Requesting logprobs: true confirms the token IDs (18, 20, 21, 19) are generated but have empty text — they were suppressed during decoding.

Expected behavior / 期望的结果

The <function>, </function>, <param>, and </param> tokens should appear in the model's output during tool calling, producing valid XML like:

<function name="get_weather"><param name="city">Tokyo</param></function>

Screenshots / 截图

No response

Environment / 环境

- OS: windows
- llama.cpp version: 9631 (6e14286ed)
- Model: MiniCPM5-1B-GGUF

Additional context / 其他信息

Root Cause

The GGUF converter (convert_hf_to_gguf.py) reads tokenizer_config.json where token IDs 18–21 are marked "special": true. The converter assigns negative scores to special tokens, which signals llama.cpp to suppress them from output. These are content tokens, not control tokens like <s> or </s>, so they should not be suppressed.

Fix

The GGUF needs to be re-converted with "special": false for token IDs 18–21 in tokenizer_config.json. I verified this fix locally — tool calling works correctly after re-conversion. The hosted GGUF files on HuggingFace need to be updated.

[zhangtao2-1]

@sobakasu
Thanks for the excellent and very thorough report — your root-cause analysis is spot on. 🙏

You're right: token IDs 18–21 (, , , ) are content tokens, not control tokens like /. Marking them "special": true in tokenizer_config.json leads the converter to assign negative scores, and llama.cpp then suppresses them during detokenization — which is exactly why the XML tags disappear and tool calling breaks.

We won't be pushing a re-converted GGUF for this right away, but we've noted it and will address this class of case (content-bearing tokens that must not be suppressed during decoding) with a dedicated design in a follow-up, rather than relying solely on the special flag. In the meantime, your local re-conversion with "special": false for those IDs is a valid workaround.

Thanks again for pinpointing the exact token IDs and verifying the fix — that made this very easy to confirm.