fix(agent): raise per-response token budget so tool calls aren't truncated#138
Merged
Conversation
bararchy
changed the title
…cated The model writes whole files inline inside <tool_call>, but the 512-token per-response cap cut generation off before the closing </tool_call> tag, so parse_tool_calls found no call and the turn silently ended. The model already stops itself on <|im_end|>, so the cap is just a safety bound: raise the default 512 -> 4096 (env SHAINET_AGENT_MAX_TOKENS). KV cache grows ~linearly with it.
bararchy
force-pushed
the
fix/agent-followthrough
branch
from
June 29, 2026 13:02
a722951 to
b71a901
Compare
bararchy
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
The agent appeared to 'stall' after announcing intent and never wrote the file. A raw-token dump on a live 30B session showed the model does emit a correct
<tool_call><function=write_file>…with the whole file inline — but the 512-token per-response cap truncated it before the closing</tool_call>tag.parse_tool_callsneeds that tag, so it matched nothing → the turn silently ended.Fix
One line: the model already self-terminates on
<|im_end|>, so the per-response cap is just a safety bound — raise the default 512 → 4096 (envSHAINET_AGENT_MAX_TOKENS). KV-cache cost grows ~linearly (~0.2-0.4 MB/token on this 30B), so a few-thousand-token ceiling is comfortable on a 16 GB card.Verified
crystal tool formatclean,ameba0.⚒ write_file(path=…, content=<full script>)now generates completely and parses — the call fires.