Skip to content

fix(agent): retry title generation with a larger budget for unconfigured reasoning models - #3387

Open
albatrossflyon-coder wants to merge 1 commit into
charmbracelet:mainfrom
albatrossflyon-coder:fix/title-generation-unconfigured-reasoning-model
Open

fix(agent): retry title generation with a larger budget for unconfigured reasoning models#3387
albatrossflyon-coder wants to merge 1 commit into
charmbracelet:mainfrom
albatrossflyon-coder:fix/title-generation-unconfigured-reasoning-model

Conversation

@albatrossflyon-coder

Copy link
Copy Markdown

Fixes #3284.

Root cause

GenerateTitle already bumps the output-token budget for models flagged CatwalkCfg.CanReason (40 → DefaultMaxTokens), specifically so a thinking model doesn't burn its whole budget on hidden reasoning content before producing a title. It also tries a /no_think prompt hack for templates that respect it.

CanReason comes from static model metadata (Catwalk's known-model registry). A hand-configured custom/local provider — like a local llama.cpp server serving a thinking-capable model — has no such metadata unless the user explicitly sets it. If it's unset, the model still reasons by its own chat template default (the /no_think hack only works for templates that specifically respect that convention) — crush just doesn't know to give it room. So it burns the tiny 40-token budget entirely on hidden reasoning, produces no title text, and every attempt (small model, then large model) falls through to the "Untitled Session" fallback — matching this issue's exact symptom.

Fix

Detect this directly instead of depending on correct prior configuration: if a title attempt hits FinishReasonLength and the response contains reasoning content even though CanReason was false, that's a template-agnostic signal the model needs more room regardless of what its config declares. Retry that same attempt once with a larger budget (DefaultMaxTokens, or a 2000-token fallback if that's also unset for a custom model) before moving to the next model attempt.

Extracted the decision into shouldRetryTitleWithLargerBudget so it's directly unit-testable without mocking the full Stream/Agent machinery.

Testing

  • Added TestShouldRetryTitleWithLargerBudget (4 cases: unconfigured reasoning model hitting the limit → retry; already-bumped budget → no further retry; hit limit with no reasoning content → not a thinking-budget problem, no retry; finished normally → no retry).
  • go build ./... and go vet ./internal/agent/... clean.
  • Confirmed no other test in the package broke: go test ./internal/agent/... -run "^$" compiles clean across all subpackages.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Summarization Request Fails with Thinking Local Llama.cpp Models

1 participant