Ollama provider: input budget collapses to 1024 tokens on 32K local models (default output reservation 64K clamps the window)

[slowly247]

Issue: Ollama provider — input budget collapses to 1024 tokens on 32K-window local models (default output reservation 64K clamps the window)

Environment

• OS: Windows 10 Pro 22H2 (zh-CN)
• Codewhale: 0.9.11 (96d13a0)
• Ollama: 0.32.3 local, http://127.0.0.1:11434
• Model: qwen2.5:7b (native 32K context), NVIDIA RTX 4060 Ti 8GB
• Config: custom [providers.ollama] route with context_window = 32768

What happened

1. Launch codewhale --provider ollama (TUI) and send any message ("hello").
2. Immediately fails with:
   • Emergency context compaction failed to reduce request below model limit (estimate ~6225 tokens, budget ~1024)
   • then Network error: SSE stream request failed after HTTP/1.1 fallback: error sending request for url (http://127.0.0.1:11434/v1/chat/completions)
3. The same happens when switching to the ollama route via /model inside an already-running session (the session shows "compacting context" and never recovers).

Root cause (source-level analysis)

For Ollama with an uncatalogued model (qwen2.5:7b has no catalogue row):

• effective_max_output_tokens_for_route (crates/tui/src/route_budget.rs) falls back to window / 2 where the assumed window is 128_000 → default output reservation = 64000.
• With the configured 32K window, clamp_output_cap (crates/tui/src/context_budget.rs) clamps that to 32768 - 2048 = 30720.
• Remaining input budget = 32768 - 30720 - 1024 = 1024 — exactly MIN_INPUT_BUDGET_TOKENS.
• The built-in system prompt alone is ~4676 tokens, so emergency compaction can never reach the 1024 budget and always fails.

The actual wire request also carries max_tokens: 30720; with ~6163 estimated input tokens that exceeds the 32K window.

Evidence

codewhale --provider ollama exec --output-format stream-json "hi" metadata reports:

"codewhale_max_output_tokens": 30720
"codewhale_max_output_tokens_source": "route-declared"
"estimated_system_tokens": 4676
"estimated_request_tokens": 6163

Two failure screenshots attached (2026-08-28 and 2026-09-02).

Workaround

Launch the ollama profile with CODEWHALE_MAX_OUTPUT_TOKENS=8192 → input budget becomes ~23552 and everything works. Note: setting this globally also clamps the default deepseek route, so it has to be applied per-launch, which is awkward.

Suggested fix

1. For uncatalogued models, do not derive the default output reservation from a fabricated 128K window (128K / 2 = 64K). Cap the automatic default by the actual window, e.g. min(8192, window / 4), or use a conservative absolute ceiling for small windows (<= 32K).
2. Consider raising MIN_INPUT_BUDGET_TOKENS relative to the window size — 1024 is smaller than the built-in system prompt on any route, which makes the emergency path mathematically impossible to satisfy on small-window self-hosted models.
3. /model hot-switch inside a running session should re-apply per-route budget math (and ideally honor CODEWHALE_MAX_OUTPUT_TOKENS for the new route), so a route switched at runtime does not hit the same trap.

---

（中文摘要：Ollama 路由 + 未收录模型（qwen2.5:7b）时，默认输出预留按 128K/2=64K 计算，在配置的 32K 窗口上被 clamp 到 30720，输入预算被压到 1024 下限（系统提示就有 4676 token），紧急压缩永远无法成功；wire 请求 max_tokens=30720 叠加 ~6163 输入也超窗口。建议：未收录模型的默认输出预留按实际窗口取保守值、MIN_INPUT_BUDGET_TOKENS 随窗口调整、/model 热切换时重新应用路由预算。）

[github-actions]

Thanks @slowly247 for the report.

This issue is staying open for maintainer triage. CodeWhale gets better because people bring us real edge cases from real machines, providers, regions, and workflows.

If you can add a reproduction, logs, version output, screenshots, or the provider/model involved, that makes it much easier for us to verify and harvest the fix. Maintainers may comment /lgtmi to mark recurring issue reporters as approved so this intake note is skipped next time.

[Hmbown]

Confirmed against current main — your arithmetic matches the source: crates/tui/src/route_budget.rs:105-106 assumes a 128k window for an uncatalogued model (unwrap_or(128_000), cap = window/2 = 64000), and crates/tui/src/context_budget.rs:269-274 then clamps to 32768 − (1024 + 1024), leaving exactly MIN_INPUT_BUDGET_TOKENS. Kept open as a real bug; the fix should derive the output reservation from the route's declared context_window when the catalog has no row, rather than from the assumed 128k.