Skip to content

fix(tui): derive local output budget from route window - #5883

Closed
dajiaohuang wants to merge 1 commit into
Hmbown:mainfrom
dajiaohuang:fix/5820-ollama-route-budget
Closed

fix(tui): derive local output budget from route window#5883
dajiaohuang wants to merge 1 commit into
Hmbown:mainfrom
dajiaohuang:fix/5820-ollama-route-budget

Conversation

@dajiaohuang

Copy link
Copy Markdown

Summary

  • derive the automatic output reservation from a route's declared context window when the model has no static catalogue row
  • preserve explicit operator overrides, concrete route output limits, compatibility caps, and existing window clamping
  • add a synthetic 32K Ollama regression covering the reported budget collapse and override precedence

Fixes #5820.

Behavior

For an uncatalogued model on a 32,768-token Ollama route, the automatic output cap is now 16,384 instead of being derived from the 128K fallback and clamped to 30,720. The resulting input ceiling is 15,360 rather than 1,024. No live Ollama process, model, credentials, or provider request is used by the test.

Validation

  • regression test against the unchanged baseline fails as expected: output cap 30,720 vs expected 16,384
  • cargo test -p codewhale-tui route_budget::tests --lib --locked with --cap-lints warn: 24 passed
  • cargo check -p codewhale-tui --locked: passed with the repository's strict warning policy
  • rustfmt --check --edition 2024 crates/tui/src/route_budget.rs
  • git diff --check

The lint cap was used only for lib-test compilation because three unrelated Windows conditional-test variables are denied as unused at baseline. The normal non-test cargo check passed without that cap.

@dajiaohuang
dajiaohuang requested a review from Hmbown as a code owner September 4, 2026 23:21
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Thanks @dajiaohuang for taking the time to contribute.

This repository is observing a maintainer-managed PR intake gate in dry-run mode, so this pull request is staying open. This note helps maintainers prepare the allowlist before any enforcement is considered.

Please read CONTRIBUTING.md for the expected contribution shape. A maintainer can grant recurring PR access by commenting /lgtm on a pull request.

@Hmbown

Hmbown commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Thank you for this — you diagnosed #5820 independently and correctly. v0.9.12 shipped the same window/2 reservation for uncatalogued local routes (with the explicit operator cap still winning) as part of a broader route-budget fix, so this branch no longer applies to main. If you see any remaining gap on a real Ollama route after updating, please reopen against main and we'll take it from there.

@Hmbown Hmbown closed this Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ollama provider: input budget collapses to 1024 tokens on 32K local models (default output reservation 64K clamps the window)

2 participants