feat: add per-model output caps and dynamic reasoning effort

[penica]

Summary

Adds configurable per-model output-token caps and dynamic reasoning-effort handling for llama.cpp-backed models.

• Applies configured output caps across Chat Completions, Completions, and Responses.
• Adds per-model reasoning-effort budgets: none, low, medium, high, and xhigh.
• Supports Chat Completions reasoning_effort and Responses reasoning.effort; top-level Responses reasoning_effort remains a compatibility alias.
• Exposes configured limits and reasoning metadata through llama-swap /v1/models extension fields.
• Documents endpoint mappings, llama.cpp dynamic-reasoning requirements, fixed-budget incompatibility, and extension-field status.
• Adds Responses alias-pipeline coverage for upstream reasoning transformation.

Compatibility

Dynamic reasoning selection requires official llama.cpp build b8605 or newer. Explicit effort is rejected when the model uses fixed --reasoning-budget or LLAMA_ARG_THINK_BUDGET.

When effort is omitted or default, llama-swap sends no thinking overrides, preserving upstream/default behavior.

Validation

• go test -count=1 ./internal/config ./internal/process ./internal/server
• git diff --check main...HEAD
• Manual validation against official llama.cpp covered Chat and Responses mappings, aliases, unsupported-upstream errors, and fixed-budget handling.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dfca131a-265c-4e80-a3a3-2acc5278a237

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Greptile Summary

This PR adds configurable output caps and reasoning-effort handling for llama.cpp-backed models. The main changes are:

• Per-model max_output_tokens config and request capping.
• Reasoning-effort budgets and llama.cpp dynamic-budget rewriting.
• /v1/models metadata for output limits and reasoning support.
• Docs, schema updates, and tests for the new config paths.

Confidence Score: 5/5

The changed flow looks mergeable after a small schema cleanup.

• Request capping and reasoning rewriting match the server path flow.
• Runtime config validation is strict enough for loaded configs.
• The schema can still approve a reasoning config that runtime loading rejects.

config-schema.json

Important Files Changed

Filename	Overview
internal/process/process_command.go	Adds llama.cpp build detection and request rewriting for explicit reasoning-effort selections.
internal/server/filters.go	Adds max-output-token capping after configured JSON filters and resolves caps through aliases.
internal/config/model_config.go	Adds reasoning config fields and runtime validation for effort names, defaults, and token budgets.
config-schema.json	Adds schema fields for output caps and reasoning, but misses the runtime dependency on max_output_tokens.
internal/server/api.go	Extends model-list metadata with output-token and reasoning fields.

_{Reviews (1): Last reviewed commit: "docs: clarify reasoning effort compatibi..." | Re-trigger Greptile}