Do not require a chat template in embeddings mode

[mcharytoniuk]

Embeddings-only agents no longer require a chat template; token generation is rejected while the cluster is configured for embeddings.

[Copilot]

Pull request overview

This PR removes the requirement to load a model chat template when the cluster is configured for embeddings, and proactively rejects token-generation requests in that configuration.

Changes:

• Skip chat-template loading in the agent when embeddings are enabled (unless an override is provided), and propagate “skipped” status through startup signaling.
• Add a dedicated terminal token result (TokenGenerationDisabled) and return it when a generation path is invoked without a chat template renderer.
• Add balancer-side guards that return HTTP 501 for token-generation endpoints when embeddings are enabled, plus add coverage tests (including an embeddings-only startup test without a chat template).

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
paddler_tests/tests/main.rs	Registers the new embeddings-without-chat-template integration test module.
paddler_tests/tests/agent_serves_embeddings_without_a_chat_template.rs	New test proving embeddings work with no chat template configured.
paddler_messaging/src/generated_token_result.rs	Adds TokenGenerationDisabled and marks it as terminal (is_done).
paddler_bootstrap/src/balancer_service_bundle.rs	Wires balancer_applicable_state_holder into OpenAIService construction.
paddler_balancer/src/require_token_generation_enabled.rs	New helper to reject token-generation when embeddings are enabled.
paddler_balancer/src/lib.rs	Exposes the new require_token_generation_enabled module.
paddler_balancer/src/inference_service/http_route/api/ws_inference_socket/mod.rs	Blocks inference websocket upgrades when embeddings are enabled; adds a test.
paddler_balancer/src/inference_service/http_route/api/post_continue_from_raw_prompt.rs	Blocks raw-prompt token streaming when embeddings are enabled.
paddler_balancer/src/inference_service/http_route/api/post_continue_from_conversation_history.rs	Blocks conversation-history token streaming when embeddings are enabled.
paddler_balancer/src/compatibility/openai_service/openai_error.rs	Treats TokenGenerationDisabled as an error description source.
paddler_balancer/src/compatibility/openai_service/mod.rs	Adds state-holder plumbing to OpenAI-compat service runtime.
paddler_balancer/src/compatibility/openai_service/http_route/post_responses.rs	Blocks /v1/responses when embeddings are enabled.
paddler_balancer/src/compatibility/openai_service/http_route/post_chat_completions.rs	Blocks /v1/chat/completions when embeddings are enabled; adds a test.
paddler_balancer/src/compatibility/openai_service/app_data.rs	Extends OpenAI-compat app data with the balancer applicable state holder.
paddler_agent/src/prepare_conversation_history_request.rs	Requires a renderer for generation; emits TokenGenerationDisabled when absent.
paddler_agent/src/lib.rs	Exposes the new chat-template load status module.
paddler_agent/src/continuous_batch_scheduler_context.rs	Makes chat_template_renderer optional to support embeddings-only startup.
paddler_agent/src/continuous_batch_arbiter.rs	Skips chat-template loading in embeddings mode and signals ChatTemplateLoadStatus.
paddler_agent/src/chat_template_load_status.rs	New enum representing loaded vs skipped-for-embeddings chat template status.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 72 out of 74 changed files in this pull request and generated 3 comments.