Skip to content

PROVIDERS.md

Latest commit

 

History

History
85 lines (73 loc) · 4.92 KB

PROVIDERS.md

File metadata and controls

85 lines (73 loc) · 4.92 KB
Raw

Provider Connections

Provider connections describe external capabilities available to WorldSharp. They are not Drivers: Providers speak to services, while Drivers decide how a feature orchestrates those services.

Current kinds

  • OpenAiCompatible: text-generation services exposing /v1/models and, /v1/chat/completions. KoboldCpp on port 5001 is the first tested implementation; live conversation generation through it is verified.
  • ComfyUi: ComfyUI's HTTP API. Connection testing uses /system_stats.

Profiles contain a display name, kind, HTTP(S) base address, optional model override, and an optional write-only API key. API keys are encrypted before SQLite persistence using a host-owned key stored beside the database. They are never returned by connection APIs. The database and .provider-credentials.key must be backed up together; losing the key requires re-entering saved credentials. OpenAI-compatible profiles also store the model context window; it defaults to 32,768 tokens and should match the context size allocated by a local server such as KoboldCpp.

The Connections page provides an OpenRouter preset. It fills https://openrouter.ai/api/v1, links to OpenRouter's key page, authenticates model discovery with the saved key, and offers the returned model IDs to the model field. OpenAI-compatible URL joining accepts both a server root such as http://localhost:5001 and a versioned base such as OpenRouter's /api/v1 without appending a second /v1.

Exactly one saved connection per Provider kind may be marked as the default. The first connection of a kind becomes its default automatically. Users can explicitly change that selection; Drivers must use the configured default and must not depend on database or load order. Profiles can be edited and deleted.

Connection tests execute on the ASP.NET host. Therefore localhost means the machine running WorldSharp, which is suitable for self-hosted KoboldCpp and ComfyUI.

Analysis output and thinking controls

World Tracker and historical reconstruction use a larger analysis budget than normal chat generation. The default is 8,192 output tokens (the model's reasoning and structured JSON share this budget), and both the World Tracker page's analysis settings let users choose a different limit from 256 to 32,768. The import wizard only stages source material and links to World Tracker; it does not start provider generation.

The World Tracker analysis settings also expose None, Minimal, Low, Medium, and High thinking effort. OpenRouter receives the selected effort as reasoning_effort. OpenAI-compatible local endpoints also receive Marinara-compatible chat_template_kwargs.enable_thinking; selecting None additionally sends reasoning_format: "none" so KoboldCpp does not spend the response budget on hidden reasoning. These settings are request-scoped and apply to manual analysis and the next imported range. OpenAI-compatible requests explicitly use non-streaming chat completions. Tracker/analysis requests, OpenRouter, and local/KoboldCpp endpoints send the output budget as max_completion_tokens (never alongside the legacy max_tokens field), matching the reasoning-capable request shape. Before the request is sent, WorldSharp estimates the prompt size and reduces the output budget when prompt plus completion would exceed the configured context window. Local endpoints also receive KoboldCpp-compatible max_context_length, so the server is explicitly asked to use the configured window. Ordinary remote generation keeps max_tokens for compatibility. Each generation attempt has a fresh ten-minute provider timeout, including a repair retry. A timeout or interrupted response is surfaced as a retryable failed run. World Tracker analysis itself is owned by a server-side operation: the browser starts it, polls short status requests, and can request cancellation without holding one HTTP request open through the model's initial and repair generations. This prevents a browser request timeout from interrupting a provider call that is still running. The response reader accepts plain-string content, typed OpenRouter output blocks, and legacy text fields. Reasoning-only, refusal, empty-choice, and unsupported response-shape results are reported with actionable diagnostics rather than being passed to the JSON Tracker as if they were a final answer.

World Tracker keeps malformed model output recoverable. It performs one bounded repair request with the validation error and a truncated copy of the rejected response, then records a durable failed run if the repair still cannot be accepted. Failed runs can be retried from the Tracker UI using the current connection, driver, and analysis settings; network, authentication, and cancellation failures are surfaced rather than silently retried.

ComfyUI image execution is deferred until WorldSharp has a user-selected workflow contract. A healthy connection alone is not enough to invent sampler, model, dimensions, or output-node choices.