Skip to content

Add direct LLM connection management - #4

Open
aaravchour wants to merge 6 commits into
10x-app-builder:mainfrom
aaravchour:codex/comprehensive-llm-connections
Open

Add direct LLM connection management#4
aaravchour wants to merge 6 commits into
10x-app-builder:mainfrom
aaravchour:codex/comprehensive-llm-connections

Conversation

@aaravchour

Copy link
Copy Markdown

Summary

This PR adds a comprehensive direct LLM connection system to the macOS app so users can bring their own providers instead of routing every generation through the hosted Claude proxy. It introduces persisted provider configuration, keychain-backed API keys, an in-app Connections settings surface, direct streaming clients, and safer unauthenticated/placeholder startup behavior.

What changed

Direct provider connections

  • Added LLMConnection and LLMConnectionService for managing saved provider connections.
  • Stores API keys in Keychain while persisting non-secret connection metadata in UserDefaults.
  • Supports OpenAI, Anthropic Claude, Ollama, and OpenRouter.
  • Tracks an active direct connection used by generation when configured and valid.
  • Validates model, base URL, enabled state, and provider-specific API key requirements.
  • Adds provider test calls for OpenAI/OpenRouter /v1/models, Anthropic /v1/models, and Ollama /api/tags.
  • Fetches installed local Ollama models from the configured local server and keeps manual custom model entry available.

Direct LLM streaming

  • Added direct streaming clients for Claude, OpenAI, Ollama, and OpenRouter.
  • Converts OpenAI-compatible streams back into the app’s Anthropic-style NDJSON event format.
  • Converts Anthropic tool definitions into OpenAI function tools where needed.
  • Converts Anthropic-style conversation history, tool use blocks, and tool result blocks into OpenAI-compatible chat messages.
  • Updates GenerationService to route through an active direct connection before using the hosted backend.

Settings UI

  • Adds a new Settings > Connections section.
  • Provides a card-based connection manager with active route status, provider summary strip, readiness validation, quick activate/deactivate, edit, delete, and test actions.
  • Adds an editor with provider-specific defaults, setup hints, custom model support, base URL configuration, enable/disable state, and test feedback.
  • Ollama model selection refreshes from local /api/tags, so users see their actual installed local models.

Guest sign-in and startup hardening

  • Adds a “Continue without signing in” option on the login screen.
  • Adds persisted guest mode and updates settings to show guest status clearly.
  • Prevents placeholder Supabase config from triggering fake network calls to your-project-ref.supabase.co.
  • Opts into Supabase auth’s newer initial-session behavior with emitLocalSessionAsInitialSession: true.
  • Prevents Sparkle from starting when the feed URL or EdDSA key is still placeholder/unconfigured.

Why

This gives users a practical local/direct model workflow, especially for Ollama, without requiring hosted account setup for every run. It also makes the app more robust in development builds where Supabase and Sparkle placeholders are expected.

Verification

  • Built successfully with:
xcodebuild -project /Users/aaravchourishi/Developer/10x/10x-macos.xcodeproj -scheme 10x-macos -configuration Debug build

Notes / follow-up areas

  • Direct OpenAI-compatible routing supports tool calls by translating the app’s Anthropic-style tool protocol into OpenAI chat completions format.
  • Ollama model fetch depends on a running local Ollama server at the configured host. If Ollama is not running, the UI shows a fetch error and still allows manual model entry.
  • Billing/hosted-only features still require a valid app auth token; guest/direct-provider mode primarily enables local app use and direct generation routing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant