Skip to content

Add LM Studio as a local model provider - #434

Open
zalex666 wants to merge 3 commits into
andrewyng:mainfrom
zalex666:feat/lmstudio-provider
Open

Add LM Studio as a local model provider#434
zalex666 wants to merge 3 commits into
andrewyng:mainfrom
zalex666:feat/lmstudio-provider

Conversation

@zalex666

@zalex666 zalex666 commented Aug 3, 2026

Copy link
Copy Markdown

Closes #122

What

Adds LM Studio as a first-class local model provider, alongside Ollama — the second keyless provider riding the OpenAI-compatible /v1 path (OpenAIProvider with a placeholder key, default http://localhost:1234).

Prior art

#184 by @sobiya-22 predates this PR and implements the same core pattern — credit to them for getting there first (I should have searched before opening this one). #206 by @hacksics independently identified the keyless-provider discovery bug before self-closing. On top of the core feature, this PR ships the brand mark, README, a recommended model, and the six correctness fixes below — the first of which is what makes "Detect → models appear" work at the default URL — and is rebased on current main. Zero attachment to which PR lands: if maintainers prefer #184, everything here can be cherry-picked onto it and I'll close this.

Users get the same experience Ollama has today: pick LM Studio in the provider gallery, optionally adjust the server URL, hit Detect, and the models already downloaded in LM Studio appear in the composer picker — gated on a liveness probe so a machine without LM Studio never shows phantom local models.

Why

LM Studio is one of the most popular ways to run local models, exposes the same OpenAI-compatible surface the provider layer already speaks, and (like the compat vendors, per the registry's own note) deserves a first-class listing rather than the "point the OpenAI slot at a custom endpoint with a dummy key" trick.

How

  • providers/registry.py — new lmstudio descriptor + builder (OpenAIProvider(api_key="lm-studio", base_url=<root>/v1)). _normalize_ollama_url is generalized to _normalize_local_url(url, default) since both servers want <root>/v1. The keyless verify branch (GET /v1/models, no auth header, friendly 404/401 copy) now covers both local providers via LOCAL_PROVIDERS.
  • providers/capabilities.pylmstudio:* models get the same conservative local-model defaults as Ollama (tools on, vision/parallel-tool-calls off) unless the matrix says otherwise.
  • server/manager.py_ollama_alive/_ollama_models generalized to _local_alive(name)/_local_models(name), driven by a small LOCAL_MODEL_SERVERS table. Ollama keeps its native /api/tags; LM Studio uses the OpenAI-shaped /v1/models (its native REST API is still beta and can sit behind an auth token). Liveness stays cached 30s, now per provider. Picker gating and add-model suggestions follow the table.
  • GUI — LM Studio mark vendored from the same MIT lobe-icons set, gallery order next to Ollama, and the keyless "install the app" help copy generalized into a LOCAL_HELP map (it previously hardcoded Ollama as the only keyless provider).
  • Recommended modelqwen/qwen3-coder-30b, the LM Studio catalog id of the same verified tool-calling pick the Ollama row recommends.

Review hardening (second commit)

An adversarial review pass (GPT-5.6 Sol @ xhigh via Codex CLI) surfaced five latent issues in the keyless-local-provider flow — all pre-existing Ollama behaviors this change inherited or generalized — fixed here since the PR owns that surface now:

  • Keyless Detect persists on every pass. Keyless providers report configured out of the box, so the GUI's dirty/configured gate skipped the first-time save: no stored profile, and set_provider's recommended-model auto-add never ran. _local_models pairs with this by treating a stored empty profile as engaged (is None gate).
  • _provider_available (local servers must actually answer; everyone else must be configured) now drives picker gating, model_ready, and the first-working-provider default handoff — a dead local server no longer reads as ready, and no longer blocks a working provider from taking over the default.
  • Probes check shape, not just status. Verify and liveness require the provider's list container (data at /v1/models, models at /api/tags) — a random service answering on the port no longer passes as a model server.
  • Explicitly-blank endpoint means "back to the default" in verify_provider, instead of resurrecting the stored URL (a passing Test used to validate a config the subsequent save then removed). Blank secrets still fall back to the stored key.
  • Liveness cache invalidates on config change, so repointing/removing a local server re-probes immediately instead of serving up-to-30s-stale results.

A follow-up confirmation review verified the fixes.

Tests

Mirrors and extends the existing Ollama coverage: URL normalization, builder base_url/placeholder-key, router prefix-strip for slash-carrying LM Studio ids, capabilities, keyless verify request shape + non-model-server rejection, liveness-gated picker entries (per-provider: one local server being up doesn't surface the other's models), both list-shape parses, cache invalidation, model_ready gating, the first-Detect end-to-end path, and the GUI hook's keyless-Detect save. The Playwright provider fixture gains an lmstudio entry.

  • pytest tests/ — 1124 passed, 1 skipped
  • tsc --noEmit clean, vitest run — 110 passed
  • Live against a real LM Studio server: verify/liveness/discovery, the first-Detect (blank endpoint) flow end-to-end, picker gating and model_ready, and tool-calling round trips on qwen-qwen2.5-coder-7b and qwen/qwen3.6-27b — JIT model load, structured get_weather(city=Paris) call, tool-result fold-in, usage metering, and streaming (a thinking model's reasoning_content flows through the reasoning sidecar correctly)

🤖 Generated with Claude Code

zalex666 and others added 3 commits August 3, 2026 12:52
Second keyless local provider alongside Ollama, riding the same
OpenAI-compatible /v1 path (OpenAIProvider with a placeholder key,
default http://localhost:1234):

- registry: lmstudio descriptor + builder; _normalize_ollama_url
  generalized to _normalize_local_url(url, default); the keyless
  verify branch (GET /v1/models, no auth) now covers both local
  providers
- capabilities: lmstudio gets the same conservative local-model
  defaults as ollama (tools yes, parallel/vision no)
- manager: _ollama_alive/_ollama_models generalized to
  _local_alive/_local_models driven by a LOCAL_MODEL_SERVERS table
  (ollama via native /api/tags, LM Studio via OpenAI-shaped
  /v1/models); picker gating and suggestions follow
- GUI: lobe-icons LM Studio mark, gallery order, and the keyless
  install-help copy generalized into a LOCAL_HELP map
- tests mirror the existing Ollama coverage (normalize/build/verify/
  capabilities/liveness gating) plus a parse test for both list shapes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y semantics

Fixes from the gpt-5.6-sol review of the LM Studio provider addition
(all pre-existing Ollama behaviors this change inherited or generalized):

- keyless Detect now persists on every pass (GUI): keyless providers
  report configured out of the box, so the dirty/configured gate skipped
  the first-time save — no stored profile, and set_provider's
  recommended-model auto-add never ran. _local_models pairs with this by
  treating a stored EMPTY profile as engaged (is None check).
- _provider_available: local servers must actually answer, everyone else
  must be configured — now drives picker gating, model_ready, and the
  first-working-provider default handoff (a dead local server no longer
  reads as ready, and no longer blocks a working provider from taking
  the default).
- probes require the provider's list shape, not just a 200: /v1/models
  must serve {"data": [...]}, /api/tags {"models": [...]} — some other
  service answering on the port no longer passes verify or liveness.
- verify_provider: non-secret fields sent explicitly blank mean 'back to
  the default' instead of resurrecting the stored value (a passing Test
  used to validate a URL the subsequent save then removed); blank
  secrets still fall back to the stored key.
- _refresh_provider drops the per-provider liveness cache, so repointing
  or removing a local server re-probes immediately.
- coverage: probe shape/URLs, cache invalidation, model_ready gating,
  first-Detect end-to-end (empty profile -> recommended auto-add ->
  default handoff), explicit-blank verify, the GUI hook's keyless
  Detect save, and an lmstudio entry in the e2e provider fixture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codex confirmation pass (round 2) verdicts: findings 1-5 FIXED, plus
three residuals addressed here:

- runTestAndSave no longer swallows a rejected/not-ok setProvider: a
  passing Detect whose save fails now shows the error instead of
  '✓ Tested & saved' (affected keyed providers too; the keyless path
  made it likelier)
- the explicit-blank verify test stubs httpx BEFORE set_provider, so
  its suggested-models pass can't hit the network
- regression test for the dead-local-default handoff: a dead local
  default yields to a newly configured provider, a live one is never
  stolen (pins _provider_available in the handoff path)
- GUI test for the failed-save path

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@zalex666
zalex666 marked this pull request as ready for review August 3, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support LM Studio as a local model provider

1 participant