feat(vllm): macOS/Metal support via vllm-metal (MLX)

[localai-bot]

Summary

Adds macOS (Apple Silicon / Metal) support to the vllm backend via vllm-metal (MLX). On darwin, vLLM runs through vllm-metal's platform-plugin; LocalAI's backend.py generation path is unchanged. Linux/CUDA/ROCm/Intel paths are untouched (all darwin logic is additive, gated behind uname -s = Darwin).

What changed

• backend/python/vllm/install.sh (darwin branch): forces PYTHON_VERSION=3.12, uses LocalAI's managed venv (ensureVenv), builds vLLM from the release source tarball, then installs the vllm-metal wheel; installRequirements is skipped on darwin.
  • One pin only — VLLM_METAL_VERSION. The coupled vLLM source version is derived at build time from vllm-metal's own installer (vllm_v=) at the pinned tag (no second hardcoded value to drift; reproducible since the tag is immutable).
  • The vllm-metal wheel is fetched via a deterministic release-asset URL (tag + cp312/arm64 wheel name), not the GitHub API — avoids the unauthenticated API rate limit (60/hr) that 403s on shared CI runners.
• .github/bump_vllm_metal.sh + bump_deps.yaml: a new autobumper job that tracks vllm-project/vllm-metal (not vllm/vllm latest — darwin can only use the vLLM vllm-metal supports) and opens a bump PR for VLLM_METAL_VERSION. Mirrors the existing bump_vllm_wheel.sh.
• .github/backend-matrix.yml: vllm added under includeDarwin:.
• backend/index.yaml: metal: capability + concrete metal-vllm / metal-vllm-development entries.
• backend.py (one defensive guard): Score now returns UNIMPLEMENTED when the engine yields no usable prompt_logprobs, instead of silently scoring every candidate as 0 (see audit). No-op on Linux/CUDA.

Validated

• CI darwin build: green — the full path runs on a macOS runner (portable cp312 → vLLM 0.23.0 built from source → vLLM version derived from the pinned tag → vllm-metal wheel installed via the deterministic URL → packaged).
• Live on a real M4 / macOS 26.5: vllm-metal registers as a platform plugin (metal -> vllm_metal:register, MetalPlatform, MLX GPU); AsyncLLMEngine.from_engine_args transparently resolves to vLLM 0.23's v1 MLX engine; async generate produced correct output on Qwen/Qwen3-0.6B.
• Autobumper dry-run: from a stale pin it advanced VLLM_METAL_VERSION to the latest release and left the derived line untouched.

backend.py audit (Score / Embeddings vs the v1 MLX engine)

• Score (prompt_logprobs): not supported on Metal. Live test: vllm-metal accepts SamplingParams(prompt_logprobs=1) but returns an all-None list rather than computing it. The previous guard treated the truthy [None] as valid → silent all-zero scores. Now it returns UNIMPLEMENTED. Generation/chat is unaffected.
• Embedding: pre-existing, platform-independent. Embedding() calls self.model.encode(), but self.model is never assigned anywhere in backend.py (identical on master) — it raises AttributeError on every platform, CUDA included. This is not a darwin regression; the vllm backend is generation-only and embeddings are served by other backends. Out of scope here.

Version coupling

vllm-metal builds against a specific vLLM release; darwin follows vllm-metal and can lag the Linux vllm pin (requirements-cublas13-after.txt, bumped independently) until vllm-metal supports a newer vLLM. The single VLLM_METAL_VERSION pin + autobumper keep this tracked and reproducible.

Assisted-by: Claude:opus-4.8 [Claude Code]