diff --git a/vllm/README.md b/vllm/README.md index f07a6b5..2bbc5e6 100644 --- a/vllm/README.md +++ b/vllm/README.md @@ -5,7 +5,7 @@ Runs an OpenAI-compatible LLM on the Livepeer network and consumes it with the * | | | | ------------ | ------------------------------------------ | | App id | `vllm/qwen2.5-0.5b-instruct` | -| Runner mode | persistent (single-shot by nature) | +| Runner mode | single-shot | | Registration | static (orchestrator config + health poll) | | Transport | HTTP + SSE (OpenAI `/v1/chat/completions`) | | Port | 8000 (vLLM), 8080 (gateway) | @@ -40,6 +40,17 @@ curl http://localhost:8080/v1/chat/completions \ -d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[{"role":"user","content":"hi"}]}' ``` +### Call it directly (no gateway) + +As a **single-shot** runner with `"routing": "label"`, vLLM is reachable straight through the orchestrator's proxy (`/apps/