A curated list of self-hosted AI tools, LLM inference engines, workflow platforms, and VPS providers optimized for AI workloads. Run AI on your own infrastructure — for privacy, cost control, and full customization.
- LLM Inference & Local Hosting
- AI Workflow & Automation
- Vector Databases & RAG
- AI Observability & Ops
- VPS Providers for AI Workloads
- GPU Cloud & Rentals
- Hosting Guides & Resources
- Contributing
Run large language models on your own hardware.
| Tool | Description | License | GitHub |
|---|---|---|---|
| Ollama | The easiest way to run LLMs locally — one command to pull and serve Llama, Mistral, Gemma, and more | MIT | GitHub |
| LM Studio | Desktop app for discovering and running local LLMs — great GUI, OpenAI-compatible server | Proprietary (free) | — |
| LocalAI | Drop-in OpenAI API replacement that runs models locally — supports LLaMA, Whisper, Stable Diffusion | MIT | GitHub |
| llama.cpp | Pure C++ LLM inference — runs on CPU with quantization, the engine behind Ollama and LM Studio | MIT | GitHub |
| vLLM | High-throughput LLM serving with PagedAttention — production-grade GPU inference | Apache-2.0 | GitHub |
| Jan | Open-source desktop AI assistant — local models, no cloud, no data collection | AGPL-3.0 | GitHub |
| GPT4All | Cross-platform local AI — runs on consumer hardware, supports dozens of models | MIT | GitHub |
| AnythingLLM | Full-stack local AI platform — document Q&A, agents, multi-user, and API | MIT | GitHub |
| Open WebUI | Feature-rich browser UI for Ollama and OpenAI-compatible APIs | MIT | GitHub |
| text-generation-webui | The original local LLM UI — supports virtually every model format and inference backend | AGPL-3.0 | GitHub |
| LibreChat | Multi-provider AI chat UI — supports OpenAI, Anthropic, Ollama, and local models | MIT | GitHub |
Build AI agents, pipelines, and production apps without vendor lock-in.
| Tool | Description | License | GitHub |
|---|---|---|---|
| Dify | LLM app development platform — RAG pipelines, AI agents, and production API in one self-hosted install | Apache-2.0 | GitHub |
| n8n | Workflow automation with 400+ integrations and native AI nodes — self-host or cloud | Fair-code | GitHub |
| Flowise | Drag-and-drop LLM flow builder built on LangChain — no code required | Apache-2.0 | GitHub |
| Langflow | Visual LangChain and LlamaIndex pipeline builder — export to Python | MIT | GitHub |
| PrivateGPT | Ask questions about your documents — 100% local, no data leaves your machine | Apache-2.0 | GitHub |
| Cheshire Cat | AI agent framework — custom memory, plugins, and REST API out of the box | GPL-3.0 | GitHub |
| Lobe Chat | Extensible AI chat platform — plugin system, knowledge base, multi-model | Apache-2.0 | GitHub |
| BerriAI LiteLLM | Unified API proxy for 100+ LLM providers — OpenAI-compatible, with load balancing and spend tracking | MIT | GitHub |
Store and retrieve embeddings for AI apps.
| Tool | Description | License | GitHub |
|---|---|---|---|
| Qdrant | High-performance vector database in Rust — best-in-class filtering and payload storage | Apache-2.0 | GitHub |
| Chroma | Simple embedding database — easy to self-host, popular in RAG tutorials | Apache-2.0 | GitHub |
| Weaviate | Full-featured vector database with multi-modal support and built-in modules | BSD-3-Clause | GitHub |
| Milvus | Enterprise-grade vector database — scales to billions of vectors | Apache-2.0 | GitHub |
| pgvector | Vector similarity search inside Postgres — no new database to manage | PostgreSQL | GitHub |
Monitor, trace, and improve your AI applications.
| Tool | Description | License | GitHub |
|---|---|---|---|
| Langfuse | Open-source LLM observability — traces, evals, prompt management, and cost tracking | MIT | GitHub |
| Phoenix (Arize) | AI observability for LLMs and ML models — experiment tracking and hallucination detection | Elastic-2.0 | GitHub |
| PromptLayer | Prompt versioning, A/B testing, and request logging for LLM apps | Proprietary | — |
Where to host your self-hosted AI stack — tested for reliability and LLM performance.
| Provider | Best For | Starting Price | Notes |
|---|---|---|---|
| Hetzner | Best price/performance in Europe — CPU inference with Ollama | From €3.79/mo | CX22 (2 vCPU, 4GB) handles small models; CAX (ARM) is very efficient |
| DigitalOcean | Developer-friendly setup — great docs and one-click apps | From $4/mo | GPU Droplets now available for $2.99/hr (H100) |
| Vultr | GPU cloud at competitive pricing — good US and EU coverage | From $2.50/mo | Cloud GPU instances from $0.20/hr |
| Linode (Akamai) | Reliable long-term hosting — strong support and SLA | From $5/mo | Stable choice for production Dify/n8n deployments |
| Contabo | Highest RAM for the price — good for RAM-hungry LLMs | From €5.99/mo | High RAM VPS (24GB) is very popular for Ollama hosting |
| OVHcloud | European data sovereignty — GDPR-compliant by default | From €3.59/mo | Bare metal GPU servers available |
Rent GPUs by the hour for training, fine-tuning, or high-throughput inference.
| Provider | Best For | Pricing | Notes |
|---|---|---|---|
| Lambda Labs | Dedicated GPU instances — H100, A100, A10 | From $0.50/hr | On-demand and reserved instances |
| Vast.ai | Cheapest GPU rentals — peer-to-peer GPU marketplace | From $0.10/hr | Variable reliability; best for experiments |
| RunPod | Balance of price and reliability — serverless GPU | From $0.19/hr | Serverless option is excellent for burst workloads |
| Paperspace | ML-focused cloud — good Jupyter and notebook experience | From $0.07/hr | Gradient notebooks included |
| CoreWeave | Enterprise GPU cloud — H100 clusters at scale | Contact for pricing | Kubernetes-native, used by major AI labs |
Deploy your self-hosted AI stack — step-by-step guides and community resources.
| Resource | Description |
|---|---|
| Dify Hosting Guide | Complete deployment guides for Dify on Hetzner, DigitalOcean, and other VPS providers — from Docker setup to reverse proxy configuration |
| Best VPS for AI Workloads | Benchmark comparisons of VPS providers for LLM inference — token/second benchmarks, cost-per-query analysis |
| Self-Host VPS Guide | Step-by-step tutorials for self-hosting Dify, Coolify, n8n, Nextcloud, and more on your own VPS — from initial setup to production deployment |
| r/LocalLLaMA | Active community for local LLM discussion — model recommendations, hardware advice, and benchmarks |
| Ollama Discord | Official Ollama community — help with setup, model selection, and API integration |
| Awesome LLM | Comprehensive list of LLM papers, tools, and frameworks |
Contributions welcome! Please read the contributing guidelines first.
- Tools must be actively maintained (commit within last 6 months for OSS)
- For VPS providers: include an honest note about limitations, not just marketing copy
- One PR per tool addition
- GPU pricing changes fast — include a note if pricing was last verified
To the extent possible under law, the contributors have waived all copyright and related rights to this work.