You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Feel free to open up a Pull request (by clicking at the "Edit page" below) to get a page for your project made or if you see a error on one of the pages!
37
+
38
+
## Configuration Guides
39
+
40
+
This section provides step-by-step instructions for configuring specific software to work with LocalAI.
41
+
42
+
### OpenCode
43
+
44
+
[OpenCode](https://opencode.ai) is an AI-powered code editor that can be configured to use LocalAI as its backend provider.
45
+
46
+
#### Prerequisites
47
+
48
+
- LocalAI must be running and accessible (either locally or on a network)
49
+
- You need to know your LocalAI server's IP address/hostname and port (default is `8080`)
50
+
51
+
#### Configuration Steps
52
+
53
+
1.**Edit the OpenCode configuration file**
54
+
55
+
Open the OpenCode configuration file located at `~/.config/opencode/opencode.json` in your editor.
56
+
57
+
2.**Add LocalAI provider configuration**
58
+
59
+
Add the following configuration to your `opencode.json` file, replacing the values with your own:
60
+
61
+
```json
62
+
{
63
+
"$schema": "https://opencode.ai/config.json",
64
+
"provider": {
65
+
"LocalAI": {
66
+
"npm": "@ai-sdk/openai-compatible",
67
+
"name": "LocalAI (local)",
68
+
"options": {
69
+
"baseURL": "http://127.0.0.1:8080/v1"
70
+
},
71
+
"models": {
72
+
"Qwen3-Coder-30B-A3B-Instruct-i1-GGUF": {
73
+
"name": "Qwen3-Coder-30B-A3B-Instruct-i1-GGUF",
74
+
"limit": {
75
+
"context": 38000,
76
+
"output": 65536
77
+
}
78
+
},
79
+
"qwen_qwen3-30b-a3b-instruct-2507": {
80
+
"name": "qwen_qwen3-30b-a3b-instruct-2507",
81
+
"limit": {
82
+
"context": 38000,
83
+
"output": 65536
84
+
}
85
+
}
86
+
}
87
+
}
88
+
}
89
+
}
90
+
```
91
+
92
+
3.**Customize the configuration**
93
+
94
+
-**baseURL**: Replace `http://127.0.0.1:8080/v1` with your LocalAI server's address and port.
95
+
-**name**: Change "LocalAI (local)" to a descriptive name for your setup.
96
+
-**models**: Replace the model names with the actual model names available in your LocalAI instance. You can find available models by checking your LocalAI models directory or using the LocalAI API.
97
+
-**limit**: Adjust the `context` and `output` token limits based on your model's capabilities and available resources.
98
+
99
+
4.**Verify your models**
100
+
101
+
Ensure that the model names in the configuration match exactly with the model names configured in your LocalAI instance. You can verify available models by checking your LocalAI configuration or using the `/v1/models` endpoint.
102
+
103
+
5.**Restart OpenCode**
104
+
105
+
After saving the configuration file, restart OpenCode for the changes to take effect.
106
+
107
+
108
+
### GitHub Actions
109
+
110
+
You can use LocalAI in GitHub Actions workflows to perform AI-powered tasks like code review, diff summarization, or automated analysis. The [LocalAI GitHub Action](https://github.com/mudler/localai-github-action) makes it easy to spin up a LocalAI instance in your CI/CD pipeline.
111
+
112
+
#### Prerequisites
113
+
114
+
- A GitHub repository with Actions enabled
115
+
- A model name from [models.localai.io](https://models.localai.io) or a Hugging Face model reference
116
+
117
+
#### Example Workflow
118
+
119
+
This example workflow demonstrates how to use LocalAI to summarize pull request diffs and send notifications:
120
+
121
+
1.**Create a workflow file**
122
+
123
+
Create a new file in your repository at `.github/workflows/localai.yml`:
- **Model selection**: Replace `qwen_qwen3-4b-instruct-2507` with any model from [models.localai.io](https://models.localai.io). You can also use Hugging Face models by using the full huggingface model url`.
212
+
- **Trigger conditions**: Customize the `if` condition to control when the workflow runs. The example only runs when a PR is merged and has a specific label.
213
+
- **API endpoint**: The LocalAI container runs on `http://localhost:8080` by default. The action exposes the service on the standard port.
214
+
- **Custom prompts**: Modify the system message in the JSON payload to change what LocalAI is asked to do with the diff.
215
+
216
+
#### Use Cases
217
+
218
+
- **Code review automation**: Automatically review code changes and provide feedback
219
+
- **Diff summarization**: Generate human-readable summaries of code changes
220
+
- **Documentation generation**: Create documentation from code changes
221
+
- **Security scanning**: Analyze code for potential security issues
222
+
- **Test generation**: Generate test cases based on code changes
LocalAI supports realtime voice interactions , enabling voice assistant applications with real-time speech-to-speech communication. A complete example implementation is available in the [LocalAI-examples repository](https://github.com/mudler/LocalAI-examples/tree/main/realtime).
233
+
234
+
#### Overview
235
+
236
+
The realtime voice assistant example demonstrates how to build a voice assistant that:
237
+
- Captures audio input from the user in real-time
238
+
- Transcribes speech to text using LocalAI's transcription capabilities
239
+
- Processes the text with a language model
240
+
- Generates audio responses using text-to-speech
241
+
- Streams audio back to the user in real-time
242
+
243
+
#### Prerequisites
244
+
245
+
- A transcription model (e.g., Whisper) configured in LocalAI
The first time you start docker compose, it will take a while to download the available models. You can follow the model downloads in real-time:
265
+
266
+
```bash
267
+
docker logs -f realtime-localai-1
268
+
```
269
+
270
+
3. **Install host dependencies**
271
+
272
+
Install the required host dependencies (sudo is required):
273
+
274
+
```bash
275
+
sudo bash setup.sh
276
+
```
277
+
278
+
4. **Run the voice assistant**
279
+
280
+
Start the voice assistant application:
281
+
282
+
```bash
283
+
bash run.sh
284
+
```
285
+
286
+
#### Configuration Notes
287
+
288
+
- **CPU vs GPU**: The example is optimized for CPU usage. However, you can run LocalAI with a GPU for better performance and to use bigger/better models.
289
+
- **Python client**: The Python part downloads PyTorch for CPU, but this is fine as computation is offloaded to LocalAI. The Python client only runs Silero VAD (Voice Activity Detection), which is fast, and handles audio recording.
290
+
- **Thin client architecture**: The Python client is designed to run on thin clients such as Raspberry PIs, while LocalAI handles the heavier computational workload on a more powerful machine.
291
+
292
+
#### Key Features
293
+
294
+
- **Real-time processing**: Low-latency audio streaming for natural conversations
295
+
- **Voice Activity Detection (VAD)**: Automatic detection of when the user is speaking
296
+
- **Turn-taking**: Handles conversation flow with proper turn detection
297
+
- **OpenAI-compatible API**: Uses LocalAI's OpenAI-compatible realtime API endpoints
298
+
299
+
#### Use Cases
300
+
301
+
- **Voice assistants**: Build custom voice assistants for home automation or productivity
302
+
- **Accessibility tools**: Create voice interfaces for accessibility applications
303
+
- **Interactive applications**: Add voice interaction to games, educational software, or entertainment apps
304
+
- **Customer service**: Implement voice-based customer support systems
0 commit comments