A self-hosted platform for real-time conversations with AI avatars.
A browser video call with a generated avatar: speak, interrupt, share your camera, and read live captions. AvatarForever runs on your GPU; OpenAI handles conversation and vision, ElevenLabs generates speech, and LiveKit Cloud carries audio/video.
Ten reference portraits and stock-voice assignments are included. Model weights, private character assets and API keys stay on your server.
Watch a conversation with live captions, interruption and an animated avatar.
chert-demo.mp4
The public demo supports one caller at a time. If it is busy, join the queue; your position updates automatically. You have 30 seconds to start when your turn arrives.
| Component | Required |
|---|---|
| GPU host | Ubuntu x86_64, H100 80 GB, at least 150 GB free storage |
| Software | Git, Python 3.11 with venv, Node.js 22.12+, FFmpeg |
| Services | LiveKit Cloud, OpenAI and ElevenLabs credentials |
| Model access | Hugging Face access to the Gemma text encoder |
Run on your GPU host:
git clone https://github.com/cherthq/realtime-research.git
cd realtime-research
python3.11 -m venv .venv-realtime
.venv-realtime/bin/python -m pip install -r realtime/requirements.txt -c realtime/constraints-linux-py311.txt
cd realtime/web
npm ci
npm run build
cd ../..- Prepare the model. Run the GPU setup script and download the pinned AvatarForever and Gemma weights.
- Add credentials. Copy realtime/.env.example to
realtime/.envand configure the providers and GPU worker. Keep this file private. - Start the application. Prepare the host, start the server and prewarm the model, then open the web app through an SSH tunnel or HTTPS.
The setup guide includes the commands and configuration for each step:
Complete setup → · Deploy on a public domain →
Provider keys stay on the server. The browser receives a short-lived room token and a key scoped to its own call.
| Directory | Contents |
|---|---|
| realtime/ | Web app, voice agent, streaming worker and tests |
| realtime/demo-avatars/ | Ten reference portraits and stock voice assignments |
| data_fetching/ | 4:3 talking-video preparation and review pipeline |
| configs/ · scripts/ | Pinned model sources and baseline tools |
| infra/ | GPU setup and deployment helpers |
For model comparisons, follow the baseline workflow and zero-shot evaluation protocol. Dataset preparation, evaluation and training are separate from running a call.
- Capacity: the current deployment serves one call per GPU, with a ten-minute call limit.
- Visual quality: long calls can drift; idle mouth motion and lip-sync remain imperfect.
- Playback: output uses 4:3 framing with a 24 FPS publication cap. Actual performance depends on the GPU and configuration; lower resolution does not resolve visual drift.
See validation results and tested scope before deploying.
Upstream models and reference images retain their own terms. Bundled voices are stock voice selections, not clones of the people in the portraits.