Skip to content
BranchesTags

Repository files navigation

Slopsmith Demucs Server

A lightweight GPU-accelerated service providing AI source separation, lyrics alignment, and per-syllable pitch extraction for Slopsmith. Designed to run on a desktop with a CUDA GPU while Slopsmith runs on a NAS or Docker host.

Docker Build

Features

Source Separation (POST /separate)

Splits audio into individual stems using Demucs:

  • Default model: htdemucs_ft (4-stem fine-tuned: drums, bass, vocals, other)
  • Other models selectable per-request: htdemucs_6s (6-stem incl. guitar/piano), mdx_extra (lighter)
  • bs_roformer_sw — BS-Roformer-SW (6-stem: vocals/drums/bass/guitar/piano/other), via audio-separator. Higher SDR than Demucs (notably bass/guitar) with far less cross-stem bleed; checkpoint (~700 MB) lazy-downloads on first use to <cache>/_roformer-models/. Stems returned as lossless FLAC.
  • File upload or URL input
  • Per-stem caching, keyed by audio and model (avoids re-processing; same song under two models caches separately)
  • WebSocket progress updates

Lyrics Alignment (POST /align)

Forced alignment of plain text lyrics against an audio file using WhisperX — Whisper transcription plus a wav2vec2 forced aligner for tighter sub-word timestamps:

  • Line, word, syllable, or phoneme granularity
  • Phoneme/character-level CTC alignment via wav2vec2 (per-language model)
  • Syllable splitting layered on word output via pyphen hyphenation (CJK character support)
  • Automatic language detection (or manual language hint)
  • Used by the Lyrics Sync plugin and the Lyrics Karaoke plugin

Per-syllable Pitch Extraction (POST /pitch)

Estimates one MIDI note per syllable from a vocals stem using CREPE via torchcrepe. Powers the karaoke pitch chart in the Lyrics Karaoke plugin:

  • CREPE neural pitch tracker — order-of-magnitude fewer octave errors than pYIN
  • Confidence-weighted mode-of-semitone aggregation per syllable
  • Song-wide range narrowing (clamps each syllable to ±12 semitones around the median)
  • Octave-error correction against the song-wide median
  • Neighbour-borrowed pitch for tokens CREPE can't lock (so whispered phrases still get bars)

Setup

Requirements

  • Python 3.10+
  • CUDA-capable GPU (recommended) or CPU fallback
  • FFmpeg (apt install ffmpeg / brew install ffmpeg)

Install (Native)

  git clone https://github.com/got-feedback/feedBack-demucs-server.git
cd feedback-demucs-server
python -m venv .venv
source .venv/bin/activate

# Step 1: Install main dependencies (fastapi, whisperx, torchcrepe, etc.)
# whisperx pins torch~=2.8.0 + torchaudio~=2.8.0
pip install -r requirements.txt

# Step 2: Install demucs SEPARATELY (torchaudio version conflict workaround)
# demucs requires torchaudio<2.1, which conflicts with whisperx.
# Installing with --no-deps bypasses the bad pin.
# dora-search is demucs's logging lib (imported as `import dora`).
pip install demucs --no-deps
pip install einops julius lameenc openunmix pyyaml tqdm dora-search

⚠️ Why two install steps? demucs (PyPI 4.0.1) pins torchaudio<2.1 while whisperx needs torchaudio~=2.8.0. These are incompatible. Installing demucs with --no-deps avoids the conflict. Demucs works fine with modern torchaudio — only the save_audio function had issues, and that's patched in run_demucs.py to use soundfile instead.

Run

python server.py --port 7865

Options:

Flag Default Description
--port 7865 Port to listen on
--host 0.0.0.0 Host to bind to
--model htdemucs_ft Demucs model (htdemucs_ft, htdemucs_6s, mdx_extra)
--device auto Force cpu or cuda
--api-key — API key for authentication
--skip-warmup — Skip startup model-weight prefetch

Environment variables override CLI defaults: SLOPSMITH_DEMUCS_MODEL, SLOPSMITH_DEMUCS_DEVICE, SLOPSMITH_API_KEY.

First-start model weight download

On first start the server pre-downloads model weights (~1.5 GB for all three endpoints: htdemucs_ft, Whisper medium, CREPE full, English wav2vec2). Subsequent restarts use cached weights.

The download runs in a background thread, so /health is queryable immediately. Each library prints its own tqdm progress bar.

/health reports per-model status:

{
  "status": "ok",
  "warmup": {
    "demucs": "ready",
    "whisperx": "downloading",
    "crepe": "pending",
    "whisperx_aligners": { "en": "ready" }
  }
}

States: pending → downloading → ready | failed: <reason> | skipped | evicted.

Pass --skip-warmup for environments without internet access.

Run as a systemd service

  1. Copy and edit the service file:
 cp feedback-demucs.service ~/.config/systemd/user/
 # Edit ~/.config/systemd/user/feedback-demucs.service
# Set User, ExecStart paths to match your setup
nano ~/.config/systemd/user/feedback-demucs.service
  1. Enable and start:
systemctl --user daemon-reload
systemctl --user enable feedback-demucs
 systemctl --user start feedback-demucs
  1. Monitor:
journalctl --user -u feedback-demucs --follow

Docker

Build

docker build -t slopsmith-demucs-server .

Run (CPU)

docker run -p 7865:7865 slopsmith-demucs-server

Run (GPU)

Requires nvidia-container-toolkit:

docker run --gpus all -p 7865:7865 slopsmith-demucs-server

Docker Compose

# Pull from GHCR and run (CPU)
docker compose up -d

# GPU mode: uncomment runtime: nvidia + NVIDIA_* env vars in compose file
docker compose up -d

Persistent model cache

Model weights are stored in /app/cache inside the container. The compose file maps this to a persistent volume so weights survive restarts:

docker compose down    # cache preserved
docker compose down -v # cache deleted (if using named volume)

To use a custom host path instead of a named volume (e.g. for Portainer or to save space on a specific drive), replace the volume in docker-compose.yml:

volumes:
  - /home/AI/feedback-demucs-cache:/app/cache

Then copy the existing cache to the new location:

# Find old volume path
docker volume inspect feedback-demucs-server_demucs-cache
 # Copy to new location
 sudo cp -a /var/lib/docker/volumes/feedback-demucs-server_demucs-cache/_data/. /home/AI/feedback-demucs-cache/

Cache environment variables (all redirect to /app/cache to prevent container root disk exhaustion):

Variable Purpose
SLOPSMITH_DEMUCS_CACHE Server cache root
HF_HOME HuggingFace model cache
TORCH_HOME PyTorch hub cache
HUGGINGFACE_HUB_CACHE HuggingFace hub downloads

Auto-update

The container can automatically check for repository updates and restart. Disabled by default (safe for Portainer/deployments without .git access).

To enable:

  1. Uncomment the .git bind mount in docker-compose.yml
  2. Set AUTO_UPDATE=true in environment
  3. Redeploy

How it works:

  1. A background daemon runs inside the container
  2. Every UPDATE_CHECK_INTERVAL seconds (default: 3600 = 1 hour), it checks if the current time matches UPDATE_TIME (default: 04:00)
  3. At the configured time, it runs git fetch origin and compares HEAD with @{upstream}
  4. If changes are detected, it pulls the new code, reinstalls dependencies, and gracefully restarts the server

Configuration via environment variables:

Variable Default Description
AUTO_UPDATE false Enable/disable auto-update
UPDATE_TIME 04:00 Time of day to check (HH:MM, 24h)
UPDATE_CHECK_INTERVAL 3600 Seconds between time checks (3600 = 1 hour)
SKIP_WARMUP false Skip model weight download on startup
SLOPSMITH_DEMUCS_MODEL — Override default Demucs model
SLOPSMITH_API_KEY — API authentication key
CACHE_TTL 24h Cache cleanup TTL (1h, 12h, 24h, or NEVER to disable auto-cleanup)
MODEL_IDLE_TIMEOUT 300 Model idle timeout in seconds (s, m, h suffixes supported; NEVER to disable). Unloads WhisperX/CREPE after inactivity to free GPU memory.

Disable auto-update (default — safe for Portainer):

docker run -e AUTO_UPDATE=false -p 7865:7865 slopsmith-demucs-server

Cache cleanup

The server automatically deletes old stem cache directories to prevent disk growth. A background thread runs every 10 minutes, checks each stem cache directory under SLOPSMITH_DEMUCS_CACHE, and removes directories older than CACHE_TTL.

Model weight caches (torch/, huggingface/, locale/) are never deleted — only the stem output cache is cleaned.

Variable Default Description
CACHE_TTL 24h Maximum age of cache entries (1h, 12h, 24h, or NEVER to disable)
MODEL_IDLE_TIMEOUT 300 Model idle timeout (s/m/h suffixes; NEVER to disable)

Disable auto-cleanup:

docker run -e CACHE_TTL=NEVER -p 7865:7865 slopsmith-demucs-server

Set custom TTL (e.g. 12 hours):

docker run -e CACHE_TTL=12h -p 7865:7865 slopsmith-demucs-server

Model idle timeout

The server automatically unloads idle ML models to free GPU memory. A background thread checks every 60 seconds and unloads any model that hasn't been used longer than MODEL_IDLE_TIMEOUT.

  • WhisperX ASR model — unloaded as a whole
  • WhisperX aligners — evicted individually by language (oldest first)
  • CREPE pitch model — unloaded as a whole

On unload, torch.cuda.empty_cache() is called to release GPU memory. Models are lazily re-loaded on the next request.

Variable Default Description
MODEL_IDLE_TIMEOUT 300 Idle timeout (s/m/h suffixes; NEVER to disable)

Disable model idle timeout:

docker run -e MODEL_IDLE_TIMEOUT=NEVER -p 7865:7865 slopsmith-demucs-server

GitHub Container Registry (CI)

The CI workflow (.github/workflows/docker-build.yml) automatically builds the Docker image, pushes it to GHCR, generates an SBOM, and runs a grype vulnerability scan on every push to main.

To enable on your fork:

  1. Go to your fork on GitHub → Actions tab
  2. Click "I understand my workflows, go ahead and enable them"
  3. Push to main — the CI builds and scans automatically

Pull the latest image:

docker pull ghcr.io/YOUR_GITHUB_USER/slopsmith-demucs-server:latest

Or from the upstream repo (once PR is merged):

docker pull ghcr.io/got-feedback/feedBack-demucs-server:latest

Build directly from git (no clone needed):

# From upstream main
docker build -t slopsmith-demucs-server https://github.com/got-feedback/feedBack-demucs-server.git#main

# From your fork
docker build -t slopsmith-demucs-server https://github.com/YOUR_USER/slopsmith-demucs-server.git#main

# Run it
docker run --gpus all -p 7865:7865 slopsmith-demucs-server

Run via Docker Compose with git build:

services:
  slopsmith-demucs:
    build: https://github.com/got-feedback/feedBack-demucs-server.git#main
    ports:
      - "7865:7865"

API

GET /health

Returns server status, model, GPU availability, cache directory, and per-model warmup state (see First-start model weight download).

POST /separate

Separate audio into stems.

Parameter Type Description
file Upload Audio file
stems Query Comma-separated stem names (default: drums,bass,vocals,other)
model Query Override model (optional)

POST /align

Forced-align lyrics against audio using WhisperX (faster-whisper transcription + wav2vec2 forced aligner).

Parameter Type Description
file Form (file) Audio file (vocals stem)
text Form Plain text lyrics
language Form ISO 639-1/2 language code hint, e.g. en, es, pt (optional, auto-detected). Must be 2–8 lowercase letters; subtags like en-US are not supported.
granularity Form line (default), word, syllable, or phoneme

Granularity behaviour:

  • line — segment-level boundaries.
  • word — wav2vec2-aligned word timestamps. The first entry in each line carries new_line: true.
  • syllable — word output split via pyphen; carries new_line on the first syllable of each line.
  • phoneme — character-level CTC token timestamps from the aligner. Each entry carries phoneme: true. With wav2vec2 character models these are letter-aligned; with phoneme-trained models they're true phonemes.

Returns: {"segments": [...], "language": "en"} where each segment is {start, end, text, ...}.

POST /pitch

Per-syllable pitch extraction using CREPE.

Parameter Type Description
file Form (file) Vocals stem (any format librosa can read)
lyrics Form JSON array of {"t": float, "d": float} — token start / duration in seconds

Returns: {"notes": [{"t": 12.34, "d": 0.5, "midi": 64}, ...]}. Tokens for which no pitch could be estimated (even after neighbour-borrow) are omitted.

GET /download/{job_id}/{stem}

Download a separated stem by job ID.

GET /jobs / GET /jobs/{job_id}

List or inspect separation jobs.

WS /ws/jobs/{job_id}

WebSocket for real-time separation progress updates.

Configure in Slopsmith

Set the Demucs Server URL to http://<your-server-ip>:7865 in Slopsmith settings.

About

Demucs source separation service for Slopsmith Studio

Resources

Readme
Activity

Stars

3 stars

Watchers

0 watching

Forks

5 forks
Report repository

Releases

Packages

Used by

Contributors

Languages