Skip to content

Stem File Format.md

Latest commit

 

History

History
476 lines (386 loc) · 21 KB

Stem File Format.md

File metadata and controls

476 lines (386 loc) · 21 KB
Raw

VirtualDJ Stem File Format

This page documents the observed .vdjstems sidecar file format used by VirtualDJ's prepared stems. It is about the container/file contract, not VDJScript stem controls.

Source labels:

  • Official: VirtualDJ public stems help describes the five stem components and prepared-stems workflow.
  • Local observation: files generated by VirtualDJ and inspected locally.
  • Local script: local helper scripts and generated files that VirtualDJ accepts.
  • Local test: tool behavior (ffmpeg/MP4Box/AtomicParsley) verified locally.
  • Inference: reasoned from evidence but not directly verified.
  • External: Matroska container references and stem-file drafts outside VirtualDJ.

Short Version

Local observation: VirtualDJ-prepared .vdjstems sidecars are open Matroska audio files, not encrypted/private blobs. The observed VirtualDJ sidecar contract is:

Stream index Matroska track ID Audio role title
0 1 vocal
1 2 hihat
2 3 bass
3 4 instruments
4 5 kick

All observed VirtualDJ-generated sidecars use five stereo AAC streams. The stream title tags are part of the role contract; do not treat the file as just "any five audio streams in Matroska."

Local test: two further contract fields are load-bearing for externally generated sidecars: the streams' sample rate must equal the original file's (see Sample Rate Contract), and the Matroska Writing application field must carry VirtualDJ's <version>.stems2 signature (see Recreating A VirtualDJ-Like Sidecar) — VirtualDJ silently ignores a sidecar without it.

Corpus Checked

Local observation: On 2026-06-04, the local folder ~/Library/Mobile Documents/com~apple~CloudDocs/Music/Tracks contained 1,505 files ending in .vdjstems.

The fast signature pass found:

Count Container/codec shape
1,476 Matroska, five stereo AAC streams at 44.1 kHz, titled vocal, hihat, bass, instruments, kick
23 Matroska, five stereo AAC streams at 48 kHz, same titles/order
5 Matroska, five stereo AAC streams at 96 kHz, same titles/order
1 MPEG-4, six stereo ALAC streams, mixed track first, not VirtualDJ-generated

The 1,504 Matroska files had a Writing application value in the form VirtualDJ ... .stems2, with examples including:

  • VirtualDJ 2026.9336.stems2
  • VirtualDJ 2026.9004.stems2
  • VirtualDJ 2026.8978.stems2
  • VirtualDJ 2025.8800.stems2
  • VirtualDJ i.stems2

The one MPEG-4/ALAC outlier had Writing application: Lavf62.3.100 and should be treated as a custom/experimental stem container, not as the VirtualDJ sidecar format described here.

Local observation: A 2026-07-18 recount of the same folder found 1,521 .vdjstems files, consistent with the corpus continuing to grow through normal VirtualDJ prepared-stems use. Additional VirtualDJ-generated sidecars with the same Matroska shape exist under ~/Library/Application Support/VirtualDJ/Sampler/ (including *.vdjsample.vdjstems files), so the format also covers sampler content.

Container Details

Local observation: A representative VirtualDJ-generated sidecar reports:

Field Observed value
Extension .vdjstems
File type Matroska audio (audio/x-matroska)
EBML doc type matroska
Matroska version Version 4
Timecode scale 1 ms
Writing application VirtualDJ <version>.stems2
Muxing/writing library Lavf...
Audio codec AAC LC (A_AAC / A_AAC-2)
Channels 2, stereo
Default audio track none
Forced audio track none

ffprobe exposes the file as format_name=matroska,webm; generic tools may warn that .vdjstems is not a normal Matroska extension. That warning is expected.

Local observation: Detailed samples had no chapters and no attachments exposed by ffprobe. Stream DURATION tags can vary by a final AAC frame; for example, kick was sometimes a few milliseconds longer than the other streams.

Metadata Contract

Local observation and Local script: VirtualDJ's role detection depends on metadata and order. The important stream tags are the lowercase titles:

0 title=vocal
1 title=hihat
2 title=bass
3 title=instruments
4 title=kick

Practical rules:

  • Use exactly five sidecar streams.
  • Do not include a mixed track stream in the sidecar.
  • Use the exact role order vocal, hihat, bass, instruments, kick.
  • Write the role names as stream titles, not only filenames.
  • Clear default dispositions on every audio stream.
  • Keep every stream stereo and start-aligned at timestamp 0.
  • Match the original file's sample rate (see next section) — this is a playback contract, not a preference.
  • AAC is what VirtualDJ itself writes, but it is not required: Local test (2026-07-18, tone probes) confirmed sidecars with FLAC (16- and 24-bit), ALAC, and PCM streams all load and play correctly in VirtualDJ 2026. This enables lossless prepared stems and does not appear to be documented anywhere else. FLAC is the natural choice (Matroska-native, smaller); the decoder is evidently plain libavformat.

Sample Rate Contract

Local observation (2026-07-18): The sidecar's sample rate must equal the original file's sample rate. Pairing every non-44.1 kHz sidecar in the corpus with its original shows an exact match in all cases:

Sidecar rate Original rate Count observed
48000 48000 all 48 kHz sidecars checked (20+)
96000 96000 all 96 kHz sidecars checked (5)
44100 44100 spot-checked sample of 40
44100 192000 1 (VirtualDJ capped it — AAC-LC tops out at 96 kHz)

Local test (2026-07-18, VirtualDJ 2026): an externally generated sidecar whose streams were 44.1 kHz against a higher-rate original loaded and was used by VirtualDJ, but muting stems played the sidecar audio pitch-shifted upward and stuttering — consistent with VirtualDJ reading sidecar frames on the original file's clock without resampling. The same failure is expected in the opposite direction (rate too high → slow/low). Externally generated packers must probe the original's rate and encode the sidecar to match, capping above 96 kHz at 44.1 kHz per the observed VirtualDJ behavior.

This also resolves part of a prior unknown: VirtualDJ 2026 does accept and play externally generated Matroska sidecars (the failure above was rate mismatch, not rejection).

Inference: The stream titles are safer to treat as required even if a particular build also appears to infer roles by order. The local scripts that work write both the order and the titles, and VirtualDJ's own sidecars include both.

Sidecar Naming

Local observation: Prepared-stem sidecars are named by appending .vdjstems to the source track path:

/path/to/original.ext.vdjstems

Examples:

Track Name.flac.vdjstems
Track Name.m4a.vdjstems
Track Name.mp3.vdjstems
Track Name.dj2.flac.vdjstems

VirtualDJ settings can also affect where prepared stems are stored; see stemsSavedFolder in Configuration Options.

Inspecting A File

Use ffprobe for a quick compatibility check:

ffprobe -v error -select_streams a \
  -show_entries stream=index,codec_name,sample_rate,channels:stream_tags=title \
  -of compact=p=0:nk=1 \
  "/path/to/track.ext.vdjstems"

Expected VirtualDJ sidecar shape:

0|aac|44100|2|vocal
1|aac|44100|2|hihat
2|aac|44100|2|bass
3|aac|44100|2|instruments
4|aac|44100|2|kick

The sample rate may be 44100, 48000, or 96000 in the local corpus, and always equals the original file's rate (see Sample Rate Contract above).

Use mediainfo when you need Matroska-level fields:

mediainfo "/path/to/track.ext.vdjstems"

Look for:

Format              : Matroska
Writing application : VirtualDJ 2026.9336.stems2
Writing library     : Lavf62.13.102
Title               : vocal
Title               : hihat
Title               : bass
Title               : instruments
Title               : kick

Recreating A VirtualDJ-Like Sidecar

Local script: The minimal packer is ordinary ffmpeg: map five prepared WAV stems, assign stream titles, encode AAC, and force a Matroska output even though the extension is .vdjstems.

#!/usr/bin/env zsh
set -euo pipefail

dir="${1:?usage: pack-vdjstems-sidecar <stem-dir> [output.vdjstems]}"
out="${2:-$dir/$(basename "$dir").vdjstems}"

for name in vocal hihat bass instruments kick; do
  [[ -f "$dir/$name.wav" ]] || {
    print -u2 "missing $dir/$name.wav"
    exit 2
  }
done

ffmpeg -y \
  -i "$dir/vocal.wav" \
  -i "$dir/hihat.wav" \
  -i "$dir/bass.wav" \
  -i "$dir/instruments.wav" \
  -i "$dir/kick.wav" \
  -map 0:a -map 1:a -map 2:a -map 3:a -map 4:a \
  -metadata:s:a:0 title="vocal" \
  -metadata:s:a:1 title="hihat" \
  -metadata:s:a:2 title="bass" \
  -metadata:s:a:3 title="instruments" \
  -metadata:s:a:4 title="kick" \
  -disposition:a:0 0 \
  -disposition:a:1 0 \
  -disposition:a:2 0 \
  -disposition:a:3 0 \
  -disposition:a:4 0 \
  -c:a aac -b:a 320k -ar "$orig_rate" \
  -f matroska \
  "$out"

$orig_rate must be the original track's sample rate (probe it with ffprobe), capped at 44100 above 96 kHz — see Sample Rate Contract.

The Writing application field must then be stamped — this is required. Local test (2026-07-18, VirtualDJ 2026, tone-probe A/B): two sidecars that were byte-identical except for the Matroska Writing application field were placed next to identical originals. The one stamped VirtualDJ 2026.9336.stems2 was used by VirtualDJ (stem pads played the sidecar streams); the one left as Lavf… was silently ignored (VirtualDJ treated the track as having no prepared stems). ffmpeg hardcodes Lavf in this field, so stamp it afterwards:

mkvpropedit "$out" --edit info \
  --set "writing-application=VirtualDJ 2026.9336.stems2"

An earlier revision of this page claimed the field did not need to be faked; that claim was wrong and is superseded by the A/B above. The likely purpose of the <version>.stems2 marker is engine-version gating — VirtualDJ can tell which stems engine produced a cache and regenerate or ignore stale ones.

The Extension Is A Container Contract

Local observation: On 2026-07-18, a six-stream MPEG-4 file named with the .vdjstems extension was not accepted by VirtualDJ as a prepared-stems file (it was either rejected or misinterpreted as a sidecar). The corpus outlier above (MPEG-4/ALAC named .vdjstems) matches the same mistake: it was produced by an earlier local script, not by VirtualDJ.

Practical rule: .vdjstems means "Matroska sidecar, five stems" to VirtualDJ. MPEG-4 stem containers must be named .m4a and used as standalone files (next section). Never emit an MP4 with a .vdjstems extension.

Standalone Six-Track M4A

Local test (2026-07-18, VirtualDJ 2026): VirtualDJ detects a multi-track MPEG-4/M4A audio file in the library as a stems-capable track. This is a community-reverse-engineered format (VirtualDJ 2023-era output) and it still works in VirtualDJ 2026 — an acceptance matrix built from one stem set confirmed the recipe below loads with working stem pads. The working recipe:

Field Working value
Extension .m4a (see extension rule above)
Container MPEG-4, brand isom (minor 512), compatible mp42 + mp41
Storage flat (MP4Box -flat; non-interleaved)
Streams exactly 6, order: mixed track, vocal, hihat, bass, instruments, kick
Track names per-track udta name atoms (MP4Box -udta N:type=name:str=...)
Disposition stream 0 (mixed track) default; all stems non-default
Codec AAC-LC 320k, ALAC, or FLAC, 44.1 kHz stereo. ALAC must be 16-bit (s16p) on every stream — Local test 2026: an s32p master plays stuttery with pitch artifacts; the identical all-s16p file plays perfectly. FLAC requires the plain mp4 muxer (ffmpeg -f mp4 -strict experimental; the ipod/.m4a muxer refuses it)
iTunes tags tool=VirtualDJ <version> (itags)

Inference: The per-track udta name atoms are the safer role signal in this container: known-working scripts wrote roles as udta names (with stream title/handler metadata absent or inconsistent), unlike the Matroska sidecar where the stream title tags carry the roles.

The mixed track stream is the full mix, so the file plays as an ordinary library track in any player. When packing from separated stems without an original mix, sum the unique stems (do not count a drums stem twice if kick and hihat were duplicated from it — see the −6 dB rule below).

Local test (2026-07-18, VirtualDJ 2026): the master must be the plain unity-gain sum of the five stem streams. When stems are active, VirtualDJ sums the stem tracks at unity gain; a master attenuated relative to that sum (e.g. ffmpeg amix default 1/n scaling) makes stem playback jump ~12 dB louder than master playback the moment any stem is muted. The legacy packer's amix=inputs=5,volume=5 idiom exists precisely to undo amix's scaling; amix=inputs=N:normalize=0 is the modern equivalent.

Standalone Acceptance Matrix (2026-07-18)

Local test (VirtualDJ 2026, macOS): five candidate containers built from the same stem set and loaded in VirtualDJ:

Candidate Result
Legacy recipe .m4a (AAC, itags incl. tempo/key, -brand isom on the ffmpeg mux) Works — loads, stem pads function
Same family, minor deltas (no ffmpeg-level isom brand, different itags) Loads from library only; drag-and-drop onto a deck fails; stems work
MP4-era .vdjstems (six-stream ALAC MP4, -inter 500, mp42 brands) Rejected — unloads the current deck track and loads nothing
Matroska 6-track .mka (mixed + 5 titled stems) Works via drag-and-drop; invisible in the browser (.mka not in VirtualDJ's extension list); renamed .mkv it appears but is not detected as a stems file
Matroska 5-track .mka/.mkv (exact sidecar shape as a file) Not treated as stems — plays the first stream (vocal) as a normal track

Local observation: There are two stem role vocabularies in local artifacts. Besides the five-stem set on this page, an October 2025 local experiment (~/Music/Stems/test-stems/TestTone.vdjstems, five-stream ALAC MP4) used the NI-style 4-stem naming as udta track names: All, Vocal, Instrument, Bass, Drums — the same set local NI-conversion scripts target. Whether the 4-stem M4A family is still recognized by VirtualDJ 2026 is under test (tests/Stems/make-diagnostic-stems.zsh builds a DIAG-4stem.m4a probe).

Notes:

  • VirtualDJ's accepted-extensions option (observed default) does not include .mka: mp3,wav,cda,wma,asf,ogg,m4a,aac,aif,aiff,flac,mpc,ape,weba,opus, vdj,vdjcache,vdjedit,vdjsample,mp4,ogm,ogv,avi,mpg,mpeg,wmv,vob,mov,divx, m4v,mkv,flv,webm,vdjcachev,apng.
  • The .mka-drag-drop vs .mkv-browser asymmetry for the 6-track Matroska candidate is unexplained; treat Matroska standalones as unreliable and use the M4A recipe.
  • The browser-only vs drag-and-drop difference between the two M4A variants is not yet isolated (candidate deltas: ffmpeg-level -brand isom, presence of INITIALKEY/tempo itags); packers should replicate the fully working variant exactly.

Local test (2026-07-18, tone-probe round — master = white noise, each stem a distinct sine, so routing is audible; harness: tests/Stems/make-diagnostic-stems.zsh):

Probe Result
5-stem standalone M4A (working recipe, noise master) Confirmed — master plays the mixed track stream; engaging stems switches to the stem streams; each pad maps to its titled stream
4-stem M4A (All, Vocal, Instrument, Bass, Drums udta names) Dead in VirtualDJ 2026 — loads into the deck but plays nothing and detects no stems (an October 2025 local artifact shows this family was previously in local use)
Matroska sidecar, writing-application=VirtualDJ 2026.9336.stems2 Works — stem pads play the sidecar's tone streams; master plays the original (noise)
Same sidecar, writing-application=Lavf… Ignored — original plays, no stems detected; proves the stamp is the acceptance gate
Sidecar with FLAC streams (stamped) Works — lossless sidecars are possible; previously undocumented
Sidecar with ALAC streams (stamped) Works
Standalone M4A in ALAC (stems s16p, master s32p) Broken — loads but playback stutters with pitch artifacts, the same audible signature as a sample-rate mismatch
Standalone M4A in ALAC, all streams s16p Works — isolates the failure above to the 32-bit master: VirtualDJ's standalone ALAC playback is 16-bit only
Sidecar with PCM (pcm_s16le) streams Works
Sidecar with 24-bit FLAC streams Works — the 16-bit limit applies only to standalone MP4 ALAC, not to the Matroska sidecar
Sidecar stamped VirtualDJ 2025.8800.stems2 (old version) Works, and VirtualDJ did not regenerate or modify the file — the stamp check is version-tolerant
Sidecar with capitalized titles (Vocal, Hihat, …) Works — role matching is case-insensitive (stream order was also canonical, so order-vs-title precedence remains unseparated)
Standalone MP4 with six FLAC streams Works — lossless FLAC standalones are possible. Note the mux: ffmpeg's .m4a/ipod muxer refuses FLAC; force -f mp4 (-strict experimental), then MP4Box processes it normally

Tagging Notes (MPEG-4 Standalone)

Local test (2026-07-18, MP4Box 2.x / ffmpeg 8.1):

  • tempo=<bpm> via MP4Box -itags is written as the iTunes tmpo atom and reads back correctly.
  • INITIALKEY=<key> via MP4Box -itags is silently dropped (MP4Box prints Unrecognize tag).
  • A key survives as an iTunes freeform atom ----:com.apple.iTunes:initialkey, e.g. written with AtomicParsley file --rDNSatom "Am" name=initialkey domain=com.apple.iTunes -W; ffprobe then reports TAG:initialkey=Am. This is the atom family other DJ taggers (Mixed In Key, Serato) use.
  • Arbitrary -metadata KEY=value on ffmpeg's MP4 muxer is dropped for unknown keys; it cannot write INITIALKEY either.

Whether VirtualDJ itself reads the freeform initialkey atom from a standalone stems file is untested; VirtualDJ analyzes key on import regardless.

Deriving Kick And Hihat From 4-Stem Sources

Local script: 4-stem separators (Demucs-style vocals/drums/bass/other) have no kick/hihat split. The working convention when duplicating the drums stem into both roles is to attenuate both copies by −6.0206 dB (a factor of exactly ½), so kick + hihat sums back to the original drums level when both stems play. When synthesizing a standalone mixed track from such stems, sum the drums source once, not both duplicates.

Local Tooling

Local script: The maintained toolchain lives in /Users/nom/config/config/zsh/bin (July 2026 overhaul):

  • vdjstems make song.flac — separate + pack in one command. Default output is the Matroska sidecar song.flac.vdjstems; --standalone builds the six-track M4A instead.
  • vdjstems-split — MLX-accelerated separation (BS-RoFormer vocals + htdemucs_ft bands + MDX23C-DrumSep kick/hihat; --sw runs the BS-Roformer-SW six-stem model in a single pass).
  • vdjstems-pack --sidecar ORIGINAL <stemdir> / vdjstems-pack --standalone — the packer implementing both formats on this page.
  • vdjstems-info, vdjstems-extract, vdjstems-from-nistem — inspection, stream extraction, and NI .stem.mp4 conversion.

Superseded experiments are parked in /Users/nom/config/config/zsh/_bin_quarantine.

Known Unknowns

The July 2026 tone-probe rounds closed every practical question (see the acceptance matrix). What remains is minor:

  • Role order vs title precedence in sidecars: every probe used canonical order AND correct titles, so which one wins on a mismatch is unobserved.
  • Whether some future engine version bump makes VirtualDJ regenerate old-stamped sidecars during library maintenance (the 2025-stamp probe was not modified during normal load/play).
  • Behavior on other platforms (all tests: macOS, VirtualDJ 2026).
  • Whether VirtualDJ reads the freeform initialkey atom from standalone files.
  • What exactly distinguishes the fully-working M4A variant from the library-only variant (see the acceptance matrix) — -brand isom at the ffmpeg layer and itag differences are the candidates.
  • Why the 6-track Matroska engaged stems when drag-and-dropped as .mka but not when browsed as .mkv.

Sources

  • VirtualDJ stems help - Official
  • Matroska stem files Internet-Draft - External
  • Local .vdjstems files under ~/Library/Mobile Documents/com~apple~CloudDocs/Music/Tracks and ~/Library/Application Support/VirtualDJ/Sampler - Local observation
  • Local vdjstems-* toolchain under /Users/nom/config/config/zsh/bin (superseded experiments in config/zsh/_bin_quarantine) - Local script
  • MP4Box itags / AtomicParsley freeform-atom behavior verified locally 2026-07-18 - Local test