music_generation.md

Music And Sound Generation

Use --task gen for models that generate music, sound effects, or audio from text and optional audio conditioning. These models are not TTS models: text chunking for speech TTS does not apply unless a model explicitly documents a long-output mode.

Common CLI shape:

audiocpp_cli --task gen --family <family> --model <model-dir> --backend cuda ...

ACE-Step

ACE-Step generates and edits music from prompts, lyrics, and optional source audio. See ace_step.md for the full route manual.

Field	Value
Family	ace_step
Model directory	models/Ace-Step1.5
Task	gen
Routes	text2music, complete, lego, extract, cover, cover-nofsq, repaint
Main inputs	Prompt text, optional lyrics, optional source audio depending on route
Languages	19+ lyric languages supported by the model

audiocpp_cli --task gen --family ace_step --model models/Ace-Step1.5 --backend cuda --task-route text2music --text "cinematic synth pop with clear vocals" --lyrics "We rise with the morning light" --duration-seconds 60 --out song.wav

Option	Values	Default	Meaning
--task-route	text2music, complete, lego, extract, cover, cover-nofsq, repaint	text2music	ACE-Step operation.
--text	text	required	Music prompt or edit instruction.
--lyrics	text	empty string	Vocal lyrics.
--audio	WAV path	not set	Source audio for edit/extract/cover routes.
--duration-seconds	float, -1 for auto	-1	Target duration.
--num-inference-steps	integer	8	Diffusion denoising steps.
--guidance-scale	float	1.0	Diffusion guidance scale.
`--session-option ace_step.mem_saver=true	false`	bool	false

Stable Audio

Stable Audio generates music or sound effects from text. It can also use source audio for init-audio or inpainting workflows. See stable_audio.md for the full Stable Audio manual.

Field	Value
Family	stable_audio
Model directories	models/stable-audio-3-small-music, models/stable-audio-3-small-sfx, models/stable-audio-3-medium
Task	gen
Modes	Text-to-music, text-to-SFX, init-audio, inpainting
Main inputs	Prompt text; optional source audio for audio-conditioned modes
Languages	en prompts

Text to music:

audiocpp_cli --task gen --family stable_audio --model models/stable-audio-3-small-music --backend cuda --text "uplifting house music with bright synths and festival drums" --duration-seconds 30 --out music.wav

Inpainting:

audiocpp_cli --task gen --family stable_audio --model models/stable-audio-3-small-music --backend cuda --audio input.wav --text "replace the masked sections with a tight snare fill" --duration-seconds 10 --request-option audio_input_kind=inpaint_audio --request-option inpaint_mask_start_seconds=2.5 --request-option inpaint_mask_end_seconds=3.5 --out inpaint.wav

Option	Values	Default	Meaning
--text	prompt text	required	Music or sound-effect prompt.
--audio	WAV path	not set	Source audio for init-audio or inpainting.
--duration-seconds	seconds[,seconds...]	120	Target duration per prompt. Use one value for all prompts, or one comma-separated value per Stable Audio batch_size item.
--num-inference-steps	integer	8	RF diffusion steps.
--guidance-scale	float	1.0	Classifier-free guidance scale.
--request-option audio_input_kind=<kind>	init_audio, inpaint_audio	init_audio when --audio is provided	How the source audio is used.
--request-option init_noise_level=<float>	0..1	1.0	Strength for init-audio conditioning.
--request-option inpaint_mask_start_seconds=<list>	comma-separated seconds	not set	Inpaint region start times.
--request-option inpaint_mask_end_seconds=<list>	comma-separated seconds	not set	Inpaint region end times.
`--session-option stable_audio.mem_saver=true	false`	bool	false

HeartMuLa

HeartMuLa generates music from lyrics and tags. The upstream reference treats tags as style/control text: it lowercases them and wraps them with special tag tokens internally. The audio.cpp integration follows that behavior and does not use speaker-reference audio.

Field	Value
Family	heartmula
Model directory	models/HeartMuLa
Task	gen
Modes	Lyrics/tags to music; optional infinite mode for longer outputs
Main inputs	--lyrics plus comma-separated tags
Languages	Multilingual lyrics supported by the model
Reference audio	Not consumed by this integration
Tag format	Free-form comma-separated descriptors such as genre, mood, instruments, tempo, and vocals

audiocpp_cli --task gen --family heartmula --model models/HeartMuLa --backend cuda --text "a bright pop chorus with drums" --lyrics "We rise with the morning light" --request-option tags=pop,bright,drums --out song.wav

Long output mode:

audiocpp_cli --task gen --family heartmula --model models/HeartMuLa --backend cuda --text "long cinematic pop song" --lyrics "Verse one begins with a quiet street. The chorus opens wide." --request-option tags=pop,cinematic,drums,vocal --duration-seconds 300 --request-option infinite_mode=true --out song_long.wav

Option	Values	Default	Meaning
--text	text	required	Music prompt or short description.
--lyrics	text	empty string	Lyrics for generated music.
--request-option tags=<text>	comma-separated text	required	Music tags; the model path wraps them as tag tokens internally.
--duration-seconds	seconds	120	Maximum generated duration.
--temperature	float	1.0	Music-token sampling temperature.
--top-k	integer	50	Music-token top-k sampling limit.
--guidance-scale	float	1.5	MuLa classifier-free guidance scale.
--num-inference-steps	integer	10	Codec flow solver steps.
--request-option codec_duration=<seconds>	seconds	29.76	Codec detokenization chunk duration.
--request-option codec_guidance_scale=<float>	float	1.25	Codec classifier-free guidance scale.
`--request-option infinite_mode=true	false`	bool	false
--text-chunk-size	chars	4096	Text chunk size for infinite mode.
--request-option infinite_chunk_audio_length_ms=<n>	milliseconds	240000	Per-chunk audio cap for infinite mode.
--seed	integer	1234	Generation seed.
`--session-option heartmula.mem_saver=true	false`	bool	false

For backend weight-type controls, use audiocpp_cli --inspect --model <model-dir> --family <family>.