Hi all, looking for a solution here or community experience/insight
System Details:
Hardware: Apple MacBook Pro M1 Max with 64GB RAM
Python Version: 3.12
Core Libraries: mlx, csm-mlx, sounddevice, webrtcvad
Model: csm-1b with custom LoRA adapters
Problem Description:
When running a real-time, multi-threaded, speech-to-speech pipeline, the audio output from the csm-mlx stream_generate function is "choppy" or "stuttery." This issue persists even when the TTS model is isolated and run on a dedicated, high-performance machine (M1 Max 64GB), indicating the problem is not a simple CPU/Memory resource bottleneck but is likely inherent to the consistency of the model's streaming output. The application is stable, but the audio playback is not smooth enough for a real-time conversational experience.
Model Optimization:
Swapped to smaller STT models (whisper-tiny.en) to reduce computational load.
Enabled TTS model quantization (nn.quantize) as recommended in the documentation to increase inference speed.
Hardware Upgrades: Migrated the application from an M2 Max 32GB machine to an M1 Max 64GB machine to eliminate memory pressure as a bottleneck. Confirmed memory usage (approx. 22GB) was well within the new hardware's limits.
Distributed Architecture: To rule out resource competition, the application was split into a client-server model across two powerful MacBooks. The choppiness persisted, proving the issue was not caused by other processes interfering with the TTS model.
Parallelism Enhancement: Identified the global mlx_lock as a potential artificial bottleneck. The lock was removed to allow the OS and MLX framework to handle parallel GPU/ANE scheduling. This provided a slight improvement but did not solve the core issue.
Jitter Buffer Implementation: Identified the root cause as the "bursty" or inconsistent nature of the stream_generate output. The tts_and_playback_thread was re-architected with a producer-consumer pattern using a queue.Queue to act as a jitter buffer, decoupling the inconsistent model generation from the real-time audio playback.
Final Result: Despite all optimizations, including the jitter buffer and running on dedicated high-end hardware, the audio remains noticeably choppy, suggesting the issue lies at the core of the model's streaming performance on the MLX framework.
Hi all, looking for a solution here or community experience/insight
System Details:
Hardware: Apple MacBook Pro M1 Max with 64GB RAM
Python Version: 3.12
Core Libraries: mlx, csm-mlx, sounddevice, webrtcvad
Model: csm-1b with custom LoRA adapters
Problem Description:
When running a real-time, multi-threaded, speech-to-speech pipeline, the audio output from the csm-mlx stream_generate function is "choppy" or "stuttery." This issue persists even when the TTS model is isolated and run on a dedicated, high-performance machine (M1 Max 64GB), indicating the problem is not a simple CPU/Memory resource bottleneck but is likely inherent to the consistency of the model's streaming output. The application is stable, but the audio playback is not smooth enough for a real-time conversational experience.
Model Optimization:
Swapped to smaller STT models (whisper-tiny.en) to reduce computational load.
Enabled TTS model quantization (nn.quantize) as recommended in the documentation to increase inference speed.
Hardware Upgrades: Migrated the application from an M2 Max 32GB machine to an M1 Max 64GB machine to eliminate memory pressure as a bottleneck. Confirmed memory usage (approx. 22GB) was well within the new hardware's limits.
Distributed Architecture: To rule out resource competition, the application was split into a client-server model across two powerful MacBooks. The choppiness persisted, proving the issue was not caused by other processes interfering with the TTS model.
Parallelism Enhancement: Identified the global mlx_lock as a potential artificial bottleneck. The lock was removed to allow the OS and MLX framework to handle parallel GPU/ANE scheduling. This provided a slight improvement but did not solve the core issue.
Jitter Buffer Implementation: Identified the root cause as the "bursty" or inconsistent nature of the stream_generate output. The tts_and_playback_thread was re-architected with a producer-consumer pattern using a queue.Queue to act as a jitter buffer, decoupling the inconsistent model generation from the real-time audio playback.
Final Result: Despite all optimizations, including the jitter buffer and running on dedicated high-end hardware, the audio remains noticeably choppy, suggesting the issue lies at the core of the model's streaming performance on the MLX framework.