Bug Report: VibeVoice Tokenizer Registration Conflict with Transformers 5.15.1
Summary
Model loading fails because the environment uses Transformers 5.15.1, which now includes a native VibeVoice tokenizer registration. The bundled legacy VibeVoice package attempts to register the same tokenizer class again, causing a registration conflict before generation begins.
Environment
- Transformers: 5.15.1
- VibeVoice: Bundled legacy package
- Affected operation: Model loading / initialization
Description
The log indicates that the issue is an incompatibility between the newer Transformers version and the bundled legacy VibeVoice package.
Transformers 5.15.1 now provides native VibeVoice tokenizer registration. However, the bundled VibeVoice package also attempts to register its own tokenizer under the same name/class.
As a result, the tokenizer is registered twice, and model loading fails before the generation step is reached.
Expected Behavior
The model should load successfully and proceed to generation without duplicate tokenizer registration.
Actual Behavior
Model loading fails during tokenizer registration because the legacy VibeVoice package attempts to register a tokenizer that is already registered by Transformers 5.15.1.
Generation is therefore never reached.
Root Cause
The bundled legacy VibeVoice implementation is incompatible with the native VibeVoice tokenizer support introduced in Transformers 5.15.1.
Specifically:
- Transformers 5.15.1 registers VibeVoice natively.
- The bundled VibeVoice package performs its own registration.
- The same-named tokenizer class is registered twice.
- The duplicate registration causes model initialization to fail.
Suggested Fix
Update the bundled VibeVoice integration to detect and use the native Transformers VibeVoice tokenizer when available, or otherwise prevent the legacy package from performing a duplicate registration.
As a compatibility workaround, using a Transformers version without the native VibeVoice registration may also avoid the conflict.
Bug Report: VibeVoice Tokenizer Registration Conflict with Transformers 5.15.1
Summary
Model loading fails because the environment uses Transformers 5.15.1, which now includes a native VibeVoice tokenizer registration. The bundled legacy VibeVoice package attempts to register the same tokenizer class again, causing a registration conflict before generation begins.
Environment
Description
The log indicates that the issue is an incompatibility between the newer Transformers version and the bundled legacy VibeVoice package.
Transformers 5.15.1 now provides native VibeVoice tokenizer registration. However, the bundled VibeVoice package also attempts to register its own tokenizer under the same name/class.
As a result, the tokenizer is registered twice, and model loading fails before the generation step is reached.
Expected Behavior
The model should load successfully and proceed to generation without duplicate tokenizer registration.
Actual Behavior
Model loading fails during tokenizer registration because the legacy VibeVoice package attempts to register a tokenizer that is already registered by Transformers 5.15.1.
Generation is therefore never reached.
Root Cause
The bundled legacy VibeVoice implementation is incompatible with the native VibeVoice tokenizer support introduced in Transformers 5.15.1.
Specifically:
Suggested Fix
Update the bundled VibeVoice integration to detect and use the native Transformers VibeVoice tokenizer when available, or otherwise prevent the legacy package from performing a duplicate registration.
As a compatibility workaround, using a Transformers version without the native VibeVoice registration may also avoid the conflict.