[Enhancement] Unified ASR timing source and reusable Qwen forced aligner

[diodiogod]

Context

The Unified ASR Transcribe node currently exposes timestamp intent, but forced-aligner ownership is engine-specific. Granite carries an asr_use_forced_aligner engine option and loads Qwen3 Forced Aligner inside its adapter. Audio.cpp ASR families preserve native timing when upstream returns it, but most families cannot obtain word timing through the Suite even though we already have the audio and transcript needed for post-alignment.

This should be revisited after the initial Audio.cpp integration is stable. The goal is not an Audio.cpp-only workaround; it is a reusable Suite-level timing architecture for every ASR engine.

Proposed direction

Move the user-facing alignment policy to Unified ASR Transcribe and treat forced alignment as an optional post-transcription stage.

Suggested timing-source choices:

• None
• Auto: prefer native timing, otherwise use Qwen forced alignment when compatible
• Native only
• Qwen forced aligner

Engine adapters/capability metadata should declare what they provide rather than owning the user policy:

• native word timestamps
• native segment timestamps
• compatible with Suite forced alignment
• no timing support

Extract the reusable Qwen alignment code currently embedded in the Granite adapter into a shared ASR alignment component. Migrate Granite without breaking saved workflows, then allow Audio.cpp and future ASR adapters to use the same component.

Important behavior

• Native timing always takes priority in Auto mode.
• Forced alignment is opt-in and must never silently download/load for timestamp-disabled runs.
• Translation mode should not use forced alignment.
• Preserve native diarization segments and attach aligned words to them rather than replacing speaker turns.
• Prefer aligning each exact Suite audio chunk against that chunk's transcript, then apply timeline offsets and overlap deduplication.
• Use explicit or trustworthy detected language metadata when available.
• Warn and return text without fabricated timing for unsupported/ambiguous languages.
• Account for the Audio.cpp server and Qwen aligner occupying GPU memory simultaneously. Support clear fallback/error behavior, CPU alignment where practical, and correct lifecycle/Clear VRAM handling.
• External Audio.cpp servers cannot be unloaded by the Suite, so memory assumptions must remain explicit.

Qwen aligner language constraints

Currently supported by the Suite integration:

Chinese, English, Cantonese, French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish.

Auto-language routing needs careful handling, especially Chinese versus Japanese; do not blindly reuse a script heuristic that treats all CJK text as Japanese.

UI/capability follow-up

• Unified ASR owns the timing-source selector and its explanatory tooltip.
• Engine nodes should not each grow another enable/disable toggle unless they need engine-specific runtime configuration.
• Capability panels should distinguish Native word timestamps, Native segment timestamps, Optional Qwen aligner, and No timestamps.
• Advanced aligner device/runtime options may remain engine-independent configuration if needed.

Acceptance criteria

• One shared forced-alignment implementation is used by Granite and at least one non-native-timestamp Audio.cpp ASR family.
• Unified ASR can select None, Auto, Native only, or Qwen forced alignment.
• Native word/segment timing and diarization remain intact.
• Chunk offsets and overlap merging produce monotonic word timings.
• Cache identity includes timing source and aligner configuration.
• Unsupported languages and translation degrade honestly to text-only results with warnings.
• Existing Granite workflows remain compatible.
• Tests cover native timing, forced timing, unsupported languages, chunk offsets, diarization attachment, cache identity, and VRAM/lifecycle failure paths.

Future exploration: reusable speaker diarization

Explore a Suite-level diarization postprocessor for ASR engines that do not provide native speaker attribution. This should be designed together with the forced-alignment work rather than implemented specifically inside Audio.cpp.

The most promising default to evaluate is the local pyannote.audio Community-1 pipeline because it provides exclusive speaker diarization intended for reconciliation with transcription timestamps and supports offline loading after the initial gated download.

Potential combined pipeline:

1. Run ASR to obtain the transcript.
2. Preserve native word timestamps or create them with the optional Qwen forced aligner.
3. Run diarization over the full recording.
4. Assign timed words to speaker turns by overlap.
5. Group consecutive words into speaker-attributed ASR segments.

Design considerations:

• Unified ASR should own the diarization policy: Off, Auto (native then Suite fallback), Native only, or a selected Suite diarizer.
• Native engine diarization must always take priority in Auto mode.
• Diarization without word timing can produce speaker turns, but cannot reliably assign transcript words. Retranscribing each speaker turn should not be the default because it loses context, handles overlap poorly, and increases runtime.
• Preserve overlapping-speaker information where available; use exclusive diarization for transcript reconciliation without discarding the richer raw output.
• Pyannote Community-1 requires accepting model conditions and an HF token for the first download. The Suite should install it directly into managed model storage and load it locally rather than depend on the Hugging Face cache.
• Account for the additional dependency/runtime, CPU/GPU selection, memory lifecycle, Clear VRAM behavior, and coexistence with native Audio.cpp GPU allocation.
• Diarization yields anonymous speaker labels. Mapping speakers to named characters using reference voices would be a separate speaker-enrollment/identification feature.
• Evaluate licensing and deployment constraints before selecting alternatives. NVIDIA Sortformer is worth comparing, but current official models have speaker-count, language, dependency, and non-commercial-license limitations that may make them unsuitable as the Suite default.

This is future exploration, not part of the initial Audio.cpp stabilization milestone.