Commit fcba3a4
committed
fix(audio8_tts): Falcon-H1 0.1B logits parity and long-run stability
Three root causes made the Falcon-H1 slow-AR path diverge from the
transformers reference (first-frame argmax 3620 vs 2732, ASR round-trip
said the wrong word) and blew the SSM state up on long sequences:
1. conv1d kernel was flipped at load time, but ggml ssm_conv and the HF
FalconH1 decode (nn.Conv1d prefill and the cached torch.sum path
alike) use the same cross-correlation orientation with window[0] the
oldest frame. Feed the GGUF kernel unflipped.
2. sx (conv window) and k_r/v (fresh K/V) were read back on the host
without ggml_set_output, so gallocr reused their buffers and the
conv/SSM/KV states were fed garbage every step. Pin exactly those
three tensors per layer.
3. The host KV cache used the current sequence length as the per-head
stride while appending only the new token, so from the second token
on the append overwrote the previous head blocks and every head past
the first read corrupted context. The resulting residual-stream
garbage - not the recurrent scan itself - drove the SSM state to
1e15..1e18 around token 180. append_falcon_kv_token now re-lays the
cached tokens into the new stride before appending.
Verified against an f32 transformers 4.57.6 reference forced onto the
recurrent path: per-layer conv/SSM/KV states match within bf16 rounding
over the whole prompt, first-frame argmax = 2732, synthesized speech
round-trips through ASR verbatim, and a 605-position generation stays
bounded (max state ~1.1e3, no NaN) on both CPU and Metal backends.
Adds a regression test for the KV cache re-stride that fails against
the previous append logic at the second token.1 parent 71325ea commit fcba3a4
5 files changed
Lines changed: 268 additions & 476 deletions
File tree
- src
- community_models/audio8_tts
- framework/modules
- tests/unittests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2089 | 2089 | | |
2090 | 2090 | | |
2091 | 2091 | | |
| 2092 | + | |
| 2093 | + | |
| 2094 | + | |
| 2095 | + | |
| 2096 | + | |
| 2097 | + | |
| 2098 | + | |
| 2099 | + | |
| 2100 | + | |
| 2101 | + | |
| 2102 | + | |
2092 | 2103 | | |
2093 | 2104 | | |
2094 | 2105 | | |
| |||
0 commit comments