Skip to content

Commit fcba3a4

Browse files
committed
fix(audio8_tts): Falcon-H1 0.1B logits parity and long-run stability
Three root causes made the Falcon-H1 slow-AR path diverge from the transformers reference (first-frame argmax 3620 vs 2732, ASR round-trip said the wrong word) and blew the SSM state up on long sequences: 1. conv1d kernel was flipped at load time, but ggml ssm_conv and the HF FalconH1 decode (nn.Conv1d prefill and the cached torch.sum path alike) use the same cross-correlation orientation with window[0] the oldest frame. Feed the GGUF kernel unflipped. 2. sx (conv window) and k_r/v (fresh K/V) were read back on the host without ggml_set_output, so gallocr reused their buffers and the conv/SSM/KV states were fed garbage every step. Pin exactly those three tensors per layer. 3. The host KV cache used the current sequence length as the per-head stride while appending only the new token, so from the second token on the append overwrote the previous head blocks and every head past the first read corrupted context. The resulting residual-stream garbage - not the recurrent scan itself - drove the SSM state to 1e15..1e18 around token 180. append_falcon_kv_token now re-lays the cached tokens into the new stride before appending. Verified against an f32 transformers 4.57.6 reference forced onto the recurrent path: per-layer conv/SSM/KV states match within bf16 rounding over the whole prompt, first-frame argmax = 2732, synthesized speech round-trips through ASR verbatim, and a 605-position generation stays bounded (max state ~1.1e3, no NaN) on both CPU and Metal backends. Adds a regression test for the KV cache re-stride that fails against the previous append logic at the second token.
1 parent 71325ea commit fcba3a4

5 files changed

Lines changed: 268 additions & 476 deletions

File tree

CMakeLists.txt

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2089,6 +2089,17 @@ if (ENGINE_BUILD_TESTS)
20892089
COMMAND safetensors_offsets_test
20902090
)
20912091

2092+
add_engine_unittest(audio8_tts_falcon_kv_cache_test tests/unittests/test_audio8_tts_falcon_kv_cache.cpp)
2093+
target_include_directories(audio8_tts_falcon_kv_cache_test PRIVATE
2094+
${CMAKE_CURRENT_SOURCE_DIR}/src/community_models/audio8_tts
2095+
${CMAKE_CURRENT_SOURCE_DIR}/tests/unittests
2096+
)
2097+
2098+
add_test(
2099+
NAME audio8_tts_falcon_kv_cache_test
2100+
COMMAND audio8_tts_falcon_kv_cache_test
2101+
)
2102+
20922103
add_engine_unittest(audio_chunking_test tests/unittests/test_audio_chunking.cpp)
20932104

20942105
add_test(

0 commit comments

Comments
 (0)