Add standalone GGUF support for Hviske v5.3 and Citrinet-256 - #46
Conversation
|
@mirek190 you moved fast 👍 I pushed the package-spec refactor, along with the other pending changes, to main. Could you please sync with the latest main (66d233b), check the current package-spec loading flow, and update your code against that version? Package-spec loading means model file layout is described in model_specs/*.json, not hardcoded in each model. There is a new doc to track Package-spec refactor and GGUF testing https://github.com/0xShug0/audio.cpp/blob/main/docs/gguf.md. Please also update this table in the PR. Thanks again for your contribution! |
Define package specs for Hviske v5.3 and Citrinet-256 so the framework resolves native safetensors, standalone GGUF weights, and named config, tokenizer, and vocabulary resources. Decode incompatible quantized convolution reshapes to F32, add native/GGUF regression coverage, document Q8_0 conversion, and record the tested GGUF status.
a811151 to
d9273d0
Compare
|
Sure All conflicts resolved and updated.
Synced exactly with the main monster commit 66d233b |
|
@mirek190 Merged! Thanks a lot! |
) * Add standalone GGUF support for Hviske and Citrinet Define package specs for Hviske v5.3 and Citrinet-256 so the framework resolves native safetensors, standalone GGUF weights, and named config, tokenizer, and vocabulary resources. Decode incompatible quantized convolution reshapes to F32, add native/GGUF regression coverage, document Q8_0 conversion, and record the tested GGUF status. * Add native q8 reshape preservation test --------- Co-authored-by: 0xShug0 <231717474+0xShug0@users.noreply.github.com>
What changed
mainat66d233bmodel_specs/hviske_asr.jsonandmodel_specs/citrinet_asr.jsondocs/gguf.mdWhy
The shared GGUF core could package these checkpoints, but these families did not have package specs. Citrinet Q8_0 also exposed a GGML row-layout mismatch when a quantized matrix was reshaped into a convolution kernel whose final dimension was not block-sized.
This change lets a completed
model.ggufbe moved, renamed, and passed directly to the CLI or server without the original model directory.Validation
asr_standalone_gguf_test, CLI, and server) build successfully on66d233basr_standalone_gguf_testpasses through the package-spec resolverThe complete default CPU build on
66d233bis currently blocked by the existing MOSS codec parity targets, which still pass filesystem paths to constructors that now requireTensorSourceobjects. Those unrelated targets are not changed by this PR.