Skip to content

Add standalone GGUF support for Hviske v5.3 and Citrinet-256 - #46

Merged
0xShug0 merged 3 commits into
0xShug0:mainfrom
mirek190:agent/hviske-citrinet-gguf
Jul 14, 2026
Merged

Add standalone GGUF support for Hviske v5.3 and Citrinet-256#46
0xShug0 merged 3 commits into
0xShug0:mainfrom
mirek190:agent/hviske-citrinet-gguf

Conversation

@mirek190

@mirek190 mirek190 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What changed

  • sync the change onto main at 66d233b
  • add model_specs/hviske_asr.json and model_specs/citrinet_asr.json
  • describe native safetensors and standalone GGUF layouts, including named configuration, tokenizer, vocabulary, and tensor resources, in those specs
  • refactor both model loaders to consume the framework-resolved named resources instead of selecting model filenames in C++
  • accept direct, renamed GGUF files while retaining native safetensors support
  • decode quantized source matrices to F32 when a model reshapes them into an incompatible Conv1D layout
  • document Q8_0 conversion commands and update the package-spec/GGUF test matrix in docs/gguf.md
  • add regression coverage for native/GGUF package resolution and the quantized matrix-to-Conv1D reshape path

Why

The shared GGUF core could package these checkpoints, but these families did not have package specs. Citrinet Q8_0 also exposed a GGML row-layout mismatch when a quantized matrix was reshaped into a convolution kernel whose final dimension was not block-sized.

This change lets a completed model.gguf be moved, renamed, and passed directly to the CLI or server without the original model directory.

Validation

  • focused CPU targets (asr_standalone_gguf_test, CLI, and server) build successfully on 66d233b
  • asr_standalone_gguf_test passes through the package-spec resolver
  • CUDA CLI and server load both real Q8_0 files successfully through their package specs
  • CUDA CLI loads both native safetensors layouts successfully through their package specs
  • Hviske native and Q8_0 produced the same transcript on the test WAV
  • Citrinet native and Q8_0 produced the same transcript on the test WAV

The complete default CPU build on 66d233b is currently blocked by the existing MOSS codec parity targets, which still pass filesystem paths to constructors that now require TensorSource objects. Those unrelated targets are not changed by this PR.

@mirek190
mirek190 marked this pull request as ready for review July 14, 2026 00:01
@0xShug0

0xShug0 commented Jul 14, 2026

Copy link
Copy Markdown
Owner

@mirek190 you moved fast 👍 I pushed the package-spec refactor, along with the other pending changes, to main. Could you please sync with the latest main (66d233b), check the current package-spec loading flow, and update your code against that version?

Package-spec loading means model file layout is described in model_specs/*.json, not hardcoded in each model.
The framework resolves configs, tokenizers, sidecars, safetensors, or GGUF sources from that spec, then the model code just consumes named resources.

There is a new doc to track Package-spec refactor and GGUF testing https://github.com/0xShug0/audio.cpp/blob/main/docs/gguf.md. Please also update this table in the PR.

Thanks again for your contribution!

@0xShug0
0xShug0 changed the base branch from dev to main July 14, 2026 04:30
Define package specs for Hviske v5.3 and Citrinet-256 so the framework resolves native safetensors, standalone GGUF weights, and named config, tokenizer, and vocabulary resources. Decode incompatible quantized convolution reshapes to F32, add native/GGUF regression coverage, document Q8_0 conversion, and record the tested GGUF status.
@mirek190
mirek190 force-pushed the agent/hviske-citrinet-gguf branch from a811151 to d9273d0 Compare July 14, 2026 08:46
@mirek190

mirek190 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Sure

All conflicts resolved and updated.

  • Refactored both loaders to consume named package resources
  • Removed hardcoded model-layout selection from C++
  • Updated the docs/gguf.md testing table

Synced exactly with the main monster commit 66d233b

@0xShug0
0xShug0 merged commit d88bab3 into 0xShug0:main Jul 14, 2026
4 checks passed
@0xShug0

0xShug0 commented Jul 14, 2026

Copy link
Copy Markdown
Owner

@mirek190 Merged! Thanks a lot!

@mirek190
mirek190 deleted the agent/hviske-citrinet-gguf branch July 14, 2026 18:13
dleiferives pushed a commit to dleiferives/audio.cpp that referenced this pull request Jul 25, 2026
)

* Add standalone GGUF support for Hviske and Citrinet

Define package specs for Hviske v5.3 and Citrinet-256 so the framework resolves native safetensors, standalone GGUF weights, and named config, tokenizer, and vocabulary resources. Decode incompatible quantized convolution reshapes to F32, add native/GGUF regression coverage, document Q8_0 conversion, and record the tested GGUF status.

* Add native q8 reshape preservation test

---------

Co-authored-by: 0xShug0 <231717474+0xShug0@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants