dlib_train

what is it?

scripts for WpfMusicPlayer to train model weights used in LRC detection.

specialized for CJK lyrics.

line_net can detect language type (8 types):

zh, latin, jp, kr, jyut, roma, onomatopoeia

song_net can detected language classification (14 types):

• 1-lang only: zh_only, latin_only, jp_only, kr_only, ru_only
• 2-lang combination: latin_zh_trans, jp_zh_trans, kr_zh_trans, ru_zh_trans, jp_roma, kr_roma, zh_jyut
• 3-lang combination: jp_zh_trans_roma, kr_zh_trans_roma

how to run?

Dependencies are restored from vcpkg.json. Place the Windows pnnx executable at:

tools/pnnx.exe

The tools directory, datasets, generated ONNX/NCNN models, intermediate files, and executable binaries are intentionally excluded from Git.

dataset format

Place the UTF-8 datasets in dlib_train/:

train.tsv
song_train.tsv

Despite the extension, both readers intentionally split each row at the first ASCII space. This historical behavior is part of the dataset contract and must not be changed to tab-separated parsing.

# train.tsv
[lang_type] [sentence]

# song_train.tsv
[song_sequence_type] [space-separated lang_type sequence]

usage

dlib_train.exe line-train
dlib_train.exe line-test
dlib_train.exe song-train
dlib_train.exe song-test

Training writes these local artifacts and immediately invokes pnnx:

lyric_lang_mlp.onnx
lyric_lang_mlp.ncnn.param
lyric_lang_mlp.ncnn.bin
lyric_lang_vocab.bin

song_structure_mlp.onnx
song_structure_mlp.ncnn.param
song_structure_mlp.ncnn.bin

The test commands use NCNN for neural-network inference. Line inference loads lyric_lang_vocab.bin through a dependency-free reader; it does not use dlib serialization.

The vocabulary uses a versioned binary format so that consumers do not need dlib or another object-serialization library. It starts with the 8-byte WMPVOCAB magic, followed by a little-endian 32-bit version and entry count, then a little-endian 64-bit bundle fingerprint. Entries are stored in feature-index order as a one-byte length plus 1-3 raw bytes. Raw bytes are required because the model operates on UTF-8 byte n-grams, including byte sequences that are not valid standalone UTF-8 strings.

The bundle fingerprint is FNV-1a 64 over the WMP-VOCAB-BUNDLE-V1 domain tag, the NCNN model fingerprint, entry count, and every encoded vocabulary entry. The model fingerprint covers both NCNN files, including domain/part labels and file sizes. This is a non-cryptographic integrity check that rejects accidentally mixed or damaged model/vocabulary artifacts; it is not an authenticity mechanism. The classifier also reads the first InnerProduct layer consuming in0 and rejects any feature vector or vocabulary whose size differs from the model input width.

For independent implementations, FNV-1a uses offset 14695981039346656037 and prime 1099511628211. The model hash input is WMP-NCNN-MODEL-V1, then for param and weights in that order: the label length as u64 LE, label bytes, file size as u64 LE, and file bytes. The bundle hash input is WMP-VOCAB-BUNDLE-V1, the model hash as u64 LE, the entry count as u32 LE, then every length-prefixed vocabulary entry in order.

Existing dlib model files can be migrated without retraining:

dlib_train.exe line-convert-legacy
dlib_train.exe song-convert-legacy

The legacy inputs are lyric_lang_mlp.dat and song_structure_mlp.dat in the dlib_train/ directory.

Licensed under the MIT license.

Feel free to use in any project if it's useful. compatible with most C++-based project.

Depends: dlib, onnx, ncnn, protobuf(managed by vcpkg. if you need to use CUDA acceleration of dlib, override features settings in vcpkg.json.)