Skip to content

feat: pick pairing from the ASR's segmentation instead of hardcoding 2 - #21

Merged
ijuinryukichi merged 1 commit into
mainfrom
feat/auto-pairing
Jul 25, 2026
Merged

ijuinryukichi merged 1 commit into
mainfrom
feat/auto-pairing

Conversation

@ijuinryukichi

Copy link
Copy Markdown
Owner

pairing — how many lyric lines make one matching unit — was fixed at 2. That
was never a property of songs. It is a property of faster-whisper medium,
which merges about two sung lines into one segment on Japanese rap. Nothing in
the code or docs said so.

large-v3 segments far finer on the same track: 41 → 64 segments, mean segment
4.73 s → 2.78 s. A two-line unit then straddles a boundary.

model segments pairing mean ≤0.5 s worst lines placed
medium 41 2 0.50 s 14/20 3.58 s 74/76
large-v3 64 1 0.31 s 15/20 0.76 s 49/76
large-v3 64 2 1.13 s 10/20 3.70 s 68/76

So upgrading the model alone makes things worse — and with the pairing it
deserves, large-v3 removes the 3.6 s outlier that five separate matcher
changes failed to fix earlier today (#13). Worst case 3.58 s → 0.76 s, mean down
39 %, outliers 1 → 0. The leverage was in the input, not the matcher.

The estimator

Lines-per-segment is exactly what pairing means, so it is also the estimate.
No audio, no ground truth, no new dependency — just the two counts we already
have.

lines / segments auto vs. fixed 2
過ぎたるもの, medium 76 / 41 2 identical
過ぎたるもの, large-v3 76 / 64 1 0.50 s → 0.31 s
黒砂, medium 80 / 46 2 identical
黒砂, large-v3 80 / 46 2 identical
過ぎたるもの, full mix (ASR collapsed) 76 / 11 3 (capped) no worse

Capped at 3 because past that the ASR has stopped producing line-like segments,
and the answer is to separate the vocal — which the CLI already tells you. Left
uncapped that case estimates 7 and is markedly worse.

What it costs, stated in the README

At pairing 1 there are only 64 segments for 76 lines, so a quarter cannot
place
: 49/76 against medium's 74/76. Both large-v3 rows match the same 18 of
the 20 measured lines, and the extra lines pairing 2 places are ones the ground
truth cannot check — while that configuration also produces six measurable
outliers against pairing 1's none. medium stays the default model.

What it does not do

It does not rescue the repeated-hook track, which was predicted before the run:
large-v3 trails medium there at every pairing, because four identical hook
lines carry no information about which repetition they are, whatever transcribes
them. Better ASR fixes outliers made of garbled text, not outliers made of
identical text.

Default changed before the first PyPI release, deliberately. 7 tests added, 57
pass.

🤖 Generated with Claude Code

The old default of 2 was not a property of songs, it was a property of
faster-whisper medium, which merges about two sung lines into one segment on
Japanese rap. Nothing said so, including this README.

large-v3 splits far finer on the same four-minute track — 41 segments to 64,
mean segment 4.73s to 2.78s — so a two-line unit straddles a boundary and the
match degrades. Measured: large-v3 at the old default is worse than medium
(mean 0.50s -> 1.13s, six outliers against one). With pairing following the
model it is better than either, and removes the 3.6s outlier this project
failed to fix five other ways today: mean 0.31s, worst 0.76s, no outliers.

Upgrading the model alone is therefore a trap, and the fix belongs in the
default. Lines-per-segment is exactly what pairing means, so it is also the
estimator. On all five measured configurations auto matches or beats the fixed
2, including the degenerate full-mix case where the ASR collapses whole verses —
capped at 3 there, since the answer to that input is to separate the vocal,
which the CLI already says.

The README states the cost as well: at pairing 1 there are only 64 segments for
76 lines, so that configuration places 49 of them against medium's 74. More
accurate on every measured line, much less complete. medium stays the default
model for that reason.

Also does not rescue the repeated-hook track, as predicted before running it:
large-v3 trails medium there at every pairing, because four identical hook lines
carry no information about which repetition they are, whatever transcribes them.

7 tests added, 57 pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ijuinryukichi
ijuinryukichi merged commit 6d03dc4 into main Jul 25, 2026
3 checks passed
@ijuinryukichi
ijuinryukichi deleted the feat/auto-pairing branch July 25, 2026 07:31
ijuinryukichi added a commit that referenced this pull request Jul 25, 2026
#21)

The old default of 2 was not a property of songs, it was a property of
faster-whisper medium, which merges about two sung lines into one segment on
Japanese rap. Nothing said so, including this README.

large-v3 splits far finer on the same four-minute track — 41 segments to 64,
mean segment 4.73s to 2.78s — so a two-line unit straddles a boundary and the
match degrades. Measured: large-v3 at the old default is worse than medium
(mean 0.50s -> 1.13s, six outliers against one). With pairing following the
model it is better than either, and removes the 3.6s outlier this project
failed to fix five other ways today: mean 0.31s, worst 0.76s, no outliers.

Upgrading the model alone is therefore a trap, and the fix belongs in the
default. Lines-per-segment is exactly what pairing means, so it is also the
estimator. On all five measured configurations auto matches or beats the fixed
2, including the degenerate full-mix case where the ASR collapses whole verses —
capped at 3 there, since the answer to that input is to separate the vocal,
which the CLI already says.

The README states the cost as well: at pairing 1 there are only 64 segments for
76 lines, so that configuration places 49 of them against medium's 74. More
accurate on every measured line, much less complete. medium stays the default
model for that reason.

Also does not rescue the repeated-hook track, as predicted before running it:
large-v3 trails medium there at every pairing, because four identical hook lines
carry no information about which repetition they are, whatever transcribes them.

7 tests added, 57 pass.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant