Repository navigation
docs: report the whole stable-ts comparison, not just their columns - #20
Merged
Merged
Conversation
Both tables showed matched / mean / <=1.0s — the three columns stable-ts wins or ties — and the prose concluded it was "slightly ahead". The median and the <=0.5s bucket, which lyric-align leads, were not shown. That is selective reporting, and it happened to run against ourselves, which does not make it more accurate. The full picture splits: they take mean (0.43 vs 0.50) and count (20 vs 19), we take median (0.36 vs 0.43) and <=0.5s (14/20 vs 12/20). Every one of those gaps is under the +-0.5s the ground truth was eyeballed to, so by this project's own rule about resolution, none of them is a measurement. The count is a contract rather than accuracy, and now says so: a forced aligner always emits, so stable-ts places all twenty — and the twentieth is the first line, 1.64s early, sitting in the intro, where lyric-align is +0.28s on the same line and declines the one the ASR collapsed on. Also dates the archive properly. Archived 2026-05-30 with a "paused development" note, but the version pip installs is 2.19.1 from 2025-08, which predates their own final alignment work (committed 2025-10, never released). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ijuinryukichi
added a commit
that referenced
this pull request
Jul 25, 2026
) Both tables showed matched / mean / <=1.0s — the three columns stable-ts wins or ties — and the prose concluded it was "slightly ahead". The median and the <=0.5s bucket, which lyric-align leads, were not shown. That is selective reporting, and it happened to run against ourselves, which does not make it more accurate. The full picture splits: they take mean (0.43 vs 0.50) and count (20 vs 19), we take median (0.36 vs 0.43) and <=0.5s (14/20 vs 12/20). Every one of those gaps is under the +-0.5s the ground truth was eyeballed to, so by this project's own rule about resolution, none of them is a measurement. The count is a contract rather than accuracy, and now says so: a forced aligner always emits, so stable-ts places all twenty — and the twentieth is the first line, 1.64s early, sitting in the intro, where lyric-align is +0.28s on the same line and declines the one the ASR collapsed on. Also dates the archive properly. Archived 2026-05-30 with a "paused development" note, but the version pip installs is 2.19.1 from 2025-08, which predates their own final alignment work (committed 2025-10, never released). Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both accuracy tables showed
matched / mean / ≤1.0 s— the three columnsstable-ts wins or ties — and the prose concluded its
align()was "slightlyahead". The median and the ≤0.5 s bucket, which lyric-align leads, were
not shown.
That is selective reporting. It ran against ourselves rather than for ourselves,
which does not make it any more accurate.
The columns split, and every gap is smaller than the ±0.5 s the ground truth
was eyeballed to — 0.07 s on both mean and median. By the rule this project
already applies elsewhere (an improvement below the ground truth's resolution is
not a measurement), neither tool is established as better.
The count is a contract, not accuracy, and the README now says so. A forced
aligner always emits, so stable-ts places all twenty — and the twentieth is that
first line, 1.64 s early, sitting in the intro.
lyric-alignis +0.28 s on thesame line and declines the one where the ASR collapsed.
Also dates the archive properly: archived 2026-05-30 with a literal "Add note
about paused development" commit, but the version
pipinstalls is 2.19.1 from2025-08, which predates their own final alignment work (committed 2025-10,
never released).
Every figure re-checked against
summary.jsonin the benchmark harness beforecommitting. 50 tests pass; no code change.
🤖 Generated with Claude Code