Skip to content

Feature/baseline updates - #69

Open
laurenceyoon wants to merge 10 commits into
pymatchmaker:developfrom
laurenceyoon:feature/baseline-updates
Open

laurenceyoon wants to merge 10 commits into
pymatchmaker:developfrom
laurenceyoon:feature/baseline-updates

Conversation

@laurenceyoon

@laurenceyoon laurenceyoon commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Baseline fixes and two new audio baselines for the TISMIR 2026 benchmark reruns. Supersedes #68.

New

  • arzt_tempo: OLTW with Arzt & Widmer (2010) simple tempo model with dynamic score alterations.
    • Modularized into ArztTempoModel in matchmaker/dp/tempo.py, cleanly integrated into OnlineTimeWarpingArztFrame(tempo_model=...).
    • Includes anti-runaway safeguards (tempo_min=0.85, tempo_max=1.20, min_advance=10) and anchor-based tempo projection in get_current_position() to prevent runaway feedback loops and discretization drift.
    • Performance: improves 146-piece piano audio benchmark TR from 71.2% (104/146, standard Arzt) to 74.7% (109/146) with lower error on Batik and common-tracked subsets.
  • arzt_multi_fold: Arzt OLTW tracking several unfolded score variants in parallel for folded scores (needs unfold_score=False).
  • matchmaker/features/audio.py: added Onset2008Processor for Arzt (2008) onset representation.
  • dtw_loop.pyx: optional backpointers record the backward path used for arzt_tempo's tempo estimate (rebuild the Cython extension).
  • run_examples.py: options for piece, files, unfolding (now off by default) and method kwargs.
  • Tests: tests/test_tempo_models.py, tests/test_oltw_arzt.py, tests/test_utils_eval_gt.py.

Bug fixes

  • Audio features: take the modulus of the complex CQT/STFT (the real dtype dropped the imaginary part).
  • Audio OPHMM: transitions follow Nakamura et al. (2016): forward-only band (D1=0, D2=2), a_{i,i+2}=1e-50, no flux boost, class default s_j=1e-100.
  • Audio OPHMM and SKF: chord durations in quarter notes; SKF score tempo in quarter-note BPM.
    • Impact of the three audio fixes above, together (develop → this branch): SKF 61.0% → 76.0% (+15.0 pt), OPHMM 53.4% → 62.3% (+8.9 pt). Arzt, Dixon and PF audio are unchanged (71.2%, ~69%, 78.8%).
  • gt_from_match: GT onsets from float64 note_on instead of float32 onset_sec. A GT onset could sit ~1e-5 s before the update that reports it, so that onset's update was missed.
  • GT onsets from the streamed performance when it holds the match notes. The .match files and the performance MIDI can use different tick resolutions (480 vs 384 ppq in (n)ASAP), which put the GT half a tick off the streamed onsets.
    • Impact on (n)ASAP MIDI (32 performances; Batik and Vienna unchanged): HMM, PTHMM, SLT_OLTW, OTM 93.8% → 100%; Arzt 96.9% → 100%; Dixon 90.6% → 96.9%; OPHMM 84.4% → 90.6%. Share of events within 0.1 beat: ~77% → ~97%.
  • PF MIDI processor: an observation is stamped with its first onset.
    • Impact on Vienna MIDI (unaffected by the GT fixes): PF 70.2% → 71.4% (+1 performance); within 0.1 beat 43.9% → 49.5%.

Notes

  • methods.yaml keeps audio OPHMM s_j: 0.0 on purpose (benchmark setting for performances without repeats).

CQTProcessor, CQTSpectralFluxProcessor (OPHMM) and RawSpectrumProcessor (SKF)
passed dtype=np.float32 to librosa.cqt / librosa.stft. librosa 0.11 then returns a
real array, dropping the imaginary part, so np.abs gave |Re X| instead of |X|
(about 36% mean magnitude error). Compute the complex transform and cast its
modulus to float32. OPHMM's score templates already used the complex modulus.
The audio OPHMM used the symmetric +-3 transition table of the MIDI model
(Nakamura et al. 2014, Table 3, with the +1 entry mistyped as 0.94531 for
0.84531), filled the whole matrix with 1e-6 outside the band that the forward
step uses, boosted the exit probability by spectral flux, and defaulted to
s_j = 1e-5. Follow the audio paper instead (Sec. III-B, IV-B2): nbh(i) =
{j; 0 <= i - j <= 2} with a_{i,i+2} = 1e-50 and no backward moves, no mass
outside the band, no flux boost, and s_j = 1e-100 (Sec. IV-C).
build_chord_sequence measured note lengths in score beats divided by 4, which is
wrong whenever the beat is not a quarter note; use onset_quarter. The tempo read
from a score marking now honours the marking's unit (e.g. half = 100 is 200
quarter-note BPM).
arzt_tempo: frame-level Arzt OLTW with the simple tempo model of Arzt & Widmer
(SMC 2010), computed from a backward path (dtw_loop records backpointers).
arzt_multi_fold: for folded scores, one Arzt matcher per unfolded version of the
score, outputting the matcher with the lowest recent cost (adapted from Arzt,
Widmer & Dixon, ECAI 2008, and Widmer & Arzt, Vienna Talk 2010).
@laurenceyoon
laurenceyoon force-pushed the feature/baseline-updates branch from 032dd83 to 0203cde Compare October 15, 2026 08:17
@laurenceyoon
laurenceyoon marked this pull request as ready for review October 15, 2026 08:52
@laurenceyoon
laurenceyoon force-pushed the feature/baseline-updates branch 4 times, most recently from d9652a4 to 8ca263f Compare October 15, 2026 10:07
gt_from_match took performance onsets from note_array()'s onset_sec, which
partitura stores as float32. A MIDI follower stamps its updates with the
float64 onset it receives, so the GT time of the same onset could fall a few
tens of nanoseconds earlier, and the perf -> score lookup ("last position at
or before t") then missed the update made at that onset and scored the
previous event. In the eval fold about a third of MIDI onsets were scored one
event late this way; 6/8 scores, where one event is up to a beat, failed the
tracking criterion on paths that are correct. Use the PerformedPart notes'
note_on (float64) instead: GT and follower times now coincide exactly.
The bottom-level self-transitions took the chord durations from score beats
times 60 / tempo. The tempo is quarter-note BPM, but score beats follow the
time signature, so in 6/8 (eighth-note beats) every chord was expected to last
twice as long, and in 2/2 half as long. Use onset_quarter; the reported score
positions stay in beats.
…notes

A .match file and the MIDI file a follower streams can store the same notes
at different tick resolutions ((n)ASAP: 480 vs 384 ppq), so their onsets differ
by up to half a tick. A GT time a fraction of a millisecond before the
follower's update was scored against the position before that update.
resolve_gt/gt_from_match take the streamed performance and use its onsets
when every matched note has the same id and pitch in it.
ParticleFilterMidiProcessor returned the middle of the polling window, which
precedes the onsets in the window's second half and follows those in its
first half. Use the first onset of the frame, as the other MIDI processors do.
…neTimeWarpingArztFrame

- Implement ArztTempoModel in matchmaker/utils/tempo_models.py (Arzt & Widmer 2010 simple tempo model with dynamic score alterations and standalone onset support)
- Integrate ArztTempoModel into OnlineTimeWarpingArztFrame with bounded backward step and tempo projection
- Support tempo model kwargs and benchmark evaluation in matchmaker/methods.yaml and utils/eval.py
- Add unit tests for ArztTempoModel and OnlineTimeWarpingArztFrame
@laurenceyoon
laurenceyoon force-pushed the feature/baseline-updates branch from 8ca263f to 6c56d3c Compare October 15, 2026 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant