TinyBooth: Chord-chart video generator tab (auto guitar-diagram sync)

[ophiocus]

Summary

A new video-generator tab in TinyBooth whose single job is: take a song (audio or a video's audio track) and emit that same song with a synchronised video track showing the guitar chord being played — one standard six-string fretboard diagram per chord, changing on the beat, over the original untouched audio.

Pipeline

audio/video in
  → E1  tempo + beat grid + chord recognition (full mix) + verb detection  →  chord grid
  → E2  editable MIDI grid panel (hear it, correct it) before render
  → E3  each chord → one EADGBE guitar voicing
  → E4  each voicing → one fretboard pictograph frame  (convention spec lives here)
  → E5  build video track, sync to grid, mux over original audio  →  output video

Scope decisions (settled during intake)

• Full-mix chord recognition, not per-instrument. Per-instrument extraction needs stem separation + per-stem chord estimation and roughly triples the effort, yet the output only ever shows one chord at a time. Dropped for v1.
• Verb = a span, at the song's current tempo, holding one recognisable melodic phrase — the repeating harmonic unit the progression cycles through. The core chord + progression are defined over a verb; the render covers the whole track.
• Pictograph = a standard six-string fretboard diagram, one guitar, standard E tuning (EADGBE).
• The editable grid (E2) is a first-class panel, not a hidden stage — auto chord detection will misfire, and letting the user hear + fix before rendering is what makes the tool trustworthy.

Epics

• E1 · Audio -> beat-quantised chord grid + verb detection #3 — E1 · Audio → beat-quantised chord grid + verb detection
• E2 · Editable MIDI chord-grid panel (playback + correction) #4 — E2 · Editable MIDI chord-grid panel (playback + correction)
• E3 · Chord -> guitar voicing resolver (EADGBE) #5 — E3 · Chord → guitar voicing resolver (EADGBE)
• E4 · Fretboard pictograph renderer + convention spec #6 — E4 · Fretboard pictograph renderer + convention spec
• E5 · Sync + mux to chord-chart video #7 — E5 · Sync + mux to chord-chart video

Done when

All five epics ship and a real song renders end-to-end: original audio plays, fretboard diagram changes in sync at each chord.

Dependency order

E1 and E4 can start in parallel. E2→needs E1. E3→needs E1. E5→needs E3 + E4.