You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The three STT behavior fixes shipped in v1.1 (#92 phantom-caption gate, #93 auto-detect CJK suppression, #94 source-language forcing) were verified only via say TTS and tiny-model integration tests — never against real varied audio on the production model. Additionally the confidence floors must be calibrated per model with real evidence, and small vs medium vs large-v3-turbo accuracy compared so the settings copy and default rest on measurement.
Floor calibration on small (then medium/large-v3-turbo): log mean per-token confidence for real speech vs any phantoms; choose floors with margin; check short-phrase real speech ("Yes." / "좋아요") survives.
Accuracy comparison small vs medium vs large-v3-turbo on the same 5-min sample-video segment: transcript diff, error notes, latency + RSS per model.
Goal
The three STT behavior fixes shipped in v1.1 (#92 phantom-caption gate, #93 auto-detect CJK suppression, #94 source-language forcing) were verified only via
sayTTS andtiny-model integration tests — never against real varied audio on the production model. Additionally the confidence floors must be calibrated per model with real evidence, and small vs medium vs large-v3-turbo accuracy compared so the settings copy and default rest on measurement.Prereqs
Verification matrix (each row = observable pass/fail, evidence via
ui-heartbeat.json+ saved session files)6ARteKgyEtw) → zero CJK output (Session start: choose spoken (source) language + target language #94).small(then medium/large-v3-turbo): log mean per-token confidence for real speech vs any phantoms; choose floors with margin; check short-phrase real speech ("Yes." / "좋아요") survives.Deliverables
docs/CALIBRATION.md(method, audio set, numbers, model recommendation).Operator-assist (called out up front)
Routing
agent/claude(PO direct on this Mac — device-bound; verification work per manual §9).Part of #107. Depends on #108, #109.