Skip to content

Calibrator wizard: camera and screen calibration with the calibrator board - #70

Merged
LeeO86 merged 14 commits into
mainfrom
feat/calibrator-wizard
Nov 9, 2026
Merged

LeeO86 merged 14 commits into
mainfrom
feat/calibrator-wizard

Conversation

@LeeO86

@LeeO86 LeeO86 commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

What this does

Adds the calibrator wizard (Settings → With calibrator) for the calibrator board (contract docs/calibrator.md, hardware in #69). The clap calibration stays as it is.

  1. Plug in – small animation: cable from phone to board.
  2. Camera and microphone – lay the board LED-down on the rear camera. The camera and mic record the flash + click train, a ring of 20 dots fills up, the result is shown as "Camera vs mic: +12.3 ms ± 0.4". Save stores it as this phone's analyser calibration (same record and validity as the clap, method calibrator-pulse).
  3. Screen and speaker – the phone's own generator opens in calibration mode: two halves at the bottom of the screen swap black/white every 500 ms, the speaker beeps 100 ms after each swap. The board clicks on each swap; the mic hears click then beep, so the mic delay cancels. "Screen vs speaker: −38.0 ms ± 0.3". Save stores it as the phone's sender calibration; the test signal then sends it in the AVS4 QR (method transponder).
    • "Measure a TV or monitor instead": the container shows the same pattern, the phone only listens; result shown, not saved.

How it is built

  • Core C++ (core/include/avsync/calibrator.hpp): sound onsets (click vs beep by order and by 2.7 kHz vs 1.5 kHz energy), flash onsets from 32 row bands on the analyser's rolling-shutter row time, pairing, outlier rejection, uncertainty (standard error of the median), the pattern on the frame grid, the beep. iOS gives no readout time and exposes for the whole frame: the readout is fitted over the pulses (the board's clock drifts against the camera, so the flash lands on different rows).
  • PhoneAnalyzer::begin_calibrator runs the camera take (and an audio-only take for an external screen); progress once a second, then the result.
  • Same path as real sending: the calibration pattern is a mode of the generators, not a React view. iPhone: generateStyled(..., {mode: 'calibration'}) uses the same display link, layers, audio engine and player node. Container: AVSYNC_PATTERN=calibration (live key, Signal API pattern, web form "Picture"). Android has no generator: skipped (shared logic is in core; the Android CMake builds it).
  • App: components/CalibratorWizard.js, pure logic in src/calibrator.mjs.
  • Analysers subtract the sender calibration (decided): once in core, Pairer::set_sender_calibration, used by the phone session (iOS + Android), the container analyzer and the file analyzer. Result = measured − sender calibration; the raw value and the calibration stay visible: phone reading rawOffsetMs / senderCalibrationMs (Measure and Results show "raw … · sender calibration …", saved results keep both), events raw_av_offset_ms / sender_calibration_ms, /api/v1/measurement raw_offset_ms / sender_calibration_ms, web UI Input card "AV sync raw", file result summary.sender_calibration_ms / median_raw_av_offset_ms. Not calibrated: no change. No double count: the phone's own analyser calibration is still subtracted only in the app (liveView); the native session never gets it.
  • Clear calibration clears both (decided): Debug → Clear calibration now asks first ("Clears both: the analyser calibration … and the sender calibration …"). There was no confirm before; this adds one.
  • Results: Title as default nickname: a sender's default name is the newest AVS4 Title from its saved results (live Δ: the Title heard now), then the short ID. A nickname you set wins. "check" also marks a different sender calibration.

Contract changes (docs/calibrator.md, on this branch; no firmware change)

  • Generator sound fixed: 4 ms beep at 1.5 kHz.
  • Camera mode uses every paired pulse (start pulses too); the start pattern is not needed.
  • Display mode does not need the swap times: the beep must come 0–450 ms after the click, so sound − picture between −100 and +350 ms (also covers a TV chain).
  • Buzzer delay and display path difference: still 0 until measured.

Tests

  • Core ctest: all pass (new test_calibrator.cpp: noise, missing pulses, stray sound, rolling shutter with known / estimated / fitted readout, 60 fps long exposure, slow screen, sensor mismatch per direction, external TV, failures, PhoneAnalyzer end to end). Synthetic accuracy about 0.03 ms.
  • node --test mobile-app/src/*.test.mjs tests/web/*.test.cjs: 58 pass (sender calibration in the reading and saved result, Title nickname, check on sender calibration).
  • Core: new test "the sender calibration from the QR is subtracted once, raw value kept"; the phone analyser end-to-end test now expects offset +32.5 (raw +20, sender −12.5).
  • phone_generator.py: the fixture sends a sender calibration (−12.5 ms) at 2 Hz; events, /api/v1/measurement and /api/v1/status checked (corrected ~40 ms, raw ~27.5 ms); 1 Hz stays not calibrated.
  • Linux MXL: new tests/integration/calibration_pattern.py (1080p50, 1080i25, 1080p59.94: swaps every 500 ms, beep 100 ms after each swap within 0.07 ms, simulated board gives 0.03 ms; live switch back to the test signal). phone_generator.py, settings.sh, api.sh, live.sh pass.
  • iOS could not be compiled here (no Xcode). mobile-app/tests/native/generator-runner.mm got a sender-calibration QR check for the Mac.

Needs the Mac / a device

  • Build iOS (Swift/ObjC++ changes in the camera view, generator controller, module, bridges; new Core_calibrator.cpp), npm run test:ios-generator, npm run test:ios-native.
  • Device: the wizard screens and animations; generator calibration mode (pattern position, outline size, beep audible, mic take works in play-and-record); a normal test signal with a saved sender calibration shows it in the analyser.
  • With a real board: camera take (exposure at lowest ISO, saturation), display take on iPhone and on a TV with the container pattern; compare the play-and-record vs playback output delay once.

Open questions

  • Should the analysers subtract the sender calibration? Decided: yes, see above.
  • Clear calibration: analyser only or both? Decided: both, with a confirm.
  • Should the normal generator also run in play-and-record, so the audio session is bit-identical to the calibration?
  • Android: add beginCalibrator to the camera view (the core is ready)?

🤖 Generated with Claude Code

https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

LeeO86 and others added 13 commits November 8, 2026 15:30
Shared C++ for the AV Sync calibrator board (docs/calibrator.md, contract v1):
- find_sound_onsets: clicks and beeps in a recording (trailing 0.25 ms RMS,
  25 % of peak), told apart by 2.7 kHz vs 1.5 kHz energy.
- find_flash_onsets: LED flash start from row-band means, on the
  rolling-shutter row the analyzer uses; readout from the band height or,
  with full-frame exposure, fitted over the pulses in camera_calibration.
- camera_calibration: pair flashes with clicks, robust median, uncertainty.
- display_calibration: click then beep D = 100 ms later, offset = gap - D,
  both swap directions weighted equally.
- calibration pattern for all generators: halves swap every 500 ms, beep.
Unit tests on synthetic signals (noise, missing pulses, stray sound, rolling
shutter with known/unknown readout, slow screen, sensor mismatch).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
PhoneAnalyzer::begin_calibrator keeps 32 row-band means per frame and the
microphone, and poll_calibrator reports progress about once a second, then
the result (20 pairs or 30 s) or an error. No live report or QR scan while
it runs. iOS: beginCalibrator(mode) on the camera view; the camera take
first re-locks exposure (frame interval) at the lowest ISO, because the LED
floods the lens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
…the QR

generateStyled(..., {mode: 'calibration'}) shows the calibration pattern in
the generator's own screen: the same display link, layers, audio engine and
player node as the test signal. Portrait halves left and right near the
bottom microphone, a red outline of the board, halves swap every 500 ms, a
1.5 kHz beep 100 ms after each swap (scheduled on the host clock minus the
reported output latency). The session is play-and-record on the built-in
speaker; the microphone take is analysed with the shared core
(display_calibration) and the result comes back when the screen closes.

The test signal sends a saved sender calibration in the AVS4 QR
(senderCalibration* options), only on the audio output it was measured on.
Core pattern helpers are now frame based (also 59.94 fps).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
Same feature as the phone: the generator can send the calibration pattern
instead of the test signal, so a phone with the calibrator board can
measure a TV/monitor chain. Left/right halves swap every 500 ms on the
grain grid (also interlaced and 59.94), a 1.5 kHz beep 100 ms after each
swap at the chirp level, a grey tick at the ends of the pattern line.
Live key, also in the Signal API (pattern) and the web form (Picture).

tests/tools/calibration_flow + tests/integration/calibration_pattern.py
read both flows, check the timing and run a simulated calibrator through
display_calibration (1080p50, 1080i25, 1080p59.94: 0.03 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
Three steps with small animations: plug in, camera (the calibrator on the
rear camera; progress ring of 20 flashes; 'Camera vs mic' result saved as
this phone's analyser calibration, method calibrator-pulse, same record and
validity as the clap), screen (the generator's calibration pattern;
'Screen vs speaker' saved as this phone's sender calibration, method
transponder). The test signal then sends it in the AVS4 QR. A TV/monitor
showing the container's calibration pattern can be measured too (result
only, not saved). The clap calibration is unchanged. iOS only for now
(Android has no generator and no calibrator take yet).

Pure logic in src/calibrator.mjs with node tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
docs/calibrator.md (copy of PR #69's contract) gets an 'App side' section
and the list of changes from the v1 proposal (none needs a firmware change).
calibration.md: user guide 'With the calibrator'. phone-generator.md:
calibration mode and the sender calibration in the QR. SPECIFICATION:
AVSYNC_PATTERN and the calibrator in 12.3. IMPLEMENTATION_PLAN: entry with
the deviations. CHANGELOG, mobile-app README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
… clap gap fill unchanged

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
Done once in core (Pairer::set_sender_calibration) for the phone session,
the container analyzer and the file analyzer. Events keep raw_av_offset_ms
and sender_calibration_ms; the phone reading has rawOffsetMs and
senderCalibrationMs; /api/v1/measurement has raw_offset_ms and
sender_calibration_ms; the web UI Input card shows AV sync raw. Not
calibrated: no change. The phone fixture sends a sender calibration at 2 Hz.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
…ickname

Measure and Results show the raw value and the sender calibration under a
corrected result; saved results keep them. The analyser calibration stays
subtracted in the app only, so nothing counts twice. Clear calibration asks
first and clears the analyser and the sender calibration. A sender's default
name is the newest Title from its QR; a set nickname wins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
… nickname

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
LeeO86 added a commit that referenced this pull request Nov 8, 2026
Same file on both branches now (BOOT button, 1 nF filter row, App side section,
changes from the v1 proposal, open points).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
Same file on both branches now, so the merge has no conflict here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4
@LeeO86
LeeO86 marked this pull request as ready for review November 9, 2026 10:22
@LeeO86
LeeO86 merged commit d57a494 into main Nov 9, 2026
11 checks passed
LeeO86 added a commit that referenced this pull request Nov 9, 2026
* Calibrator board v1 proposal: KiCad project, JLC files, firmware, signal contract

A small USB-C powered board (no battery) for the AVsync app with two automatic
modes: camera mode (LED flash and buzzer click from one register write) and
display mode (two light sensors on the comparator click at the crossing of a
black/white swap). All parts on top; the white reverse-mount LED and two
reverse-gullwing phototransistors look through holes, so the bottom stays flat.

- hardware/calibrator: KiCad 9 schematic and board generated from
  scripts/design.py, autorouted, ERC/DRC clean (one explained ERC warning),
  gerbers/BOM/CPL for JLCPCB Economic PCBA, renders, BOM and cost notes.
- hardware/calibrator/firmware: PY32F002A firmware, mode logic in plain C
  with host unit tests, Makefile. Not tested on hardware.
- docs/calibrator.md: v1 signal contract (app side not built yet).
- CI job: firmware build and tests, KiCad ERC/DRC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator v0.2: merch look, QR code, readable schematic, CERN-OHL-P-2.0

Board (60 x 22 mm now):
- AVsync logo from assets/app-icon.svg on both sides: cyan/white strokes as
  silkscreen, the orange waveform and centre ring as bare copper with a
  mask opening (no paste, no net, kept clear by keep-outs). Black mask and
  ENIG in the stackup, so the copper looks gold.
- QR code to https://leeo86.github.io/mxl-avsync/ on the bottom: version
  3-M, 0.45 mm modules, 4-module quiet zone, light silkscreen background,
  mirrored to read from below. zxing-cpp decodes the bottom render.
- Split line now also on the top (next to the LED and behind the buzzer).
- Text redone: one font, clear sizes, JLC minimum 1.0 mm for "not a toy",
  licence and pad labels.
- GND is routed as tracks before the fills are added, so no fill islands;
  ERC 0, DRC 0, 0 unconnected.

Schematic redrawn with real wires and functional blocks (power in, light
sensors, MCU, flash + click, programming), power symbols top, GND bottom,
labels only for the programming pads, custom PY32 symbol with pins grouped
by use. scripts/check_netlist.py checks the drawing against design.py
(also in CI).

Licence: board files and logo artwork CERN-OHL-P-2.0 (text added), the
firmware stays MIT. Cost table updated for ENIG (estimate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator logo: white centre dot and full dashed line like the icon

The board logo now follows assets/app-icon.svg more closely: the dashed
centre line runs from y 74 to 181 (dasharray 4 7), hidden only under the
centre dot, and the inside of the gold ring is a white silkscreen dot.
Because JLC strips silkscreen from bare copper, the gold copper and its
mask opening are cut away under the dashes and inside the ring (polygon
booleans in gen_pcb.py). ERC 0, DRC 0. The app icon file is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator v0.3: CH32X035 with USB-C firmware update

Adrian's decision (2026-11-08): switch from PY32F002A to CH32X035G8R6
(QSOP-28) so the board can be updated over its USB-C socket.

Board and schematic:
- 16-pin USB-C (C393939, HRO TYPE-C-31-M-12 footprint), 5.1k Rd on CC1/CC2
  kept, D+/D- (A6/B6, A7/B7) to PC17/PC16.
- Update button SW1 (TS-1187A, Basic) from PC17/D+ to VDD via 4.7k: the ROM
  bootloader starts only at power-on with PC17 high. No pull-down on D+ and
  no ESD part (reasons in the README).
- Sensor pair on comparator 3 with hysteresis (PA13+, PA2-), ADC on PA0/PA2.
  PB3 (comparator output, read back by the firmware) left unconnected.
- Programming pads removed (USB update, ROM bootloader). Board 66 x 22 mm,
  same mechanics and look (gold logo with white dot and dashes, QR, split
  line both sides, "hold to update" under the button).
- Schematic redrawn with a USB-C power + update block; ERC 0, netlist check
  OK, DRC 0. Router stubs/duplicates are cleaned up in a separate step.

Firmware: new hardware layer for CH32X035 (riscv-none-elf GCC, register
definitions from ch32fun, MIT); USB, its pull-up and USB PD are never
enabled. cal_logic.c and the host tests are unchanged. CI builds with
Ubuntu's gcc-riscv64-unknown-elf.

Docs: update-over-USB guide (wchisp / chprog), BOM and CHF cost table for
10 + 100 boards without cables (the user brings a USB-C cable).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator v0.3: fix render stand-ins, JLC rotations, artwork layout

- The QSOP-28 3D stand-in had its long side along x, so the render showed
  U1 turned by 90°. The footprint itself was always right (pins along y,
  pin 1 top left, as in the CH32X035 datasheet); only the render changed.
  The stand-in now matches the footprint (leads, pitch, pin-1 dot); the
  button stand-in got its legs at the pads.
- CPL: add JLC rotation offsets for the stock footprints (QSOP-28 +270,
  SOT-23 -90, HRO USB-C +180), as in the community list of JLC
  corrections. To be checked in JLC's placement preview.
- Top: logo and "AVsync calibrator" centred in the free area between the
  button and the USB-C; licence text small below the MCU.
- Bottom: 11 mm logo in the middle of the board beside the QR code, with
  "AVsync calibrator v0.3"; CAMERA / SCREEN hints and a tiny "not a toy"
  at the LED/sensor end.
ERC 0, DRC 0, netlist check OK; QR still decodes from the render.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator v0.3: layout nudges on both sides

Top: logo block 1.2 mm lower, licence text 1.3 mm to the right.
Bottom: logo block and the CAMERA / SCREEN / "not a toy" text 1.2 mm
towards the LED end, "not a toy" 1.5 mm closer to the SCREEN block.
Via keep-outs over both logo blocks so no via holes land in the artwork.
ERC 0, DRC 0, netlist check OK; the QR code still decodes from the render.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator v0.3: label the update button "BOOT"

One short "BOOT" (1.2 mm bold) centred under SW1 replaces the two-line
"hold to update". Docs now say "hold the BOOT button while plugging into
a PC". ERC 0, DRC 0, netlist check OK; QR still decodes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* Calibrator: sensor filter caps C4/C5 10 nF -> 1 nF

22 kOhm x 10 nF delayed the comparator crossing by ~0.15 ms; with 1 nF it is
~15 us. Same 0402 footprint, JLC basic part C1523. Copper and gerbers unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* docs/calibrator.md: take the app-side contract from #70

Same file on both branches now (BOOT button, 1 nF filter row, App side section,
changes from the v1 proposal, open points).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

* CHANGELOG: one current entry for the calibrator board

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ybuzgBaGxuB5UiaE8dRa4

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant