Calibrator wizard: camera and screen calibration with the calibrator board

CHANGELOG.md

@@ -2,6 +2,32 @@
 
 ## Mobile (not released yet)
 
+- Analysers subtract the sender calibration: when the AVS4 QR carries one, the
+  AV offset shown and saved is the measured value minus the sender's own
+  screen-vs-speaker offset. Measure and Results show the raw value and the
+  sender calibration in a small line; saved results keep `rawOffsetMs` and
+  `senderCalibrationMs`. Done once in core C++, so iOS and Android behave the
+  same; this phone's own camera-vs-mic calibration is still subtracted on top,
+  once. "check" in Results also marks a different sender calibration.
+- Debug → Clear calibration now asks first and says it clears both: the
+  analyser calibration and the sender calibration.
+- Results: a sender's default name is its Title from the QR (the newest one),
+  then the short ID. A nickname you set always wins.
+- iPhone test signal: Stop now clears the whole picture (the scale and the
+  marker stayed on screen before) and shows a large "Stopped".
+- Calibrator wizard (iPhone): Settings → With calibrator. Step 2 measures
+  camera vs microphone with the calibrator board's flashes and clicks and saves
+  it like a clap calibration (method `calibrator-pulse`). Step 3 shows the
+  generator's calibration pattern (halves swap every 500 ms, a beep 100 ms
+  later) and saves screen vs speaker as the phone's sender calibration; the test
+  signal then sends it in the AVS4 QR (method `transponder`) instead of "not
+  calibrated". A TV fed by the container's calibration pattern can be measured
+  too (not saved). New camera view method `beginCalibrator(mode)`, generator
+  option `mode: 'calibration'` and `senderCalibration*` options. Not tested
+  with a real board yet (`docs/calibration.md`, `docs/calibrator.md`).
+- Container generator: `AVSYNC_PATTERN=calibration` (live; Signal API
+  `pattern`, web form "Picture") sends the calibration pattern instead of the
+  test signal, for a phone with the calibrator on a TV or monitor.
 - iPhone generator: the source ID stays the same when the generator is stopped
   and started again. It changes only after a reinstall, a new app version or
   build, or another event rate; name, subtitle and image do not change it
@@ -59,6 +85,12 @@
 
 ## Container (not released yet)
 
+- Analyzer (live and file): subtracts the sender calibration from the AVS4 QR
+  from `av_offset_ms` (and `deviation_ms`, grade, metrics). Events keep
+  `raw_av_offset_ms` and `sender_calibration_ms`; `/api/v1/measurement` has
+  `raw_offset_ms` and `sender_calibration_ms`; the web UI Input card shows
+  "AV sync raw"; the file result `summary` has `sender_calibration_ms` and
+  `median_raw_av_offset_ms`. Not calibrated: no change.
 - Generator: the default `AVSYNC_TEST_ID` is now `<NMOS seed>-` plus a short
   hash of the seed, the app version and the timing settings (event rate, audio
   timecode, video format, AV offset, audio/video lead, source emit offsets),

CMakeLists.txt

@@ -59,6 +59,7 @@ add_library(avsync-core STATIC
     core/src/phone_generator.cpp
     core/src/picture_reader.cpp
     core/src/two_way.cpp
+    core/src/calibrator.cpp
     third_party/nayuki/qrcodegen.cpp
 )
 target_include_directories(avsync-core
@@ -70,7 +71,7 @@ set_source_files_properties(core/src/text.cpp PROPERTIES OBJECT_DEPENDS "${DEJAV
 target_link_libraries(avsync-core PUBLIC blend2d::blend2d)
 target_compile_options(avsync-core PRIVATE -Wall -Wextra -Wno-unused-parameter)
 
-add_executable(avsync-core-tests tests/unit/test_core.cpp tests/unit/test_stage23.cpp tests/unit/test_timecode.cpp)
+add_executable(avsync-core-tests tests/unit/test_core.cpp tests/unit/test_stage23.cpp tests/unit/test_timecode.cpp tests/unit/test_calibrator.cpp)
 target_include_directories(avsync-core-tests PRIVATE third_party/doctest)
 target_link_libraries(avsync-core-tests PRIVATE avsync-core)
 target_compile_options(avsync-core-tests PRIVATE -Wall -Wextra -Wno-unused-parameter)
@@ -182,8 +183,11 @@ if(FFMPEG_FOUND AND NOT AVSYNC_CORE_ONLY)
         add_executable(avsync-phone-fixture tests/tools/phone_fixture.cpp)
         target_link_libraries(avsync-phone-fixture PRIVATE avsync-mxlio)
         target_compile_options(avsync-phone-fixture PRIVATE -Wall -Wextra -Wno-unused-parameter)
+        add_executable(avsync-calibration-flow tests/tools/calibration_flow.cpp)
+        target_link_libraries(avsync-calibration-flow PRIVATE avsync-mxlio)
+        target_compile_options(avsync-calibration-flow PRIVATE -Wall -Wextra -Wno-unused-parameter)
         if(MXL_RUNTIME_LIB_DIRS)
-            set_target_properties(avsync-sample-flow avsync-pattern-writer avsync-phone-fixture PROPERTIES BUILD_RPATH "${MXL_RUNTIME_LIB_DIRS}")
+            set_target_properties(avsync-sample-flow avsync-pattern-writer avsync-phone-fixture avsync-calibration-flow PROPERTIES BUILD_RPATH "${MXL_RUNTIME_LIB_DIRS}")
         endif()
         message(STATUS "libmxl found: live reader and writer are linked into mxl-avsync")
     else()

IMPLEMENTATION_PLAN.md

@@ -1352,3 +1352,57 @@ only calibration is compared for them. The 100-result limit stays, but a
 reference result is never dropped by it. `sourceId` 0 (a QR without a timecode
 ID) counts as unknown sender.
 
+
+## Calibrator wizard (2026-11-08)
+
+Maintainer design (2026-11-08): Settings → With calibrator, a three-step wizard
+(plug in, camera, screen) for the calibrator board (contract
+`docs/calibrator.md`, hardware on PR #69). The clap calibration is unchanged.
+
+- Analysis in core C++ (`calibrator.hpp`): sound onsets (click/beep), flash
+  onsets from row bands, camera pairing, display pairing, robust median and
+  uncertainty; the calibration pattern (frame based) and the beep. Unit tests
+  on synthetic signals (noise, missing pulses, stray sound, rolling shutter
+  with known, estimated and fitted readout, long exposure, slow screen,
+  sensor mismatch, external TV, failures). `PhoneAnalyzer::begin_calibrator`
+  runs the camera take (and an audio-only take for an external screen).
+- Screen calibration is a mode of the generators, not a React view: iPhone
+  `generateStyled(..., {mode: 'calibration'})` (same display link, layers,
+  audio engine and player node), container `AVSYNC_PATTERN=calibration`
+  (live key, Signal API `pattern`, web form "Picture"). Android has no
+  generator (`supportsGenerator` false): skipped; its analyser could call
+  `begin_calibrator` through JNI later (not wired).
+- Results: camera → analyser calibration record (method `calibrator-pulse`,
+  same validity as the clap); screen → `senderCalibration` in the session
+  state (method `transponder`, 90 days, same OS version, sent only on the same
+  audio output) → AVS4 QR of the phone generator.
+
+Deviations and choices:
+
+- AVS4 method: `transponder` for the sender calibration (the board answers
+  each screen swap with a click). `pulse-train` stays unused: the flash train
+  of camera mode calibrates the analyser, not a sender.
+- Contract additions, no firmware change (listed in `docs/calibrator.md`):
+  the generator's sound is a 4 ms 1.5 kHz beep; camera mode uses every paired
+  pulse incl. the start pulses; display mode does not need the swap times
+  (beep 0–450 ms after the click).
+- iOS rolling shutter: no readout time from the camera, full-frame exposure;
+  the readout is fitted over the pulses (the board's clock drifts against the
+  camera). The camera take locks exposure at the lowest ISO.
+- The screen calibration needs the microphone, so the generator runs the
+  audio session in play-and-record (built-in speaker) instead of playback.
+  Same output node and latency compensation; to verify on a device (open
+  point in `docs/calibrator.md`).
+- Constants still 0: buzzer delay, display path difference. Camera mode
+  subtracts 1.5 cm sound travel.
+- Decided (Adrian, 2026-11-08): the analysers subtract the sender calibration
+  once in core (`Pairer::set_sender_calibration`), keep the raw value
+  (`raw_av_offset_ms` / `rawOffsetMs`) and the calibration next to it. The
+  phone's analyser calibration stays in the app (`liveView`), so nothing is
+  counted twice.
+- The wizard and the calibrator take are iOS only (`beginCalibrator` is not in
+  the Android camera view yet); the Settings entry is hidden on Android.
+- Debug → Clear calibration clears the sender calibration too (decided; with
+  a confirm that says both are cleared).
+- The iPhone pattern size assumes about 6 pt per mm (current iPhones); the
+  outline is a placing guide, the pattern line itself is exact.

SPECIFICATION.md

@@ -113,7 +113,7 @@ The readers MUST NOT copy whole grains. The marker detector reads only its regio
 
 | Quantity | Definition | Needs |
 | --- | --- | --- |
-| **AV offset** | `av_offset_ms = audio_event_time_ms − video_event_time_ms`, both at the measuring point | nothing beyond the signal |
+| **AV offset** | `av_offset_ms = audio_event_time_ms − video_event_time_ms`, both at the measuring point; minus the sender calibration when the AVS4 QR carries one (§5.6; the measured value is then `raw_av_offset_ms`) | nothing beyond the signal |
 | **Deviation** | `deviation_ms = av_offset_ms − expected_av_offset_ms` (the generator's set offset) | the event's expected offset (QR or manifest) |
 | **Video latency** | time the video event is seen at the measuring point − time it was emitted | a common clock |
 | **Audio latency** | same for audio | a common clock |
@@ -368,6 +368,7 @@ The rule of mxl-scopes §6, for the same reasons: a stuck reader takes down one
 | `test_id`, `event_id`, `seq`, `mode` | |
 | `video_event_time`, `audio_event_time` | ms (FILE) or TAI ns (LIVE), index times |
 | `av_offset_ms`, `expected_av_offset_ms`, `deviation_ms` | §3.1 |
+| `raw_av_offset_ms`, `sender_calibration_ms` | only when the AVS4 QR carries a sender calibration: the measured offset before it was subtracted, and the calibration (§5.6) |
 | `av_offset_arrival_ms` | LIVE only |
 | `video_latency_ms`, `audio_latency_ms`, `video_index_shift_ms`, `audio_index_shift_ms` | LIVE with a reference only; else absent |
 | `video_source_emit_offset_ms`, `audio_source_emit_offset_ms`, `video_chain_latency_ms`, `audio_chain_latency_ms` | with a latency and a source emit offset (entered, or the flow's NMOS tag; §3.3); else absent |
@@ -515,6 +516,7 @@ Precedence: environment > `AVSYNC_CONFIG_FILE` (flat JSON, keys are the variable
 | `AVSYNC_VIDEO_FORMAT` | `1080p50` | flow | §5.4 |
 | `AVSYNC_AUDIO_CHANNELS` | 2 | flow | 1–16 |
 | `AVSYNC_AV_OFFSET_MS` | 0 | live | −500…+500, set offset |
+| `AVSYNC_PATTERN` | `test` | live | `test` (the AV sync test signal) or `calibration`: the calibrator board's pattern (halves swap every 500 ms, 1.5 kHz beep 100 ms after each swap at the chirp level; docs/calibrator.md). The flows and test id stay the same |
 | `AVSYNC_CHIRP_LEVEL_DBFS` | −12 | live | −40…−1 |
 | `AVSYNC_AUDIO_ENABLE` | `true` | live | `false` mutes the chirps (silence, flows keep running) |
 | `AVSYNC_AUDIO_LEAD_MS` | 20 | restart | how far ahead of TAI audio is written |
@@ -667,6 +669,7 @@ The camera and microphone have different capture latencies (OS buffers, exposure
 - The app measures the rig, stores the capture offset per device model and OS version with the date, the method, the number of events and the spread, and subtracts it from later measurements.
 - A calibration expires after an OS update or 90 days. Uncalibrated measurements are allowed and shown as **uncalibrated** in orange, with no grade colour.
 - Expected accuracy is unknown until measured on real devices; the app shows the measured spread of its own calibration as its uncertainty.
+- **Calibrator board** (maintainer design, 2026-11-08; contract docs/calibrator.md): Settings → With calibrator. Camera mode (flash + click on the rear camera) gives the capture calibration above (method `calibrator-pulse`, same record and validity as the clap). Display mode (the generator's calibration pattern, the board clicks at each swap, the generator beeps D = 100 ms later) gives the phone's **sender calibration** (screen vs speaker, AVS4 method `transponder`), which the phone generator then sends in the QR. The pattern is a mode of the generators themselves (same display and audio path); the container has it too (`AVSYNC_PATTERN`). Analysis in `core/…/calibrator.hpp`. Every analyser (phone, container, file) subtracts the sender calibration from the measured AV offset once, in core (`Pairer`), and keeps the raw value and the calibration next to it; "not calibrated" changes nothing. The phone's own capture calibration is subtracted separately (app), so neither is counted twice. Debug → Clear calibration clears both.
 
 ---
 
@@ -785,10 +788,10 @@ Family look: header with label, role, state and registration pills, version; tab
 | Method | Path | Role | Purpose |
 | --- | --- | --- | --- |
 | GET | `/api/v1/info` | both | version, core version, MXL revision, role, label |
-| GET | `/api/v1/status` (also `/statusz`) | both | readers or writers, NMOS summary; analyzer: `signal`, the source the picture QR names (`source_id`, `test_id`, `name`, `live`, `rate`, `format`, `estimated_source`, `sender_calibration` {`method`, and when calibrated `offset_ms`, `uncertainty_ms`, `date`}), null without a QR |
+| GET | `/api/v1/status` (also `/statusz`) | both | readers or writers, NMOS summary; analyzer: `signal`, the source the picture QR names (`source_id`, `test_id`, `name`, `live`, `rate`, `format`, `estimated_source`, `sender_calibration` {`method`, and when calibrated `offset_ms`, `uncertainty_ms`, `date`}; analysers subtract it), null without a QR |
 | GET/PATCH | `/api/v1/generator` | gen | offset, level, mute, texts, rate; GET also shows `test_id` |
 | POST | `/api/v1/generator/stop`, `/start` | gen | §11 |
-| GET | `/api/v1/measurement` | ana | current grade, offset, latencies, window statistics |
+| GET | `/api/v1/measurement` | ana | current grade, offset, latencies, window statistics; with a sender calibration also `raw_offset_ms` and `sender_calibration_ms` |
 | GET | `/api/v1/events?since=<seq>&limit=` | ana | the newest matching event records, newest first, plus `total` |
 | GET | `/api/v1/statistics` | ana | window and since-start statistics (§8.2) |
 | POST | `/api/v1/statistics/reset` | ana | new statistics epoch (the event log keeps everything) |
@@ -1057,7 +1060,7 @@ integers big-endian:
 | 5 | event sequence k (40 bits) |
 | 8 | video send time T_k in ns (TAI for LIVE, media timeline for FILE) |
 | 3 | source AV offset in 1/16 ms, signed (±524 s) |
-| 6, only with bit 3 | sender calibration: method u8 (1 clap with a second phone, 2 transponder calibrator, 3 pulse-train), offset i16 in 0.1 ms (sound output minus picture output, positive: sound later; same sign as `av_offset_ms`, §3.2), uncertainty u8 in 0.5 ms, day u16 since 2020-01-01 (0 unknown) |
+| 6, only with bit 3 | sender calibration: method u8 (1 clap with a second phone, 2 transponder calibrator, 3 pulse-train), offset i16 in 0.1 ms (sound output minus picture output, positive: sound later; same sign as `av_offset_ms`, §3.2; analysers subtract it from the measured offset), uncertainty u8 in 0.5 ms, day u16 since 2020-01-01 (0 unknown) |
 | 1 + n | test id: n characters (1–64) of `0-9a-zA-Z-_`, 6 bits each; the source id is CRC-32 of the id. n = 0: the 4-byte source id alone follows (the id did not fit or does not give that source id); the analyzer then shows `source-<8 hex>` |
 | rest | generator name, UTF-8, 0–24 bytes; cosmetic, not part of the id |
 

app/config.cpp

@@ -125,7 +125,7 @@ bool is_restart_key(const std::string& key)
 bool is_live_key(const std::string& key)
 {
     return key == "AVSYNC_AV_OFFSET_MS" || key == "AVSYNC_CHIRP_LEVEL_DBFS" || key == "AVSYNC_AUDIO_ENABLE"
-        || key == "AVSYNC_EVENT_RATE" || key == "AVSYNC_AUDIO_TIMECODE"
+        || key == "AVSYNC_EVENT_RATE" || key == "AVSYNC_AUDIO_TIMECODE" || key == "AVSYNC_PATTERN"
         || key == "AVSYNC_TEST_ID" || key == "AVSYNC_TITLE" || key == "AVSYNC_SOURCE" || key == "AVSYNC_SUBTITLE" || key == "AVSYNC_NOTES"
         || key == "AVSYNC_AUDIO_CHANNEL" || key == "AVSYNC_AUDIO_THRESHOLD" || key == "AVSYNC_QR_RATE"
         || key == "AVSYNC_PASS_MIN_MS" || key == "AVSYNC_PASS_MAX_MS" || key == "AVSYNC_WARN_MIN_MS" || key == "AVSYNC_WARN_MAX_MS"
@@ -172,6 +172,7 @@ Config load_config(const std::map<std::string, std::string>& extra)
     };
     config.role = pick("AVSYNC_ROLE", "");
     config.audio_timecode = pick("AVSYNC_AUDIO_TIMECODE", "1200");
+    config.pattern = pick("AVSYNC_PATTERN", "test");
     config.event_rate = event_rate_value(pick("AVSYNC_EVENT_RATE", "2"));
     config.chirp_ms = static_cast<int>(num(pick("AVSYNC_CHIRP_MS", "30"), "AVSYNC_CHIRP_MS"));
     config.chirp_low_hz = num(pick("AVSYNC_CHIRP_LOW_HZ", "700"), "AVSYNC_CHIRP_LOW_HZ");
@@ -235,6 +236,7 @@ Config load_config(const std::map<std::string, std::string>& extra)
     config.log_format = pick("LOG_FORMAT", "json");
     config.shutdown_timeout_s = static_cast<int>(num(pick("SHUTDOWN_TIMEOUT_S", "10"), "SHUTDOWN_TIMEOUT_S"));
     require(config.audio_timecode == "legacy" || config.audio_timecode == "600" || config.audio_timecode == "1200", "AVSYNC_AUDIO_TIMECODE must be legacy, 600 or 1200");
+    require(config.pattern == "test" || config.pattern == "calibration", "AVSYNC_PATTERN must be test or calibration");
     if (config.audio_timecode != "legacy") {
         require(config.event_rate == 1 || config.event_rate == 2, "timecode supports 1 or 2 events/s");
         require(config.audio_timecode != "600" || config.event_rate == 1, "600 baud requires 1 event/s");
@@ -264,6 +266,7 @@ nlohmann::json config_settings(const Config& config)
     return {
         {"AVSYNC_ROLE", config.role},
         {"AVSYNC_AUDIO_TIMECODE", config.audio_timecode},
+        {"AVSYNC_PATTERN", config.pattern},
         {"AVSYNC_EVENT_RATE", std::to_string(config.event_rate)},
         {"AVSYNC_CHIRP_MS", std::to_string(config.chirp_ms)},
         {"AVSYNC_CHIRP_LOW_HZ", std::to_string(config.chirp_low_hz)},
@@ -314,6 +317,7 @@ void apply_settings(Config& config, const nlohmann::json& flat, std::vector<std:
         const std::string value = text(it.value());
         const std::string& key = it.key();
         if (key == "AVSYNC_AUDIO_TIMECODE") next.audio_timecode = value;
+        else if (key == "AVSYNC_PATTERN") next.pattern = value;
         else if (key == "AVSYNC_EVENT_RATE") next.event_rate = event_rate_value(value);
         else if (key == "AVSYNC_CHIRP_MS") next.chirp_ms = static_cast<int>(num(value, key.c_str()));
         else if (key == "AVSYNC_CHIRP_LOW_HZ") next.chirp_low_hz = num(value, key.c_str());
@@ -351,6 +355,7 @@ void apply_settings(Config& config, const nlohmann::json& flat, std::vector<std:
     require(windows_valid(GradeWindows{next.pass_min_ms, next.pass_max_ms, next.warn_min_ms, next.warn_max_ms}), "grade windows");
     require(next.offset_ms >= -500 && next.offset_ms <= 500, "AVSYNC_AV_OFFSET_MS");
     require(next.audio_timecode == "legacy" || next.audio_timecode == "600" || next.audio_timecode == "1200", "AVSYNC_AUDIO_TIMECODE must be legacy, 600 or 1200");
+    require(next.pattern == "test" || next.pattern == "calibration", "AVSYNC_PATTERN must be test or calibration");
     if (next.audio_timecode != "legacy") {
         require(next.event_rate == 1 || next.event_rate == 2, "timecode supports 1 or 2 events/s");
         require(next.audio_timecode != "600" || next.event_rate == 1, "600 baud requires 1 event/s");

app/config.hpp

@@ -13,6 +13,7 @@ struct Config {
     std::string role;
     int event_rate = 2;
     std::string audio_timecode = "1200"; // legacy, 600, 1200
+    std::string pattern = "test";         // generator output: test (the AV sync test signal) or calibration (calibrator board)
     int chirp_ms = 30;
     double chirp_low_hz = 700;
     double chirp_high_hz = 2500;

app/file_tool.cpp

@@ -773,6 +773,7 @@ int run_analyze(const AnalyzeRequest& request, std::string& error)
         reference.sync_state = "unknown";
         const std::string test_id = have_manifest ? manifest.value("test_id", qr ? qr->test_id : "TEST") : (qr ? qr->test_id : "TEST");
         Pairer pairer(detect.period_ms, GradeWindows{}, reference, "FILE", test_id);
+        if (qr) pairer.set_sender_calibration(qr->calibration);
         for (const auto& video : videos) {
             double expected = 0;
             if (have_manifest) {