1.0.5: interlaced video follows the shared field-per-grain convention

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 1.0.5
+
+Interlaced video (`PAL`, `NTSC`, `HD1080i50`, `HD1080i5994`) now follows the shared convention of the MXL media functions (SPECIFICATION §3.2.2): one field per grain at twice the frame rate, grain 2k = the first field in time of TAI frame k, grain 2k + 1 = the second; the flow definition keeps `grain_rate` = the frame rate and the real `interlace_mode`. Progressive modes are unchanged.
+
+- **Input (SDI → MXL):** an interlaced capture wrote one grain per frame at the field-rate index, holding frame lines 0–539 of the woven frame (the top half with both fields, cut to the grain size); every odd index stayed invalid. Every reader that follows the convention (multiviewer, webrtc-monitor, srt and 2110 egress) showed the top half stretched, with combing. Now the frame index k comes from the frame's start (one frame period before its timestamp) at the frame rate, and the bridge writes grain 2k from the first field's lines and 2k + 1 from the other lines, by row (v210, and 8-bit YUV after the expansion; for `video/v210a` the fill lines). `mxl_decklink_grains_committed_total` and `grains_committed` count grains: two per interlaced frame, one per progressive frame; `frames_total` counts frames.
+- **Field order:** `NTSC` is lower field first (`bmdLowerFieldFirst`), so its flow definition and NMOS flow and receiver caps say `interlaced_bff`; they said `interlaced_tff` for every interlaced mode. `PAL` and 1080i stay `interlaced_tff`.
+- **ANC of an interlaced input** runs at the field rate: the `video/smpte291` flow is declared at twice the frame rate (50/1 for 1080i50, 60000/1001 for 1080i59.94), field 1's packets go to grain 2k and field 2's to grain 2k + 1, the index of the video field, with the RFC 8331 `F` bits 0b10 (field 1) and 0b11 (field 2), as mxl-test-player 1.2.3 does; progressive grains keep 0b00. When only one field of a frame has packets, the other gets an empty grain; a frame without ANC packets writes no ANC grain. The field follows from the packet's SMPTE line number (line 0, unknown, counts as field 1). The NMOS ANC flow (`grain_rate`) and the ANC receiver caps follow the MXL flow definition: the field rate for interlaced modes (video keeps the frame rate). Before, the packets were written at the field-rate index into a data flow declared at the frame rate, about twice the TAI index, so readers asking for the current index found nothing. The output does not read ANC, so there is no reader to change.
+- **Output (MXL → SDI):** an interlaced flow was read one field per output frame into the top half of the DeckLink frame (the rest stale), the read position advanced at the field rate (half of what the source writes, so it fell behind and skipped ahead), and the audio per frame was half a frame (960 samples at 1080i50). Now the reader reads frame k as grains 2k and 2k + 1 and weaves them with line parity `(index & 1) ^ bff`; read position, preroll, read offset, catch-up and slips count frames, the sync-group wait waits for the second field's time, and the audio per frame is 48000 / frame rate (1920 at 25 frames/s). A frame with an invalid field repeats the last frame (`source_invalid`), without waiting for the second field when the first is invalid. A flow definition that cannot be read is read as progressive and logged (`mxl_flow_def_unreadable`, warning). A re-created flow that changes between progressive and interlaced restarts the channel. `output_channel_started` logs `interlaced`.
+- **Web UI:** the MXL tab now maps an interlaced flow to its mode: 1080 lines at 25/1 to `HD1080i50`, at 30000/1001 to `HD1080i5994`, 576 lines at 25/1 to `PAL`, 486 lines at 30000/1001 to `NTSC`. It looked for `HD1080i25` and `HD1080i2997`, which do not exist, and offered no mode.
+- **Mock card:** in interlaced modes the counter band of the odd lines is mid grey, so each field can be told apart; new `MOCK_PLAYBACK_FRAME_FILE` writes every 25th frame an output is given to a file.
+- New unit tests: field split and weave for tff and bff (every line back in place), the frame index from the frame's start and the field timestamps at 1080i50 and 1080i59.94, the newest complete frame of a field head, ANC line → field, an interlaced flow written field by field and read back as woven frames (PAL tff, NTSC bff), an output playing an interlaced flow at the frame rate with woven frames and a frame's worth of audio, an input channel capturing PAL from a fake card (field grains at 2k and 2k + 1, two grains per frame, ANC per field at 50/1 with F bits), the RFC 8331 F bits, and a frame with an invalid first field returned at once.
+
 ## 1.0.4
 
 - An output channel re-opens its MXL readers when the writer re-creates a flow under the same id. MXL then gives the flow a new data file, and a read past the old head returns `MXL_ERR_FLOW_INVALID` (dmf-mxl/mxl@218ddaa `lib/internal/src/PosixDiscreteFlowReader.cpp:96-104`, `244-254`). Before, the channel treated that like any failed read: it repeated the last frame and counted a drop, but every read of the stale reader first waited `CHx_READER_TIMEOUT_MS` twice (sync group, then grain). Only a preroll reset (25 late or dropped DeckLink frames in 10 s) re-opened the reader; when the card did not report late frames, the SDI output stayed frozen until the channel was restarted. Now the callback takes the reader out of the sync group and stops reading it; the supervisor releases it and opens it again for the same flow id at once, then after 100 ms, 200 ms, 500 ms and every 1 s while the flow is missing. Video repeats the last frame and the re-created audio flow plays silence until the new reader is open; these repeats count in `mxl_decklink_frames_repeated_total{reason="flow_reopen"}` and never trigger a preroll reset. A flow missing for more than 1 s marks the channel `degraded` until it is back. A re-created flow with another format, rate, line size or channel count restarts the channel instead. New counter `mxl_decklink_flow_reopens_total` (one per re-created flow) and log events `mxl_flow_recreated` (once per event), `mxl_flow_reopen_failed` (first failed open of an event), `mxl_flow_reopened` (with `attempts` and `after_ms`) and `mxl_flow_recreated_format_changed`. mxl-st2110-gateway, FlowXer, the multiviewer and replay already re-open on this error.

CMakeLists.txt

@@ -1,7 +1,7 @@
 cmake_minimum_required(VERSION 3.24 FATAL_ERROR)
 
 project(mxl-decklink
-    VERSION 1.0.4
+    VERSION 1.0.5
     DESCRIPTION "MXL MediaFunction container for Blackmagic DeckLink"
     LANGUAGES CXX
 )
@@ -112,6 +112,7 @@ add_library(mxl-decklink-core STATIC
     src/util/logging.cpp
     src/util/taiclock.cpp
     src/util/v210.cpp
+    src/util/interlace.cpp
     src/util/audioconv.cpp
     src/util/anc.cpp
     src/util/uuid.cpp
@@ -205,6 +206,7 @@ if(MXL_DECKLINK_BUILD_TESTS)
         tests/unit/test_route_state.cpp
         tests/unit/test_flowsync.cpp
         tests/unit/test_flow_reopen.cpp
+        tests/unit/test_interlace.cpp
     )
     target_link_libraries(unit-tests PRIVATE mxl-decklink-core)
     add_test(NAME unit-tests COMMAND unit-tests)

README.md

@@ -312,7 +312,11 @@ The build always contains a deterministic **mock DeckLink backend**
 counter band, a 1 kHz tone, an ANC test packet, TAI-paced callbacks, and
 scriptable fault injection (`MOCK_SIGNAL_LOSS_AFTER_FRAMES`,
 `MOCK_FORMAT_CHANGE_AFTER_FRAMES`, `MOCK_SUBDEVICE_COUNT`). This drives the
-identical channel/MXL code paths as real hardware.
+identical channel/MXL code paths as real hardware. In interlaced modes the
+counter band of the odd lines is mid grey, so the two fields of a frame can be
+told apart. `MOCK_PLAYBACK_FRAME_FILE=<path>` makes the mock output
+write every 25th frame it is given to that file, to check what an output
+plays.
 
 ```bash
 # unit tests

SPECIFICATION.md

@@ -73,7 +73,7 @@ Then the MXL sequence runs: `mxlFlowWriterOpenGrain(inst, writer, grainIndex, &g
 
 In parallel, `audioPacket->GetBytes(&pcm)` is read. Because DeckLink delivers interleaved PCM (`bmdAudioSampleType32bitInteger`, 48 kHz) and MXL expects `audio/float32` deinterleaved per channel in a `channels` blob, the bridge converts in flight: for each sample, `float32 = int32 / 2147483648.0f`, deinterleaving into an `mxlMutableWrappedMultiBufferSlice`. The slice structure addresses ring-buffer wraparound via two fragments. `mxlFlowWriterOpenSamples` / `CommitSamples` open and finalize the batch.
 
-For ANC data, `IDeckLinkVideoFrameAncillaryPackets` is iterated and written into a `video/smpte291` flow per RFC 8331 §2 (from the length field onward) when the channel has ANC enabled. The callback must complete well under the frame interval (at 1080p50: <20 ms) — therefore no blocking operations inside it.
+For ANC data, `IDeckLinkVideoFrameAncillaryPackets` is iterated and written into a `video/smpte291` flow per RFC 8331 §2 (from the length field onward) when the channel has ANC enabled (one grain per field for interlaced modes, §3.2.2). The callback must complete well under the frame interval (at 1080p50: <20 ms) — therefore no blocking operations inside it.
 
 ### 2.4 Data Flow, Output Path (MXL → DeckLink)
 
@@ -143,6 +143,16 @@ With an explicit `CHx_VIDEO_MODE` (e.g. `HD1080p50`, `4K2160p50`), the container
 
 The container maintains this mapping internally. Essential entries (non-exhaustive): `HD720p50 → bmdModeHD720p50`, `HD720p5994 → bmdModeHD720p5994`, `HD1080i50 → bmdModeHD1080i50`, `HD1080p24 → bmdModeHD1080p24`, `HD1080p25 → bmdModeHD1080p25`, `HD1080p2997 → bmdModeHD1080p2997`, `HD1080p50 → bmdModeHD1080p50`, `HD1080p5994 → bmdModeHD1080p5994`, `4K2160p25 → bmdMode4K2160p25`, `4K2160p50 → bmdMode4K2160p50`, `4K2160p5994 → bmdMode4K2160p5994`, `4K2160p60 → bmdMode4K2160p60`, `8K4320p50 → bmdMode8K4320p50`, `8K4320p60 → bmdMode8K4320p60`. Modes not in the table are rejected.
 
+#### 3.2.2 Interlaced Modes (`PAL`, `NTSC`, `HD1080i50`, `HD1080i5994`)
+
+The container follows the shared convention of the MXL media functions (MXL `218ddaa`). The flow definition keeps `grain_rate` = the frame rate (25/1 or 30000/1001) and the real `interlace_mode`; MXL then stores **one field per grain** (height / 2 lines) at twice that rate. Grain 2k is the first field in time of TAI frame k (k = `mxlTimestampToIndex` at the frame rate), grain 2k + 1 the second. `interlaced_tff`: the first field is the top field, the even frame lines (0, 2, 4, …); `interlaced_bff`: the bottom field, the odd lines. A reader weaves grain i into the frame lines of parity `(i & 1) ^ bff`. MXL has no per-grain field marker, so a reader cannot detect a writer that puts the first field on an odd index.
+
+- **Field order** comes from the mode table: `NTSC` is lower field first (`bmdLowerFieldFirst` in `IDeckLinkDisplayMode::GetFieldDominance`), so its flow is `interlaced_bff`; `PAL`, `HD1080i50` and `HD1080i5994` are `interlaced_tff`. The MXL flow definition and the NMOS flow and receiver caps carry it.
+- **Counters:** `mxl_decklink_grains_committed_total` (and `grains_committed` in `/statusz`) counts MXL grains: two per frame for an interlaced input, one for a progressive input. `mxl_decklink_frames_total` counts frames in both.
+- **Input:** k comes from the frame's start, one frame period before its timestamp (§3.5; the frame arrives when it is complete). The bridge writes grain 2k from the first field's lines and grain 2k + 1 from the other lines of the woven DeckLink frame, by row (v210 directly, 8-bit YUV after the expansion of §3.3; for `video/v210a` the fill lines).
+- **ANC** of an interlaced input runs at the **field rate**: the `video/smpte291` flow is declared at twice the frame rate (50/1 for 1080i50, 60000/1001 for 1080i59.94). The packets of field 1 go to grain 2k and those of field 2 to grain 2k + 1, the index of the video field, with the RFC 8331 `F` field 0b10 (field 1) or 0b11 (field 2); progressive grains keep `F` = 0b00. When only one field of a frame has packets, the other field gets an empty grain; a frame without ANC packets writes no ANC grain. The field follows from the packet's SMPTE line number: 1080i lines 564–1125, 625-line lines 313–625 and 525-line lines 266–525 and 1–3 are field 2; line 0 (unknown) counts as field 1. The NMOS ANC flow and the ANC receiver caps carry the same field rate as the MXL flow definition (video keeps the frame rate). Progressive ANC stays at the frame rate.
+- **Output:** a flow whose `interlace_mode` is `interlaced_tff` or `interlaced_bff` is read as frames: for frame k the channel reads grains 2k and 2k + 1 and weaves them into the DeckLink frame. Every frame count of §2.4 (read position, preroll, read offset, catch-up, slips, the sync-group wait, which waits for the second field's time) is in frames, and the audio per frame is 48000 / frame rate samples (1920 at 25 frames/s). A frame with an invalid field is repeated like an invalid grain (the second field is not waited for when the first is invalid). A flow definition that cannot be read or parsed is read as progressive with the warning `mxl_flow_def_unreadable`. The output does not read ANC.
+
 ### 3.3 Pixel Format Mapping
 
 MXL v1.0 supports exactly two video formats: **`video/v210`** (10-bit YUV 4:2:2, Apple/SMPTE v210 layout) and **`video/v210a`** (v210 plus alpha, for fill+key). This is fortunately congruent with the DeckLink standard format `bmdFormat10BitYUV`, which uses the same bit packing (6 pixels in 16 bytes, 128-byte row alignment). The container therefore uses **`bmdFormat10BitYUV` as the default**. `IDeckLinkOutput::RowBytesForPixelFormat` is used to compute row bytes instead of hand-rolled formulas.
@@ -165,7 +175,7 @@ In mode **`MXL_TIMESTAMP_SOURCE=hardware`**, the container uses `IDeckLinkVideoI
 
 In mode **`MXL_TIMESTAMP_SOURCE=host`**, the container takes `clock_gettime(CLOCK_TAI)` at callback entry as the frame timestamp — simpler, but subject to callback jitter (typically ±0.5 ms).
 
-The grain index follows from `mxlTimestampToIndex(&grainRate, tai_ns)`. For signal-loss frames (`bmdFrameHasNoInputSource` set), no grain is committed; a gap in the ring buffer is accepted so readers detect the outage via `MXL_GRAIN_FLAG_INVALID` or the missing sample index.
+The grain index follows from `mxlTimestampToIndex(&grainRate, tai_ns)`; interlaced modes write two field grains from the frame index of the frame's start (§3.2.2). For signal-loss frames (`bmdFrameHasNoInputSource` set), no grain is committed; a gap in the ring buffer is accepted so readers detect the outage via `MXL_GRAIN_FLAG_INVALID` or the missing sample index.
 
 ### 3.6 Behavior on Signal Loss
 
@@ -409,7 +419,7 @@ Each channel is in one of: `init` (0), `healthy` (1), `degraded` (2, e.g. signal
 
 All channel-scoped Prometheus metrics carry the labels `card_id`, `channel_index`, `channel_label`, `direction`, `subdevice_index` (plus `flow_id` where applicable). Core set:
 
-- Counters: `mxl_decklink_frames_total`, `mxl_decklink_frames_dropped_total` (inputs: frames lost; outputs: frames the card dropped or that could not be scheduled, output-side trouble only), `mxl_decklink_frames_late_total` (outputs: frames the card displayed late), `mxl_decklink_frames_repeated_total` (outputs, label `reason`: `source_stall` the source stopped writing, `source_late` its grain came after the read deadline and the read position slipped one frame, `flow_reopen` the flow is being re-opened, `read_error` an invalid grain or another read error; §2.4), `mxl_decklink_source_stalls_total` (outputs, label `essence` `video`/`audio`: a reader's source stopped writing, §2.4), `mxl_decklink_signal_lost_total`, `mxl_decklink_format_changes_total`, `mxl_decklink_reconnect_total`, `mxl_decklink_flow_reopens_total` (output: flows re-created by their writer whose reader was re-opened, §2.4), `mxl_grains_committed_total`, `mxl_grains_written_bytes_total`
+- Counters: `mxl_decklink_frames_total`, `mxl_decklink_frames_dropped_total` (inputs: frames lost; outputs: frames the card dropped or that could not be scheduled, output-side trouble only), `mxl_decklink_frames_late_total` (outputs: frames the card displayed late), `mxl_decklink_frames_repeated_total` (outputs, label `reason`: `source_stall` the source stopped writing, `source_late` its grain came after the read deadline and the read position slipped one frame, `flow_reopen` the flow is being re-opened, `read_error` an invalid grain or another read error; §2.4), `mxl_decklink_source_stalls_total` (outputs, label `essence` `video`/`audio`: a reader's source stopped writing, §2.4), `mxl_decklink_signal_lost_total`, `mxl_decklink_format_changes_total`, `mxl_decklink_reconnect_total`, `mxl_decklink_flow_reopens_total` (output: flows re-created by their writer whose reader was re-opened, §2.4), `mxl_grains_committed_total`, `mxl_grains_written_bytes_total` (inputs: per MXL grain, so two per frame for an interlaced mode, one per frame for a progressive mode; `mxl_decklink_frames_total` counts frames in both cases)
 - Gauges: `mxl_decklink_channel_state`, `mxl_decklink_signal_lock` (0/1), `mxl_ptp_locked` (IP cards), `mxl_ringbuffer_headindex`, `mxl_ringbuffer_reader_lag_grains`, `mxl_buffered_video_frames`, `mxl_buffered_audio_samples`, `mxl_flow_writer_active`, `mxl_active_video_flow_id` (info label); `mxl_decklink_read_offset_grains` (outputs: `CHx_READ_OFFSET_GRAINS` in effect after the cap, §2.4)
 - Histograms: `mxl_flow_grain_commit_latency_seconds` (buckets 100 µs … 100 ms), `mxl_callback_duration_seconds`
 
@@ -443,7 +453,7 @@ Every setting displays its **provenance** (default / file / environment). Settin
 
 #### 7.5.4 Flow-to-output assignment
 
-The MXL tab allows selecting any **video flow** (and matching audio flow) discovered in the attached domain and assigning it to an output channel. The UI derives the required `CHx_VIDEO_MODE` from the flow descriptor (`frame_width`/`frame_height`/`grain_rate`/`interlace_mode`) and warns when no DeckLink mode of the card matches the flow geometry. The assignment is a normal per-channel configuration change (§7.5.3).
+The MXL tab allows selecting any **video flow** (and matching audio flow) discovered in the attached domain and assigning it to an output channel. The UI derives the required `CHx_VIDEO_MODE` from the flow descriptor (`frame_width`/`frame_height`/`grain_rate`/`interlace_mode`; an interlaced flow's `grain_rate` is its frame rate, so 1080 lines at 25/1 map to `HD1080i50`, 30000/1001 to `HD1080i5994`, 576 lines at 25/1 to `PAL` and 486 lines at 30000/1001 to `NTSC`) and warns when no DeckLink mode of the card matches the flow geometry. The assignment is a normal per-channel configuration change (§7.5.3).
 
 #### 7.5.5 Security
 

deploy/mxl-decklink.yaml

@@ -30,7 +30,7 @@ spec:
         # feature.node.kubernetes.io/pci-1edb.present: "true"
       containers:
         - name: mxl-decklink
-          image: ghcr.io/leeo86/mxl-decklink:1.0.4
+          image: ghcr.io/leeo86/mxl-decklink:1.0.5
           resources:
             # §6.4: Guaranteed QoS — requests == limits, integer CPUs.
             # Sizing per §6.3 (Quad 2, 8× HD half duplex).