RFC: Switch a media function's input with NMOS IS-05 through nmos-cpp sidecars

[znerol2]

Summary

dmfdeploy is an independent, open prototype of a Dynamic Media Facility. It proves its claims with conformance checks against published specifications and shares what it learns upstream as evidence. It does not call itself a reference.

This RFC proposes how the v0.2 workload switches a viewer between two MXL flows with NMOS IS-05. It proposes a small nmos-cpp node beside each MXL source and the viewer, and an MXL-capable controller that sends the IS-05 requests. It also proposes a test that can prove the design wrong before anyone builds the rest.

Tracking issues: #624 (the test), #627, #628, #629 (the pieces). The milestone outcome is in docs/WORKING-MODEL.md.

The problem

The v0.2 outcome is a multi-function workload switched by NMOS IS-05. Today the console switches a source through an AWX playbook that reconnects and re-points the pod. That path works, but no receiver is controlled through IS-05. A control-plane matrix built on it would look like NMOS and behave like a restart.

BCP-007-03 defines how IS-04 and IS-05 describe MXL. In that spec:

• MXL senders and receivers use the transport urn:x-nmos:transport:mxl.
• A receiver takes mxl_domain_id and mxl_flow_id parameters.
• There is no RTP leg, no SDP, and no transport file. The /transportfile endpoint returns 404.
• Activating a receiver with master_enable true starts the MXL read.

The project needs a design that follows this spec, runs on one ARM64 node, and keeps the shipped reconnect path as a fallback.

Proposed decision

Use a small dmfdeploy-owned nmos-cpp node (a sidecar) beside each existing MXL source and beside the viewer. Pair it with an MXL-capable NMOS controller, pinned by image digest and source commit. #629 records the controller choice.

The console's new nmos-is05 actuator sends the controller's authenticated makeConnection command. It never sends a PATCH to the receiver itself. The console and the controller UI then drive the same chain:

console or controller UI
  -> controller makeConnection
  -> IS-05 PATCH on the receiver, /single/receivers/<id>/staged
  -> viewer sidecar observes the resolved /active
  -> local supervisor replaces only the preview child process with the new flow ID

Details:

• Source sidecars register IS-04 Node, Device, Source, Flow and Sender resources. The viewer sidecar registers Node, Device and Receiver resources.
• Pin an nmos-cpp commit that contains the upstream MXL pull requests #483, docs(plans): shell round 2 — rail band and message bus #503 and dmf-cms: in-flight announcements are hand-wired per call site — three paths missed it, and omission is undetectable #521. Picking and proving that commit is part of [dmf-media] Package an NMOS MXL node sidecar for sources and the viewer #627.
• The first scope is one MXL domain on one ARM64 node. IS-05 selects an MXL flow that is already readable in the viewer's domain. It does not start or retarget a Fabrics transfer.
• The viewer's preview process restarts on each switch. The pod, the MXL domain, the status endpoint and the console tile stay up. The test measures the interruption and the console shows it as "switching". Nobody claims the switch is glitch-free.
• The shipped reconnect actuator stays selectable as the default fallback until this path passes end-to-end acceptance.

Why the process restarts

The viewer reads its flow ID once, when it starts. We expect to change the ID by restarting the preview process, not by retuning it while it plays. We have not confirmed this against the MXL GStreamer elements, so the restart is part of what #624 tests.

Work breakdown

The items depend on each other in this order. Each has an acceptance test in its issue.

1. An ARM64 fixture with two source flows and one viewer ([dmf-media] Prove IS-05 switching on one node with an A/B MXL fixture #624). It comes first.
2. Pin and build an ARM64 nmos-cpp image, with digests and a provenance attestation.
3. Package the sidecar ([dmf-media] Package an NMOS MXL node sidecar for sources and the viewer #627). IS-05 /constraints, /staged and /active validate against IS-05 v1.2.
4. Viewer activation controls only its local consumer ([dmf-media] Viewer activation controls only its local consumer #628).
5. Adopt the controller behind a compatibility slice ([dmf-media] Adopt the MXL-capable crosspoint controller behind a compatibility slice #629).
6. A backend-only controller identity with least privilege: one allowed action, never held by the browser.
7. The console nmos-is05 actuator.
8. Fallback, audit and rollback ownership. On a failed activation, the actuator re-selects the previous flow through the same controller route and confirms it by reading /active. A failed switch shows as failed_rollback_required, never as success.

Items 6 to 8 count toward the v0.2 outcome only after role gating (#186, #389, #436) and per-workload operation scoping land.

Alternatives considered

• A GStreamer-plugin NMOS node (receiver as a pipeline element). It stays closer to the spec's concepts. We rejected it for the next demo because its documented build profiles do not cover ARM64, and it replaces the known preview path with a new runtime. It stays the fallback if the early test fails.
• A controller shim over the existing AWX reconnect. Its PATCH would trigger the coarse reconnect. No receiver would be controlled through IS-05, so the matrix would look real and the old mechanism would stay underneath. This fails the goal.
• The upstream nmos_crosspoint at commit 564ca41. At that commit it handles RTP transports and expects an SDP, so it cannot compose MXL transport parameters. We need a controller that handles MXL, hence [dmf-media] Adopt the MXL-capable crosspoint controller behind a compatibility slice #629.

Consequences

• The design follows the BCP-007-03 data model, so a different controller or node implementation needs no redesign.
• dmfdeploy owns a new runtime piece: the bridge between the sidecar and the supervisor. It is not a thin adapter.
• Pinning a community controller line adds a dependency. The compatibility slice in [dmf-media] Adopt the MXL-capable crosspoint controller behind a compatibility slice #629 is the boundary. Changing the line must pass that slice. It is not a values-only change. No dmfdeploy code imports controller internals.
• dmfdeploy carries no private patch. It shares reproducible traces, version and digest matrices, and narrow defect reports with the owning projects. It does not promise support or long-term maintenance of anyone else's code.

What would prove this wrong

The design is wrong if nmos-cpp accepts an MXL PATCH and reports it in /active, but the sidecar cannot reliably drive the existing consumer to the selected flow, or the preview restart is too slow or too unreliable. Then the control plane only looks compliant.

#624 tests this first. On one ARM64 node, run the new sidecar, two existing test-pattern sources and the current viewer in one MXL domain. Send the IS-05 PATCH directly to the receiver, select flow A, then flow B, and assert within a bounded interval:

1. /staged and the resolved /active hold the selected mxl_domain_id and mxl_flow_id.
2. The pod UID stays the same. Only the preview process is replaced.
3. The preview image changes from A's test pattern to B's.
4. The sidecar keeps heartbeating its registration.

A negative case changes /active without the supervisor and must fail the preview assertion. If #624 fails, we stop and decide again. We do not switch designs automatically. The test has a time box of about five working days.

The AMWA nmos-testing suites for IS-05 and BCP-007-03 check API behavior only. We plan to run them as a second gate, in warn mode first, pinned by image digest because the BCP-007-03 suites have no tagged release. These decisions are recorded on #624.

Questions for discussion

1. Which exact nmos-cpp commit should we pin, and does it build and run natively on ARM64?
2. Can both flows be readable in the viewer's domain before activation? If not, cross-node transfer is a separate requirement that this design does not hide.
3. What interruption target is acceptable for a preview restart, and how should the console tell a slow switch from a failed one?
4. Where should the controller credential live, and how is it rotated?
5. Which network rules let the controller read the sender's /active and send PATCH requests to the receiver's control address?
6. Does the status endpoint need a control socket to the supervisor, or is a static environment value enough to report the selection?
7. Should the restart-per-switch approach be raised with the MXL maintainers as a request for live retuning?
8. Is it right to run the AMWA conformance suites as a second gate, not as the primary test?

How this gets decided

Discussion here shapes the design. After #624 reports, a decision record (ADR) captures the outcome, following the RFC to ADR path. Related decisions: ADR-0046 and ADR-0047.

Evidence

• BCP-007-03
• nmos-cpp architecture
• nmos-cpp MXL pull request #483
• nmos_crosspoint issue #14
• dmfdeploy issues [dmf-media] Prove IS-05 switching on one node with an A/B MXL fixture #624, [dmf-media] Package an NMOS MXL node sidecar for sources and the viewer #627, [dmf-media] Viewer activation controls only its local consumer #628, [dmf-media] Adopt the MXL-capable crosspoint controller behind a compatibility slice #629

[znerol2]

Flow rendezvous across nodes and platforms

This extends question 2 in the RFC: how does a selected flow become readable in the viewer's domain when the writer is elsewhere? The single-node IS-05 decision stays as written. This follow-up asks whether and how dmfdeploy should add cross-node replication, and what boundary would let a separately managed platform participate.

What we know

• The A/B fixture passed on ARM64 with both source flows in one shared domain. It proved the viewer can follow an IS-05 activation; cross-node rendezvous remains open.
• Issue #654 assessed Qvest's mxl-k8s: a Kubernetes operator, per-node agents and gateways that coordinate MXL Fabrics mirrors. Its earlier decision deferred adoption for v0.2 and called for a later cross-node decision. The issue also records two upstream contributions.
• The existing dmfdeploy Fabrics demo uses its own initiator, target, and Kubernetes ConfigMap handshake. The historical MXL review records a cross-host ARM64 TCP demonstration. Neither proves cross-vendor interoperability or RDMA performance.
• BCP-007-03 defines NMOS resources and transport parameters for MXL, including replicated flows in different domains. It does not define the complete cross-platform Fabrics setup. MXL v1.1 includes inter-host Fabrics replication, so ADR-0017 needs a factual review.

Candidate direction

Evaluate mxl-k8s as an optional Media Exchange component inside a dmfdeploy-managed cluster. Keep NMOS connection intent, the function lifecycle, Kubernetes scheduling, and flow replication under explicit ownership. Compare it with the existing demo and a no-cross-node baseline before choosing an implementation.

This is an evaluation proposal. The current operator decision still defers adoption in v0.2. The evaluation would not itself change that decision, the v0.1 freeze, or the committed AWX actuator. An adopted design may need an ADR-0017 amendment and a review of the AWX/NetBox boundaries in ADR-0037.

Questions

1. When does the product need cross-node MXL, and what remains viable on one node? Should intra-cluster replication be an optional profile?
2. If IS-05 selects a flow, which component requests a mirror, reports readiness and tears it down? How do we avoid conflicting desired state between NMOS, AWX and the mxl-k8s operator?
3. Can the node-shared tmpfs and hostPath model meet the required tenant and workload isolation? What privileges, Kubernetes RBAC, networking and resource limits does the selected release actually need?
4. Which pinned MXL and mxl-k8s versions, images and architectures are compatible? The Qvest build chain uses a downstream MXL fork; compare its exact changes and image provenance. Track dmfdeploy image provenance work in #745.
5. What should be exposed to monitoring and operators when a mirror cannot be created, loses its producer, falls behind or recovers?
6. For a vendor-managed platform, what interface should dmfdeploy consume for discovery and control? Can the vendor retain its own nodes, scheduling and internal replication while exposing NMOS and a mutually supported media transport?
7. For replication between independently managed platforms, is ST 2110/AES67 the appropriate boundary, or is there a concrete need to evaluate direct MXL Fabrics? What common signaling, versioning, transport capabilities, network and security agreements would make the latter interoperable? Running mxl-k8s on both sides does not establish that agreement.

Evidence before a decision

On an isolated two-node ARM64 k3s environment, compare a pinned mxl-k8s release with the existing TCP demo. Record image digests and MXL versions, CPU and memory cost, placement and isolation, flow visibility, mirror readiness, writer/gateway restart recovery, teardown and failure reporting. Test an NMOS A/B selection against a flow that must be mirrored, including the case where replication cannot be established. A separate RDMA test needs suitable hardware and a stated network profile; no RDMA or cross-vendor claim follows from the TCP test.

The separate inter-platform question needs a named partner and a concrete interface to test. A standards-based media connection is one candidate. Direct Fabrics needs an agreed setup mechanism and a cross-implementation test before any interoperability claim.

What evidence or constraints would change this direction? In particular, is there already a suitable platform-level interface in the MXL or AMWA work that this proposal should follow?