RFC: First-class Media Workload entity (assembly of Media Functions) via NetBox tags

[znerol2]

Continues the EBU framing in #112. This is an RFC (→ ADR when it converges, per CONTRIBUTING RFC→ADR). Adversarially cross-checked with codex against the whitepaper PDF (3 rounds → PASS).

RFC: First-class Media Workload entity (assembly of Media Functions) via NetBox tags

Category: RFC (→ ADR when converged, per CONTRIBUTING RFC→ADR pipeline)
Author: @znerol2 (with Claude)
Date: 2026-07-08 (rev 3 — codex R2 consistency fixes: #185 state, vertical acceptance)
Amends: ADR-0037 (Media Workloads = NetBox Media Function instance inventory) — as a tag-derived virtual grouping; no NetBox graph object
Touches: ADR-0003 (EBU v2 taxonomy), ADR-0013 (catalog model), ADR-0027 (catalog/instance separation), ADR-0032 (scoped NetBox writer)
Status: Draft for discussion

---

Summary

Introduce a Media Workload as a first-class grouping in DMF: an assembly of
Media Functions that together serve a specific production (EBU DMF Reference
Architecture V2.0: "Media Functions are combined to implement Media Workloads";
Annex B Figure B1). Membership is a derived grouping over existing NetBox
instance records, keyed by a workload:<slug> tag — not a new NetBox graph
object. The videotest workload (mxl-videotestsrc + mxl-videotest-view) is
the first worked example.

The RFC also corrects two vocabulary drifts (loosely "L1/L2") — but narrowly,
per the whitepaper: the six-stage lifecycle is workload-scoped (Fig 1 is titled
Media Workload Lifecycle), and the catalog vertical bug is only the
non-vertical values on media-processing entries, not a claim that Layer-5
functions can't carry a vertical.

Context / problem

Canon agrees with the whitepaper (docs/architecture/DMF EBU Mapping): a Media
Workload = assembly of Media Functions for a production; a Media Function =
a unit acting on media streams/files/objects, provisioned as one or more
Containers.

DMF has no Media Workload entity. ADR-0037 deliberately modelled the
console's "Media Workloads" page as a NetBox-driven Media Function instance
inventory and deferred the assembly (it explicitly declines to model a workload
graph object). As the MXL demo grows past one function, the assembly is the unit
the operator reasons about — its lifecycle, its future switch target, its
resource budget — and there is nowhere to hang those.

Proposal

1. The entity (a derived grouping, not a graph object)

A Media Workload is a named assembly of Media Functions serving one
production. It is derived from instance records; it owns no persisted graph.
It has: a scoped identity, its member functions, and (as a non-persisted
overlay) the flows among them.

2. Membership via NetBox tag — with a durability contract

Each member service carries workload:<slug> alongside app:/lifecycle: tags.
This fits ADR-0037's three-store model (NetBox owns the instance record).

P1 blocker to resolve before implementation — tag durability. The current
launchers PATCH exact tag lists, so a separately-stamped workload: tag is
erased mid-lifecycle. Concretely (dmf-runbooks/roles/mxl): provision PATCHes
netbox_service_tags + netbox_service_monitoring_tags; configure PATCHes
netbox_service_active_tags + netbox_service_monitoring_tags; finalise PATCHes
netbox_service_tags — none preserve a foreign workload: tag. Acceptance
requirement: either (a) make every launcher tag-flip preserve unknown/
non-owned tags and only replace its owned prefixes (lifecycle:* + the
deliberate monitoring stamp), or (b) thread workload through launcher
defaults/catalog metadata so it is re-stamped on every flip. Ship a
backfill/migration test proving workload:videotest survives a full
provision→configure→finalise cycle on both MXL services.

3. Identity & cardinality contract

• Identity is (tenant/site/scope, workload_slug); workload:<slug> is a
  grouping key applied only after the console's existing tenant/site scoping.
• Cardinality: exactly zero or one workload:* per service for v0.2. Two or
  more is invalid/degraded, surfaced as such — never silently multi-member.
• Zero means one thing, chosen now: a service with no workload: tag is
  shown in an explicit unassigned bucket (not a synthetic single-function
  workload). One code path, not two.

4. Flows are an overlay, never the membership source (ADR-0037 preserved)

Membership is the tag. Flow edges are a separate overlay and are never
persisted to NetBox:

• Declared flow contract comes from the catalog (the MXL entries already
  carry mxl_flows.produces/consumes on source/view).
• Observed flow state comes from the sidecar / Prometheus / NMOS at runtime.

Do not derive membership from live flow observation. And do not define a
Workload as only the flows between its members — Annex B Figure B2 shows
workloads interconnecting/feeding each other, so future flow edges may cross
workload boundaries.

5. Worked example — the videotest workload

workload: videotest                (NetBox tag workload:videotest, tenant-scoped)
├── mxl-videotestsrc   — Media Function, role: source   (catalog mxl_flows.produces)
└── mxl-videotest-view — Media Function, role: view      (catalog mxl_flows.consumes)
    └── flow overlay: src → view   (declared in catalog; observed at runtime; not in NetBox)

Two functions + one flow: the smallest demonstrator of the B1 pattern — not a
literal miniature (B1 is a many-role, multi-vendor production with UI surfaces and
outputs).

6. Console (reframes L1) — with a rollup caveat

The "Media Workloads" page becomes workload-first: top level lists Workloads;
each expands to its member Functions (WP-C's tile/table becomes per-function
detail inside a workload). The workload shows an EBU lifecycle stage; each
function keeps its run-state.

• Lifecycle is a console rule, softened: Fig 1 is the Media Workload
  Lifecycle, but the whitepaper also defines function-orchestration activities
  (identify → download/update → assign licences → launch → configure →
  stop/release). So: a function-instance lifecycle:* tag is desired/run intent;
  the six-stage journey is the workload's. This is a useful console framing, not
  an absolute ontology claim.
• Derivation has limits: running/failing cannot yield Design/Plan/ Finalise & Review. v0.2 derives only the stages telemetry can support (roughly
  Provision/Configure/Operate) and shows the rest as declared or unknown — or
  defers workload-stage derivation entirely until instance-level telemetry exists.
• Rollup blocker (P2): dmf-cms/media_workloads.py currently rolls observed
  state up by Prometheus app label and collapses multiple instances per app.
  That cannot drive per-workload health/lifecycle once two instances of one
  function key live in different workloads. Acceptance gate: workload grouping
  and health/lifecycle rollups must join by NetBox service/instance identity,
  or the RFC must explicitly defer lifecycle derivation until instance-level
  telemetry lands.

7. Classification (reframes L2) — narrowly

Verticals are cross-cutting and apply at every layer (mapping doc §verticals;
the whitepaper's Media Functions chapter itself has Orchestration/Control/
Monitoring/Security subsections). So:

• Layer-5 support/control functions may legitimately carry a vertical —
  nmos-cpp (layer:5, orchestration) and nmos-crosspoint (layer:5,
  control) are not invalid.
• The bug is narrower: the media-processing entries carry non-vertical values
  (mxl-videotestsrc/view → media-functions; mxl-hello → media-processing).
• Rule: a Layer-5 media-processing function gets layer:5 + a
  media_function_type/role (source, view, mixer, output, render, …) and
  omits vertical; the catalog schema check enforces this without breaking
  the legitimate support-function verticals.
• mxl-hello is a separate case: it is layer:4, vertical:media-processing
  — decide whether it is reclassified (layer + type) or merely has its invalid
  vertical removed. Call it out explicitly in the migration.

Consequences

• Positive: honest vocabulary; a real home for the lifecycle, the future
  switch target, and L3 preflight/rollback; the catalog stops mis-tagging
  media-processing functions; minimal new infrastructure (one tag).
• Negative: launcher tag-flip hardening (P1) is a prerequisite, not free; a
  console IA change (workload-first) and a catalog schema migration; a one-time
  backfill of the two MXL services.
• Neutral: Flows stay runtime/overlay (ADR-0037 preserved); single-membership
  assumed for v0.2.

Alternatives considered

• Catalog-level workload manifest (declare members in YAML) — richer, but
  adds a new store/authority duplicating NetBox's instance record. Rejected for
  the demo; revisit if workloads need design-time definition before deployment.
• Keep the page as a function inventory — leaves the conflation and gives the
  lifecycle/switch/budget nowhere to live. Rejected.

Open questions

1. Single vs multi workload membership (proposal: single, 0-or-1, for v0.2).
2. Exactly which lifecycle stages v0.2 derives vs shows as declared/unknown.
3. Console IA: rename the per-function surface to "Media Functions" and reserve
   "Media Workloads" for the assembly? Backwards-compatible /api/media-workloads
   payload transition vs a new grouping endpoint?
4. Does the launcher preserve foreign tags (option a) or re-stamp workload
   (option b)? (a) is more general; (b) is more explicit.

Scope / non-goals

• In: the derived entity, tag membership + durability contract, identity/
  cardinality, flow-as-overlay, the videotest bundle, workload-first console
  grouping, the narrow L1 and L2 reframes, the catalog schema check.
• Out: composition-graph authoring (a B1 editor), multi-vendor orchestration,
  Design/Plan authoring, switching (the v0.2b connection-intent seam), licence
  enforcement.

Relationship to #185 / v0.2a

This supersedes the loose "L1/L2" additions and pulls them into this RFC → ADR.
State (resolved 2026-07-08): the A–E demo set is complete (dmf-cms#26 merged),
so #185 is closed; a fresh tracking issue #189 (v0.2 EBU re-anchor) now
carries L3 + this Media Workload entity (+ reframed L1/L2) + v0.2b + licensing, and
the re-anchor plan's tracking_issue was moved to #189. No active plan points at a
closed issue. This RFC's own implementation work opens its own issue under #189
once it converges to an ADR.

[znerol2]

My lean: adopt this RFC, with two constraints that should be explicit in the ADR:

1. Keep workload membership single for v0.2.
2. Treat the launcher tag-preservation change as a hard prerequisite, not implementation detail.

Reasoning, anchored on the EBU whitepaper as north star:

• The whitepaper is unambiguous that a Media Workload is the production-level assembly, and that Media Functions are combined to implement that workload. That means ADR-0037's current page shape is honest as a function-instance inventory, but not enough as the long-term meaning of "Media Workloads".
• The six-stage lifecycle belongs primarily to the Media Workload. A function instance can still carry lifecycle intent/run state (lifecycle:bootstrapped, lifecycle:active, observed running/failing), but the console should not pretend that a single function instance has enough semantic weight to be in Design, Plan, or Finalise & Review.
• The whitepaper also defines Media Function orchestration activities, so I would avoid an overcorrection like "functions have no lifecycle". Better wording: function lifecycle tags are desired/run intent; workload lifecycle is the production journey.

My answers to the open questions:

1. Single vs multi workload membership

Use exactly zero or one workload:* tag per service for v0.2. Zero goes to an explicit unassigned bucket. More than one is degraded/invalid and should be visible as such.

If a future function genuinely serves multiple productions, I would model that through connection/flow intent or explicit shared-resource semantics, not by making one instance silently belong to multiple workloads. Shared support/control systems like NMOS should remain shared facility/support functions unless deliberately instantiated into one production workload.

2. Which lifecycle stages v0.2 derives

Derive only stages the current signals can support:

• Provision: member services exist / are bootstrapped.
• Configure: active intent exists but observed health/declared-flow readiness is incomplete or mixed.
• Operate: required member instances are active and observed healthy; declared flow overlay is present or at least not failing.

For Design, Plan, and Finalise & Review, show declared/unknown, or defer entirely. Do not infer finalise from absence. Do not ship workload-stage derivation off the current Prometheus app rollup; once two instances of the same function key live in different workloads, health must join by NetBox service/instance identity.

3. Console IA / API

Yes: reserve Media Workloads for the assembly. Rename the flat per-function surface to Media Functions or Function Instances.

API preference: do not silently reshape the existing /api/media-workloads payload in place. Add the grouped API first, migrate the UI, then deprecate the flat shape. Clean end state:

• /api/media-workloads = grouped workload-first payload.
• /api/media-function-instances = flat instance inventory.

During migration, either keep the current flat /api/media-workloads response briefly or expose /api/media-workloads/grouped until frontend/tests are moved. But the ADR should state the target split.

4. Preserve foreign tags vs re-stamp workload

Choose option a: launchers preserve foreign/non-owned tags. The launcher owns lifecycle replacement plus deliberate static/monitoring stamp domains; it should not erase workload:*.

Threading workload through launcher defaults is acceptable as a seed/backfill convenience for videotest, but not as the authority. Workload membership belongs to the workload/design/NetBox layer, not to each function launcher.

Last point: I agree with the narrow vertical correction. The EBU verticals are cross-cutting, so nmos-cpp as orchestration and nmos-crosspoint as control are legitimate. The actual catalog bug is the media-processing entries using non-vertical pseudo-values; those need a function type/role field instead.