Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ pip install pytest
python -m pytest # fast, no GStreamer/Docker needed (tests/ uses a stubbed gi)
```

CI runs the same suite plus `bash -n` on every shell script and a docs-link check
([.github/workflows/ci.yml](.github/workflows/ci.yml)). These tests exist to protect
CI runs the same suite plus `node --test tests/js/*.test.js` and `bash -n` on every
shell script ([.github/workflows/ci.yml](.github/workflows/ci.yml)). There is no
automated docs-link check yet — if you touch docs, click through the links you changed. These tests exist to protect
the hard-won media core (the restamp/conform behaviour in
[`tools/contribution_core.py`](tools/contribution_core.py)) from well-meaning cleanup.
**If a test fails, re-read the cited FINDINGS section before changing the test** — the
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ You get a cuttable, keyed, browser-watchable MXL switcher:
- ✓ Test-pattern source + file playout
- ✓ HTML5 lower-third graphics keyer
- ✓ WebRTC program output (watch in any browser)
- ✓ **Two SRT contribution slots** — point Larix / OBS / vMix at `srt://YOUR-IP:8890`
- ✓ **Two SRT contribution slots** — point Larix / OBS / vMix at `srt://YOUR-IP:8890` (Guest 1) or `srt://YOUR-IP:8891` (Guest 2), Caller mode, no stream ID
- ✓ **A browser switcher** — preview/program with TAKE + a live multiview of every source

Then check it's healthy and cut between sources:
Expand Down
15 changes: 10 additions & 5 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,18 @@ down. In particular:
that browser's localStorage). Set `MXL_CONTROL_REQUIRE_TOKEN=1` to fail closed (503) if
the token is ever missing. `/repair` is also rate limited (default 10 per 60 s per client;
tune with `MXL_REPAIR_RATE_MAX` / `MXL_REPAIR_RATE_WINDOW_S`, `0` disables). A shared
**Behind a reverse proxy / tunnel, set `MXL_TRUST_PROXY=1`** so limits key on the real
client (`CF-Connecting-IP` / `X-Forwarded-For`); it is OFF by default because on a
directly exposed server those headers are client-controlled. A shared
token is a speed bump, not a substitute for network controls — still keep the VM
control ports behind a firewall/NSG. Always serve the backend over HTTPS when a token is in use.
- **The SRT guest slots accept any publisher** that knows the stream id — that's the
point (open contribution), but it means anyone who learns `srt://your-ip:8890` +
`publish:guest1` can push video. Rotate/scope stream ids for anything beyond a demo.
**Optional hardening (off by default):** run the quickstart with a per-guest SRT
passphrase and mediamtx will reject any publisher on that slot that doesn't encrypt with it:
- **The SRT guest slots accept any publisher** — that's the point (open contribution).
In the default `srt-listen` transport each guest owns a UDP port (`8890` = Guest 1,
`8891` = Guest 2) and anyone who can reach that port can push video; the passphrase
options below are **not applied** in this mode, so restrict UDP 8890/8891 to known
source IPs in your cloud firewall for anything beyond a demo.
**Optional hardening (off by default, `MXL_GUEST_TRANSPORT=srt-direct` only):** run the
quickstart with a per-guest SRT passphrase and mediamtx will reject any publisher on that slot that doesn't encrypt with it:
```bash
sudo MXL_GUEST1_SRT_PASSPHRASE='long-secret-for-guest-1' \
MXL_GUEST2_SRT_PASSPHRASE='different-secret-guest-2' scripts/quickstart.sh
Expand Down
20 changes: 13 additions & 7 deletions backend/mxl-auth.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,20 @@ function tokenFromRequest(req) {
}

// Identify the real client. Behind cloudflared (or most CDNs) req.ip is the tunnel's
// address, so every visitor would share one rate-limit bucket (Review R6a). Prefer the
// CDN's real-client header when present: CF-Connecting-IP (Cloudflare) or the first hop
// of X-Forwarded-For. These are only trustworthy when the request genuinely comes
// through your proxy — which is the intended deployment (the control UI sits behind the
// tunnel) — so this is opt-out via MXL_TRUST_PROXY_HEADERS=0 for a direct-exposure case.
// address, so every visitor would share one rate-limit bucket (Review R6a). When the
// server sits behind a TRUSTED proxy, opt in to the CDN's real-client header:
// CF-Connecting-IP (Cloudflare) or the first hop of X-Forwarded-For.
//
// DEFAULT OFF: those headers are client-controlled. On a directly exposed server an
// attacker could send a fresh X-Forwarded-For on every request, get a new bucket each
// time, and sidestep both the /repair limiter and the failed-auth lockout. Opt in with
// MXL_TRUST_PROXY=1 (alias: MXL_TRUST_PROXY_HEADERS=1) ONLY when every request really
// arrives through your proxy (e.g. a cloudflared tunnel to a localhost-bound backend).
function _truthy(v) {
return ['1', 'true', 'yes', 'on'].includes(String(v ?? '').trim().toLowerCase());
}
function clientKeyFactory(env = process.env) {
const trustHeaders = String(env.MXL_TRUST_PROXY_HEADERS ?? '1').toLowerCase() !== '0'
&& String(env.MXL_TRUST_PROXY_HEADERS ?? '1') !== 'false';
const trustHeaders = _truthy(env.MXL_TRUST_PROXY) || _truthy(env.MXL_TRUST_PROXY_HEADERS);
return function _clientKey(req) {
const h = req.headers || {};
if (trustHeaders) {
Expand Down
2 changes: 1 addition & 1 deletion companion-module-mxl-switcher/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ class MXLSwitcherInstance extends InstanceBase {
id: 'baseurl',
label: 'Switcher base URL',
width: 8,
default: 'https://prodbots.com',
default: 'http://127.0.0.1:3100',
},
{
type: 'textinput',
Expand Down
21 changes: 14 additions & 7 deletions docs/CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,16 +34,16 @@ and Node loaders both honour them):
| `MXL_SSH_KEY` | SSH private key | **required** |
| `MXL_SITE_IP` | the one source IP the VM's NSG allows (only used in an error message) | `203.0.113.50` (placeholder; set `MXL_SITE_IP`) |
| `MXL_MAKITO_IP` | CAM 2 Makito X4 encoder (informational) | `192.168.8.177` |
| `MXL_BACKEND_URL` | facility backend base URL (kiosk page + `/api/mxl/*`); also exported to the VM-side `run-cam*.sh` as `MXL_REPAIR_URL=$MXL_BACKEND_URL/api/mxl/repair` | `https://prodbots.com` |
| `MXL_FEED_URL` | public WebRTC feed tunnel | `https://mxl-feed.cochran.cloud` |
| `MXL_BACKEND_URL` | facility backend base URL (kiosk page + `/api/mxl/*`); also exported to the VM-side `run-cam*.sh` as `MXL_REPAIR_URL=$MXL_BACKEND_URL/api/mxl/repair` | `http://127.0.0.1:3100` (the local control plane — set it for a real facility) |
| `MXL_FEED_URL` | public WebRTC feed tunnel | `http://127.0.0.1:8889` (local mediamtx — set it for a real facility) |
| `MXL_AZ_RESOURCE_GROUP`, `MXL_AZ_VM_NAME` | `az vm start/deallocate` target | **required** (both) |
| `MXL_HTML` | path of the deployed kiosk page | `$HOME/prodbots-backend/public/mxl.html` |

## Python / shell tools

| Variable | Used by | Default |
|---|---|---|
| `MXL_BACKEND_URL` | `audio_pgm.py`, `layout_pgm.py`, `mxl_multiview.py`, `nmos_node.py` (`--facility`), `selector-doctor.sh`, `guest-leg-doctor.sh` | `https://prodbots.com` |
| `MXL_BACKEND_URL` | `audio_pgm.py`, `layout_pgm.py`, `mxl_multiview.py`, `selector-doctor.sh`, `guest-leg-doctor.sh`, `nmos_node.py` (`--facility`) | `http://127.0.0.1:3100` (the local control plane). `nmos_node.py` still carries the original facility default for now — set `MXL_BACKEND_URL` / `--facility` explicitly |
| `TAMS_HOST` | `tams_shipper.py`, `backfill-mini.py` — the TAMS/MinIO box | `203.0.113.140` (placeholder; set `TAMS_HOST`) |
| `TAMS_URL` | `tams_shipper.py` TAMS API (pre-existing) | `http://$TAMS_HOST:8000` |
| `TAMS_S3_ENDPOINT` | `tams_shipper.py`, `backfill-mini.py` MinIO endpoint | `http://$TAMS_HOST:9000` |
Expand Down Expand Up @@ -74,7 +74,7 @@ report-only and is separate from this watcher.
| `MXL_CONTROL_REQUIRE_TOKEN` | same — `1` = answer 503 if no token configured (fail closed) | unset |
| `MXL_REPAIR_RATE_MAX`, `MXL_REPAIR_RATE_WINDOW_S` | same — `/repair` calls per window per client (`0` = no limit) | `10`, `60` |
| `MXL_AUTH_FAIL_MAX`, `MXL_AUTH_FAIL_WINDOW_S` | `backend/mxl-auth.js` — failed-auth attempts per client before a `429` lockout (throttles brute-forcing the shared token; `0` = no limit) | `20`, `60` |
| `MXL_TRUST_PROXY_HEADERS` | `backend/mxl-auth.js` — use `CF-Connecting-IP` / `X-Forwarded-For` as the rate-limit client key (so visitors behind the tunnel get separate buckets, not the tunnel's one IP). Set `0` only if the server is exposed directly, not behind a trusted proxy | `1` (on) |
| `MXL_TRUST_PROXY` (alias `MXL_TRUST_PROXY_HEADERS`) | `backend/mxl-auth.js` — `1` = use `CF-Connecting-IP` / `X-Forwarded-For` as the rate-limit / failed-auth client key (so visitors behind a tunnel get separate buckets, not the tunnel's one IP). Set it **only** when every request arrives through a trusted proxy (e.g. a cloudflared tunnel to a localhost-bound backend): on a directly exposed server those headers are client-controlled and would let a caller dodge the throttles | unset (off) |
| `MXL_GUEST1_SRT_PASSPHRASE`, `MXL_GUEST2_SRT_PASSPHRASE`, `MXL_GUEST_SRT_PASSPHRASE` | `scripts/quickstart.sh` — require an SRT passphrase to publish on a guest slot | unset = open |
| `MXL_GRAPHICS_BIND` | `scripts/quickstart.sh` — bind address of the `:8085` graphics server | docker bridge gateway (e.g. `172.17.0.1`); `0.0.0.0` = old behaviour |

Expand All @@ -97,6 +97,13 @@ and therefore must present the token once the backend enforces it. Each of them
## Not (yet) configurable

Intentionally left as literals — flagged for a follow-up, not changed here: the `/home/guy/...`
paths in `guest-leg-doctor.sh`, the static web pages' own hostnames (`web/*.html`:
`mxl-feed.cochran.cloud`, `mxlinfo.cochran.cloud`, canonical/og URLs), the exported Companion
page (`*.companionconfig`) and the Companion module's editable default host.
paths in `guest-leg-doctor.sh`, the backend host in `web/lower-third.html` (it runs inside the
keyer, not same-origin with the backend), canonical/og URLs in the static pages, and the
exported Companion page (`*.companionconfig`).

Web pages: `web/commentary.html` posts WHIP to the same-origin `/mxlfeed/` proxy (make
sure your reverse proxy forwards WHIP `POST`/`PATCH`/`DELETE` there); the kiosk's DOMAIN
link in `web/mxl.html` points at the serving host on `:9608`; its Larix QR codes come
live from `GET /api/mxl/ingest/qr/guestN.png` (hidden if the backend doesn't serve it).
Neither page takes a host override from the URL, by design.
The Companion module's editable base URL defaults to `http://127.0.0.1:3100`.
2 changes: 1 addition & 1 deletion docs/GCP-BUILD-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ worth knowing before you try):
`/dev/shm`. Pulled a full clean 1080p frame from the domain flow
(`gcp-real-ptz-on-gcp.jpg`).
- **Makito X4 Cam 2 (192.168.8.177)** repointed Azure→GCP via its REST API
(`PUT /apis/streams/2`, address→34.83.212.72, latency 200) — held **13+ min
(`PUT /apis/streams/2`, address→203.0.113.10, latency 200) — held **13+ min
stable, 12.7 Mbps, 1 reconnection**. (Stream 1 "Extra Hours SRT"=YouTube, untouched.)
- **SRT latency for WAN:** the Azure 20ms (tuned for ~6ms studio↔VM LAN) was too
tight for the studio→GCP-Oregon internet hop; **200ms** made both feeds hold.
Expand Down
20 changes: 15 additions & 5 deletions docs/QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@ upstream with `MXL_BLEEDING_EDGE=1`).
(the CEF-based keyer alone is 3.3 GB), so the image pull dominates the run
time.
- **Cloud firewall / NSG open:** `8889/tcp` (WebRTC page + signaling),
`8189/udp` (WebRTC media), and `8890/udp` (SRT guest contribution — the phone
slots are now part of the quickstart, so open this too). The unauthenticated
`8189/udp` (WebRTC media), and **both** `8890/udp` + `8891/udp` (SRT guest
contribution — one listener port per guest in the default `srt-listen` transport:
8890 = Guest 1, 8891 = Guest 2). Only `8890/udp` is needed if you opt into
`MXL_GUEST_TRANSPORT=srt-direct`. The unauthenticated
media-function **control APIs (`9600`–`9605`) bind to `127.0.0.1` only** — drive
them from the box (`curl 127.0.0.1:…`) or over an SSH tunnel; never expose them.

Expand Down Expand Up @@ -107,9 +109,16 @@ The quickstart arms two SRT guest slots. Point any SRT encoder — the free
**Larix Broadcaster** app, OBS, or vMix — at the publish point from the banner:

```
srt://<your-ip>:8890 streamid: publish:guest1 (or publish:guest2)
srt://<your-ip>:8890 (Guest 1) srt://<your-ip>:8891 (Guest 2)
Mode: Caller · Stream ID: leave blank — the port picks the guest
```

That's the default `srt-listen` transport (the ingest itself is the SRT listener, one
UDP port per guest — open both in your firewall). If you opted into
`MXL_GUEST_TRANSPORT=srt-direct`, both guests share one port and are told apart by
stream id instead: `srt://<your-ip>:8890` with streamid `publish:guest1` (or
`publish:guest2`).

The watcher attaches the slot automatically the moment the stream connects; then
cut to it:

Expand All @@ -118,8 +127,9 @@ cut to it:
curl -X POST -H 'Content-Type: application/json' -d '{"slot":2}' http://127.0.0.1:9604/pipeline/active-input
```

Larix note: free Larix needs the stream id as its dedicated `srtstreamid` field
(not baked in the URL). The banner prints a scannable QR with the exact scheme.
Larix note (srt-direct only): free Larix needs the stream id as its dedicated
`srtstreamid` field (not baked in the URL). In either mode the browser UI's
**Add your camera** button shows a scannable QR for this box with the exact scheme.

**Slot numbers are stable.** Guest 1 is always slot 2 and Guest 2 always slot 3 —
whether or not the other is streaming. An absent guest's slot is held by a safe
Expand Down
14 changes: 7 additions & 7 deletions scripts/bring-up-mxl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ VM_USER=${MXL_VM_SSH_USER:-guy} # SSH user on the VM
SSH_KEY=$MXL_SSH_KEY
SITE_IP=${MXL_SITE_IP:-203.0.113.50} # placeholder (RFC 5737); your NSG-allowed source IP — set MXL_SITE_IP
MAKITO_IP=${MXL_MAKITO_IP:-192.168.8.177} # CAM 2 Makito X4 encoder (informational)
BACKEND_URL=${MXL_BACKEND_URL:-https://prodbots.com} # facility backend (kiosk + /api/mxl/*)
BACKEND_URL=${MXL_BACKEND_URL:-http://127.0.0.1:3100} # facility backend (kiosk + /api/mxl/*) — set MXL_BACKEND_URL for a real facility
BACKEND_URL=${BACKEND_URL%/}
FEED_URL=${MXL_FEED_URL:-https://mxl-feed.cochran.cloud} # public WebRTC feed tunnel
FEED_URL=${MXL_FEED_URL:-http://127.0.0.1:8889} # public WebRTC feed (tunnel) — set MXL_FEED_URL for a real facility
AZ_RG=$MXL_AZ_RESOURCE_GROUP
AZ_VM=$MXL_AZ_VM_NAME
SSH="ssh -i $SSH_KEY -o BatchMode=yes -o ConnectTimeout=8 $VM_USER@$VM_IP"
Expand Down Expand Up @@ -99,7 +99,7 @@ for p in 9600 9601 9602 9603 9604 9605; do # 9606 = parked TG2, don't wait on i
for i in $(seq 1 30); do curl -s -m 2 -o /dev/null http://127.0.0.1:$p/pipeline/status && break; sleep 2; done
done
# mxl-info-gui needs a domain_def.json to recognize the domain; /dev/shm is
# volatile so recreate it every bring-up (2026-09-13; GUI at mxlinfo.cochran.cloud)
# volatile so recreate it every bring-up (2026-09-13; GUI on :9608)
echo '{"id":"domain_1"}' | sudo tee /dev/shm/mxl/domain_1/domain_def.json >/dev/null
# hands-on extras (mxl-info-gui :9608, spx-server :5660, webrtc2mxl :9609)
# are restart-unless-stopped docker containers — nothing to relaunch here
Expand Down Expand Up @@ -175,7 +175,7 @@ post 9603/pipeline/stop '{}'
# The live OHG facility opts INTO the ProdBots repair announce explicitly — the
# contribution core now defaults to NO announce (open-core boundary), so this env
# var is what keeps the selector auto-reattaching the cam here. (See contribution_core.py.)
sudo docker exec hls2mxl sh -c 'printf "#!/bin/sh\nexport MXL_REPAIR_URL=%s/api/mxl/repair\nwhile :; do python3 /tmp/cam_ingest.py >> /tmp/cam-ingest.log 2>&1; echo RESTART >> /tmp/cam-ingest.log; sleep 2; done\n" "$0" > /tmp/run-cam1.sh && chmod +x /tmp/run-cam1.sh' "${MXL_BACKEND_URL:-https://prodbots.com}"
sudo docker exec hls2mxl sh -c 'printf "#!/bin/sh\nexport MXL_REPAIR_URL=%s/api/mxl/repair\nwhile :; do python3 /tmp/cam_ingest.py >> /tmp/cam-ingest.log 2>&1; echo RESTART >> /tmp/cam-ingest.log; sleep 2; done\n" "$0" > /tmp/run-cam1.sh && chmod +x /tmp/run-cam1.sh' "$BACKEND_URL"
sudo docker exec -d hls2mxl /tmp/run-cam1.sh
sleep 6

Expand All @@ -188,7 +188,7 @@ sleep 6
# (slices=4 broke mediamtx's TS parsing). Runner keeps cmdline pkill-safe.
sudo docker cp /srv/mxl-tools/cam2_ingest.py hls2mxl:/tmp/cam2_ingest.py
sudo docker exec hls2mxl sh -c 'pkill -f run-cam2.sh; pkill -f cam2_ingest.py; true'
sudo docker exec hls2mxl sh -c 'printf "#!/bin/sh\nexport MXL_REPAIR_URL=%s/api/mxl/repair\nwhile :; do nice -n 10 python3 /tmp/cam2_ingest.py >> /tmp/cam2-ingest.log 2>&1; echo RESTART >> /tmp/cam2-ingest.log; sleep 2; done\n" "$0" > /tmp/run-cam2.sh && chmod +x /tmp/run-cam2.sh' "${MXL_BACKEND_URL:-https://prodbots.com}"
sudo docker exec hls2mxl sh -c 'printf "#!/bin/sh\nexport MXL_REPAIR_URL=%s/api/mxl/repair\nwhile :; do nice -n 10 python3 /tmp/cam2_ingest.py >> /tmp/cam2-ingest.log 2>&1; echo RESTART >> /tmp/cam2-ingest.log; sleep 2; done\n" "$0" > /tmp/run-cam2.sh && chmod +x /tmp/run-cam2.sh' "$BACKEND_URL"
sudo docker exec -d hls2mxl /tmp/run-cam2.sh
sleep 6

Expand Down Expand Up @@ -227,7 +227,7 @@ pgrep -f "http.server 8086" >/dev/null || \
# Layout compositor (2-up / PiP, selector slot 6, added 2026-09-10): always-on,
# all six sources behind two internal selectors -> compositor; layout changes
# are live pad-property flips (no flow recreation -> no slot-6 wedges). Polls
# prodbots.com/api/mxl/layout-state for desired state (browser-UA header —
# $MXL_BACKEND_URL/api/mxl/layout-state for desired state (browser-UA header —
# the CF zone 403s python-urllib).
sudo docker cp /srv/mxl-tools/layout_pgm.py hls2mxl:/tmp/layout_pgm.py
sudo docker exec hls2mxl sh -c 'pkill -9 -f run-layout.sh; pkill -9 -f layout_pgm.py; true'
Expand Down Expand Up @@ -306,7 +306,7 @@ $SSH 'for u in ca111e00:Cam1 ca222e00:Cam2 1a900700:Layout; do uuid=${u%%:*}; nm
step "Fresh layout output lock"
$SSH 'sudo docker exec hls2mxl sh -c "pkill -9 -f \"[r]un-layout\"; pkill -9 -f \"[l]ayout_pgm.py\"; sleep 2"; sudo docker exec -d hls2mxl /tmp/run-layout.sh; echo " ✓ layout respawned"'

# ── 4. Feed tunnel (named: mxl-feed.cochran.cloud, systemd on the VM) ────────
# ── 4. Feed tunnel (named tunnel serving $MXL_FEED_URL, systemd on the VM) ────
# Tunnel UUID e5ec74db-8e99-4232-b5a4-4e240f88572b; config /etc/cloudflared/mxl-feed.yml.
# systemd auto-starts it on boot — this just makes sure and verifies.
step "Ensuring mxl-feed named tunnel on VM"
Expand Down
Loading
Loading