Support metadata passing to converters

samples/mjpeg_usb_cam/README.md

@@ -2,7 +2,7 @@
 
 A pipeline demonstrating how to capture MJPEG from a USB camera. MJPEG is a common format for USB/MIPI CSI-2 cameras providing compressed, low-latency video streaming.
 
-The resulting stream can be accessed via LL-HLS on `http://locahost:888/stream/video`
+The resulting stream can be accessed via LL-HLS on `http://localhost:888/stream/video`
 
 Tested on platforms:
 

samples/multiple_gige/README.md

@@ -2,7 +2,7 @@
 
 A simple pipeline demonstrates how GigE Vision Source Adapter works in Savant. In the demo video from one GigE Vision camera is passed as raw-rgba frames, and another one is passed as HEVC-encoded frames. Both streams are passed to an Always-On-RTSP sink.
 
-The resulting streams can be accessed via LL-HLS on `http://locahost:888/stream/gige-raw` (raw-rgba frames) and `http://locahost:888/stream/gige-encoded` (HEVC-encoded frames).
+The resulting streams can be accessed via LL-HLS on `http://localhost:888/stream/gige-raw` (raw-rgba frames) and `http://localhost:888/stream/gige-encoded` (HEVC-encoded frames).
 
 Tested on platforms:
 

samples/multiple_rtsp/README.md

@@ -2,7 +2,7 @@
 
 A simple pipeline demonstrates how multiplexed processing works in Savant. In the demo, two RTSP streams are ingested in the module and processed with the PeopleNet model. 
 
-The resulting streams can be accessed via LL-HLS on `http://locahost:888/stream/city-traffic` and `http://locahost:888/stream/town-centre`.
+The resulting streams can be accessed via LL-HLS on `http://localhost:888/stream/city-traffic` and `http://localhost:888/stream/town-centre`.
 
 Tested on platforms:
 

samples/output_converter_metadata/README.md

@@ -0,0 +1,81 @@
+# Per-source Converter Configuration from Etcd
+
+A simple pipeline demonstrates how metadata processing in output converters works in Savant. In the demo, two RTSP streams are ingested in the module and processed with the YOLO11n model. The output converter is configurable via etcd.
+
+The resulting streams can be accessed via LL-HLS on `http://localhost:888/stream/city-traffic` and `http://localhost:888/stream/town-centre` or via RTSP on `rtsp://127.0.0.1:554/stream/city-traffic` and `rtsp://127.0.0.1:554/stream/town-centre`.
+
+Two RTSP streams (`city-traffic` and `town-centre`) are ingested by a single module and
+processed with a YOLO11n detector. The detector's output converter is a custom subclass
+of the built-in YOLO converter that, for every frame, reads the frame's `source_id` from
+the converter `metadata` argument and looks up a per-source configuration object in Etcd
+(e.g. the detection `confidence_threshold`). Values can be changed **live** with
+`etcdctl` — no pipeline restart required.
+
+This relies on the output-converter `metadata` argument: when a converter's `__call__`
+declares a `metadata` parameter it receives the frame's `NvDsFrameMeta` wrapper
+(`source_id`, `pts`, `video_frame`, objects, tags). Converters that do not declare it keep
+working unchanged. See `samples/output_converter_metadata/converter.py`.
+
+The resulting streams can be accessed via LL-HLS on
+`http://localhost:888/stream/city-traffic` and `http://localhost:888/stream/town-centre`.
+
+Tested on platforms:
+
+- Nvidia Ampere
+
+## Prerequisites
+
+```bash
+git clone https://github.com/insight-platform/Savant.git
+cd Savant
+git lfs pull
+./utils/check-environment-compatible
+```
+
+**Note**: Ubuntu 22.04 runtime configuration [guide](https://insight-platform.github.io/Savant/develop/getting_started/0_configure_prod_env.html) helps to configure the runtime to run Savant pipelines.
+
+## Build Engines
+
+The demo uses models that are compiled into TensorRT engines the first time the demo is run. This takes time. Optionally, you can prepare the engines before running the demo by using the command:
+
+```bash
+# you are expected to be in Savant/ directory
+
+./scripts/run_module.py --build-engines samples/output_converter_metadata/module.yml
+```
+
+## Run Demo
+
+```bash
+# you are expected to be in Savant/ directory
+
+# if x86
+docker compose -f samples/output_converter_metadata/docker-compose.x86.yml up
+
+# if Jetson
+docker compose -f samples/output_converter_metadata/docker-compose.l4t.yml up
+
+# open 'rtsp://127.0.0.1:554/stream/city-traffic' in your player
+# or visit 'http://127.0.0.1:888/stream/city-traffic' (LL-HLS)
+
+# open 'rtsp://127.0.0.1:554/stream/town-centre' in your player
+# or visit 'http://127.0.0.1:888/stream/town-centre' (LL-HLS)
+
+# Ctrl+C to stop running the compose bundle
+```
+
+## Per-source Configuration
+
+The converter reads the Etcd key `savant/source/<source_id>` as a JSON object.
+Supported fields: `confidence_threshold` and `nms_iou_threshold`. Use the helper script to
+set or update a source's configuration (the `etcd` service must be running):
+
+```bash
+# you are expected to be in Savant/samples/output_converter_metadata/ directory
+
+# keep low-confidence detections on city-traffic (more boxes)
+./set-config.sh city-traffic '{"confidence_threshold": 0.2}'
+
+# require high confidence on town-centre (fewer boxes)
+./set-config.sh town-centre '{"confidence_threshold": 0.7}'
+```

samples/output_converter_metadata/converter.py

@@ -0,0 +1,80 @@
+"""Detector output converter that pulls per-source config from Etcd."""
+
+import json
+from typing import Optional, Tuple
+
+import numpy as np
+from savant_rs.utils import eval_expr
+
+from savant.base.model import ObjectModel
+from savant.converter.yolo import TensorToBBoxConverter
+from savant.deepstream.meta.frame import NvDsFrameMeta
+
+# how long a fetched Etcd value stays cached locally (seconds)
+CONFIG_CACHE_TTL = 5
+
+
+class EtcdConfigurableConverter(TensorToBBoxConverter):
+    """YOLO bbox converter whose thresholds are overridden per source_id from Etcd."""
+
+    def __init__(self, **kwargs):
+        self._default_confidence_threshold = kwargs.get('confidence_threshold', 0.25)
+        self._default_nms_iou_threshold = kwargs.get('nms_iou_threshold', 0.0)
+        self._configs = {}
+        super().__init__(**kwargs)
+
+    def _load_source_config(self, source_id: str) -> dict:
+        expr = f'etcd("source/{source_id}", "")'
+        val, is_cached = eval_expr(expr, ttl=CONFIG_CACHE_TTL, no_gil=True)
+        if not is_cached:
+            if val:
+                try:
+                    parsed_config = json.loads(val)
+                    self._configs[source_id] = (
+                        parsed_config if isinstance(parsed_config, dict) else {}
+                    )
+                except json.JSONDecodeError:
+                    self.logger.warning(
+                        'Invalid JSON in Etcd config for source %s: %r', source_id, val
+                    )
+                    self._configs[source_id] = {}
+            else:
+                self._configs[source_id] = {}
+
+        return self._configs.get(source_id, {})
+
+    def __call__(
+        self,
+        *output_layers: np.ndarray,
+        model: ObjectModel,
+        roi: Tuple[float, float, float, float],
+        metadata: Optional[NvDsFrameMeta] = None,
+    ) -> Optional[np.ndarray]:
+        """Converts detector output layer tensor to bbox tensor.
+
+        :param output_layers: Output layer tensor
+        :param model: Model definition, required parameters: input tensor shape,
+            maintain_aspect_ratio
+        :param roi: [left, top, width, height] of the rectangle
+            on which the model infers
+        :param metadata: Frame metadata.
+        :return: BBox tensor, see the base converter.
+        """
+
+        config = {}
+        if metadata is not None:
+            config = self._load_source_config(metadata.source_id)
+            self.logger.debug(
+                'Source %s converter config: %s', metadata.source_id, config
+            )
+
+        # per-source override with fallback to construction-time defaults
+        self.confidence_threshold = config.get(
+            'confidence_threshold', self._default_confidence_threshold
+        )
+        self.nms_iou_threshold = config.get(
+            'nms_iou_threshold', self._default_nms_iou_threshold
+        )
+
+        # reuse the parent's YOLO tensor decoding / NMS / coordinate transform
+        return super().__call__(*output_layers, model=model, roi=roi)

samples/output_converter_metadata/docker-compose.l4t.yml

@@ -0,0 +1,87 @@
+services:
+
+  rtsp-city-traffic:
+    image: ghcr.io/insight-platform/savant-adapters-gstreamer-l4t:latest
+    restart: unless-stopped
+    volumes:
+      - zmq_sockets:/tmp/zmq-sockets
+    environment:
+      - RTSP_URI=rtsp://hello.savant.video:8554/stream/city-traffic
+      - ZMQ_ENDPOINT=pub+connect:ipc:///tmp/zmq-sockets/input-video.ipc
+      - SOURCE_ID=city-traffic
+    entrypoint: /opt/savant/adapters/gst/sources/rtsp.sh
+    depends_on:
+      module:
+        condition: service_healthy
+
+  rtsp-town-centre:
+    image: ghcr.io/insight-platform/savant-adapters-gstreamer-l4t:latest
+    restart: unless-stopped
+    volumes:
+      - zmq_sockets:/tmp/zmq-sockets
+    environment:
+      - RTSP_URI=rtsp://hello.savant.video:8554/stream/town-centre
+      - ZMQ_ENDPOINT=pub+connect:ipc:///tmp/zmq-sockets/input-video.ipc
+      - SOURCE_ID=town-centre
+    entrypoint: /opt/savant/adapters/gst/sources/rtsp.sh
+    depends_on:
+      module:
+        condition: service_healthy
+
+  module:
+    privileged: true
+    image: ghcr.io/insight-platform/savant-deepstream-l4t:latest
+    restart: unless-stopped
+    volumes:
+      - zmq_sockets:/tmp/zmq-sockets
+      - ../../cache:/cache
+      - ..:/opt/savant/samples
+    command: samples/output_converter_metadata/module.yml
+    environment:
+      - MODEL_PATH=/cache/models/yolo11
+      - DOWNLOAD_PATH=/cache/downloads/yolo11
+      - ZMQ_SRC_ENDPOINT=sub+bind:ipc:///tmp/zmq-sockets/input-video.ipc
+      - ZMQ_SINK_ENDPOINT=pub+bind:ipc:///tmp/zmq-sockets/output-video.ipc
+      - METRICS_FRAME_PERIOD=1000
+      - CODEC=jpeg
+    depends_on:
+      etcd:
+        condition: service_healthy
+    runtime: nvidia
+
+  always-on-sink:
+    image: ghcr.io/insight-platform/savant-adapters-deepstream-l4t:latest
+    restart: unless-stopped
+    ports:
+      - "554:554"    # RTSP
+      - "1935:1935"  # RTMP
+      - "888:888"    # HLS
+      - "8889:8889"  # WebRTC
+    volumes:
+      - zmq_sockets:/tmp/zmq-sockets
+      - ../assets/stub_imgs:/stub_imgs
+    environment:
+      - ZMQ_ENDPOINT=sub+connect:ipc:///tmp/zmq-sockets/output-video.ipc
+      - SOURCE_IDS=city-traffic,town-centre
+      - FRAMERATE=25/1
+      - STUB_FILE_LOCATION=/stub_imgs/smpte100_1280x720.jpeg
+      - DEV_MODE=True
+    command: python -m adapters.ds.sinks.always_on_rtsp
+
+  etcd:
+    container_name: etcd
+    image: bitnamilegacy/etcd:3.6.4-debian-12-r4
+    restart: unless-stopped
+    environment:
+      - ALLOW_NONE_AUTHENTICATION=yes
+      - ETCD_ADVERTISE_CLIENT_URLS=http://etcd:2379
+    ports:
+      - "2379:2379"
+    healthcheck:
+      test: [ "CMD", "/opt/bitnami/scripts/etcd/healthcheck.sh" ]
+      interval: 5s
+      timeout: 5s
+      retries: 3
+
+volumes:
+  zmq_sockets:

samples/output_converter_metadata/docker-compose.x86.yml

@@ -0,0 +1,52 @@
+services:
+
+  rtsp-city-traffic:
+    image: ghcr.io/insight-platform/savant-adapters-gstreamer:latest
+    extends:
+      file: docker-compose.l4t.yml
+      service: rtsp-city-traffic
+
+  rtsp-town-centre:
+    image: ghcr.io/insight-platform/savant-adapters-gstreamer:latest
+    extends:
+      file: docker-compose.l4t.yml
+      service: rtsp-town-centre
+
+  module:
+    privileged: true
+    image: ghcr.io/insight-platform/savant-deepstream:latest
+    extends:
+      file: docker-compose.l4t.yml
+      service: module
+    runtime: runc
+    environment:
+      - CODEC=h264
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: 1
+              capabilities: [ gpu ]
+
+  always-on-sink:
+    privileged: true
+    image: ghcr.io/insight-platform/savant-adapters-deepstream:latest
+    extends:
+      file: docker-compose.l4t.yml
+      service: always-on-sink
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: 1
+              capabilities: [ gpu ]
+
+  etcd:
+    extends:
+      file: docker-compose.l4t.yml
+      service: etcd
+
+volumes:
+  zmq_sockets: