From 7cd20785d9ead975c64e48b2e5f8c8119413e218 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 21 Jul 2026 06:33:04 +0000 Subject: [PATCH 1/3] docs: document frame metadata access in output converters Document the optional, opt-in `metadata` argument for model output converters (#1172), which passes the frame's NvDsFrameMeta (source_id, pts, video_frame, objects, tags) to a converter's __call__ when it declares the parameter. Added to the detector-model chapter's Converter section: usage, backward-compatibility/signature-inspection note, the convert-output telemetry span, and a pointer to the output_converter_metadata sample and Etcd reconfiguration. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TbsaV4NhYriTyUqxuzKd8Q --- docs/source/savant_101/30_dm.rst | 48 ++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/source/savant_101/30_dm.rst b/docs/source/savant_101/30_dm.rst index 8f55a90db..68fd7e80b 100644 --- a/docs/source/savant_101/30_dm.rst +++ b/docs/source/savant_101/30_dm.rst @@ -193,6 +193,54 @@ An example of the converter for YOLOv4 listed below. The YOLOv4 model has two ou ) +Accessing Frame Metadata In A Converter +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +By default a converter sees only the model output tensors, the model definition, +and the ROI it ran on. A converter can also optionally receive the **frame +metadata**, which lets its post-processing depend on the context of the frame — +for example, applying different thresholds per source, or adapting to objects and +tags already present on the frame. + +To opt in, add a ``metadata`` parameter to the converter's ``__call__``: + +.. code-block:: python + + from typing import Optional + from savant.deepstream.meta.frame import NvDsFrameMeta + + class TensorToBBoxConverter(BaseObjectModelOutputConverter): + def __call__( + self, + *output_layers: np.ndarray, + model: ObjectModel, + roi: Tuple[float, float, float, float], + metadata: Optional[NvDsFrameMeta] = None, + ) -> np.ndarray: + source_id = metadata.source_id if metadata is not None else None + ... + +The argument is passed a :py:class:`~savant.deepstream.meta.frame.NvDsFrameMeta` +wrapper for the frame, exposing ``source_id``, ``pts``, the underlying +``video_frame``, the objects already on the frame, and its tags. + +.. note:: + + The ``metadata`` argument is **opt-in and backward compatible**. Savant + inspects the converter's ``__call__`` signature once (the result is cached per + converter class, so there is no per-frame overhead) and passes ``metadata`` + only when the parameter is declared. Converters written for earlier versions of + Savant — including all the built-in ones — keep working unchanged. When + ``metadata`` is used, the output conversion is additionally wrapped in a + ``convert-output`` telemetry span nested under the frame span, so it remains + traceable. + +The `output_converter_metadata sample `__ +demonstrates this by combining it with :doc:`/advanced_topics/4_etcd`: a YOLO +output converter reads ``metadata.source_id`` and looks up per-source detection +thresholds in Etcd, which can be changed live without restarting the pipeline. + + Object Filtering ^^^^^^^^^^^^^^^^ From bd53ab18d1f552ceb28c79bbf16e0b39b1108275 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 21 Jul 2026 06:53:03 +0000 Subject: [PATCH 2/3] docs: clarify converter metadata nullability and per-frame cost Address review feedback on the output-converter metadata docs: - Note that `metadata` can be None at call time even when the converter declares the parameter (the framework passes None when the frame's VideoFrame context is unavailable), so converters should guard for None. - Correct the performance wording: signature inspection is cached per converter class and not repeated per frame, but a cached lookup still happens per frame (not "no per-frame overhead"). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TbsaV4NhYriTyUqxuzKd8Q --- docs/source/savant_101/30_dm.rst | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/docs/source/savant_101/30_dm.rst b/docs/source/savant_101/30_dm.rst index 68fd7e80b..f347117b6 100644 --- a/docs/source/savant_101/30_dm.rst +++ b/docs/source/savant_101/30_dm.rst @@ -220,20 +220,24 @@ To opt in, add a ``metadata`` parameter to the converter's ``__call__``: source_id = metadata.source_id if metadata is not None else None ... -The argument is passed a :py:class:`~savant.deepstream.meta.frame.NvDsFrameMeta` -wrapper for the frame, exposing ``source_id``, ``pts``, the underlying -``video_frame``, the objects already on the frame, and its tags. +When available, ``metadata`` is a +:py:class:`~savant.deepstream.meta.frame.NvDsFrameMeta` wrapper for the frame, +exposing ``source_id``, ``pts``, the underlying ``video_frame``, the objects +already on the frame, and its tags. It is optional at call time: even when the +converter declares the parameter, the framework may still pass ``None`` (for +example, when the frame's ``VideoFrame`` context is unavailable), so a converter +that uses it should guard against ``None`` — as the snippet above does. .. note:: The ``metadata`` argument is **opt-in and backward compatible**. Savant - inspects the converter's ``__call__`` signature once (the result is cached per - converter class, so there is no per-frame overhead) and passes ``metadata`` - only when the parameter is declared. Converters written for earlier versions of - Savant — including all the built-in ones — keep working unchanged. When - ``metadata`` is used, the output conversion is additionally wrapped in a - ``convert-output`` telemetry span nested under the frame span, so it remains - traceable. + inspects the converter's ``__call__`` signature only once and caches the result + per converter class, so the signature is not re-inspected on every frame (the + per-frame cost is just a cached lookup). ``metadata`` is passed only when the + parameter is declared, so converters written for earlier versions of Savant — + including all the built-in ones — keep working unchanged. When ``metadata`` is + used, the output conversion is additionally wrapped in a ``convert-output`` + telemetry span nested under the frame span, so it remains traceable. The `output_converter_metadata sample `__ demonstrates this by combining it with :doc:`/advanced_topics/4_etcd`: a YOLO From b9cf7b09f76bbc257b74a4204c3c2403a702eb09 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 21 Jul 2026 06:58:58 +0000 Subject: [PATCH 3/3] docs: add comma after introductory "By default" Minor grammar fix per review, matching the style used elsewhere in the docs. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TbsaV4NhYriTyUqxuzKd8Q --- docs/source/savant_101/30_dm.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/savant_101/30_dm.rst b/docs/source/savant_101/30_dm.rst index f347117b6..370150636 100644 --- a/docs/source/savant_101/30_dm.rst +++ b/docs/source/savant_101/30_dm.rst @@ -196,7 +196,7 @@ An example of the converter for YOLOv4 listed below. The YOLOv4 model has two ou Accessing Frame Metadata In A Converter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -By default a converter sees only the model output tensors, the model definition, +By default, a converter sees only the model output tensors, the model definition, and the ROI it ran on. A converter can also optionally receive the **frame metadata**, which lets its post-processing depend on the context of the frame — for example, applying different thresholds per source, or adapting to objects and