diff --git a/docs/source/savant_101/30_dm.rst b/docs/source/savant_101/30_dm.rst index 8f55a90db..370150636 100644 --- a/docs/source/savant_101/30_dm.rst +++ b/docs/source/savant_101/30_dm.rst @@ -193,6 +193,58 @@ An example of the converter for YOLOv4 listed below. The YOLOv4 model has two ou ) +Accessing Frame Metadata In A Converter +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +By default, a converter sees only the model output tensors, the model definition, +and the ROI it ran on. A converter can also optionally receive the **frame +metadata**, which lets its post-processing depend on the context of the frame — +for example, applying different thresholds per source, or adapting to objects and +tags already present on the frame. + +To opt in, add a ``metadata`` parameter to the converter's ``__call__``: + +.. code-block:: python + + from typing import Optional + from savant.deepstream.meta.frame import NvDsFrameMeta + + class TensorToBBoxConverter(BaseObjectModelOutputConverter): + def __call__( + self, + *output_layers: np.ndarray, + model: ObjectModel, + roi: Tuple[float, float, float, float], + metadata: Optional[NvDsFrameMeta] = None, + ) -> np.ndarray: + source_id = metadata.source_id if metadata is not None else None + ... + +When available, ``metadata`` is a +:py:class:`~savant.deepstream.meta.frame.NvDsFrameMeta` wrapper for the frame, +exposing ``source_id``, ``pts``, the underlying ``video_frame``, the objects +already on the frame, and its tags. It is optional at call time: even when the +converter declares the parameter, the framework may still pass ``None`` (for +example, when the frame's ``VideoFrame`` context is unavailable), so a converter +that uses it should guard against ``None`` — as the snippet above does. + +.. note:: + + The ``metadata`` argument is **opt-in and backward compatible**. Savant + inspects the converter's ``__call__`` signature only once and caches the result + per converter class, so the signature is not re-inspected on every frame (the + per-frame cost is just a cached lookup). ``metadata`` is passed only when the + parameter is declared, so converters written for earlier versions of Savant — + including all the built-in ones — keep working unchanged. When ``metadata`` is + used, the output conversion is additionally wrapped in a ``convert-output`` + telemetry span nested under the frame span, so it remains traceable. + +The `output_converter_metadata sample `__ +demonstrates this by combining it with :doc:`/advanced_topics/4_etcd`: a YOLO +output converter reads ``metadata.source_id`` and looks up per-source detection +thresholds in Etcd, which can be changed live without restarting the pipeline. + + Object Filtering ^^^^^^^^^^^^^^^^