diff --git a/docs/source/getting_started/3_samples.rst b/docs/source/getting_started/3_samples.rst index aa91fb4e3..3ed78a675 100644 --- a/docs/source/getting_started/3_samples.rst +++ b/docs/source/getting_started/3_samples.rst @@ -657,6 +657,30 @@ Auxiliary Streams **Link**: `Auxiliary Streams `_ +PyGroup: Colocated PyFuncs +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**Platform Support**: X86 + L4T + +**Main Features**: + +- Multiple sequential PyFuncs colocated in a single ``pygroup`` element +- Sequential per-frame execution without inter-element queues +- Per-stage OpenTelemetry spans preserved for each colocated PyFunc + +**Auxiliary Features**: + +- Overlay drawing on the main frame +- Per-stage auxiliary stream generation +- Preconfigured Jaeger/OTLP tracing + +**Adapters Used**: + +- :ref:`Video loop source adapter ` +- :ref:`Always On RTSP sink adapter ` (multi-stream) + +**Link**: `PyGroup `_ + Data Integration and APIs ------------------------- diff --git a/docs/source/reference/api/module_config.rst b/docs/source/reference/api/module_config.rst index d3220171c..b6c397bf9 100644 --- a/docs/source/reference/api/module_config.rst +++ b/docs/source/reference/api/module_config.rst @@ -10,7 +10,7 @@ Main module configuration entities .. _pipeline_element_hierarchy: -.. inheritance-diagram:: PipelineElement ModelElement PyFuncElement +.. inheritance-diagram:: PipelineElement ModelElement PyFuncElement PyGroupElement :parts: 1 :caption: PipelineElement hierarchy @@ -25,6 +25,7 @@ Main module configuration entities SourceElement SinkElement PyFuncElement + PyGroupElement ModelElement ElementGroup GroupCondition diff --git a/docs/source/savant_101/70_python.rst b/docs/source/savant_101/70_python.rst index 5d817aad4..e168b2b36 100644 --- a/docs/source/savant_101/70_python.rst +++ b/docs/source/savant_101/70_python.rst @@ -1,7 +1,7 @@ Python Function Unit ==================== -The Python Function Unit is used to include arbitrary custom Python code in the pipeline. To work with ``pyfunc``, custom code must be implemented by specifying :py:class:`~savant.deepstream.NvDsPyFuncPlugin` as the parent class, which exposes two methods: the first allows frames for each source to be handled separately, the second supports processing the frames for the whole batch. +The Python Function Unit is used to include arbitrary custom Python code in the pipeline. To work with ``pyfunc``, custom code must be implemented by specifying :py:class:`~savant.deepstream.pyfunc.NvDsPyFuncPlugin` as the parent class, which exposes two methods: the first allows frames for each source to be handled separately, the second supports processing the frames for the whole batch. Per-source processing, normally you want to use this method: @@ -103,3 +103,84 @@ Also, the ``pyfunc`` unit configuration allows setting an arbitrary set of user config_path: /opt/savant/samples/traffic_meter/line_crossing.yml Parameters defined with ``kwargs`` are available as ``pyfunc`` class instance attributes. + +Grouping PyFuncs Into a Single Element +-------------------------------------- + +When a pipeline contains several Python Function Units that always run one +after another, each of them is normally instantiated as a standalone +GStreamer element, and the framework inserts a queue between neighboring +units. Every element boundary adds an extra thread and buffer hand-off, which +is pure overhead for short, strictly sequential PyFuncs. + +The ``pygroup`` unit removes this overhead by *colocating* multiple PyFuncs in +a single GStreamer element. The colocated PyFuncs are executed sequentially on +every frame, in the listed order, within the same element — without queues in +between. Each colocated PyFunc keeps its own telemetry span, so per-stage +observability is preserved (see below). + +A ``pygroup`` unit is declared with the ``elements`` key, which holds a list of +regular PyFunc definitions. Every sub-element uses the same ``module``, +``class_name``, and (optionally) ``kwargs`` keys as a standalone ``pyfunc``: + +.. code-block:: yaml + + - element: pygroup + name: my_group + elements: + - module: module.pyfunc_module_1 + class_name: PyFuncClass1 + - module: module.pyfunc_module_2 + class_name: PyFuncClass2 + kwargs: + key: value + +Each sub-element must implement :py:class:`~savant.deepstream.pyfunc.NvDsPyFuncPlugin`, +exactly like a standalone ``pyfunc``. Everything a regular PyFunc can do also +works inside a group: receiving its own ``kwargs``, reading and writing object +metadata, drawing on frames, and creating :doc:`auxiliary video streams +`. Savant inserts the necessary +queues *around* the group automatically, just as it does for a standalone +``pyfunc``. + +.. note:: + + ``pygroup`` is intended for short chains of PyFuncs that are effectively + serial and benefit from lower per-element overhead. Heavy PyFuncs that + should run on their own threads, in parallel with the rest of the pipeline, + are better kept as standalone ``pyfunc`` units. + +Telemetry +^^^^^^^^^ + +Colocating PyFuncs does not merge them into an opaque block for tracing +purposes. For every frame, ``pygroup`` opens a ``process-frame`` span for the +group and wraps each colocated PyFunc in its own nested span named +``.``: + +.. code-block:: text + + process-frame + ├── module.pyfunc_module_1.PyFuncClass1 + └── module.pyfunc_module_2.PyFuncClass2 + +PyFunc code may open further nested spans inside its own span for finer-grained +profiling. As a result, you can still measure each stage independently in +Jaeger or any other OTLP backend. See :doc:`/advanced_topics/9_open_telemetry` +for details on tracing in Savant. + +Development Server +^^^^^^^^^^^^^^^^^^ + +The ``pygroup`` unit honors the :doc:`Development Server +`: when the module runs in dev mode, the +colocated PyFuncs are reloaded on source changes just like standalone +``pyfunc`` units. + +Sample +^^^^^^ + +The `pygroup sample `__ +colocates two overlay PyFuncs that draw sequentially on the frame, each feeding +its own auxiliary stream, and ships a preconfigured Jaeger/OTLP setup so the +per-stage spans can be inspected out of the box. diff --git a/samples/pygroup/README.md b/samples/pygroup/README.md new file mode 100644 index 000000000..9efe8303d --- /dev/null +++ b/samples/pygroup/README.md @@ -0,0 +1,75 @@ +# PyGroup: Colocating Multiple PyFuncs + +A pipeline demonstrating the `pygroup` element, which colocates multiple +sequential PyFuncs into a single GStreamer element. The colocated PyFuncs run +one after another on every frame, without queues in between, while each keeps +its own OpenTelemetry span so the stages can still be profiled individually. + +The sample colocates two overlay PyFuncs defined in [overlays.py](overlays.py): + +- `HorizontalLineOverlay` ("Step 1") draws a horizontal line and a label on the + main frame, and feeds a `-h` auxiliary stream. +- `VerticalLineOverlay` ("Step 2") draws a vertical line and a label on top of + Step 1's result — visibly confirming the group runs sequentially — and feeds + a `-v` auxiliary stream. + +Both stages are declared under a single `pygroup` unit in +[module.yml](module.yml). The bundle also ships a preconfigured Jaeger/OTLP +setup so you can inspect the per-stage spans. + +Tested on platforms: + +- Nvidia Turing, Ampere +- Nvidia Jetson Orin family + +## Prerequisites + +```bash +git clone https://github.com/insight-platform/Savant.git +cd Savant +git lfs pull +./utils/check-environment-compatible +``` + +**Note**: Ubuntu 22.04 runtime configuration [guide](https://insight-platform.github.io/Savant/develop/getting_started/0_configure_prod_env.html) helps to configure the runtime to run Savant pipelines. + +## Run Demo + +```bash +# you are expected to be in Savant/ directory + +# if x86 +docker compose -f samples/pygroup/docker-compose.x86.yml up + +# if Jetson +docker compose -f samples/pygroup/docker-compose.l4t.yml up + +# open the main stream and the two auxiliary streams in your player: +# 'rtsp://127.0.0.1:554/stream/video' (main frame, both overlays) +# 'rtsp://127.0.0.1:554/stream/video-h' (Step 1 auxiliary stream) +# 'rtsp://127.0.0.1:554/stream/video-v' (Step 2 auxiliary stream) +# or visit 'http://127.0.0.1:888/stream/video' (LL-HLS) + +# Ctrl+C to stop running the compose bundle +``` + +## Inspect the Telemetry + +The bundle runs an all-in-one Jaeger instance and configures the module to +export traces to it over OTLP. Open the Jaeger UI to see, per frame, a +`process-frame` span with one nested span per colocated PyFunc +(`samples.pygroup.overlays.HorizontalLineOverlay` and +`samples.pygroup.overlays.VerticalLineOverlay`), each of which further nests its +own `draw-*` and `aux-stream-publish` spans: + +``` +http://127.0.0.1:16686 +``` + +**Note**: On x86 the sample encodes output with H.264. The Jetson +(`docker-compose.l4t.yml`) bundle uses the JPEG codec instead, because +entry-level devices such as the Jetson Orin Nano do not provide an NVENC +hardware encoder. + +See the [Python Function Unit](https://insight-platform.github.io/Savant/develop/savant_101/70_python.html) +documentation for more details on the `pygroup` element.