Publish serialized FlatBuffers from the tracker API

docs/source/device/add_device.rst

@@ -65,12 +65,9 @@ Reference schema: :code-file:`src/core/schema/fbs/pedals.fbs`
 
 - **Output table** — The primary payload type (e.g. ``Generic3AxisPedalOutput``) with the
   device fields. This is what the plugin serializes and pushes.
-- **Tracked wrapper** — A table that wraps the output in an optional ``data`` field
-  (e.g. ``Generic3AxisPedalOutputTracked``). Used by the in-memory tracker API so that
-  ``data`` can be null when no sample is available.
 - **Record wrapper** — A table that wraps the output plus ``DeviceDataTimestamp``
   (e.g. ``Generic3AxisPedalOutputRecord``). This is the root type written to MCAP channels
-  by the recorder; trackers serialize into this type in ``serialize_all()``.
+  by the recorder.
 - **root_type** — Set to the Record type (e.g. ``root_type Generic3AxisPedalOutputRecord;``).
 
 Include ``timestamp.fbs`` for ``DeviceDataTimestamp``; include other shared types (e.g.
@@ -126,32 +123,35 @@ tensor samples from OpenXR. Implement a concrete tracker class (e.g.
 - **Factory registration** — Register your tracker in the live factory dispatch table
   (see ``LiveDeviceIOFactory``). The factory constructs an ``ITrackerImpl`` that holds
   a ``SchemaTracker``, builds a ``SchemaTrackerConfig`` from the tracker's stored
-  configuration, and implements ``update(XrTime)`` and
-  ``serialize_all(channel_index, callback)``.
+  configuration, and implements ``update(int64_t monotonic_time_ns)``.
 
 In the **Impl**:
 
-- **update()** — Call ``m_schema_reader.read_all_samples(pending_records)``. If the
-  collection is not present, clear the tracked state (e.g. set ``m_tracked.data = nullptr``).
-  Otherwise, deserialize the latest sample (or all samples) into your tracked type and
-  keep the last one for ``get_data()``.
-- **serialize_all()** — For each sample in the pending batch, deserialize, build the
-  Record FlatBuffer (output table + ``DeviceDataTimestamp``), and invoke the callback with
-  ``(log_time_ns, buffer_ptr, size)``. The buffer is only valid during the callback. If the
-  device disappeared and there are no samples, you may emit one record with null data and
-  the update-tick timestamp so the MCAP stream marks absence.
+- **Construction** — Build the ``SchemaTrackerConfig`` from the tracker's configuration and
+  hand it to the ``SchemaTracker``, along with the MCAP channels (or ``nullptr`` when
+  recording is disabled) and the sub-channel indices to write.
+- **update()** — Call ``m_schema_reader.update(m_tracked)``, where ``m_tracked`` is the
+  published ``Serialized<Generic3AxisPedalOutput>`` handle. That one call reads the pending
+  samples, writes each of them to MCAP when channels are attached, and publishes the final
+  one. A tick with no new samples leaves the last-known handle in place; an absent
+  collection empties it.
+- **get_data()** — Return the published handle. See
+  :ref:`Reading a payload <data-schema-convention>` for what a consumer may assume about it.
+
+Recording needs no per-tracker serialization code: ``SchemaTracker`` writes through
+``McapTrackerChannels``, which wraps the payload in the Record type with its
+``DeviceDataTimestamp``.
 
 Reference implementation — split across facade and live backend:
 
 - **Tracker facade** — :code-file:`src/core/deviceio_trackers/cpp/generic_3axis_pedal_tracker.cpp`
   (class ``Generic3AxisPedalTracker``): holds collection configuration, implements ``ITracker``, and
   exposes ``get_data(session)`` returning
-  ``Generic3AxisPedalOutputTrackedT`` by dispatching to the session’s
+  ``Serialized<Generic3AxisPedalOutput>`` by dispatching to the session’s
   ``IGeneric3AxisPedalTrackerImpl`` (see :code-file:`src/core/deviceio_base/cpp/inc/deviceio_base/generic_3axis_pedal_tracker_base.hpp`).
 - **Live backend** — :code-file:`src/core/live_trackers/cpp/live_generic_3axis_pedal_tracker_impl.cpp`
-  (``LiveGeneric3AxisPedalTrackerImpl``): composes ``SchemaTracker``, implements ``update()`` and
-  ``serialize_all()``, and uses ``SchemaTracker::read_all_samples()`` with
-  ``std::vector<SchemaTracker::SampleResult>`` for the pending batch. See
+  (``LiveGeneric3AxisPedalTrackerImpl``): composes ``SchemaTracker``, owns the MCAP channels, and
+  implements ``update()`` as a single ``SchemaTracker::update()`` call. See
   :code-file:`src/core/live_trackers/cpp/inc/live_trackers/schema_tracker.hpp`
   for ``SchemaTracker`` and ``SampleResult`` (buffer + timestamp metadata).
 
@@ -166,8 +166,8 @@ the collection and prints samples. Pattern (see :code-file:`examples/schemaio/pe
 2. Get required extensions with ``DeviceIOSession::get_required_extensions(trackers)`` and
    create an ``OpenXRSession``.
 3. Create a ``DeviceIOSession`` with ``DeviceIOSession::run(trackers, oxr_session->get_handles())``.
-4. Loop: call ``session->update()``, then read ``tracker->get_data(*session)``. If
-   ``tracked.data`` is non-null, use the latest sample; otherwise sleep briefly and repeat.
+4. Loop: call ``session->update()``, then read ``tracker->get_data(*session)``. If the
+   returned handle is non-empty, use the latest sample; otherwise sleep briefly and repeat.
 
 Use the same ``collection_id`` (and optionally ``tensor_identifier``) as the plugin. See
 :ref:`Schema IO example: build and run <schema-io-example>` above for building and running
@@ -214,12 +214,13 @@ Both exit after 100 samples, or press Ctrl+C to exit early.
   the configured identifier, provides ``push_buffer()`` for raw serialized data. Use composition
   to create typed wrappers (e.g. ``Generic3AxisPedalPusher`` in :code-file:`examples/schemaio/pedal_pusher.cpp`).
 - **SchemaTracker** (``live_trackers``) — Helper for reading FlatBuffer schema data via
-  OpenXR tensor collections: discovers collections by identifier, exposes ``read_all_samples()`` into
-  ``SampleResult`` values. Live tracker implementations (e.g. ``LiveGeneric3AxisPedalTrackerImpl``)
-  compose a ``SchemaTracker`` and implement ``ITrackerImpl::update()`` / ``serialize_all()``.
+  OpenXR tensor collections: discovers collections by identifier, reads pending ``SampleResult``
+  values, records them when MCAP channels are attached, and publishes the final one as a
+  ``Serialized<...>``. Live tracker implementations (e.g. ``LiveGeneric3AxisPedalTrackerImpl``)
+  compose a ``SchemaTracker`` and implement ``ITrackerImpl::update()`` on top of it.
 - **Generic3AxisPedalTracker** (tracker facade in ``deviceio_trackers``) — Concrete ``ITracker`` for
   ``Generic3AxisPedalOutput``: holds configuration and
-  ``get_data(session)`` returning ``Generic3AxisPedalOutputTrackedT`` via the session’s
+  ``get_data(session)`` returning ``Serialized<Generic3AxisPedalOutput>`` via the session’s
   ``IGeneric3AxisPedalTrackerImpl``.
 - **DeviceIOSession** — Session manager: collects required OpenXR extensions from registered
   trackers, creates tracker implementations with session handles, and calls ``update()`` on all

docs/source/device/oglo.rst

@@ -28,7 +28,7 @@ Components
 ----------
 
 - **Schema** — :code-file:`src/core/schema/fbs/oglo_tactile.fbs`
-  (``OgloGloveSample`` / ``OgloGloveSampleTracked`` / ``OgloGloveSampleRecord``).
+  (``OgloGloveSample`` / ``OgloGloveSampleRecord``).
 - **Plugin** — :code-dir:`src/plugins/oglo_tactile` (BLE read → parse → OpenXR push).
 - **Tracker** — ``OgloTactileTracker``
   (:code-file:`src/core/deviceio_trackers/cpp/inc/deviceio_trackers/oglo_tactile_tracker.hpp`)

docs/source/device/trackers.rst

@@ -44,23 +44,17 @@ Data Schema Convention
 ----------------------
 
 Every tracker's data is defined by a FlatBuffers schema under
-:code-dir:`src/core/schema/fbs`. Each schema follows a three-tier convention:
+:code-dir:`src/core/schema/fbs`. Each schema follows a two-tier convention:
 
 .. code-block:: idl
 
-   // 1. Inner data table -- the actual payload
+   // 1. Payload table -- the actual data, and what trackers hand to consumers.
    table Xxx {
        field_a: SomeType (id: 0);
        field_b: AnotherType (id: 1);
    }
 
-   // 2. Tracked wrapper -- used by the in-memory tracker API.
-   //    data is null when the tracked entity is inactive.
-   table XxxTracked {
-       data: Xxx (id: 0);
-   }
-
-   // 3. Record wrapper -- used as the MCAP recording root type.
+   // 2. Record wrapper -- used as the MCAP recording root type.
    //    Adds a DeviceDataTimestamp alongside the payload.
    table XxxRecord {
        data: Xxx (id: 0);
@@ -69,18 +63,39 @@ Every tracker's data is defined by a FlatBuffers schema under
 
    root_type XxxRecord;
 
-- **Inner data table** (e.g. ``HeadPose``, ``HandPose``, ``ControllerSnapshot``) --
-  contains the device-specific fields. All fields are present when the parent
-  wrapper's ``data`` pointer is non-null.
-
-- **Tracked wrapper** (e.g. ``HeadPoseTracked``) -- wraps the inner data in an
-  optional ``data`` field. The in-memory ``get_*()`` accessors return a reference
-  to this wrapper. When ``data`` is ``nullptr`` (C++) or ``None`` (Python), the
-  device is inactive or no sample has arrived yet.
+- **Payload table** (e.g. ``HeadPose``, ``HandPose``, ``ControllerSnapshot``) --
+  contains the device-specific fields. All fields are present whenever the table
+  itself is present.
 
-- **Record wrapper** (e.g. ``HeadPoseRecord``) -- wraps the inner data plus a
+- **Record wrapper** (e.g. ``HeadPoseRecord``) -- wraps the payload plus a
   ``DeviceDataTimestamp``. This is the ``root_type`` written to MCAP channels by
-  the recorder via ``serialize_all()``.
+  the recorder.
+
+Reading a payload
+~~~~~~~~~~~~~~~~~
+
+The ``get_*()`` accessors hand out the payload table itself as an owning handle
+over the encoded bytes -- ``Serialized<HeadPose>`` in C++
+(:code-file:`src/core/schema/cpp/inc/schema/serialized.hpp`), a read-only view
+class (``HeadPose``) in Python. Reads go straight into the buffer, so there is no
+unpack step and joint arrays come back as zero-copy NumPy views.
+
+An **empty handle is the absent payload**: the device is inactive, no sample has
+arrived yet, or replay hit a gap. Test it with ``if (handle)`` in C++; in Python
+the accessor returns ``None``.
+
+Each ``session.update()`` publishes a *new* buffer rather than refilling the
+previous one, so a handle read this frame keeps its values after the next update.
+
+To build a payload from Python, pass every field to its constructor -- these
+views are immutable, so there are no setters.
+
+.. note::
+
+   ``MessageChannelMessagesTracked`` wraps its payload in a table, because that
+   payload is a **list** and something has to hold the vector. ``get_messages()``
+   always returns a non-empty handle; an absent ``data`` vector means no messages
+   arrived this frame.
 
 Shared Types
 ~~~~~~~~~~~~

examples/lerobot/record.py

@@ -120,41 +120,37 @@ def main():
                         session.update()
 
                         # Get hand data
-                        left_tracked: schema.HandPoseTrackedT = (
-                            hand_tracker.get_left_hand(session)
-                        )
-                        right_tracked: schema.HandPoseTrackedT = (
-                            hand_tracker.get_right_hand(session)
+                        left_tracked: schema.HandPose = hand_tracker.get_left_hand(
+                            session
                         )
-                        head_tracked: schema.HeadPoseTrackedT = head_tracker.get_head(
+                        right_tracked: schema.HandPose = hand_tracker.get_right_hand(
                             session
                         )
+                        head_tracked: schema.HeadPose = head_tracker.get_head(session)
 
                         # Extract positions and orientations (with defaults for invalid data)
                         left_pos = np.zeros(3, dtype=np.float32)
                         right_pos = np.zeros(3, dtype=np.float32)
 
-                        if left_tracked.data is not None and left_tracked.data.joints:
-                            wrist = left_tracked.data.joints.poses(deviceio.JOINT_WRIST)
+                        if left_tracked and left_tracked.joints:
+                            wrist = left_tracked.joints.poses(deviceio.JOINT_WRIST)
                             if wrist.is_valid:
                                 pos = wrist.pose.position
                                 left_pos = np.array(
                                     [pos.x, pos.y, pos.z], dtype=np.float32
                                 )
 
-                        if right_tracked.data is not None and right_tracked.data.joints:
-                            wrist = right_tracked.data.joints.poses(
-                                deviceio.JOINT_WRIST
-                            )
+                        if right_tracked and right_tracked.joints:
+                            wrist = right_tracked.joints.poses(deviceio.JOINT_WRIST)
                             if wrist.is_valid:
                                 pos = wrist.pose.position
                                 right_pos = np.array(
                                     [pos.x, pos.y, pos.z], dtype=np.float32
                                 )
 
                         head_pos = np.zeros(3, dtype=np.float32)
-                        if head_tracked.data is not None and head_tracked.data.is_valid:
-                            pos = head_tracked.data.pose.position
+                        if head_tracked and head_tracked.is_valid:
+                            pos = head_tracked.pose.position
                             head_pos = np.array([pos.x, pos.y, pos.z], dtype=np.float32)
 
                         # STEP 3: Record frame to dataset

examples/mcap_record_replay/cpp/record_full_body.cpp

@@ -26,6 +26,7 @@
 #include <deviceio_trackers/full_body_tracker.hpp>
 #include <oxr/oxr_session.hpp>
 #include <schema/full_body_generated.h>
+#include <schema/serialized.hpp>
 
 #include <chrono>
 #include <cstdint>
@@ -70,12 +71,12 @@ std::string resolve_output_path(const std::string& arg)
     return arg;
 }
 
-uint32_t count_valid_joints(const core::FullBodyPoseT& data)
+uint32_t count_valid_joints(const core::FullBodyPose& data)
 {
     uint32_t valid_count = 0;
     for (uint32_t i = 0; i < core::FullBodyTracker::JOINT_COUNT; ++i)
     {
-        if ((*data.joints->joints())[i]->is_valid())
+        if ((*data.joints()->joints())[i]->is_valid())
         {
             ++valid_count;
         }
@@ -123,12 +124,11 @@ try
 
         if (frame_count % 60 == 0)
         {
-            const auto& tracked = tracker->get_body_pose(*session);
+            const auto* body = tracker->get_body_pose(*session).get();
             std::cout << "[record] t=" << std::fixed << std::setprecision(2) << elapsed_s << "s  frame=" << frame_count;
-            if (tracked.data)
+            if (body != nullptr)
             {
-                std::cout << "  joints=" << count_valid_joints(*tracked.data) << "/"
-                          << core::FullBodyTracker::JOINT_COUNT;
+                std::cout << "  joints=" << count_valid_joints(*body) << "/" << core::FullBodyTracker::JOINT_COUNT;
             }
             else
             {

examples/oglo_tactile/oglo_teleop_record.py

@@ -140,9 +140,9 @@ def _make_overlay_layer(
 
 
 def _taxels(tracked) -> np.ndarray | None:
-    if tracked is None or tracked.data is None:
+    if not tracked:
         return None
-    t = tracked.data.taxels
+    t = tracked.taxels
     if not t or len(t) < NUM_TAXELS:
         return None
     return np.asarray(t, dtype=np.float32)[:NUM_TAXELS]

examples/oxr/cpp/oxr_session_sharing.cpp

@@ -5,6 +5,7 @@
 #include <deviceio_trackers/hand_tracker.hpp>
 #include <deviceio_trackers/head_tracker.hpp>
 #include <oxr/oxr_session.hpp>
+#include <schema/serialized.hpp>
 
 #include <chrono>
 #include <iostream>
@@ -85,12 +86,13 @@ try
 
         if (i % 3 == 0)
         {
+            const bool head_valid = head_tracked && head_tracked->is_valid();
             std::cout << "Frame " << i << ": "
-                      << "Hands=" << (left_tracked.data ? "ACTIVE" : "INACTIVE") << " | "
-                      << "Head=" << ((head_tracked.data && head_tracked.data->is_valid) ? "VALID" : "INVALID");
-            if (head_tracked.data && head_tracked.data->is_valid && head_tracked.data->pose)
+                      << "Hands=" << (left_tracked ? "ACTIVE" : "INACTIVE") << " | "
+                      << "Head=" << (head_valid ? "VALID" : "INVALID");
+            if (head_valid && head_tracked->pose() != nullptr)
             {
-                const auto& pos = head_tracked.data->pose->position();
+                const auto& pos = head_tracked->pose()->position();
                 std::cout << " [" << pos.x() << ", " << pos.y() << ", " << pos.z() << "]";
             }
             std::cout << std::endl;

examples/oxr/cpp/oxr_simple_api_demo.cpp

@@ -1,10 +1,11 @@
-// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
 #include <deviceio_session/deviceio_session.hpp>
 #include <deviceio_trackers/hand_tracker.hpp>
 #include <deviceio_trackers/head_tracker.hpp>
 #include <oxr/oxr_session.hpp>
+#include <schema/serialized.hpp>
 
 #include <iostream>
 #include <memory>
@@ -84,14 +85,13 @@ try
         const auto& head_tracked = head_tracker->get_head(*session);
 
         std::cout << "Frame " << i << ":" << std::endl;
-        std::cout << "  Left hand:  " << (left_tracked.data ? "ACTIVE" : "INACTIVE") << std::endl;
-        std::cout << "  Right hand: " << (right_tracked.data ? "ACTIVE" : "INACTIVE") << std::endl;
-        std::cout << "  Head pose:  " << ((head_tracked.data && head_tracked.data->is_valid) ? "VALID" : "INVALID")
-                  << std::endl;
+        std::cout << "  Left hand:  " << (left_tracked ? "ACTIVE" : "INACTIVE") << std::endl;
+        std::cout << "  Right hand: " << (right_tracked ? "ACTIVE" : "INACTIVE") << std::endl;
+        std::cout << "  Head pose:  " << ((head_tracked && head_tracked->is_valid()) ? "VALID" : "INVALID") << std::endl;
 
-        if (head_tracked.data && head_tracked.data->is_valid)
+        if (head_tracked && head_tracked->is_valid())
         {
-            const auto& pos = head_tracked.data->pose->position();
+            const auto& pos = head_tracked->pose()->position();
             std::cout << "    Position: [" << pos.x() << ", " << pos.y() << ", " << pos.z() << "]" << std::endl;
         }
         std::cout << std::endl;