feat(examples): reading-data stage + numbered examples taxonomy (phase 1)

AGENTS.md

@@ -43,9 +43,10 @@ Working implementations to port from: multibase `src/post7_hand_tracking/egodex_
   batched video decode) - tracked with full steps in
   [#17](https://github.com/Eventual-Inc/daft-physical-ai/issues/17).
 
-# Regenerating the examples demo
+# Regenerating the hand-tracking demo
 
-`examples/{demo.py,demo.ipynb,demo.md,demo_keypoints.png}` are **generated** -
+`examples/04_episode_operations/hand_tracking/{demo.py,demo.ipynb,demo.md,demo_keypoints.png}`
+are **generated** -
 don't hand-edit them. They all render from one shared cell list in
 `daft_physical_ai/_render.py`, so editing the source keeps the three formats in
 sync. To rebuild them:

README.md

@@ -81,13 +81,13 @@ list[list[float32]], kp3d: list[list[float32]] }]`, defined as `HANDS_DTYPE` in
 A complete walkthrough - read a dataset, run `track_hands` (MediaPipe), draw the
 keypoints, and score against EgoDex ground truth:
 
-![track_hands keypoints](examples/demo_keypoints.png)
+![track_hands keypoints](examples/04_episode_operations/hand_tracking/demo_keypoints.png)
 
 Available in three equivalent forms:
 
-- **[examples/demo.md](examples/demo.md)** - read it start to finish; code and outputs inline.
-- **[examples/demo.ipynb](examples/demo.ipynb)** - runnable notebook (outputs included).
-- **[examples/demo.py](examples/demo.py)** - plain script.
+- **[examples/04_episode_operations/hand_tracking/demo.md](examples/04_episode_operations/hand_tracking/demo.md)** - read it start to finish; code and outputs inline.
+- **[examples/04_episode_operations/hand_tracking/demo.ipynb](examples/04_episode_operations/hand_tracking/demo.ipynb)** - runnable notebook (outputs included).
+- **[examples/04_episode_operations/hand_tracking/demo.py](examples/04_episode_operations/hand_tracking/demo.py)** - plain script.
 
 Generate your own (other methods, a Modal GPU runtime, with/without eval) with the
 `daft-physical-ai hands` command - run it with no flags for an interactive

examples/01_reading_data/README.md

@@ -0,0 +1,16 @@
+# 01 - Reading data
+
+Get robot datasets into Daft. Daft's native readers do the heavy lifting;
+these scripts show the minimal, copyable pattern per source.
+
+- `droid_episode_index.py` - `daft.datasets.droid.raw()`: filter successful
+  episodes and project an operational episode index, lazily.
+- `lerobot_episode_index.py` - `daft.datasets.lerobot` (Daft >= 0.7.17):
+  episode/task/frame views of a LeRobot v3 dataset, filtered without decoding
+  any video.
+- `egodex_raw_hdf5_video.py` - raw EgoDex episodes from a locally extracted
+  release via `daft_physical_ai.datasets.egodex`: lazy `hdf5_file` /
+  `video_file` access, no conversion step.
+
+Planned: `mcap_topics.py` - Daft reads MCAP natively as of 0.7.19; a robot-log
+topic-extraction example is queued.

examples/01_reading_data/droid_episode_index.py

@@ -0,0 +1,34 @@
+from __future__ import annotations
+
+import daft
+from daft.datasets import droid
+
+
+def build_episode_index() -> daft.DataFrame:
+    """Build a lazy DROID episode index using released Daft APIs."""
+    episodes = droid.raw()
+
+    successful_episodes = episodes.where(daft.col("success") == daft.lit(True))
+
+    return successful_episodes.select(
+        "uuid",
+        "scene_id",
+        "building",
+        "current_task",
+        "success",
+        "trajectory_length",
+        "wrist_cam_video",
+        "ext1_cam_video",
+        "ext2_cam_video",
+    )
+
+
+def main() -> None:
+    episode_index = build_episode_index()
+
+    # Inspect the lazy plan before materializing remote data.
+    episode_index.explain(show_all=True)
+
+
+if __name__ == "__main__":
+    main()

examples/01_reading_data/lerobot_episode_index.py

@@ -0,0 +1,46 @@
+"""Index a LeRobot v3 dataset without decoding a single video frame.
+
+`daft.datasets.lerobot` (Daft >= 0.7.17) reads a LeRobot dataset lazily:
+`read_episodes` gives one row per episode straight from the metadata,
+`read_tasks` the task table, and `read` one row per frame with episode
+metadata broadcast on - video stays undecoded until you ask for it with
+``load_video_frames``. The dataset here is a tiny EgoDex sample (3 episodes /
+632 frames) in LeRobot v3 format.
+"""
+
+from __future__ import annotations
+
+import argparse
+
+from daft import col
+from daft.datasets import lerobot
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Episode/task/frame views of a LeRobot dataset.")
+    parser.add_argument("--dataset", default="pepijn223/egodex-test", help="HF repo id or path")
+    parser.add_argument("--min-length", type=int, default=100, help="episode-length filter to demo")
+    args = parser.parse_args()
+
+    episodes = lerobot.read_episodes(args.dataset)
+    index = episodes.select("episode_index", "tasks", "length").sort("episode_index").to_pydict()
+    print(f"{args.dataset}: {len(index['episode_index'])} episodes")
+    for episode_index, tasks, length in zip(index["episode_index"], index["tasks"], index["length"]):
+        print(f"  episode {episode_index}: {length:4d} frames  {tasks[0][:70]}")
+
+    tasks = lerobot.read_tasks(args.dataset).to_pydict()
+    print(f"\n{len(next(iter(tasks.values()), []))} distinct tasks in the task table")
+
+    long_episodes = episodes.where(col("length") >= args.min_length)
+    frames = lerobot.read(args.dataset).join(
+        long_episodes.select("episode_index"), on="episode_index", how="semi"
+    )
+    print(
+        f"frames in episodes with >= {args.min_length} steps: {frames.count_rows()} "
+        f"(selected without touching any video)"
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

examples/02_episode_data/README.md

@@ -0,0 +1,10 @@
+# 02 - Episode data
+
+Episode-level views over robot datasets.
+
+- `merge_lerobot_datasets.py` - merge two LeRobot recording sessions into one
+  training table: re-index `episode_index` and the global frame `index`, then
+  concat - the collision-prone part of combining recordings, as one Daft query.
+
+Planned: normalizing demonstrations into the canonical one-row-per-step
+contract (lands with the episode-contract PR), `episode_stats.py`.

examples/02_episode_data/merge_lerobot_datasets.py

@@ -0,0 +1,65 @@
+"""Merge two LeRobot recording sessions into one training table.
+
+Robot data arrives in batches - yesterday's teleop session and today's land as
+two LeRobot datasets, and both number their episodes from zero. Merging them
+is an index problem: ``episode_index`` and the global frame ``index`` collide,
+so the second session must be re-indexed before the tables can stack. With the
+frames as a lazy dataframe that is an offset and a concat.
+
+This demo reads the same tiny v3 dataset twice as "session A" and "session B"
+(a second public v3 dataset is not available yet); the mechanics are exactly
+those of merging distinct recordings from one rig. Task strings ride along on
+every frame, so task identity survives the merge without a task_index remap.
+"""
+
+from __future__ import annotations
+
+import argparse
+
+from daft import col, lit
+from daft.datasets import lerobot
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Merge two LeRobot sessions with re-indexed episodes.")
+    parser.add_argument("--session-a", default="pepijn223/egodex-test")
+    parser.add_argument("--session-b", default="pepijn223/egodex-test")
+    args = parser.parse_args()
+
+    frames_a = lerobot.read(args.session_a)
+    frames_b = lerobot.read(args.session_b)
+
+    # Session A's extent decides session B's offsets.
+    extent = frames_a.agg(
+        (col("episode_index").max() + lit(1)).alias("n_episodes"),
+        (col("index").max() + lit(1)).alias("n_frames"),
+    ).to_pydict()
+    episode_offset, frame_offset = extent["n_episodes"][0], extent["n_frames"][0]
+
+    merged = frames_a.concat(
+        frames_b.with_column("episode_index", col("episode_index") + lit(episode_offset)).with_column(
+            "index", col("index") + lit(frame_offset)
+        )
+    )
+
+    lengths = (
+        merged.groupby("episode_index")
+        .agg(col("frame_index").count().alias("frames"))
+        .sort("episode_index")
+        .to_pydict()
+    )
+    print(f"merged: {len(lengths['episode_index'])} episodes / {sum(lengths['frames'])} frames")
+    for episode_index, frames in zip(lengths["episode_index"], lengths["frames"]):
+        source = "A" if episode_index < episode_offset else "B"
+        print(f"  episode {episode_index} (session {source}): {frames} frames")
+
+    n_frames = merged.count_rows()
+    n_distinct = merged.select("index").distinct().count_rows()
+    assert n_frames == n_distinct, "global frame index must stay unique after the merge"
+    print(f"\nglobal frame index unique after re-indexing: {n_distinct}/{n_frames}")
+    print("write it back out: merged.write_parquet(...) - one table, ready for training prep.")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

examples/04_episode_operations/hand_tracking/README.md

@@ -0,0 +1,32 @@
+# Hand tracking (EgoDex, MediaPipe)
+
+A complete hand-tracking walkthrough, generated by the `daft-physical-ai` CLI:
+read a LeRobot dataset, run `track_hands` (MediaPipe, CPU), draw the keypoints
+against the EgoDex ground truth, and score them (detect% + PCK).
+
+![ground truth vs predictions](demo_keypoints.png)
+
+Three equivalent forms:
+
+- **`demo.md`** - read it start to finish; code and outputs inline, nothing to run.
+- **`demo.ipynb`** - the same, executed (outputs included); open in JupyterLab.
+- **`demo.py`** - plain script.
+
+Run them:
+
+```bash
+pip install "daft-physical-ai[mediapipe]" matplotlib scipy
+python examples/04_episode_operations/hand_tracking/demo.py
+# or: jupyter lab examples/04_episode_operations/hand_tracking/demo.ipynb
+```
+
+Want a different setup (WiLoR, both methods, a Modal GPU runtime, with/without
+eval)? Generate your own:
+
+```bash
+daft-physical-ai            # interactive
+daft-physical-ai --method wilor --runtime modal --mano-path MANO_RIGHT.pkl --no-input
+```
+
+> The committed files here are *executed* (so outputs and the image show without
+> running). The CLI generates the same structure as a fresh starting point.

examples/README.md

@@ -1,33 +1,22 @@
 # Examples
 
-A complete hand-tracking walkthrough, generated by the `daft-physical-ai` CLI:
-read a LeRobot dataset, run `track_hands` (MediaPipe, CPU), draw the keypoints
-against the EgoDex ground truth, and score them (detect% + PCK).
-
-![ground truth vs predictions](demo_keypoints.png)
-
-Three equivalent forms:
-
-- **`demo.md`** - read it start to finish; code and outputs inline, nothing to run.
-- **`demo.ipynb`** - the same, executed (outputs included); open in JupyterLab.
-- **`demo.py`** - plain script.
-
-Run them:
-
-```bash
-# deps fetched on the fly, nothing to install
-uv run --with "daft-physical-ai[mediapipe]" --with matplotlib --with scipy examples/demo.py
-# or in JupyterLab:
-uvx --from jupyterlab --with "daft-physical-ai[mediapipe]" --with matplotlib --with scipy jupyter-lab examples/demo.ipynb
-```
-
-Want a different setup (WiLoR, both methods, a Modal GPU runtime, with/without
-eval)? Generate your own:
-
-```bash
-daft-physical-ai hands      # interactive
-daft-physical-ai hands --method wilor --runtime modal --mano-path MANO_RIGHT.pkl --no-input
-```
-
-> The committed files here are *executed* (so outputs and the image show without
-> running). The CLI generates the same structure as a fresh starting point.
+Runnable physical-AI data recipes on Daft, numbered as the workflow a
+researcher actually runs: read datasets, inspect episode data, transform,
+run episode operations, label with inference, write outputs, hand off to
+training, analyze policy evals. Stages land incrementally; directories
+marked *planned* are reserved, with their target scripts named here.
+
+| # | Stage | What it covers | Status |
+|---|---|---|---|
+| 01 | [Reading data](01_reading_data/) | Robot datasets into Daft: DROID metadata, LeRobot v3 episode/task/frame views, raw EgoDex HDF5+video | `droid_episode_index.py` · `lerobot_episode_index.py` · `egodex_raw_hdf5_video.py` |
+| 02 | [Episode data](02_episode_data/) | Episode-level views and dataset combination | `merge_lerobot_datasets.py`; normalization lands with the episode contract |
+| 03 | Transforms | Deterministic NumPy features as episode passes and in-plan expressions | planned |
+| 04 | [Episode operations](04_episode_operations/) | Packaged robotics ops over episodes | [`hand_tracking/`](04_episode_operations/hand_tracking/); motion trim and pose queries planned |
+| 05 | Inference | Model-backed labeling with Daft AI functions | planned |
+| 06 | Writing data | Curated training artifacts as views | planned |
+| 07 | Training handoff | Curated dataframes into `to_torch_dataloader` | planned |
+| 08 | Policy evals | Benchmark reproduction and failure mining over rollout parquet | planned |
+
+Every landed example runs first-try on a clean environment against public
+data (EgoDex raw reading expects a locally extracted release - see its
+docstring).

pyproject.toml

@@ -9,7 +9,7 @@ description = "Physical-AI data processing on Daft, starting with hand tracking.
 # (Daft#7184) the demo depends on for reasonable remote-read performance. The
 # [video] brings av + pillow for video decode; [hdf5] supports raw EgoDex
 # metadata and trajectory files. Both are part of the normal package surface.
-dependencies = ["daft[hdf5,video]>=0.7.18", "numpy"]
+dependencies = ["daft[hdf5,video]>=0.7.19", "numpy"]
 dynamic = ["version"]
 license = "Apache-2.0"
 authors = [{name = "Eventual"}]

scripts/regen_demo.py

@@ -137,7 +137,11 @@ def _execute(nb_path: Path) -> None:
 
 def main(argv: list[str] | None = None) -> int:
     p = argparse.ArgumentParser(description="Regenerate the examples/ demo programmatically.")
-    p.add_argument("--output-dir", default="examples", help="where to write the demo (default: examples)")
+    p.add_argument(
+        "--output-dir",
+        default="examples/04_episode_operations/hand_tracking",
+        help="where to write the demo (default: the committed hand-tracking example)",
+    )
     p.add_argument("--skip-exec", action="store_true", help="reuse --source instead of executing a fresh notebook")
     p.add_argument("--source", help="executed notebook to reuse with --skip-exec (default: <output-dir>/demo.ipynb)")
     args = p.parse_args(argv)