Cairn is a scenario mining tool for autonomous vehicle data built in Rust. It lets engineers query a large multi-sensor driving dataset using human-readable conditions and instantly replay the matching clips as a synchronized 3D visualization of ego trajectory, LiDAR point clouds, and labeled bounding boxes.
A developer runs the Cairn backend once against a local copy of the NVIDIA PhysicalAI Autonomous Vehicles dataset. On startup, the backend registers all sensor data — ego motion, LiDAR, camera timestamps, and obstacle detections — as queryable SQL tables backed by Parquet files on disk. No data is copied or transformed; the raw dataset files are indexed in place using DataFusion, an embedded columnar query engine. This means the first query is fast — there is no ingestion pipeline to wait for.
An engineer opens the Cairn frontend, a native desktop app built in Rust with egui. The frontend fetches the available schema and obstacle classes from the backend on startup and presents them as interactive filters. The engineer selects one or more obstacle classes — e.g. car and person — sets a minimum deceleration threshold, and clicks Replay.
The frontend sends that query to the backend as a single HTTP request. The backend translates it into a DataFusion SQL query that joins ego motion, obstacle detections, and LiDAR availability across the dataset, and applies a HAVING COUNT(DISTINCT label_class) condition to find clips containing all selected classes. It returns matching clip UUIDs in a few seconds — scanning potentially hundreds of gigabytes of Parquet with predicate pushdown rather than loading it all into memory.
For each matching clip the backend fetches ego motion samples, Draco-decoded LiDAR point clouds, and per-frame obstacle bounding boxes, then streams them all into a running Rerun viewer. The result is a scrubable 3D timeline showing:
- The vehicle's path as a line strip
- Rotating LiDAR scans as point clouds
- Tracked obstacle bounding boxes that appear, move, and disappear in sync with detections
All sensor streams are locked to the same ego_time timeline so they are temporally aligned. The engineer can scrub through the clip, inspect individual frames, and immediately understand the driving context of the scenario they queried for.
The primary objective of Cairn is scenario mining. Training multimodal machine learning models for autonomous vehicles requires large amounts of diverse data. If the scenarios a model trains on are not sufficiently diverse, it leads to overfitting — high accuracy on training data but poor generalization to new scenarios.
Cairn addresses this by making it fast and interactive to find specific driving scenarios within a large dataset. For example: find all clips where the vehicle was decelerating above 2.5 m/s² that contain both a car and a person. This is done using a compiled, async Rust service with DataFusion as an embedded columnar query engine. This means:
- Fast predicate pushdown over large Parquet datasets — filtering is performed on the source files, not in memory
- Concurrent queries without the overhead of a Python runtime
- Live streaming of query results directly into a 3D visualization tool for immediate validation
- Rust 1.88+
- cmake — required for Draco C++ point cloud bindings:
brew install cmake - A HuggingFace account with the NVIDIA PhysicalAI-AV license accepted
- Rerun viewer:
cargo install rerun-cli --locked
Clone the repo
git clone https://github.com/Vinnstah/cairn
cd cairnDownload a subset of the dataset from HuggingFace and place it under ./data/nvidia_physical_dataset.
Run the backend
cargo run -p cairnOnce the backend has registered its tables, run the frontend
cargo run -p cairn-uiNote: Further improvements to the startup procedure are planned.
Search for matching clip IDs or replay matching clips into Rerun
curl "http://localhost:3000/clips/search?min_speed=15.0&min_decel=2.5"
curl "http://localhost:3000/clips/replay?min_speed=15.0&min_decel=2.5"Each request queries the Parquet files via DataFusion SQL, finds matching clip UUIDs, then streams their ego motion trajectory and Draco-decoded LiDAR point clouds into the running Rerun viewer.
Cairn is configured via a dataset.toml file. This tells the backend which datasets to register, where to find them on disk, and what semantic meaning their columns carry.
Each dataset is declared as an entry in the [[datasets]] array. The backend iterates these on startup, registers each folder as a queryable SQL table, and injects clip_id from filenames automatically.
[[datasets]]
name = "ego_motion" # SQL table name used in queries
path = "/path/to/egomotion.chunk_0000"
description = "Ego motion for the NVIDIA PhysicalAI-AV dataset"
file_ext = ".egomotion.parquet" # only files with this suffix are registered
[datasets.characteristics.semantics]
timestamp = "timestamp" # which column holds the relative timestampThe optional [datasets.characteristics] section tells Cairn how to treat this dataset during replay and scenario mining:
| Key | Type | Effect |
|---|---|---|
contains_lidar |
bool | Marks this dataset as the LiDAR source. Clips must have a matching entry here to be returned by search queries. |
contains_classes |
bool | Marks this dataset as the obstacle/classification source. The label_class semantic column is used to populate the frontend filter pills and label_classes search params. |
[datasets.characteristics.semantics] maps logical roles to actual column names in the Parquet files:
| Key | Effect |
|---|---|
timestamp |
Column used to align this dataset to ego_time during replay |
label_class |
Column queried for DISTINCT values shown as filter pills in the frontend, and used in WHERE label_class IN (...) search queries |
[[datasets]]
name = "ego_motion"
path = "/data/nvidia_physical_dataset/egomotion.chunk_0000"
description = "Ego motion — pose, velocity, acceleration at ~100 Hz"
file_ext = ".egomotion.parquet"
[datasets.characteristics.semantics]
timestamp = "timestamp"
[[datasets]]
name = "lidar"
path = "/data/nvidia_physical_dataset/lidar.chunk_0000"
description = "LiDAR point clouds — Draco-encoded, ~200 spins per clip at 10 Hz"
file_ext = ".lidar_top_360fov.parquet"
[datasets.characteristics]
contains_lidar = true
[datasets.characteristics.semantics]
timestamp = "spin_start_timestamp"
[[datasets]]
name = "obstacles"
path = "/data/nvidia_physical_dataset/obstacle.offline.chunk_0000"
description = "Obstacle detections with bounding boxes and class labels"
file_ext = ".obstacle.offline.parquet"
[datasets.characteristics]
contains_classes = true
[datasets.characteristics.semantics]
label_class = "label_class"pathshould be an absolute path to the chunk folder. All.parquetfiles matchingfile_extinside that folder will be registered.clip_iddoes not need to be declared — it is injected automatically from the filename prefix (<uuid>.<file_ext>).- Datasets without a
characteristicssection are registered and queryable but are not used for search filtering or replay routing.
The app is split into 3 different crate: backend, frontend and shared.
shared— types shared between frontend and backend (ClipSearchParams,ColumnInfo,SchemaResponse,CairnError)backend— ports-and-adapters (hexagonal) architecture; core domain has no infrastructure dependenciesfrontend— native desktop GUI built with eframe and egui
Clip ID injection — The NVIDIA dataset stores clip identity only in filenames (<uuid>.egomotion.parquet), not as a column inside the Parquet files. Cairn works around this by registering each file individually as a sub-table and combining them into a CREATE VIEW with the UUID injected as a literal clip_id column, making cross-table joins possible without modifying the raw data.
Draco decoding — LiDAR point clouds are stored as Draco-compressed binary blobs in BinaryView Arrow columns. The spatial_codec_draco crate requires a color attribute that this dataset does not provide. Cairn uses draco-rs (Rust bindings to the C++ Draco library, requires cmake) which decodes geometry-only point clouds via PointCloud::from_buffer and get_point_alloc::<f32, 3>.
Obstacle tracking in Rerun — Each tracked obstacle is logged to its own entity path (world/obstacles/<label_class>/<track_id>). This uses Rerun's "latest at" semantics so each box persists until its next update. A Clear::flat() is logged just before the next detection for that track so boxes disappear when the object leaves the sensor's field of view rather than persisting indefinitely.
Orphan rule workaround — Implementing TryFrom<RecordBatch> for Vec<PointCloud> violates Rust's orphan rule since both types are foreign. Cairn uses a PointClouds(Vec<PointCloud>) newtype wrapper to implement the conversion cleanly.
Arrow version isolation — Rerun and DataFusion both re-export Arrow but pin different versions. All RecordBatch processing uses datafusion::arrow types exclusively. Rerun types (Points3D, Transform3D, Boxes3D) are constructed from plain Rust primitives at the SceneLogger adapter boundary.