feat(pose): add model-free pose geometry, features, and scenario queries - #29
feat(pose): add model-free pose geometry, features, and scenario queries#29everettVT wants to merge 1 commit into
Conversation
Ports daft_physical_ai.pose from the e2e demo: - state/skeleton: vectorized NumPy geometry over the 48-D hand state and 204-D body skeleton (curl, pinch, palm normal, finger flexion, arm extension, forearm axis, hand-local joints) - features: EpisodeFeatureComputer — one NumPy pass per episode assembles state-only and full skeleton-derived track sets; TemporalFeatureComputer provides forward-difference rates - temporal: in-DAG twin via @daft.func + window expressions (lead(1), euclidean_distance, rows_between smoothing); stays lazy end to end - query: scenario predicates (grasping, lifting, writing/hammer grip, reaching, in-hand, twisting, openness) + percentile calibration + segment stitching - examples/03_transforms/: pose_features_numpy.py (episode NumPy pass) and pose_rates_in_dag.py (distributed window-expression twin) - 19 new tests covering all geometry primitives, track assembly, scenarios, calibration, and temporal plan construction Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ab461b58a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for side in SIDES: | ||
| joint_names += [side + part for part in ("Hand", "Forearm", "Arm", "Shoulder")] |
There was a problem hiding this comment.
For real EgoDex skeleton arrays this indexes every joint from the wrong offset: the existing package order in daft_physical_ai/datasets/egodex.py starts hip, leftArm, leftForearm, leftHand, while this new JOINT_NAMES builder places leftHand at index 0. Any 204-D vector flattened in the repo's EgoDex order will make joint_position(..., "leftHand") read the hip coordinates, so all skeleton-derived features (closure, reach, roll, etc.) are corrupted rather than just noisy.
Useful? React with 👍 / 👎.
| ``hand`` is ``'left'`` / ``'right'`` / ``'either'`` (masks OR together). | ||
| """ | ||
| scenario = pose if callable(pose) else SCENARIOS[pose] | ||
| tags = {"left": ("L",), "right": ("R,")}.get(hand, ("L", "R")) |
There was a problem hiding this comment.
When callers request hand="right", this returns the tag "R," and the loop immediately indexes tracks_by_tag["R,"], but EpisodeFeatureComputer and the docstring use "R". Right-hand-only scenario queries therefore raise KeyError instead of evaluating the right hand; the existing test only covers the default either path.
Useful? React with 👍 / 👎.
| sums = np.convolve(values, kernel, mode="same") | ||
| counts = np.convolve(np.ones_like(values), kernel, mode="same") | ||
| return sums / counts |
There was a problem hiding this comment.
For episodes shorter than the default five-frame smoothing window, np.convolve(..., mode="same") returns the kernel length rather than len(values), so centered_mean returns 5 roll samples for a 1–4 frame episode. EpisodeFeatureComputer.compute then emits roll_L/roll_R tracks longer than num_frames and the other tracks, which breaks twisting masks and any downstream per-frame alignment on short clips.
Useful? React with 👍 / 👎.
1 participant
Summary
daft_physical_ai.pose.state— pure-NumPy per-frame geometry over the 48-D two-hand state vector: wrist position, rot6d palm normal, fingertip distances (curl, pinch, aperture)daft_physical_ai.pose.skeleton— per-frame geometry over the 204-D body skeleton: finger flexion, palm plane normal, arm extension, forearm axis, hand-local joint frame (68-joint EgoDex convention)daft_physical_ai.pose.features—EpisodeFeatureComputer: one vectorized NumPy pass per episode assembles state-only and full skeleton track sets;TemporalFeatureComputeradds forward-difference ratesdaft_physical_ai.pose.temporal— in-DAG twin via@daft.func+ window expressions (lead(1),euclidean_distance,rows_betweensmoothing); stays lazy from reader to single collectdaft_physical_ai.pose.query— scenario predicates (grasping, lifting, writing grip, hammer grip, reaching, in-hand manipulation, twisting, openness), percentile calibration over a corpus, and segment stitchingexamples/03_transforms/pose_features_numpy.py— episode NumPy pass over a public EgoDex LeRobot sampleexamples/03_transforms/pose_rates_in_dag.py— distributed window-expression twin; one lazy plan to a single collecttests/test_pose.py— 19 tests covering all geometry primitives, track assembly, scenario predicates, calibration, and temporal plan constructionValidation
uv run pytest tests/ -v— 72 passed, 2 skippedpre-commit run --all-files— all 13 hooks pass