Fine-grained Yoga-82 recognition with frozen HMR, SMPL pose features, RGB appearance, and hierarchical classification
本项目面向 Yoga-82 细粒度瑜伽体式识别，研究冻结的人体网格恢复模型能否提供比二维关键点更有判别力的三维姿态表示。系统覆盖数据完整性审计、HMR2/SMPL 特征提取、6/20/82 类层次分类、RGB 基线、后期融合、误差分析与三维网格可视化。
This is a reproducible feasibility study—not a medical, coaching, or ground-truth 3D pose system.
- 🎯 Hierarchical Yoga-82 recognition — predicts coarse, intermediate, and fine-grained labels across the 6/20/82-class hierarchy.
- 🧍 Multiple pose representations — compares projected HMR joints (48D), canonicalized 3D joints (72D), SMPL joint rotations (144D), and a full 3D representation (216D).
- 🧹 Leakage-aware evaluation — audits URLs, decoded files, exact duplicates, and cross-split contamination before training.
- 🔀 RGB + 3D late fusion — combines image appearance with recovered body geometry for the strongest result.
- 📊 Matched and reproducible experiments — reports mean ± sample standard deviation over seeds 2026, 2027, and 2028.
- 🧊 Inspectable 3D output — exports SMPL estimates as multiview renders, GLB meshes, and an interactive browser viewer.
Yoga-82 images
↓
Integrity audit → primary-person detection → frozen HMR2 / SMPL extraction
↓
┌────────────┬────────────┬────────────┬────────────┐
│ 48D HMR 2D │ 72D joints │ 144D rot. │ 216D full │
└────────────┴────────────┴────────────┴────────────┘
↓
Hierarchical 6 / 20 / 82 classifier
↓
RGB image → fine-tuned ResNet-50 ───────→ score-level late fusion
↓
metrics · error analysis · 3D visualization
HMR2 acts as a frozen representation extractor. The project evaluates how useful its recovered geometry is for downstream classification; it does not train HMR2 end to end or claim ground-truth reconstruction accuracy.
Results use the leakage-audited Yoga-82 clean-v2 split. Values are mean ± sample standard deviation across three random seeds.
| Representation | Dimensions | Top-1 (%) ↑ | Top-5 (%) ↑ | Macro-F1 (%) ↑ |
|---|---|---|---|---|
| Projected HMR joints | 48 | 88.28 ± 0.59 | 97.16 ± 0.15 | 86.34 ± 0.80 |
| Canonicalized 3D joints | 72 | 88.46 ± 0.85 | 97.51 ± 0.15 | 87.00 ± 0.57 |
| SMPL joint rotations | 144 | 90.48 ± 0.69 | 97.47 ± 0.00 | 89.29 ± 0.84 |
| Full 3D representation | 216 | 91.05 ± 0.28 | 97.92 ± 0.18 | 89.81 ± 0.29 |
| Fine-tuned ResNet-50 | RGB | 82.59 ± 0.96 | 95.86 ± 0.08 | 79.70 ± 0.94 |
| RGB + full 3D late fusion | — | 92.87 ± 0.23 | 98.79 ± 0.20 | 92.06 ± 0.29 |
- The 216D full 3D representation improves Top-1 accuracy by 2.77 percentage points over its matched HMR-projected 2D representation (91.05% vs. 88.28%).
- On the stricter 10,263-image common subset, full SMPL/3D reaches 91.52% Top-1, exceeding the strongest independent 2D baseline in this study (MediaPipe) by 6.87 points.
- RGB + full 3D late fusion achieves the best overall result: 92.87% Top-1, 98.79% Top-5, and 92.06% Macro-F1.
- The gains measure downstream representation utility, not absolute 3D reconstruction accuracy.
Detailed tables are available in experiments/clean_v2/paper_tables/, with pose-estimation comparisons in experiments/pose2d_benchmarks_matched_clean_v2/.
The montage compares three independent 2D pose estimators with the HMR2 mesh overlay and recovered SMPL body from the same inputs. Chair Pose, Handstand, and Crow Pose illustrate successful recovery patterns; the final Yogic Sleep row is intentionally retained as an honest HMR failure case. These examples support qualitative inspection only and are not evidence of ground-truth 3D reconstruction accuracy.
The repository owner supplied this composite for research presentation. Verify redistribution rights for the underlying source photographs before reusing it outside this repository.
Explore the included interactive 3D viewer or download the mesh.glb asset. This example is a frozen HMR/SMPL estimate rather than ground-truth anatomy. The Yoga-82 source photograph is intentionally excluded; SMPL-Body attribution is documented here.
- Python 3.10+
- PyTorch 2.7+
- A CUDA-capable GPU is recommended for HMR extraction and training
- Licensed SMPL assets are required only for mesh recovery/rendering workflows
git clone https://github.com/BoPythonAI/3DYoga.git
cd 3DYoga
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install -e .pytest -qThe public snapshot was verified with 53 passing tests on 2026-09-02.
The pipeline is split into focused Python modules. Each command exposes its own arguments:
python -m yoga3d.download_yoga82 --help
python -m yoga3d.clean_yoga82 --help
python -m yoga3d.prepare_manifest --help
python -m yoga3d.train_classifier --help
python -m yoga3d.late_fuse_rgb_pose --help
python -m yoga3d.generate_paper_figures --helpHMR2 extraction additionally needs the external 4D-Humans/HMR2 environment described in docs/FULL_EXTRACTION_V1.md.
Optional independent 2D baselines require:
python -m pip install -e '.[pose2d-benchmarks]'- Data audit — download metadata, validate decoded images, remove exact duplicates, and freeze the leakage-audited
clean-v2split. - Human recovery — detect the primary person and extract frozen HMR2/SMPL outputs.
- Feature construction — derive projected 2D, canonicalized 3D joints, SMPL rotations, and the full 216D representation.
- Matched training — train hierarchical pose classifiers and the RGB ResNet-50 baseline under aligned splits and seeds.
- Fusion and evaluation — tune score-level late fusion, calculate Top-1/Top-5/Macro-F1, and analyze per-class errors and HMR quality.
- Reporting — generate publication tables, figures, mesh views, and blinded review material.
For exact experiment settings, read the full Chinese report and the versioned protocols in docs/.
3DYoga/
├── configs/ # Storage and runtime configuration
├── docs/ # Protocols, audits, stage reports, 3D demo
│ └── assets/ # README architecture assets
├── experiments/ # Metrics, tables, predictions, and figures
│ ├── clean_v2/ # Main leakage-audited experiments
│ └── pose2d_benchmarks_*/ # Independent 2D comparisons
├── paper/ # Chinese final report and evidence maps
├── scripts/ # Environment and report utilities
├── src/yoga3d/ # Data, HMR, training, fusion, evaluation
├── tests/ # Unit and integrity tests
├── pyproject.toml # Package metadata and dependencies
└── README.md
Datasets, virtual environments, HMR checkpoints, cached features, SMPL model files, logs, per-seed histories, per-image predictions, and other large outputs are intentionally excluded from Git. Curated aggregate metrics, tables, and figures remain versioned.
To reproduce the full pipeline, obtain Yoga-82, HMR2/4D-Humans, and licensed SMPL assets from their respective official sources. Do not commit private model files, dataset images, credentials, or machine-specific paths.
This repository establishes an engineering and representation-learning baseline. It does not claim:
- medical effectiveness or corrective coaching;
- ground-truth 3D pose accuracy;
- multi-person interaction support;
- end-to-end HMR training.
External HMR ratings and final adjudication of the remaining perceptual near-duplicate candidates are still pending.
Issues and pull requests are welcome. Please keep experiments reproducible, add tests for behavior changes, and avoid committing datasets or licensed model assets.
This project builds on Yoga-82, HMR2/4D-Humans, SMPL, PyTorch, and torchvision. Please follow the original licenses and citation requirements of each upstream project and dataset.