Direct extraction of clinical joint angles from parametric body-model rotation matrices. Companion code release for the paper Direct Clinical Joint Angle Extraction from Parametric Body Model Rotation Matrices (Kardolus et al., 2026).
License: Babon Noncommercial Research License 1.0 — free for personal use and non-commercial academic research; commercial and clinical use need a license from support@babon.eu. Not a medical device. See License below.
You give Movalytics per-segment rotation matrices that a body model already produces (one 3×3 per segment per frame). You get back clinical joint angles in ISB-compatible conventions — knee flexion, hip rotation, shoulder abduction, etc. — sub-millisecond per joint per frame, no inverse-kinematics or musculoskeletal-modelling stage.
The procedure is three small steps (paper §2.2):
-
Step 1 — isolate joint motion:
$R_{\text{motion}} = R_{\text{local}},R_{\text{rest,local}}^{-1}$ -
Step 2 — swing-twist decompose about the calibrated axes: a twist about the bone long axis
$e_1$ plus a swing projected onto$e_2, e_3$ , giving three magnitudes$(c_0, c_1, c_2)$ -
Step 3 — linear projection to the clinical convention:
$\theta = w_1 c_0 + w_2 c_1 + w_3 c_2 + \delta$
The parameters calibrations/).
Each calibration is fit for one (body model, skeleton, validation
cohort) triple. The shipped default (soma_gemx_v14) covers GEM-X
with the SOMA skeleton on OpenCap LabValidation.
On the OpenCap LabValidation cohort (9 subjects, 720 trials, leave-one-subject-out cross-validation), pooled mean absolute error on the 15 rotational degrees of freedom comparable to the OpenCap Monocular reference set is 4.50° raw / 3.11° centred, with correlations of 0.98–0.99 (knee flexion), 0.98 (hip flexion), and 0.90–0.93 (shoulder flexion) on the strong-amplitude DOFs. Full per-DOF table + protocol in the paper.
Repeating the identical procedure on a second body model, SAM 3D Body
(MHR skeleton), changing only the calibration table, gives 4.66°,
statistically indistinguishable from GEM-X: evidence that the
extraction is body-model agnostic. That table ships here too
(calibrations/mhr_sam3dbody_v1).
For the CPU-only path (calibration + extraction, no body model):
git clone https://github.com/Babon-Innovations-b-v/movalytics
cd movalytics
pip install -e .For the full path (running GEM-X end-to-end on video), you also
need the body-model install. Python 3.10 is required (GEM-X pins
numpy==1.23.5 which doesn't build on 3.12+):
# 1. fresh Py3.10 venv (uv installs the interpreter automatically)
uv venv .venv --python 3.10
source .venv/bin/activate
uv pip install -e .
# 2. CUDA-matched torch. GEM-X pins torch==2.10.0+cu126, so install that
# exact build from the cu126 index (runs on any recent NVIDIA driver):
uv pip install torch==2.10.0+cu126 torchvision==0.25.0+cu126 \
--index-url https://download.pytorch.org/whl/cu126
# 3. submodules + body model
cd lib/gemx
# Pinned to the commits validated for the paper; upstream can move.
git clone https://github.com/NVlabs/SOMA-X.git third_party/soma
git -C third_party/soma checkout 8663276
git clone https://github.com/facebookresearch/sam-3d-body.git third_party/sam-3d-body
git -C third_party/sam-3d-body checkout b5c765a
uv pip install -e third_party/soma
(cd third_party/soma && git lfs pull) # SOMA neutral assets (~830 MB)
# GEM-X loads the SOMA template rig + procedural transforms from the
# nvidia/soma-x HF cache, but they ship in the SOMA-X clone. Seed the
# cache dir, then copy those assets in so GEM-X can find them:
SOMA_HF=$(python -c "from huggingface_hub import snapshot_download as s; print(s('nvidia/soma-x'))")
cp -n third_party/soma/assets/SOMA_template_rig.usda \
third_party/soma/assets/SOMA_procedural_transforms.json \
third_party/soma/assets/SOMA_neutral.npz "$SOMA_HF"/
# system libs OpenCV needs on a headless box (skip on desktop Linux):
# sudo apt-get install -y libgl1 libglib2.0-0
# install_env.sh builds detectron2 from source. With a CUDA toolkit present
# it builds GPU ops; without one (no nvcc), unset CUDA_HOME first so it
# builds CPU-only — detection then runs on CPU, fine for short clips.
bash scripts/install_env.sh # gem + detectron2
cd ../..
# 4. GEM-X weights (~12 GB from huggingface.co/nvidia/GEM-X)
python tools/fetch_weights/fetch_weights.py
# 5. WSL2 only: warp needs libcuda visible
export LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATHCPU is enough for everything Movalytics-authored (the calibration
pipeline, extraction, validation). The CUDA dependency lives in
lib/gemx/ — only needed when running the body model on raw video.
The always-works path. No data downloads, no GPU.
movalytics list-calibrations
# → mhr_sam3dbody_v1
# → soma_gemx_v14 (default)
movalytics extract my_session.npz --out-dir angles/
movalytics extract my_session.npz --calibration soma_gemx_v14
movalytics extract my_session.npz --mapping ./my_custom.jsonThe .npz input carries the arrays the extraction needs:
| Array | Shape | Dtype | Required? |
|---|---|---|---|
global_rots |
(T, J, 3, 3) |
float | yes |
rest_global_rots |
(J, 3, 3) |
float | yes |
joint_names |
(J,) |
str | yes |
com_velocity |
(T, 3) |
float | optional — for the pelvis transverse-rotation channel |
This is the same schema tools/run_gemx/run_gemx.py emits, so its per-trial
.npz files feed movalytics extract directly — no conversion step.
Output: one CSV per joint in --out-dir. Each CSV has time_s, one
column per clinical DOF (<prefix>_<dof_short>_deg), informational
off-axis columns suffixed _info, and <prefix>_lock_flag.
For library use:
from movalytics import extract_clinical_angles, load_calibration
angles_per_joint = extract_clinical_angles(
global_rots=global_rots, # (T, J, 3, 3)
rest_global_rots=rest_global_rots, # (J, 3, 3)
joint_names=joint_names, # list of J strings
fps=60.0,
com_velocity=com_velocity, # (T, 3), optional
mapping=load_calibration(), # default: soma_gemx_v14
)To use a custom calibration off disk instead of one from the registry:
from movalytics import load_mapping
mapping = load_mapping("./my_custom_calibration.json")Honest end-to-end from the source data. Download OpenCap LabValidation (~30 GB) from https://simtk.org/projects/opencap, download GEM-X weights (~12 GB), run GEM-X on the videos (needs a CUDA GPU, 4–12 h on an L4), build a local bundle, then run LOSO / validate / waveforms on CPU.
This reproduces the GEM-X headline (matched-15 MAE 4.50° / cMAE 3.11° / r 0.695). The SAM 3D Body / MHR result (Appendix C, 4.66° / 3.46° / 0.615) uses a different estimator whose real-time engine is not part of this release, so it reproduces from a shipped rotation-matrix bundle instead of raw video:
python tools/fetch_bundle/fetch_bundle.py --out ./appendix_c_bundle
movalytics loso ./appendix_c_bundle --output loso_c.json
movalytics validate ./appendix_c_bundle loso_c.json --matched-camera Cam1Full walkthrough with hardware / disk / time per step, plus the Appendix C
detail:
docs/reproduce_paper.md.
The end-to-end command sequence:
# After downloading OpenCap LabValidation manually from simtk.org
# and converting its .mot files to the canonical GT CSV schema
# (see docs/calibrate.md):
python tools/fetch_weights/fetch_weights.py
python tools/run_gemx/run_gemx.py --opencap-root ./opencap_LabValidation \
--output-dir ./gemx_output
movalytics build-bundle --gemx-output ./gemx_output \
--gt-csv-dir ./opencap_gt_csvs --bundle-out ./validation_data
movalytics loso ./validation_data --output loso_report.json
movalytics validate ./validation_data loso_report.json --output validation_tables.json
movalytics waveforms ./validation_data --output-dir waveforms/
# Or in one shot, with skip-if-exists per step:
python tools/reproduce_paper/reproduce_paper.py \
--opencap-root ./opencap_LabValidation \
--gt-csv-dir ./opencap_gt_csvscalibration will need a lot of compute.
Full walkthrough with the bundle schema:
docs/calibrate.md.
movalytics calibrate ./my_bundle --output my_calibration.json
# Optionally publish it:
mv my_calibration.json calibrations/<skeleton>_<estimator>_v1.jsonmovalytics list-calibrations
movalytics extract <input.npz> [--calibration NAME | --mapping PATH]
movalytics calibrate <bundle> [--cohort COHORT] [--held-out N]
movalytics loso <bundle> [--cohort COHORT] [--subjects ...]
movalytics validate <bundle> <loso> [--matched-camera Cam1]
movalytics waveforms <bundle> [--camera Cam1] [--calibration NAME | --mapping PATH]
list-calibrations and extract are the CPU-only subcommands; the
rest operate on a validation bundle (three parquet files
described in docs/calibrate.md).
| Path | Contents |
|---|---|
src/movalytics/kinematics/ |
Step 1, 2, 3 extraction (rotations → clinical angles) |
src/movalytics/calibrate/ |
Three-phase calibration fit |
src/movalytics/validation/ |
Bundle reader, bundle builder, LOSO orchestrator, Table 1 aggregator, Appendix B waveforms |
src/movalytics/signals/ |
Per-trial primitives (lag, filter, spike, metrics) |
src/movalytics/sources/ |
gemx + csv adapters feeding bundle_build |
src/movalytics/mapping.py |
Calibration schema + JSON IO |
src/movalytics/calibrations.py |
Registry — list_calibrations(), load_calibration(name) |
calibrations/ |
Registered calibrations, one JSON per (body model, skeleton, cohort) |
lib/gemx/ |
Vendored GEM-X body model (Apache 2.0; weights fetched separately) |
tools/ |
External workflow scripts — run_gemx/, fetch_weights/, fetch_bundle/ (Appendix C bundle), reproduce_paper/, test_e2e/ (each one self-contained, see tools/README.md) |
docs/ |
reproduce_paper.md, calibrate.md |
LICENSE |
Babon Noncommercial Research License 1.0 — free for personal + non-commercial academic use; commercial and clinical via support@babon.eu |
ATTRIBUTIONS.md |
Third-party component credits |
@article{kardolus2026movalytics,
title = {Direct Clinical Joint Angle Extraction from Parametric Body
Model Rotation Matrices},
author = {Kardolus, J. and Hendriks, D. and Jansen, J.},
year = {2026},
note = {Companion repository: https://github.com/Babon-Innovations-b-v/movalytics},
}Movalytics is under the Babon Noncommercial Research License 1.0. The line is drawn by what the use is, not who you are or whether money changes hands.
- Free: personal use, and non-commercial academic research and teaching. You may modify it and pass it on, as long as the same license travels with it and any published results cite the paper.
- Needs a license from support@babon.eu (which also carries the patent rights): commercial use, and any clinical use, meaning any use that touches the care of an identifiable person. Clinical use is reserved even for a non-profit hospital.
The LICENSE file has the exact terms.
This work builds on third-party models and tools, each under its own
license: GEM-X (Apache 2.0, vendored in lib/gemx/), SAM 3D Body
by Meta (SAM License), the real-time Fast-SAM-3D-Body C++ engine by
Ammar Qammaz (MIT), ViTPose (Apache 2.0), and the OpenSim /
OpenCap reference standard. Model weights are fetched from NVIDIA's
Hugging Face repository, not hosted here. Full credits, links, and the
required SAM 3D Body attribution are in ATTRIBUTIONS.md.
The extraction method is the subject of Netherlands patent application
no. 4001444 (reference P62056409NL); see NOTICE.
This is not a medical device. It is research software: not CE-marked, not cleared or approved by any regulator, not validated for clinical decision-making.
Research questions and commercial licensing: support@babon.eu.