Foundation models for movement data: Are they ready for prime-time?

Official repository for Foundation models for movement data: Are they ready for prime-time? (arXiv 2026)

Abstract

Foundation models (FMs) trained on large-scale accelerometer data have been proposed as general-purpose feature extractors for health monitoring, but systematic evidence of their advantages is lacking. We present the first comprehensive evaluation of four open-source accelerometer FMs against supervised baselines covering 19 tasks across the domains of activity recognition including activities of daily living, clinical monitoring, and physiological inference. We find task-dependent performance results: supervised models remain competitive with FMs on human action recognition (HAR), with no consistent advantage for either, while selected FMs lead on fall and stress detection and are the most robust to sensor-placement variation. As frozen feature extractors, FMs are strongest for demographic inference, whereas sleep staging performance remains near chance level for all models. The internal FM representations show strong similarity across layers, highlighting potential for future FM improvements. Linear and frozen probing reveals that UniMTS provides the strongest representations and is the only FM that surpasses the supervised baselines without finetuning. Concept discovery analysis shows all models capture high-intensity activities clearly but struggle with sedentary, complex or ambiguous activities. We provide scenario-based deployment recommendations. Furthermore, we identify FM-derived activity profile inference—moving beyond fixed category classification—as a promising research direction.

Environment setup

The code uses two conda environments. For Linux + CUDA 12.8 (Python 3.12.12, PyTorch 2.9.0).

Concept discovery also depends on the NLMCD-ALIGN package (jvielhaben/NLMCD-ALIGN).

fm_movement_benchmark/       # this repo
├── concept_analysis/
└── NLMCD-ALIGN/             # from github.com/jvielhaben/NLMCD-ALIGN

Model checkpoints

The two foundation models with released weights must be downloaded and placed under checkpoints/ at the exact paths the configs/models/<model>.yaml files expect:

Model	Place at	Download
NormWear	checkpoints/normwear/normwear_last_checkpoint-15470-correct.pth	NormWear v1.0.0-alpha release
UniMTS	checkpoints/unimts/UniMTS.pth	xiyuanz/UniMTS on HuggingFace

NormWear's checkpoint was renamed but the code uses the previous name -> after downloading, the checkpoint needs to be renamed to 'normwear_last_checkpoint-15470-correct.pth'.

OxfordSSL and ElderNet need no manual download — their weights load automatically via torch.hub from OxWearables/ssl-wearables and yonbrand/ElderNet respectively (cached under ~/.cache/torch/hub/).

Reproducing the experiments

lightning3 — main environment for dataset prep, training, and the efficiency benchmark (steps 1–3 below):

conda env create -f lightning3.yml
conda activate lightning3

1. Prepare a (non-demographic) dataset

Raw-data download links are in datasets/README.md; prepped datasets are written to processed_data and read from there by the experiment runners. datasets/prep_all.py rebuilds every prepared dataset from raw into processed_data, converting units to g at prep time:

# Check which commands will be run
python -m datasets.prep_all --only pamap2 --dry-run

# Prep just PAMAP2 (all three sensor positions: wrist, chest, ankle)
python -m datasets.prep_all --only pamap2

1.1 Prepare a demographic dataset

Demographic-task variants (age/weight/height regression) are derived from an already-prepped dataset:

# Derive the demographic variants (creates pamap2_demographics_{age,weight})
python -m datasets.create_all_demographics_datasets --only pamap2

1.2 Run a dataset through one FM and one baseline

bin/run_experiment.py in single-cell mode runs one (dataset, model, eval_mode) combination through main.py. Add --dry-run to print the composed main.py command without executing it. Model names are normwear, unimts, oxfordssl, eldernet (foundation models) and s4, inception1d, xresnet1d50, tinyhar (supervised baselines). Evaluation modes are frozen_linear, frozen_query, and full_finetune (baselines only run full_finetune).

# Foundation model (NormWear) — frozen linear probe on PAMAP2 (chest)
python bin/run_experiment.py --dataset pamap2 --model normwear --eval-mode frozen_linear

# Baseline (S4) — full finetune on PAMAP2 (baselines only support full_finetune)
python bin/run_experiment.py --dataset pamap2 --model s4 --eval-mode full_finetune

# Same, but on a demographic dataset from step 2
python bin/run_experiment.py --dataset pamap2_demographics_age --model normwear --eval-mode frozen_linear

2. Sensor-position experiment (RealWorld2016)

Prep the seven per-position RealWorld variants, then run the full sensor_position experiment matrix (7 positions × 8 models, full_finetune):

# Prep the 7 per-position datasets (chest, head, shin, thigh, upperarm, waist, wrist)
python -m datasets.prep_all --only realworld2016

# Run the whole position matrix (add --dry-run to preview, --slurm to emit array jobs)
python bin/run_experiment.py --experiment sensor_position

# Build the position-analysis report/figures from the results
python bin/generate_position_report.py

3. Computational-efficiency experiment

# All models, GPU 0 (paper defaults: batch 64, 100 timing iters)
python bin/benchmark_efficiency.py --gpu 0

# Subset with a custom timing budget
python bin/benchmark_efficiency.py --models normwear s4 --batch-size 64 --iters 100

4. Concept discovery (CBA + CKA)

Two steps. First, the layer-wise pipeline extracts backbone features from every layer, runs NLMCD concept discovery per layer, and computes intra-model CBA alignment. Then the figure generator writes the standard outputs: per-model UMAP + concept-activity heatmap triptychs, and the CKA / CBA layer-alignment heatmaps.

nlmcd — separate environment for concept discovery:

conda env create -f concept_analysis/nlmcd.yml
conda activate nlmcd

# Extract features + discover concepts + align, for all models on realworld2016
python concept_analysis/run_layerwise_analysis.py --model all --dataset realworld2016 --gpu 0

# Generate the standard figures: UMAP/heatmap triptychs + CKA & CBA layer-alignment plots
python concept_analysis/generate_realworld_figures.py --pretrained --sections cka cba triptych latex

Use --gpu 0 1 2 3 on the layer-wise step to parallelise across GPUs. Drop --pretrained on the figure step (and add --finetuned on the layer-wise step) to run the analysis on the finetuned checkpoints instead of the pretrained FMs.

Cite as

@article{braeuer2026foundationmodelsmovementdata,
      title={Foundation models for movement data: Are they ready for prime-time?}, 
      author={Alexander Br\"{a}uer and Benjamin Cauchi and Nils Strodthoff},
      journal={arXiv preprint 2608.13316},
      year={2026},
      eprint={2608.13316},
      archivePrefix={arXiv},
      primaryClass={eess.SP},
      url={https://arxiv.org/abs/2608.13316}, 
}