Adapting Pretrained Large Vision Models for Sensor-based Activity Recognition [IMWUT 2026]

This repository contains the official implementation of our paper, “Adapting Pretrained Large Vision Models for Sensor-based Activity Recognition”, accepted by the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2026).

Overview

VisionHAR is a wearable sensor-based Human Activity Recognition (HAR) framework that leverages pretrained Large Vision Models (LVMs) to enhance the generalization capability of HAR models. It bridges the modality gap between wearable sensor data and visual foundation models through three key components:

• Sensor-to-Image Transformation: Converts multi-channel IMU data into visual forms that are compatible with pretrained LVMs.
• Sensor Knowledge Adaptation: Introduces a lightweight Sensor-to-Vision Adapter to inject sensor-specific knowledge into LVMs, enabling effective HAR learning.
• Efficiency-Aware Distillation: Transfers the generalization capability of LVMs to a lightweight edge model using class-relation-aware knowledge distillation for efficient on-device deployment.

Usage

Environment Setup

Install the required dependencies:

pip install -r requirements.txt

Sensor Image Generation

python ConstructImage.py -d SBHAR

Sensor Knowledge Adaptation

python train_teacher.py -d SBHAR

Efficiency-Aware Distillation

python train_student.py -d SBHAR

Cross-dataset Evaluation

# Teacher model
python cd_test_teacher.py -sd SBHAR -td HHAR

# Student model
python cd_test_student.py -sd SBHAR -td HHAR