Autobehaver is a Python package for processing and analyzing animal behavior videos. The current implementation was developed for high-throughput analysis of Drosophila recordings, but the package could be adapted for other projects.
Autobehaver is designed for experiments where the goal is to compare behavioral signatures between groups. The pipeline converts videos and keypoints into per-animal behavioral feature vectors, trains classifier ensembles to distinguish reference groups such as control and mutant animals, and thus creates a behavioral axis upon which animals with intermediate phenotypes can be placed. Autobehaver also uses feature-importance analysis to identify the behavioral features that separate groups, providing some interpretability. Our goal is to use Autobehaver for rescue-style experiments, where the key question is whether a treatment, genotype, or condition shifts an individual mutant's behavioral profile away from the mutant state and towards the control state.
The accompanying preprint is available on bioRxiv: Autobehaver: An AI-Based Pipeline for Animal Behavior Analysis.
Autobehaver is under active development. The repository is intended to make the current codebase usable, inspectable, and reproducible.
autobehaver/ Python package source
configs/ Example YAML configuration files
environment/ Conda environment recipes and environment checks
tests/ Package tests
assets/ README images and other small repository artwork
Autobehaver is intended to be run from a cloned copy of this repository.
Use this environment for most of the pipeline:
git clone https://github.com/NHLBI/autobehaver.git
cd autobehaver
conda env create -f environment/environment.yml
conda activate autobehaver
python -m pip install -e .
python environment/check_environment.py
python -m autobehaverpython -m pip install -e . installs the local source tree in editable mode. After this, the environment can find Autobehaver reliably, and the top-level roadmap command is available:
python -m autobehaverThe main environment does not install DeepLabCut or TensorFlow. Use a separate DLC-capable environment only when running the DLC stage itself:
conda env create -f environment/environment-dlc-tf.yml
conda activate autobehaver-dlc
python -m pip install -e .
python environment/check_environment.py --dlc-only --require-dlc
python -m autobehaver.preinf.dlc --helpOn an HPC system that already provides a DeepLabCut module, you can use that module instead. For example:
module load DeepLabCut/2.3.9
cd /path/to/autobehaver
python-DLC -m autobehaver.preinf.dlc --helpA typical workflow is to run DLC once in the DLC-capable environment, then switch back to the main environment for keypoint preprocessing, PosePred inference, master-matrix generation, feature-importance analysis, and overlay movie rendering.
For more detail, see environment/environment_README.md.
The configs/ folder contains example YAML files:
configs/smoke_config.yaml Small example configuration showing stage wiring
configs/pipeline_template.yaml Template for adapting the pipeline to a new project
Each command-line stage also accepts explicit arguments. Run the relevant --help command before adapting a stage to a new dataset.
Autobehaver is organized into pre-inference, inference/training, and post-inference stages. For a command-line roadmap of the full workflow, run:
python -m autobehaverEach stage has its own --help page. The stage summaries below are meant to orient new users; run the stage-specific help command before adapting a stage to a new dataset.
Module:
autobehaver.preinf.moviechecker
Moviechecker records which source movies are available and can create exclusion metadata used by downstream stages.
python -m autobehaver.preinf.moviechecker --helpModule:
autobehaver.preinf.autocropper
Autocropper detects chambers in full-frame movies and writes cropped per-chamber movies for individual-animal analysis.
python -m autobehaver.preinf.autocropper --helpModule:
autobehaver.preinf.dlc
The DLC wrapper runs DeepLabCut on cropped movies and writes keypoint CSV files. This stage should be run from a DLC-capable environment or from an HPC DLC module. Downstream stages do not require DLC once the keypoint CSVs exist.
python -m autobehaver.preinf.dlc --helpModule:
autobehaver.preinf.keypre
Keypre converts DLC keypoint CSVs into positional and kinematic feature files. These files provide the per-frame keypoint-derived measurements used by later inference and feature-matrix stages.
python -m autobehaver.preinf.keypre --helpModule:
autobehaver.preinf.keyprenorm
Keyprenorm normalizes keypoint-derived feature files so that they can be passed into PosePred inference using the same feature scale used during model training.
python -m autobehaver.preinf.keyprenorm --helpModule:
autobehaver.preinf.posepred_inference
PosePred inference applies a trained behavior/orientation model to normalized feature files and writes per-frame behavior and orientation predictions.
python -m autobehaver.preinf.posepred_inference --helpModule:
autobehaver.postinf.mmgen
MMGen combines pose/orientation inference outputs with preprocessed keypoint features to produce individual-level behavioral feature matrices. These matrices summarize time budgets, bout structure, transitions, kinematics, posture, spatial features, sleep-related features, and related measurements.
python -m autobehaver.postinf.mmgen --helpModule:
autobehaver.postinf.featureimp
FeatureImp reads the master feature matrix, trains classifier ensembles to distinguish reference groups, computes feature-importance outputs, and can run reference-axis PCA and feature set enrichment analysis.
python -m autobehaver.postinf.featureimp --helpModule:
autobehaver.postinf.infmovie
InfMovie renders overlay movies showing cropped videos together with predicted behavior and orientation labels.
python -m autobehaver.postinf.infmovie --helpPosePred training has a separate entry point:
python -m autobehaver.training.posepred_training --helpThe public example workflow is focused on inference and post-analysis. Training a production PosePred model requires labeled training data and additional project-specific setup.
From the main environment:
python environment/check_environment.py
python -m compileall -q autobehaver tests
pytest -qFrom a DLC-capable environment or HPC DLC module:
python environment/check_environment.py --dlc-only --require-dlc
python -m autobehaver.preinf.dlc --helpIf you use Autobehaver, please cite the preprint:
O'Neill, R.S., Aviles, S., and Rusan, N.M. Autobehaver: An AI-Based Pipeline for Animal Behavior Analysis. bioRxiv, 2026. doi: 10.64898/2026.05.12.724596
Autobehaver is distributed as a U.S. Government Work. It was prepared by NIH/NHLBI employees as part of their official duties and is not subject to U.S. copyright protection under 17 U.S.C. 105. See LICENSE.txt for details.
Third-party software and resources remain under their own licenses; see THIRD_PARTY_NOTICES.md.