Deep-Pose-Tracker (DPT)

A unified YOLO-based framework for multi-animal pose detection, tracking, and behavioural analysis of Caenorhabditis elegans.

📄 Preprint

Deep-Pose-Tracker: a unified model for behavioural analysis of Caenorhabditis elegans

Saha D., Chaudhary S., Vyas D., Roy A.G., Sharma R.

Model features

The model is composed of the following features:

1. Pose detection of C. elegans with head-tail identification
2. Quantification of eigenworms
3. Worm tracking and speed (average and instantaneous) measurement
4. Spatial exploration or trajectory extent measurement
5. Orientation of motion detection
6. Identifying forward-reverse movement
7. Detection of omega turns
8. Multi-class (worms and eggs) detection and counting

Note:

1. All the codes are optimized such that they can be used for a single worm as well as multiple worms.
2. The codes work on videos as well as images (wherever applicable).
3. You can keep multiple files in a folder, and give the folder path as input. That will work as well.
4. All the analysis results will be saved inside the outputs folder. For example, if you are running the pose.ipynb file on some video, the outputs will be saved in the outputs/pose/run1 folder. If you run this file several times, respective outputs will be saved in run2, run3, ... inside the main outputs/pose folder.

Repository Structure

The repository consists of the following files-

1. All the codes for the analysis of different behavioural features.
2. Pretrained weights for pose detection, detection of worms and eggs, detecting and counting C. elegans worms, and bacteria counting.
3. Sample input videos for practice.

Important Links

1. Pose detection sample examples.
2. Training dataset for pose estimation and worms and eggs detection.
3. Spreadsheet containing video-level splitting of the training dataset for pose detection.
4. Weight files for pose estimation and worms and eggs detection.
5. Tracking metrics for evaluating MOTA, IDF1, etc.
6. Optimization datasets for omega turn and reversal detection.
7. Eigenworm dataset.

Installation steps

Here are the steps to install Deep-Pose-Tracker. We highly recommend installing the dependencies in a separate environment. Here we have shown the steps using conda. One can use python3 as well for creating the environments.

1. Clone the repository

   git clone https://github.com/cebpLab/Deep-Pose-Tracker.git

   cd Deep-Pose-Tracker

2. Create the Conda environment

   conda env create -f environment.yml

3. Activate the environment

   conda activate deep-pose-tracker

4. Verify installation

   yolo help
python -c "import motmetrics; print('motmetrics OK')"

Annotations

Annotations are performed in Roboflow. It is suitable for labelling images to train models for object detection, pose detection, image segmentation, and classification tasks. But there are other platforms like CVAT where you can label your images.

Training dataset

The complete training datasets for "pose detection" and "worms and eggs detection" are made available in this Google Drive link. Training images and labels are both provided.

Dataset Description

The datasets are divided into three subsets:

• Training set: used for model training
• Validation set: used for evaluation
• Test set: used exclusively for final evaluations, like tracking and pose accuracies.

Split Details

Deep-Pose-Tracker:

• train: 7383 images
• valid: 400 images
• test: 157 images

Worms and eggs detection dataset:

• train: 963 images
• valid: 49 images
• test: 39 images

Annotation Format

Deep-Pose-Tracker dataset:

• Each annotation contains:
  class id, x, y, w, h, keypoints
• keypoints are stored as (x, y, v), where the visibility flag v is used as
  v = 0: keypoint not labelled (out-of-frame or deleted)
  v = 1: keypoint labelled but occluded
  v = 2: keypoint visible
• total 11 keypoints

During evaluation, keypoints with v = 1 and 2 are considered. Keypoints with v = 0 are excluded from evaluation.

Worms and eggs detection dataset:

• Each annotation contains:
  class id, x, y, w, h

Training model

The training process is the same as the standard training procedure of YOLO. In this model, we have used YOLOv8 for all the analysis. The training was performed on a custom dataset with labelled images of C. elegans. Here are the following details that we followed during training:

• A total of 3018 images were taken for training different YOLOv8 architectures.
• We trained different YOLOv8 architectures (medium, large, and extra) for pose detection.
• Training was performed on $1024 \times 1024$ input image size.

Here is the detailed training procedure. We assume that you have properly installed untralytics.

1. Import YOLO.

   from ultralytics import YOLO

2. Define the model on which you want to perform the training. Here is the list of different YOLOv8 models for pose detection.

   model = YOLO("yolo8m-pose.pt")

3. Once the model is defined, you can now start the training.

   model.train(data="/path/to/data.yaml", epochs=100, save=True, batch=32, val=True, device=[0,1], imgsz=640)

A snapshot of the training is shown below:

That's it. Now let's unpack each of the parameters defined here.

• The data.yaml file consists of the details of the training dataset, which includes the names of individual classes with class indices, and paths of the folders containing training, validation and test datasets. In our case, there are only one class worm with class index 0.

• epochs is the number of times you want to repeat or iterate the training process. Ideally, we start with a small number (such as 100, 200), but we can increase this value as required.

• The save flag is used to save the training outputs.

• batch is an important parameter which defines the number of images on which we want the model to be trained at once. This factor depends on the hardware configuration (GPU memory), network architecture and input image size.

• val is set to be True to ensure the model is performing well on the validation dataset, and does not overfit.

• device is used to specify where you want to perform the training. If you are training on a GPU, define device = 0. If you are training on multiple GPUs, define device = [0, 1, 2, 3] (depending upon the number of GPUs available). It is highly recommended to train on GPUs only. Make sure that CUDA and compatible PyTorch are properly installed, which are necessary for GPU training.

• imgsz defines the image size on which you want to train the network.

In point 2, we defined a model, which is basically a YOLO architecture for pose detection with pretrained weights. Here, we have chosen yolo8m-pose.pt as we are interested in training YOLO on posture data. If you are working with a detection problem, you would have to choose 'yolo8m.pt'. Here you can choose any architecture (small, medium, large, etc.), according to the requirements.

Some important tips for training

• Resize the images (by cropping) before annotations. In Roboflow, you can resize images after annotations; however, try to avoid squeezing the images to fit within the given dimensions. It would distort the spatial features in the training data, resulting in poor performance during predictions.
• Resize images into square shapes, because the YOLO architectures perform best when the inputs are given in the form of a square matrix.
• Training images should contain as much variability as possible in the data. It helps the model to learn about all the differences that can appear in an image. If you are working with pose detection, take images with complex structures as well as normal sinusoidal shapes. It helps in predicting complex shapes and enhances accuracy.
• Use augmentations to increase the number of training images by various transformations. It also increases the variability in the images, which makes the model more robust.

Where are the training outputs saved?

The training outputs are by default saved in the runs folder. Every time you run the training, a new folder with the name train1, train2, ... is created, which contains all the training outputs, including weights (inside the weights folder). The weights folder contains two weight files: best.pt (corresponding to the best performance on validation data) and last.pt (corresponding to the performance on the last training epoch). We have used the best.pt file and renamed it to yolov8x-832.pt according to the network architecture and training image size, for convenience.

Deployments

This is the most important step, where you use different algorithms on real experimental data (videos or images), focusing on various assays. Here we show how to use the code and how to read the outputs. We will use the pose detection code pose.ipynb for demonstration. The same follows for all the other codes.

The first few lines of the code are shown here. Once the libraries are imported, we do the following:

1. Let's first define the model

   model = YOLO("/path/to/yolov8x-832.pt")

   which is the YOLO architecture itself, with trained weights on our own custom data for posture detection of C. elegans. The weights can be found in the weights folder. Choose the appropriate weight for your work. Here we are working with the pose detection, which is why we chose the pose weights, which are in the weights/pose folder.

2. Define the source on which you want to run the program. You can define the input in several ways.

   • You can run on a video file. Videos with .avi, .mp4, .mov and .mkv are supported.

     source = "/path/to/video.mp4"

   • You can run on an image file. The following image formats are supported: .jpg, .jpeg, .png and .bmp.

     source = "/path/to/image.jpg"

   • If you want to run it on multiple input files in a single run, just provide the folder path which contains all the files. It may contain images as well as videos.

     source = "/path/to/the/folder/"

With these changes, you can now run the program. The outputs will be saved in the outputs folder with the pose subfolder in it, which is defined in the base_out_dir variable. You can change the folder names, but that is optional.

Evaluation of tracking accuracy

The tracking metrics are evaluated to test the tracking performance of the model in overlapping conditions. Standard tracking metrics, such as MOTA, MOTP and IDF1, are measured. These metrics are calculated by comparing the model predictions with the manually annotated ground truths using the MOTChallenge algorithm, implemented through the motmetrics library.

Ground truth and predicted bounding boxes are matched for each frame of the video using the intersection-over-union (IoU). An IoU threshold of 0.5 is used, below which predictions are ignored from evaluation. The assignment of one-to-one matching between the predictions and ground truths is performed using the Hungarian algorithm as implemented in motmetrics. The evaluation code and the necessary datasets are provided here, ensuring reproducibility.

Some important notes

Some of the codes may need optimization before using them to get better accuracy. For example,

1. The omega turns detection algorithm contains two adjustable parameters: proximity distance and bending angle, which need to be tuned on a ground truth dataset.
2. The script for forward-reversal detection has three parameters: the angle threshold ($H$), body index ($\alpha$), and buffer, which should be optimized on a ground truth dataset before using.

This is necessary because these parameters are not fixed and can vary from one experimental condition to another.

Support

1. If you are using DPT and find any issue that needs to be solved, please feel free to email us mentioning them.
2. If you want to use DPT, but are having difficulties in installing the packages or understanding the workflow (annotation, training and prediction), please do not hesitate to reach out to us. We will be happy to help you.

Acknowledgements

• Ultralytics for YOLO, a state-of-the-art computer vision model, making it all open-source.
• Roboflow and CVAT for annotations facility.

Cite this

If you find this work useful, please cite it with the following:

@article {Saha2026DPT,
	author = {Saha, Debasish and Chaudhary, Shivam and Vyas, Dhyey and Ghosh-Roy, Anindya and Sharma, Rati},
	title = {Deep-Pose-Tracker: a unified model for behavioural analysis of Caenorhabditis elegans},
	elocation-id = {2025.11.23.689997},
	year = {2026},
	doi = {10.1101/2025.11.23.689997},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2026/05/24/2025.11.23.689997},
	eprint = {https://www.biorxiv.org/content/early/2026/05/24/2025.11.23.689997.full.pdf},
	journal = {bioRxiv}
}

Contact us

Contact us through this email address: rati@iiserb.ac.in