TrackAid-DT: An Egocentric Vision Dataset and Benchmarks for Lane Detection in Visually Impaired Athlete Guidance
TrackAid-DT introduces:
- A pixel-annotated egocentric dataset for athlete guidance.
- A benchmark of modern semantic segmentation architectures.
- A complete training, evaluation, and inference framework for lane perception in assistive sports applications.
The project focuses on autonomous lane guidance for visually impaired athletes through egocentric vision. TrackAid-DT provides a dataset collected from chest-mounted cameras during real running sessions and evaluates multiple deep learning architectures for detecting the walkable running lane under realistic athletic conditions.
The dataset contains pixel-level lane annotations acquired under diverse environmental and motion conditions, including daytime, nighttime, low-light environments, sprint sessions, and challenging viewpoints. Together with the dataset, this repository provides reproducible pipelines for training, evaluating, and benchmarking semantic segmentation models for assistive sports applications.
TrackAid-DT consists of 747 pixel-annotated images collected from chest-mounted cameras during real running sessions on athletics tracks. The original annotations are provided in COCO Segmentation format and can be converted to binary segmentation masks for training with this repository. All other datasets used in the study were collected from publicly available resources. We share the TrackAid-DT dataset; for details about the additional public data, please refer to the full study: TrackAid-DT.
For the training and evaluation code, images and binary masks are expected to follow the folder structure defined in config.py. The following augmentation was applied to create 3 versions of each source image:
- 50% probability of horizontal flip
- Random rotation of between -10 and +10 degrees
- Random exposure adjustment of between -15 and +15 percent
- Random Gaussian blur of between 0 and 1.5 pixels
- Salt and pepper noise was applied to 1.01 percent of pixels
TrackAid-DT was collected on standard outdoor athletics tracks using a chest-mounted monocular camera to capture an egocentric view of the running lane. Data acquisition involved multiple athletes performing running sessions under diverse environmental and motion conditions to reflect realistic deployment scenarios.
The dataset includes recordings acquired during daytime, nighttime, and low-light conditions, as well as straight and curved track segments. Additional variability was introduced through different running speeds, athlete positions within the lane, and viewpoint changes naturally occurring during athletic activity. To further improve diversity and robustness, supplementary recordings were collected using smartphone cameras under different orientations and acquisition settings.
From the recorded videos, frames were uniformly sampled and manually annotated with pixel-level masks representing the walkable running lane. The final dataset comprises 747 annotated images, providing a challenging benchmark for semantic segmentation models operating in real-world athletic environments.
- Pixel-level lane annotations
- Egocentric chest-mounted viewpoint
- Multiple athletes and recording sessions
- Daytime and nighttime acquisitions
- Low-light scenarios
- Sprint and jogging conditions
- Straight and curved track segments
Each image is paired with a binary segmentation mask representing the walkable running lane.
The benchmark includes the following segmentation architectures. All networks are implemented using TensorFlow and Keras:
| Model | Description |
|---|---|
UNet |
Baseline encoder-decoder architecture with skip connections |
UNet++ |
Enhanced variant with nested skip connections |
UNetL |
Lightweight version of UNet |
UNet++L |
Lightweight version of UNet++ |
TransUNet |
Hybrid CNN and Transformer architecture |
LightMUNet |
Uses Mamba State Space Models instead of Transformers |
🔍 The TransUNet model implementation was adapted from the repository TransUNet-tf.
All models are trained and evaluated under a unified experimental protocol to enable a fair comparison across different architectural families.
This project requires Python 3.10.
Clone the repository and install the required dependencies:
git clone https://github.com/vrai-group/TrackAid-DT.git
cd TrackAid-DT
pip install -r requirements.txtThe requirements.txt file contains all dependencies required to reproduce the experiments presented in the paper, including libraries for:
- Deep learning model training
- Dataset loading and preprocessing
- Data augmentation
- Model evaluation and benchmarking
- Segmentation inference
- Visualization and result generation
TrackAid-DT/
├── main.py # Main entry point for training, evaluation, and inference
├── train.py # Training, evaluation, inference, and cross-validation sessions
├── predict.py # Prediction, evaluation metrics, and mask-saving utilities
├── test.py # Optional batch inference utility over saved best models
├── config.py # Dataset paths, input settings, and training parameters
├── dataclass.py # Dataset and parameter data structures
├── models/ # Segmentation architectures
├── utils/ # Dataset loading, saving, logging, denoising, and helper utilities
├── images/ # Figures used in this README
└── requirements.txt # Python dependencies
The standard entry point of the repository is:
python main.pyTraining, evaluation, and inference are selected by enabling the corresponding function call inside main.py. In the current workflow, train.py and predict.py provide the underlying classes and utilities, while main.py orchestrates the experiment.
Dataset paths, model-saving folders, input size, number of channels, and hyperparameters are configured in:
config.pyTo train the selected segmentation models and evaluate them after training, enable start_model_wise() inside train_eval_session() in main.py:
TrainingAndEvaluationSession(
logger,
dataset,
models,
params,
False,
denoised,
extra_filter
).start_model_wise()Then run:
python main.pyThis workflow trains the models listed in models, evaluates each trained model on the configured test split, saves predicted masks, and stores evaluation metrics such as IoU, F1-score, precision, recall, and accuracy.
To train only one model, modify the models list in main.py. For example:
models = [UNetPP]To train all benchmark models, use:
models = [UNetL, UNet, UNetPPL, UNetPP, TransUNet, PretrainedTransUNet, LightMUNet]To test an already trained model from main.py, disable the training call and enable the evaluation/inference call with the path of the saved model:
TrainingAndEvaluationSession(
logger,
dataset,
[UNetPP],
params,
True,
denoised,
extra_filter
).start_inference(
model_path="./saves_check_Unet/lane/UNetPP/model.hdf5",
model_dir_name="UNetPP"
)Then run:
python main.pyThe model_path argument must point to the saved model file, usually model.hdf5. The model_dir_name argument is used to name the output folder containing the predicted masks and inference summary.
For a quick single-image sanity check, disable train_eval_session(logger) in main.py and enable:
test_prediction()Then run:
python main.pyBy default, this loads the model path configured in EvaluationSession.test_prediction() and saves the input image, ground-truth mask, and predicted mask in the prediction output folder.
Predicted masks are saved under the prediction path configured in config.py:
Paths.PREDICTIONSFor each model, the framework creates a dedicated output directory and writes the predicted binary masks together with an inference summary.
The repository may also include test.py, which can run batch inference over all models stored inside a best_models/ directory. However, the primary documented workflow uses main.py as the single entry point.
Stability is reported as the standard deviation of IoU across the 8 cross-validation folds; lower values indicate more stable performance.
| Model | IoU (μ ± σ) | F1 (μ ± σ) | Recall (μ ± σ) | Precision (μ ± σ) | Accuracy (μ ± σ) | Stability |
|---|---|---|---|---|---|---|
| U-Net | 0.854 ± 0.102 | 0.918 ± 0.059 | 0.921 ± 0.072 | 0.917 ± 0.063 | 0.973 ± 0.027 | 0.102 |
| U-Net-L | 0.738 ± 0.178 | 0.837 ± 0.121 | 0.813 ± 0.143 | 0.869 ± 0.113 | 0.953 ± 0.039 | 0.178 |
| UNet++ | 0.859 ± 0.113 | 0.920 ± 0.068 | 0.906 ± 0.101 | 0.940 ± 0.050 | 0.972 ± 0.033 | 0.113 |
| UNet++-L | 0.763 ± 0.165 | 0.855 ± 0.109 | 0.836 ± 0.132 | 0.882 ± 0.102 | 0.957 ± 0.038 | 0.165 |
| TransUNet | 0.865 ± 0.120 | 0.923 ± 0.073 | 0.940 ± 0.061 | 0.910 ± 0.099 | 0.978 ± 0.019 | 0.120 |
| Pretrained TransUNet* | 0.929 ± 0.064 | 0.962 ± 0.036 | 0.958 ± 0.037 | 0.965 ± 0.036 | 0.988 ± 0.011 | 0.064 |
| LightM-UNet | 0.697 ± 0.182 | 0.808 ± 0.128 | 0.777 ± 0.156 | 0.851 ± 0.113 | 0.944 ± 0.046 | 0.182 |
| Model | IoU (μ ± σ) | F1 (μ ± σ) | Recall (μ ± σ) | Precision (μ ± σ) | Accuracy (μ ± σ) | Stability |
|---|---|---|---|---|---|---|
| U-Net | 0.921 ± 0.064 | 0.958 ± 0.037 | 0.960 ± 0.035 | 0.956 ± 0.041 | 0.987 ± 0.011 | 0.064 |
| U-Net-L | 0.884 ± 0.089 | 0.936 ± 0.052 | 0.936 ± 0.044 | 0.937 ± 0.065 | 0.981 ± 0.016 | 0.089 |
| UNet++ | 0.917 ± 0.067 | 0.956 ± 0.038 | 0.957 ± 0.033 | 0.954 ± 0.044 | 0.986 ± 0.011 | 0.067 |
| UNet++-L | 0.815 ± 0.121 | 0.893 ± 0.076 | 0.881 ± 0.081 | 0.908 ± 0.081 | 0.968 ± 0.024 | 0.121 |
| TransUNet | 0.796 ± 0.136 | 0.880 ± 0.086 | 0.877 ± 0.123 | 0.895 ± 0.089 | 0.962 ± 0.036 | 0.136 |
| Pretrained TransUNet* | 0.899 ± 0.073 | 0.945 ± 0.042 | 0.943 ± 0.049 | 0.948 ± 0.044 | 0.983 ± 0.014 | 0.073 |
| LightM-UNet | 0.578 ± 0.244 | 0.702 ± 0.198 | 0.647 ± 0.216 | 0.793 ± 0.204 | 0.918 ± 0.066 | 0.244 |
* Pretrained TransUNet denotes the model initialized with ImageNet-pretrained weights.
TrackAid-DT was evaluated using 8-fold cross-validation across multiple semantic segmentation architectures, including convolutional, transformer-based, and state-space models. Among the models evaluated without task-specific KITTI pretraining, Pretrained TransUNet achieved the best overall performance, reaching an IoU of 0.929 ± 0.064 and an F1-score of 0.962 ± 0.036. Task-specific pretraining on the KITTI dataset significantly improved the performance of convolutional architectures. In this setting, U-Net achieved the highest performance with an IoU of 0.921 ± 0.064 and an F1-score of 0.958 ± 0.037, closely followed by UNet++. These results demonstrate that TrackAid-DT provides a challenging benchmark for egocentric lane perception while highlighting the effectiveness of modern segmentation architectures for assistive athlete guidance under diverse real-world conditions.
Example predictions demonstrate accurate lane segmentation across a wide range of running conditions, supporting reliable perception for athlete guidance applications.
To evaluate robustness beyond the cross-validation protocol, all trained models were additionally assessed on an independent external test set that was not used during training or validation. This evaluation provides insight into the ability of the models to generalize to unseen running conditions, athletes, viewpoints, and environmental scenarios. The results indicate that the top-performing architectures maintain strong segmentation performance under previously unseen conditions, demonstrating the potential of TrackAid-DT for real-world deployment in assistive athlete guidance applications.
Beyond offline benchmarking, the proposed framework was deployed and evaluated on multiple embedded hardware platforms to assess its suitability for real-time operation in wearable assistive systems. The trained models were converted and optimized for edge inference, enabling efficient execution on resource-constrained devices while maintaining reliable segmentation performance. To validate the practical feasibility of the system, a complete prototype integrating egocentric vision, onboard processing, and vibrotactile feedback was tested in real outdoor running scenarios. During these trials, athletes navigated standard athletics tracks while receiving real-time haptic guidance based on the estimated lane position. The system demonstrated stable lane perception under varying illumination conditions, curved track geometries, and different running speeds, providing timely feedback that supported lane maintenance throughout the running sessions. These experiments highlight the potential of TrackAid-DT not only as a benchmark dataset for semantic segmentation research, but also as a foundation for developing real-world assistive technologies that promote greater autonomy and independence for visually impaired athletes.
To read more about these tests, we recommend the following resources, and read the full paper.
Dr. Alessandro Galdelli: a.galdelli@univpm.it Dr. Gagan Narang: g.narang@pm.univpm.it
Full paper: TrackAid-DT
If you use this repository or the TrackAid-DT dataset in your research, please cite:
@article{TrackAidDT,
title={TrackAid-DT: An Egocentric Vision Dataset and Benchmarks for Lane Detection in Visually Impaired Athlete Guidance},
journal={Computer Vision and Image Understanding},
volume={269},
pages={104788},
year={2026},
author={Gagan Narang and Alessandro Galdelli and Oleksandr Kuznetsov and Primo Zingaretti and Adriano Mancini},
doi={10.1016/j.cviu.2026.104788}
}