A hands-on learning path with 15 end-to-end Computer Vision projects built with PyTorch Lightning. Each project covers a real-world task — from license plate recognition to CLIP-based image search — with training, evaluation, and inference code you can run locally.
Source repository: AppliedAI-Lab/Computer-Vision-Roadmap
| Skill Area | Projects |
|---|---|
| Image Classification | Traffic signs, crop diseases, satellite imagery, medical X-rays, multi-label tagging |
| Object Detection & Localization | YOLO real-time detection, license plate localization |
| Segmentation | Medical CT segmentation (U-Net++, TransUNet) |
| OCR & Document AI | Handwritten text recognition (CRNN, TrOCR) |
| Face & Pose | Face recognition (FaceNet), human pose estimation |
| Vision + NLP | Image captioning, CLIP image-to-text search |
| Retrieval & Recommendation | Fashion visual similarity (DINOv2, OpenFashionCLIP) |
| Anomaly Detection | Industrial defect detection (PatchCore, EfficientAD) |
| Requirement | Notes |
|---|---|
| Python | 3.9+ recommended |
| PyTorch | CPU works for most projects; GPU strongly recommended for training |
| Git | To clone this repository |
| Kaggle account | Required for automatic dataset downloads (13 of 15 projects) |
| Tesseract OCR | Required only for Project 1 |
- CUDA-capable GPU — speeds up training significantly
- TensorBoard — monitor training metrics (
pip install tensorboard) - 8 GB+ RAM — some datasets are large (e.g., crop disease ~87K images)
git clone https://github.com/AppliedAI-Lab/Computer-Vision-Roadmap.git
cd Computer-Vision-RoadmapMost projects download datasets automatically via the Kaggle CLI.
- Create a Kaggle account at kaggle.com
- Go to Account → Create New Token to download
kaggle.json - Place the token file:
# macOS / Linux
mkdir -p ~/.kaggle
mv ~/Downloads/kaggle.json ~/.kaggle/
chmod 600 ~/.kaggle/kaggle.json
# Windows
# Place kaggle.json at C:\Users\<YourUser>\.kaggle\kaggle.json- Install the Kaggle CLI:
pip install kaggleEach project lives in its own folder. The standard workflow is:
cd "1. License Plate Recognition System" # pick any project folder
python -m venv .venv # optional but recommended
source .venv/bin/activate # macOS/Linux
# .venv\Scripts\activate # Windows
pip install -r requirements.txt
python utils/download_data.py # download dataset (see notes below)
python train.py --config config.json # train the model
python test.py --resume saved/models/.../best-checkpoint-....ckpt # evaluateOpen TensorBoard during training:
tensorboard --logdir saved/log
# Then visit http://localhost:6006Projects are numbered, but you can follow this progression based on difficulty and concept building:
Beginner ──────────────────────────────────────────────────────────────► Advanced
3. Traffic Sign 4. Crop Disease 5. Satellite
Classification Classification Classification
│ │ │
▼ ▼ ▼
1. License Plate 6. YOLO Detection 7. Face Recognition
│ │ │
▼ ▼ ▼
2. OCR / HTR 9. Pose Estimation 8. Image Captioning
│ │ │
▼ ▼ ▼
10. Medical Classify 11. Segmentation 12. Multi-Label
│ │ │
▼ ▼ ▼
14. Defect Detection 13. Fashion Rec. 15. CLIP Search
| Level | Suggested Projects | Why Start Here |
|---|---|---|
| Beginner | 3, 4, 5 | Classic image classification with well-known datasets |
| Intermediate | 1, 6, 7, 9 | Detection, localization, and embedding-based tasks |
| Advanced | 2, 8, 10, 11, 12 | Transformers, medical imaging, multi-label, segmentation |
| Expert | 13, 14, 15 | CLIP/DINOv2 retrieval, anomaly detection, vision-language |
Each project has its own detailed README.md with models, metrics, and step-by-step instructions.
| # | Project | CV Task | Key Models | Dataset |
|---|---|---|---|---|
| 1 | License Plate Recognition | Localization + OCR | ResNet18/34, Tesseract | Car Plate Detection |
| 2 | OCR + Document Understanding | Handwritten Text Recognition | CRNN, TrOCR | Handwriting Recognition (Kaggle) |
| 3 | Traffic Sign Recognition | Classification + STN | Simple CNN, STN-CNN | GTSRB |
| 4 | Crop Disease Detection | Multi-class Classification | MobileNetV2/V3, EfficientNet-B0 | Plant Diseases |
| 5 | Satellite Image Classification | Land Cover Classification | ResNet18, MobileNetV3 | Satellite Images |
| 6 | YOLO Object Detection | Real-Time Detection | YOLOv8, YOLOv10 | COCO128 |
| 7 | Face Recognition | Metric Learning | FaceNet | LFW / custom face dataset |
| 8 | Image Captioning | Vision + Language | CNN-RNN, ViT-GPT2 | Flickr8k |
| 9 | Human Pose Estimation | Keypoint Detection | ViTPose, RTMPose | COCO Keypoints |
| 10 | Medical Image Classification | Binary Classification | ConvNeXt V2, MedMamba | Chest X-Ray Pneumonia |
| 11 | Medical Segmentation | Pixel-Level Segmentation | UNet++, TransUNet | SIIM Medical Images |
| 12 | Multi-Label Classification | Multi-Tag Prediction | ConvNeXt V2, Swin V2 | Flickr30k |
| 13 | Fashion Recommendation | Visual Similarity / KNN | DINOv2, OpenFashionCLIP | Fashion Product Images |
| 14 | Industrial Defect Detection | Anomaly Detection | PatchCore, EfficientAD | MVTec AD |
| 15 | Image-to-Text Search | Vision-Language Retrieval | CLIP | Flickr8k |
Almost all projects (1–14) share the same structure and commands. Project 15 uses a slightly different entry point — see its README.
# 1. Enter the project folder
cd "<project folder name>"
# 2. Create and activate a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate # macOS / Linux
# 3. Install dependencies
pip install -r requirements.txt
# 4. Download the dataset
python utils/download_data.py # see exceptions below
# 5. Train
python train.py --config config.json
# 6. Monitor training (optional, in a separate terminal)
tensorboard --logdir saved/log
# 7. Evaluate with the best checkpoint
python test.py --resume saved/models/<ProjectName>/latest/best-checkpoint-epoch=XX-val_loss=X.XXX.ckpt
# 8. Run inference (if the project provides inference.py)
python inference.py --resume saved/models/.../best-checkpoint-....ckpt --image path/to/image.jpg| Project | Extra Steps | Download Command |
|---|---|---|
| 1 License Plate | Install Tesseract OCR on your system | python utils/download_data.py |
| 2 OCR | Dataset included or fetched during training setup — no separate download script | — |
| 6 YOLO | Uses root-level download script | python download_dataset.py |
| 12 Multi-Label | Run label preparation after download | python utils/download_data.py then python utils/prepare_dataset.py |
| 13 Fashion | Download OpenFashionCLIP weights to weights/openfashionclip.pt |
python utils/download_data.py |
| 15 CLIP Search | Manual dataset placement; uses main.py instead of train.py |
See Project 15 README |
cd "15_Image-to-Text Search Engine (CLIP-based)"
pip install -r requirements.txt
# Manually download Flickr8k and place at:
# data/Flickr8k_Dataset/captions.txt
# data/Flickr8k_Dataset/images/
python main.py --model clip
python main.py --model clip --batch_size 32 --num_epochs 20 --lr 5e-5Most projects follow this layout:
<Project Name>/
├── README.md # Detailed docs, models, and results
├── config.json # Hyperparameters and model selection
├── requirements.txt # Python dependencies
├── train.py # Training entry point
├── test.py # Evaluation entry point
├── inference.py # Real-time / single-image inference (some projects)
├── parse_config.py # Config parser
├── base/ # Base trainer, model, data loader classes
├── model/ # Model architecture, loss, metrics
├── data_loader/ # Dataset and DataLoader
├── trainer/ # PyTorch Lightning trainer logic
├── logger/ # Logging and TensorBoard visualization
├── utils/ # Helpers (download_data.py, etc.)
├── saved/
│ ├── models/ # Checkpoints (best-checkpoint-*.ckpt)
│ ├── log/ # TensorBoard logs
│ └── visual_results/ # Prediction visualizations
└── data/ # Datasets (created after download)
Many projects let you swap architectures by editing config.json — no code changes needed. Look for keys like "model_type", "arch", or "type" in each project's README.
Example (Project 6 — YOLO):
"model_type": "YOLOv8"Change to "YOLOv10" to switch architectures.
| Problem | Solution |
|---|---|
kaggle: command not found |
Run pip install kaggle and verify kaggle.json is in ~/.kaggle/ |
| Kaggle download fails (403) | Accept the dataset license on Kaggle's website before downloading |
| CUDA out of memory | Reduce batch_size in config.json or use CPU |
| Missing checkpoint on test | Train first, or use the path printed at the end of training |
| Tesseract not found (Project 1) | Install Tesseract and add it to your system PATH |
| Project 8 runs without real data | Project 8 auto-generates a small synthetic dataset if Flickr8k is missing |
- Deep Learning: PyTorch, PyTorch Lightning
- Model Libraries:
timm,transformers,ultralytics,open-clip-torch - Computer Vision: OpenCV, Pillow, Albumentations
- Medical Imaging:
pydicom - Monitoring: TensorBoard
- Datasets: Kaggle (primary source)
Contributions are welcome! To add or improve a project:
- Fork the repository
- Create a feature branch
- Add your changes with a clear README in the project folder
- Open a pull request against
main
Built by AppliedAI-Lab. Each project README contains paper references and dataset citations for the models and benchmarks used.
No license file is included in the repository. Contact the maintainers before using this code in commercial products.