|
|
| RF-DETR detection of Satellite Ground Stations from aerial imagery | RF-DETR detection of Nuclear Reactor infrastructure from aerial imagery |
All detections shown above were generated using the fine-tuned RF-DETR model developed in this project.
This project explores aerial and drone-based detection of critical infrastructure using RF-DETR (Roboflow Detection Transformer).
The objective was to build a robust object detection system capable of identifying multiple infrastructure categories from high-altitude imagery while addressing one of the biggest challenges in computer vision projects:
Data Scarcity
Rather than focusing solely on model architecture, this project follows a data-centric workflow, emphasizing:
✅ Dataset Creation ✅ Annotation Quality Control ✅ Synthetic Data Generation ✅ Error Analysis ✅ Iterative Retraining ✅ Class-Specific Optimization
The model was trained to detect 19 infrastructure categories:
| Infrastructure | Infrastructure |
|---|---|
| Airport Runway | Oil Refinery |
| Bridge | Satellite Dish / Ground Station |
| Cargo Ship | Seaport |
| Cooling Tower | Shipping Containers |
| Dam | Solar Power Plant |
| Electrical Substation | Thermal Power Plant |
| Energy Storage Infrastructure | Transmission Tower |
| Mobile Tower | Water Tower |
| Nuclear Reactor | Wind Turbine |
| Mobile Harbour Cranes |
The dataset was created using a combination of:
- Real aerial imagery
- Satellite imagery
- Synthetic imagery
┌──────────────────────┐
│ Real Aerial Imagery │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Synthetic Imagery │
│ (Nano Banana Pro, │
│ Seedream 4) │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Manual QA & Review │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Label Studio │
│ Annotation │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ COCO Dataset Export │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ RF-DETR Training │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Evaluation & │
│ Error Analysis │
└──────────────────────┘
To address limited availability of infrastructure imagery from rare aerial viewpoints, synthetic samples were generated using:
- Nano Banana Pro
- Seedream 4
- ChatGPT-assisted prompt engineering
Synthetic data was used to create:
- Oblique drone perspectives
- Complex environmental conditions
- Low-light scenarios
- Rare infrastructure viewpoints
- Additional examples for underrepresented classes
All images underwent manual review before training.
✅ Removal of unrealistic synthetic outputs ✅ Visual inspection for generation artifacts ✅ Annotation consistency checks ✅ Duplicate image filtering ✅ Class balance verification ✅ Manual edge-case review
The goal was to maximize annotation quality before model training rather than relying solely on augmentation techniques.
Label Studio
COCO JSON
- Manual bounding-box annotation
- Class-level verification
- Edge-case review
- Annotation consistency validation
| Category A | Category B |
|---|---|
| Thermal Power Plant | Nuclear Reactor |
| Cargo Ship | Mobile Harbour Cranes |
| Shipping Containers | Airport Runway Structures |
| Electrical Substation | Energy Storage Infrastructure |
These categories received additional review due to visual similarity and contextual overlap.
RF-DETR was selected because transformer-based detectors can leverage broader contextual information across aerial scenes.
This is particularly useful when distinguishing visually similar infrastructure categories where surrounding environmental context contributes to classification accuracy.
- Google Colab
- PyTorch
- RF-DETR
- CUDA GPU Acceleration
✅ Transfer Learning ✅ Multi-Stage Fine-Tuning ✅ Early Stopping ✅ EMA Evaluation ✅ Targeted Retraining of Weak Classes
| Metric | Value |
|---|---|
| Best mAP50-95 | 0.531 |
| Best Model | EMA Checkpoint |
| Early Stopping | Enabled |
The final model demonstrated strong performance on visually distinctive infrastructure classes while highlighting several challenging categories that remain active areas of improvement.
| Category |
|---|
| Oil Refinery |
| Water Tower |
| Satellite Dish / Ground Station |
| Mobile Tower |
| Transmission Tower |
| Category |
|---|
| Shipping Containers |
| Electrical Substations |
| Solar Power Plants |
| Dams |
Common challenges include:
- Small object size
- Dense object clustering
- High visual similarity
- Scale variation across aerial viewpoints
A major outcome of this project was confirming that:
Dataset quality, annotation consistency, and targeted data collection produced larger gains than additional training epochs alone.
Performance improvements were primarily achieved through:
- Additional class-specific data collection
- Synthetic data augmentation
- Error analysis
- Annotation refinement
- Focused retraining of underperforming classes
This reinforced the importance of a data-centric AI workflow over purely model-centric optimization.
Critical-Infrastructure-Detection-using-RF-DETR/
├── assets/
├── dataset_info/
├── notebooks/
├── weights/
├── README.md
├── requirements.txt
└── LICENSE
🔗 Google Colab
https://colab.research.google.com/drive/167xgsFcpFqzVfAbT88IFdUYeM_X0h1um
Available through the repository Releases section.
Planned improvements include:
- Hard-negative mining
- Expansion of Shipping Container samples
- Improved Substation annotation consistency
- Video-based infrastructure tracking
- Integration with segmentation pipelines
- Multi-object tracking support
Computer Vision Practitioner | Data Annotation Specialist
📧 Open to collaboration on:
- Computer Vision
- Object Detection
- Dataset Development
- Annotation Workflows
- Synthetic Data Generation
GitHub https://github.com/sayanc227
LinkedIn https://www.linkedin.com/in/sayan-chakraborty-595a45382