MarocExplorer 🇲🇦

MarocExplorer is a geographic prediction project that estimates the GPS coordinates (latitude, longitude) of an image taken in Morocco, leveraging both its visual content and metadata associated with the corresponding city (climate, architecture, altitude, environment type, etc.).

The approach uses an architecture inspired by GeoNet (e.g. GeoGuessr, IM2GPS, PlaNet) to extract deep visual representations via a CNN backbone (ResNet50). These features are then combined with city characteristics in a tabular branch, followed by an attention mechanism and a deep fully-connected regression module to predict GPS coordinates.

🎯 Designed for researchers, computer vision engineers, and students — MarocExplorer highlights the impact of contextual metadata on the performance of a geographic regression model.

---

Table of Contents

• Key Objectives
• Global Pipeline
• Model Architecture
• Mathematical Foundations
• Project Structure
• Documentation
• Authors

---

🚀 Key Objectives

• 🖼 Predict GPS coordinates from urban images
• 🏙 Compare model performance with and without city features
• 🧠 Combine CNN-based image extraction with a tabular feature sub-network
• 📉 Analyze prediction accuracy with relevant GPS metrics (MSE, Haversine, R²)

---

🔧 Global Pipeline

1. Data Loading

Data is loaded from a file containing image metadata (names, GPS coordinates, associated city) and descriptive city characteristics. Images are organized in a dedicated folder.

2. Preprocessing

A set of transformations is applied: processing of numerical and categorical variables, and image transformations to make them compatible with the model.

3. Modeling

The model automatically extracts relevant visual information from images, optionally integrating tabular data from cities. The goal is to learn a regression function that estimates geographic location.

4. Training

The model is trained in a supervised manner on annotated data. The impact of city metadata on accuracy can be tested by enabling or disabling their use in the pipeline.

---

🧠 Model Architecture

The model is composed of 5 sequential modules:

Image ──────────────► ResNet50 Backbone ──────────────► Visual Features (2048)
                                                                │
City Features ──► Dense Encoder (3 layers) ──────────► City Embedding (128)
                                                                │
                                          ┌─────────────────────┘
                                          ▼
                                  Concatenation (2176)
                                          │
                                  Attention Module
                                          │
                                  Deep Regression (4 layers)
                                          │
                                  GPS Output (lat, lon)

1. Visual Feature Extraction

• Backbone: ResNet50 pre-trained on ImageNet — output: 2048 neurons
• All layers frozen except layer4 (fine-tuned)

2. City Feature Encoder

A 3-layer dense network:

Linear(num_city_features → 128)  →  ReLU
Linear(128 → 256)                →  ReLU
Linear(256 → 128)

Output: 128-dimensional city embedding

3. Attention Mechanism

Image (2048) + City (128) are concatenated → combined vector of size 2176

Then passed through:

Linear(2176 → 512)  →  ReLU
Linear(512 → 2176)  →  Sigmoid

The attention mask is applied element-wise:

$$\text{combined_att} = \text{combined} \odot \sigma!\left(W_2 \cdot \text{ReLU}(W_1 \cdot \text{combined})\right)$$

Where:

• $\odot$ — element-wise (Hadamard) product
• $\sigma$ — Sigmoid activation
• $W_1 \in \mathbb{R}^{512 \times 2176}$, $W_2 \in \mathbb{R}^{2176 \times 512}$ — learned weight matrices

4. Deep Regression Network

A 4-layer regression head with residual connection:

Layer	In → Out	Activation	Dropout
fc1	2176 → 1024	ReLU	0.15
fc2	1024 → 512	ReLU	0.10
fc3	512 → 256	ReLU	0.05
Residual	2176 → 256	—	—

Final formula with residual connection:

$$x = \text{fc3}(\text{fc2}(\text{fc1}(\text{combined_att}))) + W_{\text{res}} \cdot \text{combined}$$

5. Final Prediction

$$\hat{y} = \text{fc_final}(x) \in \mathbb{R}^2 \quad \Longrightarrow \quad (\hat{\text{lat}},\ \hat{\text{lon}})$$

---

📐 Mathematical Foundations

Loss Function — MSE

The model is trained by minimizing the Mean Squared Error between predicted and true GPS coordinates:

$$\mathcal{L}_{\text{MSE}} = \frac{1}{N} \sum_{i=1}^{N} \left[(\hat{\text{lat}}_i - \text{lat}_i)^2 + (\hat{\text{lon}}_i - \text{lon}_i)^2\right]$$

Evaluation Metric — Haversine Distance

The geographic error is measured using the Haversine formula (great-circle distance in km):

$$d = 2R \cdot \arcsin!\left(\sqrt{\sin^2!\left(\frac{\Delta\phi}{2}\right) + \cos\phi_1 \cos\phi_2 \sin^2!\left(\frac{\Delta\lambda}{2}\right)}\right)$$

Where:

• $R = 6371$ km — Earth's radius
• $\phi_1, \phi_2$ — latitudes in radians
• $\Delta\phi = \phi_2 - \phi_1$ — latitude difference
• $\Delta\lambda = \lambda_2 - \lambda_1$ — longitude difference

Evaluation Metric — R² Score

$$R^2 = 1 - \frac{\sum_{i=1}^{N}(\hat{y}_i - y_i)^2}{\sum_{i=1}^{N}(\bar{y} - y_i)^2}$$

Where $\bar{y}$ is the mean of the true coordinates. A score of $R^2 = 1$ means perfect prediction.

Attention Weight Formula

The attention gate produces a per-dimension importance score:

$$\alpha = \sigma(W_2 \cdot \text{ReLU}(W_1 \cdot z)) \in [0, 1]^{2176}$$

$$z_{\text{att}} = z \odot \alpha$$

Where $z \in \mathbb{R}^{2176}$ is the concatenated feature vector.

---

📂 Project Structure

MarocExplorer/
├── data/                        # Training data
│   ├── images/                  # Urban images folder
│   ├── coords.csv               # GPS coordinates (latitude, longitude, city)
│   └── city_features.csv        # City characteristics (climate, altitude, etc.)
│
├── model/                       # Saved model
│   └── model.pth                # Model weights
│
├── interface/                   # Visualization or demo interface
│
├── doc/                         # Documentation
│
├── notebooks/                   # Exploratory analyses and tests
│   ├── MarocExplorer_Data.ipynb
│   ├── MarocExplorer_Model.ipynb
│   └── Usage_Evaluation.ipynb
│
└── README.md                    # Project presentation

---

📚 Documentation

Full project documentation is available at:

👉 https://maroxexplorer.readthedocs.io/en/latest/index.html

---

Last updated: 2025 | Version: 1.0.0