A complete, offline-first, intelligence-grade platform for searching satellite imagery archives using natural language and detecting structural changes over time using deep learning.
- Overview
- Key Features
- System Architecture
- Project Structure
- Technology Stack
- AI Models
- Database Schema
- API Reference
- Getting Started
- Training the AI Models
- Data Ingestion Pipeline
- Frontend Dashboard
- Datasets
- Performance Tuning
- License
SentinelSearch solves a critical bottleneck in geospatial intelligence: finding what you need inside massive satellite imagery archives without knowing exactly when or where it is.
Traditional GIS tools require analysts to manually pan and zoom through thousands of satellite tiles using rigid filters (date, bounding box, sensor). SentinelSearch replaces this workflow entirely.
An analyst can simply type:
"Find areas where new concrete structures appeared near rivers between 2023 and 2024"
The platform will:
- Parse the intent using a local AI language model (Ollama/Phi-3).
- Convert the query into a 256-dimensional semantic embedding vector.
- Instantly search millions of pre-indexed tile embeddings stored in FAISS (Facebook AI Similarity Search).
- Return ranked, geolocated results directly on an interactive map.
The system operates entirely offline — all AI models, the vector index, and the spatial database are self-contained. No data ever leaves your local machine or network.
Instead of relying on rigid keyword tags or metadata, SentinelSearch uses a custom-trained Lightweight Convolutional Neural Network (CNN) to convert each satellite tile into a 256-dimensional semantic embedding vector that captures the visual meaning of the imagery.
When an analyst submits a query, the local Ollama language model (phi3) extracts structured intent (object types, temporal keywords, geographic hints), which is then matched against the vector archive using FAISS approximate nearest neighbor search. Results are ranked by cosine similarity and returned with geospatial bounding boxes that are visualized directly on the Leaflet map.
What this means in practice: You can find a tile showing a newly built warehouse near a highway even if the tile was never manually tagged with those keywords. The CNN embedding captures the visual pattern, not just the text metadata.
The Siamese Change Detection Model takes two satellite images of the exact same location at different timestamps and identifies where things have changed at the pixel level.
The model architecture uses a shared-weight convolutional encoder for both images, followed by a differencing layer and a decoder that produces a binary change mask (same resolution as the input). This change mask is then overlaid on the map and classified into change categories (Construction, Vegetation Loss, Water Encroachment, etc.).
Use Case: Monitor a city perimeter over 12 months. The system will automatically highlight every new building, road extension, or cleared forest patch.
All natural language understanding happens locally using Ollama, a self-hosted LLM runtime. The lightweight phi3 model (Microsoft's 3.8B parameter model, quantized to ~2GB) is bundled inside a Docker container and runs without internet access.
The NLP layer is responsible for:
- Intent Extraction: Pulling out the core subject (e.g., "buildings", "water body")
- Temporal Parsing: Identifying date ranges (e.g., "between 2023 and 2024")
- Geographic Hints: Identifying spatial qualifiers (e.g., "near rivers", "in coastal areas")
- Change Type Classification: Identifying if the query is about changes or static features
We use Facebook's FAISS library to index and search millions of 256D tile embeddings at sub-millisecond latency. The FAISS IndexIVFFlat index uses inverted file quantization to organize the embedding space into Voronoi cells, enabling approximate nearest-neighbor search without scanning every embedding linearly.
Practical throughput: On a CPU, FAISS can search 1 million 256D vectors in under 5ms.
All scene and tile metadata is stored in a PostgreSQL 15 database with the PostGIS extension. This enables:
- Spatial Filtering: Filter results by bounding box, radius, or polygon intersection.
- Temporal Filtering: Query tiles by acquisition date.
- Sensor Filtering: Filter by satellite platform (Sentinel-2, Landsat-8, PlanetScope).
- Review Queue: Track analyst review states (PENDING, APPROVED, REJECTED).
The analyst dashboard is built as a React (Vite + TailwindCSS v4) single-page application styled as an advanced tactical HUD interface. It features:
- Leaflet Map Integration: All search results are rendered as bounding box overlays directly on an interactive dark-mode map.
- Real-time UTC Clock: Live system timestamp in the header for mission-critical timestamping.
- System Status Panel: Right-side telemetry readout showing FAISS, Ollama, and database status.
- Live API Log: Animated terminal-style log window showing every backend event.
- Mode Switching: The left navigation panel allows switching between Semantic Search, Temporal Change Analysis, and Similar Scene Detection modes.
- CRITICAL / MONITOR / LOGGED tagging: Every result is tagged with a risk status based on its confidence score.
┌─────────────────────────────────────────────────────────────────┐
│ Analyst Browser (Port 5173) │
│ React + Vite + TailwindCSS + Leaflet │
└──────────────────────────┬──────────────────────────────────────┘
│ HTTP (REST)
▼
┌─────────────────────────────────────────────────────────────────┐
│ FastAPI Backend (Port 8000) │
│ │
│ ┌──────────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ /api/search │ │ /api/change │ │ /api/ingestion│ │
│ │ /text │ │ /analyse │ │ /scene │ │
│ └────────┬─────────┘ └──────┬────────┘ └──────┬────────┘ │
│ │ │ │ │
│ ┌────▼────┐ ┌──────▼──────┐ ┌───────▼──────┐ │
│ │ Ollama │ │ Siamese │ │ LightweightCNN│ │
│ │ (NLP) │ │ Change │ │ (Ingestor) │ │
│ └────┬────┘ │ Model │ └───────┬──────┘ │
│ │ └──────┬──────┘ │ │
│ ┌────▼────┐ │ ┌───────▼──────┐ │
│ │ FAISS │◄──────────────┘ │ FAISS Index │ │
│ │ Search │ │ (Write) │ │
│ └────┬────┘ └──────────────┘ │
│ │ │
│ ┌────▼────────────────────────────────────────────────┐ │
│ │ PostGIS Database (Port 5432) │ │
│ └─────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Semantic Retrieval/
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI app with all API endpoints
│ │ ├── core/
│ │ │ └── vector_index.py # FAISS wrapper for embedding storage & search
│ │ └── models/
│ │ ├── semantic_encoder.py # LightweightCNN - generates 256D embeddings
│ │ └── siamese_change_model.py # Siamese network for pixel-level change detection
│ ├── Dockerfile # Backend container definition
│ └── requirements.txt # Python dependencies
├── frontend/
│ ├── src/
│ │ ├── App.tsx # Main React component (tactical HUD interface)
│ │ └── index.css # TailwindCSS v4 + tactical theme variables
│ ├── Dockerfile # Frontend container definition
│ └── vite.config.ts # Vite configuration with Tailwind v4 plugin
├── db/
│ └── init/
│ └── schema.sql # PostgreSQL + PostGIS table definitions
├── scripts/
│ ├── train_semantic_encoder.py # EuroSAT training loop for the CNN
│ ├── train_siamese_change.py # LEVIR CD training loop for the Siamese model
│ ├── ingest_scene.py # GeoTIFF ingestion pipeline (rasterio + GDAL)
│ └── generate_synthetic_data.py # Synthetic data generator for unit testing
├── data/
│ ├── EuroSAT/ # EuroSAT dataset (downloaded separately)
│ ├── LEVIR CD/ # LEVIR Change Detection dataset
│ ├── spacenet/ # SpaceNet 7 multi-temporal dataset
│ ├── raw/ # Raw GeoTIFFs for ingestion
│ └── weights/ # Saved PyTorch model weights (.pt files)
├── docker-compose.yml # Multi-container Docker orchestration
├── README.md # This document
├── .gitignore # Git exclusion rules
└── architecture.md # High-level architecture notes
| Layer | Technology | Reason |
|---|---|---|
| Frontend Framework | React 18 + Vite | Fast HMR development, modern bundler |
| Frontend Styling | TailwindCSS v4 | Utility-first, fast design iteration |
| Map Library | React Leaflet | Lightweight, no API key required |
| HTTP Client | Axios | Promise-based, simple API integration |
| Backend Framework | FastAPI | Async-first, auto-generates Swagger docs |
| ORM / DB Client | psycopg2 | Direct PostgreSQL driver, JSONB support |
| Database | PostgreSQL 15 + PostGIS | Geospatial indexing, ST_ functions |
| Vector Search | FAISS | Sub-millisecond nearest-neighbor search |
| Deep Learning | PyTorch 2.x | Dynamic computation graphs, CUDA support |
| NLP Engine | Ollama (Phi-3) | Offline LLM, lightweight, quantized |
| Containerization | Docker + Docker Compose | Reproducible offline deployment |
| Raster Processing | Rasterio + GDAL | GeoTIFF reading, tiling, CRS transforms |
File: backend/app/models/semantic_encoder.py
A custom convolutional neural network designed for fast feature extraction from satellite imagery tiles. It accepts a (B, 3, 256, 256) RGB tensor and produces a (B, 256) L2-normalized embedding vector.
Architecture:
Input (3, 256, 256)
→ Conv2d(3, 32) + BatchNorm + ReLU + MaxPool → (32, 128, 128)
→ Conv2d(32, 64) + BatchNorm + ReLU + MaxPool → (64, 64, 64)
→ Conv2d(64, 128) + BatchNorm + ReLU + MaxPool → (128, 32, 32)
→ AdaptiveAvgPool(4, 4) → (128, 4, 4)
→ Flatten → Linear → L2 Normalize → (256,)
Training Dataset: EuroSAT (27,000 Sentinel-2 images across 10 land-use classes).
Output: A 256-dimensional embedding that is indexed into FAISS for semantic retrieval.
File: backend/app/models/siamese_change_model.py
A Siamese network architecture that accepts two temporally separated satellite images of the same location and produces a pixel-level binary change mask.
Architecture:
Image_A (3, 256, 256) ──► Shared Encoder ──► Feature Map A (512, 8, 8)
↓
Image_B (3, 256, 256) ──► Shared Encoder ──► Feature Map B (512, 8, 8)
↓
Feature Difference
↓
Decoder (Upsampling)
↓
Change Mask (1, 256, 256) [0-1 probability]
Training Dataset: LEVIR-CD (637 pairs of 1024×1024 image pairs covering construction changes).
Loss Function: Binary Cross-Entropy with Logits (BCEWithLogitsLoss), treating each pixel as an independent binary classification (changed vs. unchanged).
The PostGIS database contains the following core tables:
| Table | Description |
|---|---|
scenes |
Top-level satellite image metadata (date, sensor, CRS, bounding box geometry) |
tiles |
256x256 sub-tiles derived from each scene, storing tile-level bounding boxes |
vectors |
Stores the FAISS vector IDs linking tiles to their semantic embeddings |
changes |
Stores the output of change detection analysis runs with pixel masks |
reviews |
Analyst review queue entries (status: PENDING, APPROVED, REJECTED) |
All geometric columns use EPSG:4326 (WGS84) coordinate reference system.
The FastAPI backend auto-generates interactive Swagger documentation at http://localhost:8000/docs.
| Method | Endpoint | Description |
|---|---|---|
POST |
/api/search/text |
Natural language semantic search over tile embeddings |
POST |
/api/search/similar |
Find visually similar tiles given a tile ID |
POST |
/api/change/analyse |
Run Siamese change detection between two temporal tiles |
POST |
/api/ingestion/scene |
Ingest a new GeoTIFF scene into the database and index |
GET |
/health |
Service health check |
GET |
/docs |
Interactive Swagger UI |
{
"query": "Find newly built structures near rivers",
"date_start": "2024-01-01",
"date_end": "2024-12-31",
"top_k": 20
}{
"date_start": "2024-01-01",
"date_end": "2024-12-31",
"aoi": {
"type": "Polygon",
"coordinates": [[[lng, lat], ...]]
}
}- Docker Desktop with WSL2 enabled
- Node.js v18+ and npm (for local frontend dev)
- Python 3.11 (for running training scripts on your local GPU)
- 4GB+ NVIDIA GPU (highly recommended for training; CUDA 12.1 required)
git clone https://github.com/your-username/sentinel-search.git
cd sentinel-searchdocker-compose up --build -dThis command will download and start all four services:
sentinel_db— PostgreSQL + PostGIS atlocalhost:5432sentinel_backend— FastAPI atlocalhost:8000sentinel_ollama— Ollama LLM engine atlocalhost:11434sentinel_frontend— React dashboard atlocalhost:5173
docker exec -it sentinel_ollama ollama run phi3Wait for the model to download, then type /bye to exit the interactive prompt.
Open your browser and go to: http://localhost:5173
Access the backend API documentation at: http://localhost:8000/docs
py -3.11 -m venv venv311
.\venv311\Scripts\activate
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install tqdm pillowEnsure EuroSAT is extracted so each class is a subfolder inside data/EuroSAT/.
data/EuroSAT/
├── AnnualCrop/
├── Forest/
├── HerbaceousVegetation/
├── Highway/
├── Industrial/
├── Pasture/
├── PermanentCrop/
├── Residential/
├── River/
└── SeaLake/
Run:
python scripts/train_semantic_encoder.pyExpected Output: A progress bar for each epoch showing the training loss. The final weights are saved to data/weights/semantic_encoder.pt.
Ensure LEVIR CD is structured as:
data/LEVIR CD/
└── train/
├── A/ ← Before images (PNG)
├── B/ ← After images (PNG)
└── label/ ← Binary change masks (PNG, 0=no change, 255=change)
Run:
python scripts/train_siamese_change.pyExpected Output: Training loss decreasing across 10 epochs. Weights saved to data/weights/siamese_change.pt.
Once your models are trained, you can ingest real GeoTIFF satellite imagery into the platform.
- Place your
.tiffiles insidedata/raw/. - Run the ingestion script:
python scripts/ingest_scene.py
- The script will:
- Validate the GeoTIFF and read its coordinate reference system (CRS).
- Reproject the imagery to EPSG:4326 if necessary.
- Cut the full scene into 256x256 pixel tiles.
- Run each tile through the
LightweightCNNto generate a 256D embedding. - Save the embedding to the FAISS vector index.
- Store scene metadata and tile bounding boxes in the PostGIS database.
After ingestion, every tile will be immediately searchable from the React dashboard!
The analyst dashboard is a modern tactical HUD interface. Here is a description of each UI component:
| Component | Description |
|---|---|
| Header | Shows the app name, active node ID, database status, and a live UTC clock. |
| Left Navigation | Mode switcher: Semantic Search, Temporal Change, and Similar Scene finder. |
| Query Panel | Natural language input, date range pickers, and the EXECUTE_QUERY button. |
| Results Feed | Shows ranked search results, each with a confidence score and CRITICAL/MONITOR/LOGGED status badge. |
| Leaflet Map | Full-screen interactive map. Results are visualized as bounding box overlays (red for CRITICAL, cyan for others). |
| System Status Panel | Right-side HUD showing FAISS index status, vector count, Ollama status, and the AI model state. |
| Live Log | Animated terminal-style log window that updates in real time with backend events. |
| Dataset | Usage | Size | License | Link |
|---|---|---|---|---|
| EuroSAT | Train Semantic Encoder | ~2GB | MIT | github.com/phelber/eurosat |
| LEVIR CD | Train Change Model | ~3GB | CC BY-NC 4.0 | rcdaudt.github.io/oscd |
| SpaceNet 7 | Multi-temporal building tracking | ~15GB | CC BY-SA 4.0 | spacenet.ai |
| Onera OSCD | Urban change detection | ~1GB | Free | rcdaudt.github.io/oscd |
Both training scripts are pre-optimized for 4GB VRAM:
- Automatic Mixed Precision (AMP): Uses
torch.cuda.amp.autocast()to halve memory by computing in fp16. - Micro-Batching: Batch size of 16 for the Semantic Encoder and 4 for the Siamese Model.
- Image Downscaling: All inputs are automatically resized to 256×256 before GPU transfer.
- Pin Memory:
pin_memory=Truein DataLoaders to speed up CPU-to-GPU transfers.
- Increase
BATCH_SIZEto 64+ if you have a 16GB+ GPU for faster training. - Use
IndexIVFPQin FAISS instead ofIndexFlatL2for compressed 100M+ scale indices. - Use PgBouncer connection pooling in front of PostgreSQL for high-concurrency inference requests.
This project was developed for the Smart India Hackathon (SIH). All open-source datasets belong to their respective creators and are governed by their individual licenses (see Datasets table above). The code in this repository is released under the MIT License.