techstack.md

Technology Stack

Offline AI-Powered Satellite Imagery Semantic Search and Change Detection Platform

1. Technology Selection Principles

The technology stack is selected around five constraints:

1. Complete offline operation after staging.
2. Ability to train and run custom AI models.
3. Support for GeoTIFF/COG and geospatial metadata.
4. Incremental vector indexing.
5. Reproducible deployment on a local workstation or server.

The recommended stack prioritizes open-source software and local execution.

---

2. High-Level Stack

Frontend
├── React
├── TypeScript
├── Vite
├── Leaflet or OpenLayers
└── Local UI assets

Backend
├── Python
├── FastAPI
├── Pydantic
└── Uvicorn

AI / ML
├── PyTorch
├── torchvision
├── Custom CNN / Vision Transformer
├── Custom Siamese Change Detection Model
└── Optional local text model

Geospatial
├── GDAL
├── Rasterio
├── GeoPandas
├── Shapely
├── pyproj
└── NumPy

Vector Search
├── FAISS
└── Optional HNSW/other local index

Database
├── PostgreSQL
├── PostGIS
└── SQLite fallback for prototype

Deployment
├── Docker
├── Docker Compose
└── Offline package/model bundles

---

3. Frontend

3.1 React + TypeScript

Recommended for:

• Component-based dashboard.
• Search workflows.
• Review queues.
• Interactive state management.
• Typed API integration.

Suggested libraries:

• React.
• TypeScript.
• Vite.
• TanStack Query for API state.
• Zustand or Redux Toolkit for application state.
• Tailwind CSS or another locally bundled styling approach.

3.2 Mapping

Option A: Leaflet

Best for a fast prototype.

Advantages:

• Easy integration.
• Lightweight.
• Good marker/layer support.

Option B: OpenLayers

Better for more advanced geospatial workflows.

Advantages:

• Strong raster/vector support.
• Advanced projections.
• More GIS-oriented capabilities.

Important offline requirement:

Do not depend on a public online tile provider for core functionality. Use:

• Local raster layers.
• Local MBTiles/PMTiles where supported.
• A simple offline basemap packaged with the system.
• Coordinate/grid fallback.

---

4. Backend

4.1 Python

Python is recommended because the ecosystem supports:

• Deep learning.
• Raster processing.
• Geospatial analysis.
• Vector indexing.

4.2 FastAPI

Recommended API modules:

/api/search
/api/similarity
/api/change
/api/ingestion
/api/scenes
/api/review
/api/clusters
/api/export
/api/system

Example service structure:

backend/
├── app/
│   ├── main.py
│   ├── api/
│   ├── services/
│   ├── repositories/
│   ├── schemas/
│   └── core/

---

5. Geospatial Technology

5.1 GDAL

Use GDAL for:

• Raster inspection.
• Format support.
• Metadata extraction.
• Reprojection.
• Conversion where required.

5.2 Rasterio

Use Rasterio for Python-native raster processing:

• Reading GeoTIFF.
• Window/tile extraction.
• Band access.
• Transform preservation.
• Bounding-box calculations.

5.3 GeoPandas and Shapely

Use for:

• AOI polygons.
• Spatial filtering.
• Footprints.
• Intersections.
• Exporting GeoJSON-like results.

5.4 pyproj

Use for:

• Coordinate transformation.
• CRS handling.

---

6. AI / Machine Learning Stack

6.1 PyTorch

Primary framework for:

• Custom model development.
• Training.
• GPU inference.
• Checkpointing.
• Offline execution.

The final model should be stored locally, for example:

models/
├── semantic_encoder/
│   ├── model.pt
│   ├── config.json
│   └── metadata.json
└── change_detector/
    ├── model.pt
    ├── config.json
    └── metadata.json

6.2 Custom Semantic Image Encoder

Recommended prototype options:

Lightweight CNN

Input Tile
   ↓
Conv Block 1
   ↓
Conv Block 2
   ↓
Conv Block 3
   ↓
Global Average Pooling
   ↓
Dense Projection
   ↓
Embedding Vector

Advantages:

• Easier to train from scratch.
• Lower GPU requirements.
• Faster inference.

Small Vision Transformer

Image
 ↓
Patch Embedding
 ↓
Transformer Encoder Blocks
 ↓
Pooling
 ↓
Projection Head
 ↓
Embedding

Advantages:

• More expressive global context.
• Natural embedding output.

Disadvantage:

• More training data and compute may be required.

6.3 Recommended Embedding Size

Start with:

• 256 dimensions for lightweight indexing, or
• 512 dimensions for richer representations.

Example:

Tile → Custom Encoder → 512 float values

Normalize vectors before cosine/inner-product search.

---

7. Change Detection Model

7.1 Recommended Architecture: Siamese Encoder

Before Image ──┐
               ├── Shared CNN Encoder
After Image  ──┘
       ↓
Feature Comparison
       ↓
Fusion Layers
       ↓
Decoder / Classifier
       ↓
Change Mask + Change Class

The model can produce:

1. Pixel-level change mask.
2. Tile-level change probability.
3. Change class.

Suggested classes:

0 = No Change
1 = Construction
2 = Clearance
3 = Water Increase
4 = Water Decrease
5 = Road Development
6 = Other Supported Change

7.2 Feature Comparison Options

• Absolute feature difference.
• Concatenation.
• Element-wise multiplication.
• Attention-based fusion.

A simple prototype can use:

abs(features_before - features_after)

followed by convolutional classification.

---

8. Natural Language Search

8.1 Important Architecture Decision

The project should separate:

Language understanding

from

Satellite visual understanding.

A local LLM should not be treated as the core vision model.

8.2 Option A: Custom Text-Image Dual Encoder

Architecture:

Text Query ─────> Text Encoder ───> Text Embedding
                                      │
                                      │ similarity
                                      │
Satellite Tile ─> Image Encoder ──> Image Embedding

Training requires image-text pairs.

Example labels/captions:

"new building near river"
"dense road network"
"large water body"
"cleared vegetation"
"industrial structures"

A contrastive objective can train matching text/image pairs closer together in embedding space.

8.3 Option B: Structured Query Parsing

A local language component can transform:

"Find new construction near a river between 2023 and 2025"

into:

{
  "concepts": ["construction", "river"],
  "date_start": "2023-01-01",
  "date_end": "2025-12-31",
  "change_required": true
}

The structured output is then executed by the retrieval and geospatial systems.

8.4 Ollama

Ollama can be installed locally and used offline after model staging.

Recommended role:

• Query interpretation.
• Filter extraction.
• Result explanation.
• Natural-language report generation.

Not recommended as the main model for:

• Satellite tile embeddings.
• Change masks.
• Geospatial image analysis.

---

9. Training Stack

Suggested structure:

training/
├── datasets/
│   ├── semantic_dataset.py
│   ├── change_dataset.py
│   └── augmentations.py
├── models/
│   ├── semantic_encoder.py
│   ├── text_encoder.py
│   └── siamese_change_model.py
├── losses/
│   ├── contrastive_loss.py
│   └── classification_loss.py
├── train_semantic.py
├── train_change.py
├── evaluate.py
└── export_model.py

Useful components:

• PyTorch Dataset/DataLoader.
• Mixed precision when supported.
• TensorBoard or local logs.
• Checkpoint saving.
• Deterministic seeds for reproducibility.

---

10. Dataset Pipeline

Primary sources specified for the challenge may include:

• Copernicus Sentinel-2 optical imagery.
• Copernicus Sentinel-1 SAR imagery.
• USGS Landsat Collection 2.
• NRSC/ISRO Bhuvan open Earth-observation products.

All datasets and licenses should be documented separately.

10.1 Dataset Structure

datasets/
├── raw/
│   ├── sentinel2/
│   ├── sentinel1/
│   ├── landsat/
│   └── bhuvan/
├── processed/
│   ├── tiles/
│   └── temporal_pairs/
├── labels/
│   ├── semantic/
│   └── changes/
└── manifests/

10.2 Manifest

Each record should include:

{
  "scene_id": "example",
  "tile_id": "example_tile_001",
  "sensor": "Sentinel-2",
  "acquisition_time": "2025-01-15T00:00:00Z",
  "bbox": [0, 0, 0, 0],
  "crs": "EPSG:XXXX",
  "labels": ["construction", "river"],
  "quality_score": 0.92
}

---

11. Image Processing and Quality Handling

11.1 Optical Imagery

Possible processing:

• Band alignment.
• Reflectance scaling.
• Normalization.
• Cloud detection.
• Shadow detection.
• Haze estimation.
• Seasonal/context-aware comparison.

11.2 SAR Imagery

Possible processing:

• Speckle reduction.
• Calibration.
• Normalization.
• Geometric alignment.

SAR should not simply be mixed with optical data without sensor-aware processing.

11.3 Registration

Before/after images should be aligned.

Possible methods:

• Metadata/geotransform alignment.
• Feature-based alignment.
• Translation/affine refinement.
• Rejecting pairs with excessive registration error.

---

12. Vector Search

12.1 FAISS

FAISS is recommended because it:

• Runs locally.
• Does not require a cloud service.
• Supports efficient nearest-neighbour search.
• Supports adding vectors incrementally.

Prototype Index

For small-to-medium datasets:

IndexFlatIP

Advantages:

• Exact search.
• Simple.
• Easy to debug.

For larger datasets, evaluate approximate indexes such as IVF or HNSW implementations available in the selected environment.

12.2 Vector Storage

Store separately:

FAISS Index
      +
Metadata Database

FAISS contains:

vector_id → embedding

Database contains:

vector_id → tile metadata + provenance

---

13. Metadata Database

13.1 PostgreSQL + PostGIS

Recommended for the full prototype.

Core tables:

scenes

scene_id
source
sensor
acquisition_time
crs
footprint
file_path
metadata_json

tiles

tile_id
scene_id
vector_id
geometry
tile_path
quality_score
processing_version

changes

change_id
before_tile_id
after_tile_id
change_type
confidence
earliest_supported_observation
status

reviews

review_id
candidate_id
decision
timestamp
notes

processing_runs

run_id
input_id
step_name
parameters
model_version
output_id
timestamp

13.2 Prototype Fallback

SQLite can be used for a small demonstration if PostGIS setup becomes too heavy, but PostgreSQL/PostGIS is preferred for strong geospatial filtering.

---

14. API Design

Search

POST /api/search/text

Input:

{
  "query": "newly built structures near a river",
  "date_start": "2023-01-01",
  "date_end": "2025-12-31",
  "sensor": ["Sentinel-2"],
  "top_k": 20
}

Similarity

POST /api/search/similar

Input:

{
  "tile_id": "tile_123",
  "top_k": 20
}

Change Detection

POST /api/change/analyse

Input:

{
  "aoi": {},
  "date_start": "2023-01-01",
  "date_end": "2025-12-31",
  "change_types": ["construction", "clearance"]
}

Ingestion

POST /api/ingestion/scene

Workflow:

• Validate.
• Register.
• Tile.
• Preprocess.
• Embed.
• Index.
• Commit.

---

15. Offline Deployment

15.1 Preferred Option: Docker

Services:

docker-compose.yml
├── frontend
├── backend
└── database

AI models may be mounted or copied into the image/container.

15.2 Offline Asset Bundle

Before evaluation, stage:

offline_bundle/
├── docker_images/
├── python_wheels/
├── model_weights/
├── local_llm_models/
├── datasets/
└── configs/

15.3 Python Package Bundle

Download packages beforehand:

pip download -r requirements.txt -d offline_packages

Install offline:

pip install --no-index --find-links=offline_packages -r requirements.txt

15.4 Model Loading

Custom PyTorch checkpoint:

model.load_state_dict(torch.load("models/model.pt", map_location=device))

All model paths should be local and configurable.

---

16. Hardware Recommendations

Development Minimum

• Modern 4-core CPU.
• 16 GB RAM.
• 50+ GB available storage.
• Optional NVIDIA GPU.

Recommended Training System

• NVIDIA GPU with at least 8 GB VRAM.
• 32 GB RAM preferred.
• SSD storage.
• CUDA-compatible PyTorch environment.

Inference-Only Demo

CPU execution may be possible for lightweight models, but GPU acceleration will improve:

• Embedding generation.
• Change detection.
• Query response under load.

Hardware used must be documented in the evaluation report.

---

17. Recommended Requirements Files

Core Python dependencies should include the versions tested by the team.

Example categories:

fastapi
uvicorn
torch
torchvision
numpy
pandas
rasterio
geopandas
shapely
pyproj
gdal
faiss-cpu or GPU-compatible alternative
scikit-learn
opencv-python
pydantic
sqlalchemy
psycopg

Exact versions must be frozen before final packaging.

---

18. Development Phases

Phase 1: Data Foundation

• Collect public imagery.
• Validate licenses.
• Build scene manifest.
• Implement GeoTIFF reading.
• Implement tiling.

Phase 2: Semantic Model

• Define classes/concepts.
• Label training tiles.
• Train custom image encoder.
• Generate embeddings.
• Build FAISS index.

Phase 3: Retrieval

• Implement metadata filtering.
• Implement nearest-neighbour search.
• Add result ranking.

Phase 4: Change Model

• Build temporal pairs.
• Label change/no-change data.
• Train Siamese model.
• Add quality filtering.

Phase 5: UI

• Build search.
• Build map.
• Build before/after comparison.
• Build review queue.

Phase 6: Offline Packaging

• Freeze dependencies.
• Package weights.
• Build containers.
• Disconnect network.
• Test from clean start.

---

19. Technology Decisions Summary

---

Requirement Recommended Technology

---

Frontend React + TypeScript

Backend FastAPI

Custom model training PyTorch

Semantic embeddings Custom CNN / Small ViT

Change detection Custom Siamese CNN

Natural-language parsing Custom text encoder or optional local Ollama

GeoTIFF Rasterio + GDAL

Vector search FAISS

Geospatial DB PostgreSQL + PostGIS

Clustering scikit-learn / HDBSCAN if staged locally

Containerization Docker + Docker Compose

Offline packaging Local models + wheelhouse/container images

---

20. Recommended Final Stack

For the SIH prototype, the balanced stack is:

React + TypeScript
        │
        ▼
FastAPI Backend
        │
        ├── Rasterio / GDAL
        ├── PostgreSQL + PostGIS
        ├── FAISS
        │
        ├── Custom Semantic CNN/ViT
        │
        └── Custom Siamese Change Model
                  │
                  ▼
            Local Model Weights

Optional:
Local Ollama
for query parsing and explanations

This design allows the team to demonstrate custom AI development while keeping the entire application reproducible and offline.