SENTINEL-X God's Eye

SENTINEL-X God's Eye is an AI-first UAV mission-awareness platform for search-and-rescue and disaster-response decision support. It combines real-time mission state streaming, aerial computer vision workflow design, operator-facing uncertainty handling, 3D mission visualization, and grounded copilot/reporting logic.

The project is intentionally scoped as human-in-the-loop mission awareness. It surfaces visual evidence, coverage, confidence, uncertainty, and handoff context for operator review; it does not claim autonomous field decisioning.

Highlights

• Real-time FastAPI backend with REST APIs and full-state WebSocket mission snapshots.
• React + TypeScript tactical HUD with a React Three Fiber 3D mission map.
• 4x4 sector coverage model with drone path, scan cone, visual contact state, last-known positions, and uncertainty zones.
• Scripted V0.1 mission stream for stable demos and portfolio review.
• Deterministic Mission Copilot grounded only in MissionState, with safety language filtering and no-invention constraints.
• Read-only markdown handoff report generation for mission summaries.
• AI workspace for DOTA-based aerial asset experiments, YOLO-OBB training/export, ONNX serving direction, artifact metadata, and evaluation.
• Docker Compose setup for running backend and frontend together.

Current Status

V0.1 is complete as a stable mission-awareness demo foundation.

• Backend runs on port 8001.
• Frontend runs on port 5173.
• Demo stream reaches 11 of 16 sectors, or 68.75% coverage.
• Mission Copilot is deterministic and grounded in mission state.
• Handoff report generation is read-only.
• Early AI provider seams and remote-sensing artifacts are prepared for the V0.2 AI track.

V0.2 is the active AI engineering track.

• Ultralytics YOLO-OBB training and export for aerial asset detection.
• DOTA128 smoke-test, followed by DOTA-v1.5 subset fine-tuning.
• Safe app-facing class map: aircraft, vessel, small_vehicle, large_vehicle, bridge, storage_facility.
• Planned ONNX Runtime inference provider with OpenCV preprocessing and postprocessing.
• Model Evaluation Center, AI Evidence Panel, MLflow tracking, optional FiftyOne review, and grounded LLM copilot upgrade.

AI Smoke-Test Metrics

The current tracked metrics come from a DOTA128 YOLO-OBB smoke-test run. They are kept as smoke-test evidence, not final production performance claims.

Metric	Value
mAP50	95.01%
mAP50-95	79.23%
Precision	93.24%
Recall	88.92%
Model	yolo11n-obb.pt
Image size	640
Epochs	3

Model weights are intentionally excluded from Git. Lightweight artifact metadata is tracked under ai/artifacts/sentinelx_remote_sensing/.

Architecture

flowchart LR
  Demo["demo_events.json"] --> DemoProvider["DemoDetectionProvider"]
  DemoProvider --> State["MissionStateService"]
  State --> REST["FastAPI REST"]
  State --> WS["WebSocket snapshots"]
  REST --> HUD["React HUD"]
  WS --> HUD
  State --> Copilot["Grounded Mission Copilot"]
  State --> Report["Read-only handoff report"]

  Datasets["DOTA128 / DOTA-v1.5 subset"] --> Train["YOLO-OBB training"]
  Train --> Artifacts["AI artifacts + metrics"]
  Artifacts --> Provider["AIDetectionProvider"]
  Provider --> State

Loading

Tech Stack

Layer	Tools
Backend	Python, FastAPI, Pydantic, WebSocket
Frontend	React, TypeScript, Vite
3D Mission View	Three.js, React Three Fiber, drei
AI Workflow	Ultralytics YOLO-OBB, DOTA, ONNX Runtime direction, OpenCV direction
DevOps	Docker Compose

Repository Layout

backend/        FastAPI mission backend
frontend/       React/Vite tactical HUD
ai/             AI training, notebooks, configs, artifact metadata
docs/           Project plan and AI workflow documentation
CLAUDE.md       Agent handoff notes for future continuation

Run Locally

Backend:

cd backend
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8001

Frontend:

cd frontend
npm install
npm run dev

Open:

http://localhost:5173

Docker Compose:

docker compose up --build

Useful Endpoints

• GET /health
• GET /mission/state
• POST /mission/start
• POST /mission/reset
• POST /copilot/ask
• POST /copilot/briefing
• GET /report/markdown
• WS /ws/mission

Verification

Backend import check:

cd backend
python -c "from app.main import app; print(app.title)"

Frontend type/build check:

cd frontend
npm run build

AI config dry-run:

python ai\training\train.py --config ai\training\configs\dota_obb_smoke.yaml --dry-run

Safety Scope

SENTINEL-X uses mission-awareness wording by design: human candidate, visual contact, last-known position, uncertainty, operator review, rescan, handoff report, and decision support. Copilot/report outputs are grounded in current mission state and should not invent detections, sectors, confidence values, or recommendations.

See docs/PROJECT_PLAN.md and docs/AI_WORKFLOW.md for the full roadmap, safety policy, AI workflow, and continuation plan.