Models

Models

Overseer bundles or fetches a set of models. Weights live under models/ (git ignored, so they never bloat the repo) or in the relevant model-hub cache. If any download is skipped or fails, the app still runs at reduced accuracy and says so on each result, so a missing model never crashes anything.

Re-run the fetch at any time:

uv run python -m match.tools.export_models
# or a single step:
uv run python -m match.tools.export_models --only clip

Model catalogue

Model	What it does	How it arrives
YOLO11 detector (yolo11s.pt)	People, vehicles, animals, weapons on every analysed frame	Automatic (first analysis)
YOLO11-seg (yolo11n-seg.pt)	Foreground / instance masks for cleaner geometry and matching	Automatic
YOLO11n (yolo11n.pt)	Lightweight detector for the background roster harvester	Automatic
Depth Anything V2	Dense monocular depth for the spatial 3D view	Automatic (Hugging Face hub, first spatial build)
DINOv2 ViT-S/14 (dinov2_vits14.torchscript)	Generic appearance embedding; person/vehicle ReID fallback	Automatic (torch.hub)
CLIP ViT-B/32	Zero-shot vehicle body type (sedan / hatchback / SUV / ...)	Automatic (Hugging Face hub, first vehicle)
Vehicle brand ViT (vehicle_make.torchscript)	Confidence-gated vehicle make (55 brands)	Automatic (release weights)
Whisper large-v3-turbo	Offline speech-to-text (Turkish + English) for the AI Operator	Automatic (first voice command)
Real-ESRGAN general x4 (realesr-general-x4v3.pth)	Learned super-resolution for reconstruction and live enhance	Automatic (first use)
YuNet (face_detection_yunet.onnx)	Face detection for face crops in dossiers	Automatic
EasyOCR	Licence-plate reading (ANPR)	Automatic with uv sync --extra ai-extras
OSNet-AIN (osnet_ain_x1_0.torchscript)	Dedicated person re-identification	Manual checkpoint, see below
VeRi R50-ibn (veri_sbs_R50-ibn.torchscript)	Dedicated vehicle re-identification	Manual checkpoint, see below

Most models are fully automatic. The two dedicated re-identification models give the best identity accuracy but are hosted behind model zoos, so you download the checkpoint once by hand and the same fetch command converts it. The app already works well without them because the general DINOv2 embedder stands in.

Where things run (GPU vs CPU)

• On the GPU when present: detection, depth, super-resolution and the ReID encoders run half-precision on CUDA, with graceful CPU fallback.
• Deliberately on the CPU: the vehicle brand ViT and the body-type CLIP run in throttled background readers on the CPU. This keeps them off the GPU hot path and, importantly, avoids starving depth / detection / speech-to-text of VRAM. On a machine with spare VRAM you can move body type to the GPU with OVERSEER_CLIP_DEVICE=cuda.
• Speech-to-text prefers the GPU (float16) and falls back to CPU (int8) automatically if the GPU load fails.

Environment overrides

Variable	Default	Purpose
OVERSEER_STT_MODEL	large-v3-turbo	Whisper size: tiny / base / small / medium / large-v3 / large-v3-turbo. Turbo is the default: near large-v3 accuracy for Turkish + English at a fraction of the VRAM/latency. Drop to small only on a very tight GPU.
OVERSEER_STT_DIR	models/whisper	Where the Whisper model is cached.
OVERSEER_CLIP_MODEL	openai/clip-vit-base-patch32	CLIP model id for zero-shot body type.
OVERSEER_CLIP_DEVICE	cpu	Set to cuda to run body-type CLIP on the GPU (needs spare VRAM).

Manual re-identification models (optional)

Person re-identification (OSNet-AIN). Download a cross-domain OSNet-AIN checkpoint from the torchreid model zoo, drop it into models/, then run the fetch command to convert it to models/osnet_ain_x1_0.torchscript.

Vehicle re-identification (VeRi). Download the fast-reid VeRi model, drop it into models/, and convert it. This one needs the fast-reid framework, so clone it and point FASTREID_ROOT at it before running the fetch.

See the repository README.md for the exact checkpoint links.