Continuous Tracking System

Multi-camera person tracking and dementia signal detection for senior care. Processes RTSP camera feeds through a 15-stage ML pipeline: YOLO detection, spatial floor projection, SOLIDER-REID and RTMPose inference, cross-camera pre-association dedup, floor-plane Kalman world tracker (Hungarian association over PersonHypothesis), Bayesian identity resolution, posture classification, trajectory writing, and event publishing.

Detects clinically relevant behavioral patterns from trajectory and dwell data: pacing, sundowning, bathroom dwell anomalies, prolonged stillness, nighttime movement, and unexplained absence.

Results are streamed via Redis Streams (protobuf) to Cognitive Companion, the BFF gateway that serves the Vue admin UI and MCP tools.

Documentation: silvermind-project.github.io

Architecture reference: docs/systems-architecture.md covers the PersonHypothesis world-tracker model, the identity resolver, home-camera nuances, and how CTS feeds Cognitive Companion.

Services

Service	Port	Description
go2rtc	1984	RTSP proxy sidecar
rtsp-ingress	8090	Go RTSP ingest: camera registration, motion gating, MinIO upload
tracking-orchestrator	8500	Python ML pipeline: detection, tracking, identity, signals
triton	8701	Model serving for YOLO, SOLIDER, RTMPose, and the five Buffalo_L face graphs

Key design points

• PersonHypothesis (PH) is the world-level tracked-person entity. ph_id (UUID) is the single cross-camera identifier on the wire.
• Cross-camera dedup: before the Hungarian assignment runs, a pre-association floor-point pass collapses same-person observations from overlapping cameras into one representative, preventing duplicate PHs for a person seen at a hallway/bathroom boundary.
• PH evidence is first-class: caregiver correction UIs consume real tagged_keyframes and repository-validated metadata. Do not fabricate image keys or coerce invalid JSONB into empty objects.
• Quality capture: each PH carries a mean_quality field (EMA of observation quality scores) that travels to Cognitive Companion for display in location envelopes.
• No silent fallbacks: stream consumers dead-letter unprocessable messages with a metric and warning log; they never silently skip.

Inference deployment profiles

The default DGX profile serves canonical full-precision Buffalo_L models from triton-models. The Jetson Orin Nano Super profile serves explicit-Q/DQ models and target-built TensorRT plans from triton-models-jetson.

The Jetson repository contains eight required graphs:

• YOLO26L person detection
• RTMPose-m pose estimation
• SOLIDER body re-identification
• SCRFD face detection
• ArcFace recognition
• 2D106 and 3D68 facial landmarks
• gender and age attributes

Depth Anything V2 is not loaded. The person-identification-service uses Triton as its only inference backend and switches profiles by changing TRITON_GRPC_URL and PERSON_ID_MODEL_PROFILE. Startup fails unless all five Buffalo_L models are ready.

Public deployment and qualification guides:

• Jetson INT8 deployment runbook
• Jetson CTS deployment
• Model quantization and accelerator portability

Quick start

docker compose -f ../docker-compose.db.yml -p nanai up -d  # shared Postgres
cp rtsp-ingress/config/settings.yaml rtsp-ingress/config/settings.local.yaml
docker compose up -d

See the documentation site for camera setup, model export, and operations guides.

Development

cd tracking-orchestrator && uv sync --frozen --extra dev
make check          # ruff + mypy + import-linter + pytest (fast gate)
make check-all      # Python + Go + buf lint
make ci             # authoritative gate: check-all + integration proofs (requires Docker)

License

AGPL-3.0-or-later