Urbanis — Crime Mapping for São Paulo

Interactive crime map of the city of São Paulo, Brazil. Combines official monthly statistics per police station (SSP-SP) as region choropleths with community-reported point incidents (Onde Fui Roubado), behind an explicit source filter — so official and community data are never conflated.

Features

• Street / satellite basemaps (CARTO Positron + Esri World Imagery raster tiles, no API keys)
• Choropleth over civil police jurisdiction areas (DP) or the 96 city districts
• Point incidents with clustering, category-group colors (against persons / against property / others) and popups
• Filters: crime category (grouped in the three categories), period (3m/6m/12m/24m), region type and data source (official vs community)
• Ranking of most unsafe regions by occurrence count or rate per 100k inhabitants
• Region details panel with per-category breakdown and monthly time series
• Plugin architecture: each data source is a plugin implementing one shared contract

Architecture

┌────────────────────┐      ┌────────────────────────────┐
│  web (Next.js 16)  │ HTTP │  api (ASP.NET Core, .NET 10)│
│  Leaflet + cluster │─────▶│  ├─ REST endpoints         │
│  App Router + TS   │      │  ├─ Ingest runner + worker │
└────────────────────┘      │  └─ Plugin registry (DI)   │
                            │     ├─ GeoSampa (geo)      │
                            │     ├─ IBGE (population)   │
                            │     ├─ SSP-SP (official)   │
                            │     └─ OndeFuiRoubado      │
                            │  EF Core + SQLite          │
                            └─────────────┬──────────────┘
                                          │ volume ./data
                                data/raw/<plugin>/ + urbanis.db

Repository layout

backend/           .NET 10 solution (Urbanis.slnx)
  src/Urbanis.Core               canonical schema + crime taxonomy
  src/Urbanis.Plugin.Abstractions plugin contract, GeoJSON/text utils, raw cache
  src/Urbanis.Infrastructure     EF Core DbContext (SQLite), options
  src/Urbanis.Api                minimal-API endpoints, ingest runner/CLI
  plugins/Urbanis.Plugin.*       one project per data source
  tests/                         xUnit tests
frontend/          Next.js 16 + TypeScript + Tailwind + Leaflet
data/              gitignored: SQLite database + raw plugin downloads
docker-compose.yml api (8080) + web (3000), ./data volume

Quick start (Docker)

docker compose up --build

The API boots with an empty database and runs the full ingest in the background (a few minutes for ~570 SSP requests). Open http://localhost:3000 once the map data appears. The SQLite database and raw downloads live in the urbanis-data named volume (a Windows bind mount is not used because SQLite journals misbehave there).

To place a manually downloaded file (e.g. the Kaggle CSV) inside the volume:

docker compose cp ./sp-crimes.csv api:/app/data/raw/ondefuiroubado/
docker compose run api --ingest --plugins ondefuiroubado

To run the ingest manually:

docker compose run api --ingest                    # all plugins
docker compose run api --ingest --plugins ssp-sp   # one plugin

Local development

Backend (from the repository root, so data/ lands in the repo):

dotnet run --project backend/src/Urbanis.Api
# or run the ingest as a one-shot:
URBANIS__DATAROOT=$(pwd)/data dotnet run --project backend/src/Urbanis.Api -- --ingest
dotnet test backend/tests/Urbanis.Core.Tests

Frontend:

cd frontend && npm install && npm run dev   # http://localhost:3000

The frontend proxies /backend/* to the API (API_PROXY_TARGET, default http://localhost:8080).

Data sources & plugins

Plugin	Type	What it provides	Source
geosampa	geo reference	96 district polygons, DP jurisdiction polygons, police station points	GeoSampa WFS (CC BY-SA 4.0)
ibge	geo reference	Censo 2022 resident population per district; DP-area population estimated by proportional split	IBGE agregados API
ssp-sp	official aggregates	Monthly occurrences per DP (last 6 years) — homicide, robbery, theft, rape, etc.	SSP-SP statistics portal
ondefuiroubado	community points	Geolocated self-reported robberies (2013–2018, ~12.9k)	Kaggle export of ondefuiroubado.com.br — manual download

Data honesty

• Official Brazilian crime data is aggregated by police station — no government source publishes incident coordinates. The choropleth is the official view; the point layer is community data and is labeled as such everywhere in the UI.
• The ondefuiroubado plugin requires a manual Kaggle download (authentication required): download the CSV from danlessa/geospatial-sao-paulo-crime-database into data/raw/ondefuiroubado/ and run the ingest again. Community data is old (2013–2018) and self-selection-biased; the UI shows a permanent warning.
• DP-area population (used for rate/100k) is an approximation that splits district population proportionally across DP polygons — commercial districts with low residential population (e.g. Pari) will show inflated rates.

Writing a new plugin

Implement IDataSourcePlugin (see Urbanis.Plugin.Abstractions), register it in Program.cs, and append rows to the canonical entities (Region, Incident, AggregateRecord). Ingest must be idempotent; raw downloads go through RawFileCache into data/raw/<plugin-id>/ and every run is recorded in the PluginRuns history.

API overview

Endpoint	Purpose
GET /api/health	health check
GET /api/plugins	plugin manifests + last run
GET /api/regions?type=dp|distrito|delegacia	region geometries (GeoJSON)
GET /api/aggregates?regionType=dp&from=2025-08&to=2026-08&categories=	per-region totals for the choropleth
GET /api/rankings?regionType=dp&metric=count|rate	top regions
GET /api/incidents?source=community&categories=&from=&bbox=	point incidents (GeoJSON)
GET /api/regions/{id}/details	per-region breakdown + monthly series
GET /api/meta/categories	canonical taxonomy for filter UIs
POST /api/ingest/run	trigger ingest (guard with Urbanis:IngestToken)

Configuration

Setting	Default	Notes
Urbanis:DataRoot	data	database + raw downloads root
Urbanis:IngestOnStartup	true	first boot with empty DB triggers full ingest
Urbanis:IngestToken	empty	bearer token for POST /api/ingest/run
Urbanis:AllowedOrigins	http://localhost:3000	CORS allowlist
SspSp:YearsBack	5	years of DP history to ingest
API_PROXY_TARGET	http://localhost:8080	frontend → API proxy target

Attribution

Secretaria de Segurança Pública do Estado de São Paulo · Prefeitura de São Paulo (GeoSampa/SMDU, CC BY-SA 4.0) · IBGE Censo Demográfico 2022 · Onde Fui Roubado contributors · © OpenStreetMap contributors · © CARTO · Esri World Imagery. This project is not affiliated with any of these organizations.