OpenMV.net Datasets

The Django site behind https://openmv.net — a public catalogue of small, real-world datasets used as worked examples in the textbook Process Improvement using Data by Kevin Dunn.

The site lets visitors:

• Browse every public dataset in one sortable table.
• Filter datasets by tag.
• Read a per-dataset detail page with description, source, shape, usage restrictions, and contact info.
• Download a dataset in CSV / XLSX / XML / MAT format. Each download increments a privacy-respecting hit counter (timestamp + dataset only — no IP, user-agent, or referrer is stored).

Layout

.
├── manage.py
├── Makefile                  # dev tasks (install, migrate, test, lint, debug, docker-up, ...)
├── pyproject.toml            # uv-managed dependencies + pytest config
├── uv.lock                   # committed lockfile
├── Dockerfile                # multi-stage image for local dev (and future prod)
├── docker-compose.yml        # local dev (SQLite, runserver)
├── docker-compose.prod.yml   # production (Postgres, gunicorn)
├── .github/workflows/ci.yml  # pre-commit + pytest on push and PR
├── openmv/                   # Django project (settings, root URLs, WSGI/ASGI)
│   ├── settings/             # base.py + dev.py + prod.py
│   ├── urls.py
│   ├── wsgi.py
│   └── asgi.py
├── datasetapp/               # the only Django app
│   ├── models.py             # Tag, Dataset, DataFile, Hit
│   ├── views.py              # display_all, display_by_tag, about_dataset, download_dataset
│   ├── urls.py
│   ├── admin.py
│   ├── apps.py
│   ├── migrations/
│   ├── templates/datasetapp/ # base.html, all_datasets.html, dataset_info.html
│   └── templatetags/         # `slice_string` filter
├── .pre-commit-config.yaml
├── .flake8
├── .gitignore
├── .env.example              # copy to .env and fill in
├── README.md
├── CLAUDE.md                 # repo orientation + roadmap
└── LICENSE

Requirements

• Python 3.11+
• uv for dependency management
• PostgreSQL (production only, via openmv.settings.prod). Local dev uses SQLite via openmv.settings.dev.

Dependencies are declared in pyproject.toml and pinned in uv.lock.

Local development

Native (uv)

# 1. Clone and enter the repo
git clone <repo-url> && cd Django-dataset-download-app

# 2. Install dependencies into a managed venv
uv sync --dev

# 3. Configure environment
cp .env.example .env
# Edit .env: set SECRET_KEY to any random string. The Postgres keys are
# only consulted by openmv.settings.prod, so they can stay as the
# placeholders for local dev. (Settings read process env first, so you
# can also export SECRET_KEY directly and skip the .env file entirely.)

# 4. Run the dev server (collectstatic + migrate + createcachetable + runserver:8080)
make debug

Docker compose

cp .env.example .env   # set SECRET_KEY
make docker-up         # builds + runs runserver against SQLite

Both paths use openmv.settings.dev (SQLite) and serve http://127.0.0.1:8080/. To rehearse the production stack locally (Postgres + gunicorn + openmv.settings.prod), use docker compose -f docker-compose.prod.yml up --build instead.

Create a superuser with uv run python manage.py createsuperuser (native) or docker compose exec web python manage.py createsuperuser (Docker) to log into /admin/ and add Tags / Datasets / DataFiles.

Testing & CI

• make test — runs the smoke-test suite (uv run pytest).
• make lint — runs pre-commit run --all-files.
• .github/workflows/ci.yml runs both on every PR and on pushes to master.

Production notes

• Production sets DJANGO_SETTINGS_MODULE=openmv.settings.prod (wired in docker-compose.prod.yml), which selects PostgreSQL via POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD, SQL_HOST, SQL_PORT from the environment (or .env) and turns off DEBUG. Settings read process environment first, then fall back to .env, so containers/CI can inject config directly.
• ALLOWED_HOSTS reads from the ALLOWED_HOSTS env var (comma-separated). Defaults: .openmv.net,127.0.0.1 in prod, 127.0.0.1,localhost in dev. Override the env var to deploy under a different hostname.
• Static files land in BASE_DIR / 'static' after python manage.py collectstatic. Admin-uploaded dataset files land in BASE_DIR / 'media' (MEDIA_ROOT), reachable at /media/ (MEDIA_URL). On Hetzner the host-installed Caddy serves /static/ and /media/ directly from bind-mounted host directories; the download_dataset view returns a 302 to the /media/... URL rather than streaming the file through Django. Locally, runserver serves /media/ only when DEBUG=True (see openmv/urls.py); in production Caddy intercepts /static/ and /media/ before the request reaches gunicorn.
• The Hit table grows with every download. There is no automatic pruning.

Tooling

• make install — uv sync --dev.
• make migrate — uv run python manage.py migrate.
• make collectstatic — uv run python manage.py collectstatic --no-input.
• make test — uv run pytest.
• make lint — uv run pre-commit run --all-files.
• make debug — collectstatic + migrate + createcachetable + runserver on :8080.
• make docker-up / make docker-down — wrappers over docker compose.
• make clean — remove __pycache__, caches, etc.

License

BSD 3-Clause — see LICENSE. © 2010–present Kevin Dunn.

Contributing / roadmap

See CLAUDE.md for an architectural overview and the prioritised list of future improvements.