Skip to content

dev-native.md

Latest commit

 

History

History
175 lines (124 loc) · 6.8 KB

dev-native.md

File metadata and controls

175 lines (124 loc) · 6.8 KB
Raw

Native dev environment (Docker fallback)

This is the Plan B dev path. Docker remains the primary way to run SPECTRA-Lab — it gives you parity with production, isolation, and one-command bring-up of the whole stack. But Docker breaks sometimes (Docker Desktop wedging on disk pressure, daemon getting stuck, build cache I/O errors), and when it does, you don't want to lose a workday before you can write code again.

This doc describes how to run Postgres, Redis, and the FastAPI services natively on macOS via Homebrew. Both paths are kept in working order at all times; switching between them is just choosing a different make target.

When to use which

Situation Path
Normal development, want production parity Docker (make smoke-test-build / make dev-up)
Docker is broken right now and you need to ship Native (make dev-native-up)
Iterating fast on one service, want zero startup cost Native (run serve-<svc> against the host)
Need Celery worker, MinIO, Prometheus, or Grafana Docker (native mode doesn't run those)
CI / pre-deploy testing Docker (the canonical environment)

First-time setup (~5 minutes)

brew bundle install

That installs everything the Brewfile declares: PostgreSQL 15, Redis, Python 3.11, Node 20, uv, and (optionally) colima + Docker CLI.

Bring up the native datastores

make dev-native-up

This runs brew services start postgresql@15 redis, ensures the spectra role + database exist, and applies Alembic migrations. (The migrate step currently fails until Phase 3 / Session 3.1 fixes the migration drift; services still boot against an empty schema.)

To stop:

make dev-native-down       # stops Postgres + Redis; data preserved
make dev-native-status     # show what's running
make dev-native-reset      # DROP DATABASE spectra and recreate (destroys data)

Run a service natively

Each FastAPI service has its own venv because requirements pin different versions of starlette, httpx, etc., that can't coexist:

make venv-analysis         # creates .venv-analysis with uv
make serve-analysis        # uvicorn against localhost:5432, port 8001

Same pattern for the others:

make venv-lims              && make serve-lims
make venv-process_control   && make serve-process_control

Or build all three venvs in parallel:

make venv-all

For the frontend:

make serve-web              # cd apps/web && npm install && npm run dev (port 3012)

Port mapping (memorize this)

Service Native port Docker port
Postgres 5432 (Homebrew default) 5435 (docker-compose.yml)
Redis 6379 (Homebrew default) 6381
Analysis API 8001 8001
LIMS API 8002 8002
Process Control API 8003 8003
Web 3012 3012

Native and Docker can both be installed simultaneously without colliding because the datastore ports differ. The DATABASE_URL env var that the make targets set automatically reflects which path you're on:

  • make dev-up (Docker) and make migrate use localhost:5435
  • make dev-native-up and make serve-* use localhost:5432

If you ever need to run Alembic or a one-off script against the native DB:

export DATABASE_URL="postgresql+psycopg://spectra:spectra@localhost:5432/spectra"

What native mode does NOT give you

The Docker stack includes services that aren't in the native path:

  • Celery worker / beat / Flower — background job queue. Diffusion and oxidation runs depend on this in production. Use Docker if you need to test async run lifecycles.
  • MinIO — S3-compatible object storage for ELN signature blobs and SOP documents. Native mode falls back to filesystem stubs where the code allows it.
  • Prometheus + Grafana — observability scaffolding. Phase 5 wires the metrics; native mode skips them entirely.
  • (process_control was added to compose in Session 2.2 and is now part of the Docker stack.)

If your task touches any of those, switch back to Docker:

make dev-native-down
make dev-up

Both are idempotent; switching back and forth is safe.

Optional: switch your Docker daemon to colima

Docker Desktop is the macOS app most people install. It's heavy, the daemon occasionally wedges (the bug that caused this whole fallback to be written), and Docker Inc.'s licensing changed in 2022 such that orgs >250 employees need a paid plan.

colima is a free, lightweight alternative. Same docker compose up workflow; no project changes needed.

brew install colima      # already in Brewfile
colima start --cpu 4 --memory 8 --disk 40
docker compose up -d     # works exactly the same

To stop:

colima stop

If you switch, you can uninstall Docker Desktop entirely (brew uninstall --cask docker) — it's not needed once colima is providing the daemon.

Troubleshooting

pg_isready not found : Run brew services start postgresql@15 and ensure /opt/homebrew/opt/postgresql@15/bin is on your PATH. The dev-native-up target falls back to a fully-qualified path if the bare pg_isready isn't available.

role "spectra" does not exist : The dev-native-up target creates this on first run. If it failed silently, run it explicitly:

psql -h localhost -p 5432 -d postgres -c "CREATE ROLE spectra WITH LOGIN SUPERUSER PASSWORD 'spectra'"
psql -h localhost -p 5432 -d postgres -c "CREATE DATABASE spectra OWNER spectra"

uv not found : Run brew bundle install (it's declared in the Brewfile but may have been skipped if you set up earlier). Or fall back to pip: python3.11 -m venv .venv-analysis && .venv-analysis/bin/pip install -r services/analysis/requirements.txt.

Service starts but can't connect to DB : Confirm the env var is set to 5432 (native) not 5435 (Docker). The serve-* targets set this automatically; manual invocations need to remember.

Port already in use : Most likely Docker is also running. Either stop Docker (make dev-down) or run native services on different ports. The serve-* targets accept a PORT=... override:

PORT=18001 make serve-analysis

(note: the Makefile targets currently hard-code 8001/8002/8003 — to override, pass it via env to uvicorn directly).

Native DB is corrupt / I want a clean slate :

make dev-native-reset
make dev-native-up

Going back to Docker

Whenever you want:

make dev-native-down       # stop native datastores
make smoke-test-build      # bring up the Docker stack

No state migrates between modes — Docker has its own data volumes. If you need to copy data over, pg_dump from one and psql into the other, or just re-run make seed-db against the target.