From 8bd7aa43cf6f56046a18f77494c6c9800f78f14c Mon Sep 17 00:00:00 2001 From: Dean Kayton Date: Tue, 14 Jul 2026 20:54:24 +0200 Subject: [PATCH] Restructure docs around a runnable local dev environment - Rewrite README: what DTrack is, verified local quickstart (debug mode, no Azure), doc index, repo layout, project status (mvp-baseline) - Add docs/architecture.md: schemas, api view pattern, role/RLS model, request lifecycle, onboarding, cyanaudit, initdb conventions, infra, version pins cross-referenced against July 2026 releases - Add docs/development.md: prerequisites (incl. colima), one-time setup, dev vs debug modes, bootstrap order and its container-recreate quirk, everyday commands, direct API access with debug JWTs, troubleshooting - Add docs/deployment.md: provisioning/deploy content moved out of README - Move dtrack/dev/deploy.md to docs/runbooks/backup-restore.md; fold dtrack/dev/debug.md into docs/development.md - Add make dev/debug/initdb/down/clean convenience targets Co-Authored-By: Claude Fable 5 --- Makefile | 27 ++- README.md | 69 ++++-- docs/architecture.md | 222 ++++++++++++++++++ docs/deployment.md | 61 +++++ docs/development.md | 126 ++++++++++ .../runbooks/backup-restore.md | 7 + dtrack/dev/debug.md | 19 -- 7 files changed, 491 insertions(+), 40 deletions(-) create mode 100644 docs/architecture.md create mode 100644 docs/deployment.md create mode 100644 docs/development.md rename dtrack/dev/deploy.md => docs/runbooks/backup-restore.md (93%) delete mode 100644 dtrack/dev/debug.md diff --git a/Makefile b/Makefile index 53c00ca..3bc3b3b 100644 --- a/Makefile +++ b/Makefile @@ -2,16 +2,36 @@ TERRAFORM_CMD = terraform ANSIBLE_CMD = ansible-playbook ANSIBLE_DIR = config/ansible +COMPOSE_DEV = docker compose -f docker-compose.yaml \ + -f config/docker-compose.overrides/dev.yaml +COMPOSE_DEBUG = $(COMPOSE_DEV) \ + -f config/docker-compose.overrides/debug.yaml + env ?= null tf_env = $(if $(filter $(env),staging prod),$(env),$(if $(filter $(env),null),staging,$(error Invalid environment: $(env)))) play_env = $(if $(filter $(env),staging prod),$(env),null) args ?= -.PHONY: precommit infra infra-destroy play help +.PHONY: precommit infra infra-destroy play help dev debug initdb down clean precommit: pre-commit run --all-files +dev: + $(COMPOSE_DEV) up -d --build + +debug: + $(COMPOSE_DEBUG) up -d --build + +initdb: + ./initdb.sh + +down: + $(COMPOSE_DEBUG) down + +clean: + $(COMPOSE_DEBUG) down -v --remove-orphans + infra-init: $(TERRAFORM_CMD) init -backend-config=backend.tfconf @@ -30,6 +50,11 @@ play: help: @echo "Available targets:" + @echo " dev Start the local stack in dev mode (real Azure AD login)." + @echo " debug Start the local stack in debug mode (local JWT login, no Azure)." + @echo " initdb Bootstrap the database (runs ./initdb.sh)." + @echo " down Stop the local stack (data is kept)." + @echo " clean Stop the local stack and delete its data volume." @echo " precommit Manually run pre-commit hooks on all files." @echo " infra-init Initialize Terraform with backend configuration." @echo " infra Apply infrastructure configuration using Terraform." diff --git a/README.md b/README.md index 09d55f3..8fb9f14 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,51 @@ # DTrack -## Architecture - -## Using this repo -- Dependencies: - - Setup AWS account, and IAM user, with policies (attach EC2 Full Access) -- TODO: Manage Terraform policies better - - User AWS CLI to configure account - - Install Terraform - - Install Ansible - - Github CLI (or otherwise get Github API token) - - Setup Cloudflare and add a domain as zone (get Cloudflare API token) -- Clone repo (potentially some permissions are required if not public, and potentially you would want to fork it first, and clone your fork) -- If setting up for the first time, generate key pairs (else get existing keys securely): - - `mkdir -p private/keys/instance private/keys/deploy` - - `ssh-keygen -t ed25519 -C "deankayton@gmail.com" -f private/keys/instance/id` - - `ssh-keygen -t ed25519 -C "deankayton@gmail.com" -f private/keys/deploy/id` -- `cp terraform.tfvars.tpl terraform.tfvars` and modify what is applicable (mainly Cloudflare and Github token) -- Provision Infrastructure (`terraform init` and `terraform apply` from root of repo) -- Change workspace to 'prod' `terraform workspace new prod` and `terraform workspace select prod` and repeat last step -- Staging is the default, so to go back, `terraform workspace select default` -- It is possible to use Ansible to provision staging and prod simultaneously +Time tracking for teams, built SQL-first: the database **is** the application. + +- **PostgreSQL 15** holds the schema, business logic (functions + triggers) and authorization (roles + row-level security). There is no application server. +- **[PostgREST](https://postgrest.org)** exposes the `api` schema as a REST API. +- **[react-admin](https://marmelab.com/react-admin/)** provides the single-page UI. +- **Azure AD (MSAL)** authenticates users in production; a self-contained **debug login** makes local development possible without Azure. +- **Terraform + Ansible** provision and deploy staging/production to AWS behind Cloudflare. + +## Quickstart (local, no Azure required) + +Prerequisites: Docker with the compose plugin (on macOS e.g. `colima start`), and this repo cloned. + +```sh +cp .env.tpl .env # dummy credentials are fine locally +mkdir -p keys && printf 'Dummy5ecr3t4D3bug0n1yN0T4Pr0D123' > keys/jwt-secret +./initdb.sh # build + start postgres, bootstrap the database +make debug # start the full stack in debug mode +``` + +Then log in at as `Dummy.User@example.com` (any password field shown is ignored — debug mode signs a local JWT). The API is at , Swagger UI at . + +See [docs/development.md](docs/development.md) for the full guide, including dev-vs-debug modes and troubleshooting. + +## Documentation + +| Doc | What it covers | +| --- | --- | +| [docs/architecture.md](docs/architecture.md) | System design: schemas, roles, RLS, request lifecycle, audit | +| [docs/development.md](docs/development.md) | Running and working on DTrack locally | +| [docs/deployment.md](docs/deployment.md) | Provisioning infrastructure and deploying | +| [docs/runbooks/backup-restore.md](docs/runbooks/backup-restore.md) | Backup, test-restore and live-restore procedures | + +## Repository layout + +``` +dtrack/db/sql/ Database bootstrap SQL, executed in order by initdb.sh +dtrack/ui/ react-admin single-page app (Vite + TypeScript) +dtrack/dev/queries/ Helper SQL for operations (also used by restore automation) +config/ Docker images, compose overrides, Ansible playbooks +.github/ CI/CD workflows and their Ansible playbooks +main.tf Terraform: AWS EC2 + Cloudflare DNS + GitHub deploy keys +``` + +## Project status + +The current codebase is a deliberate stability baseline (tag: `mvp-baseline`). The +focus is consolidation — documentation, developer experience, CI and test +coverage — not new features. Application code (SQL and UI) is intentionally +frozen while the safety net is built around it. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..f78b306 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,222 @@ +# Architecture + +DTrack is a SQL-first application. All business logic, data validation and +authorization live in PostgreSQL; PostgREST turns the database's `api` schema +into a REST API; a react-admin SPA consumes it. There is no hand-written +application server. + +``` + Azure AD (production) ┌──────────────┐ + │ OIDC / MSAL │ swagger-ui │ :8080 + ▼ └──────┬───────┘ +┌─────────────┐ Bearer JWT ┌───────────┐ SQL ┌──────▼───────┐ +│ react-admin ├──────────────►│ PostgREST ├───────►│ PostgreSQL │ +│ (nginx) │ :5174 → :3000 │ │ │ + cyanaudit │ +└─────────────┘ └───────────┘ └──────────────┘ + SPA serves schema `api` schema, logic, RLS +``` + +## Components + +| Service | Image | Pinned | Latest (July 2026) | Notes | +| --- | --- | --- | --- | --- | +| `postgres` | custom, from `postgres:15` | PG 15 | PG 18 (15 supported until Nov 2027) | Adds cyanaudit (compiled from source), plpython3u + `guardpost` (JWT validation), perl deps for cyanaudit tools. [Dockerfile](../config/dockerfiles/postgres/Dockerfile) | +| `postgrest` | `postgrest/postgrest` | v10.1.2 | v14 (v11–v14 released since) | Serves schema `api`, anon role `anon`, pre-request `pre.request` | +| `react-admin` | custom, node 18 build → nginx | react-admin 4.9, Vite 4, TS 4.6 | react-admin 5.15, Vite 7, TS 5.x | Built with `VITE_*` build args; served as static SPA. [Dockerfile](../config/dockerfiles/react-admin/Dockerfile) | +| `swagger` | `swaggerapi/swagger-ui` | v4.18.1 | v5.x | Reads PostgREST's OpenAPI output | + +Version pins are part of the `mvp-baseline` line in the sand; upgrades are +deliberate future work, not drive-by changes. + +## Database layout + +The database (default name `dtrack`) is organized by schema, with privileges +increasing from left to right in the pipeline: + +| Schema | Purpose | +| --- | --- | +| `internal` | Base tables — the source of truth. Never exposed directly. | +| `auth` | `users` table plus SECURITY DEFINER functions for user/role management | +| `api` | Views + `INSTEAD OF` triggers — the only schema PostgREST serves | +| `pre` | `pre.request()`, PostgREST's pre-request hook (sets `cyanaudit.uid`) | +| `admin` | Cyanaudit inspection helpers (transactions, slowest, details) | +| `cyanaudit` | Column-level audit log of all data changes | + +### Tables (`internal` + `auth`) + +``` +auth.users ─────────────┬────────────────────────────────┐ + │ │ +internal.projects ──┬─ internal.user_project_roles │ (is_lead flags PMs) + ├─ internal.project_activities ── internal.activities + ├─ internal.project_areas_of_work ── internal.areas_of_work + │ │ +internal.time_trackings ── (user, project, activity, aow)│ + │ +internal.areas_of_expertise ── internal.aoe_hierarchies ─┘ (aoe, lead, member) +internal.faqs +``` + +Key constraints: +- `time_trackings.duration` must be a multiple of 15 minutes (CHECK constraint). +- A member may belong to only one team (lead) per area of expertise (trigger + `check_unique_member_per_aoe`). +- All FKs are `ON DELETE RESTRICT ON UPDATE CASCADE`; deletes cascade manually + inside the `INSTEAD OF DELETE` trigger functions. + +### The api view pattern + +Every resource the UI sees is an `api` view with `security_invoker = on` +(so RLS applies to the calling user), plus `INSTEAD OF` triggers translating +INSERT/UPDATE/DELETE on the view into writes against `internal` tables: + +| api view | write triggers | backing data | +| --- | --- | --- | +| `employees`, `current_employee`, `all_employees` | upsert + delete | `auth.users`, project/team memberships | +| `projects` | upsert + delete (also grants/revokes `pm`) | `internal.projects` + join tables | +| `activities`, `areas_of_work` | upsert + delete | respective tables + project links | +| `areas_of_expertise` | upsert + delete | `internal.areas_of_expertise` | +| `aoe_teams` | upsert + delete (also grants/revokes `lead`) | `internal.aoe_hierarchies` | +| `time_trackings` | insert + update + delete | `internal.time_trackings` | +| `faqs` | none (plain view, RLS on table) | `internal.faqs` | +| `timetracking_summary(...)`, `duration_calendar` | read-only | dynamic aggregation for dashboard plots | + +Views aggregate related entities as JSONB (`{id, name}` pairs and `*_ids` +arrays) so react-admin can render and edit relationships without extra +round-trips. `aoe_teams` has a synthetic text ID: `lpad(aoe_id)||lpad(lead_id)` +(4+4 digits), since a team is identified by (area of expertise, lead). + +## Identity, roles and authorization + +**Each user is a PostgreSQL role**, named by their email address. Group roles +define capability tiers, granted cumulatively: + +``` +anon < basic < pm ─┐ + basic < lead ─┴─< power ( power BYPASSRLS; admin is member of power ) +``` + +- `authenticator` (`$POSTGRES_USER_APP`) is the only login role PostgREST + uses; it can `SET ROLE` to any user role. +- PostgREST maps JWT claim `preferred_username` (an email) to the role — + `PGRST_JWT_ROLE_CLAIM_KEY=.preferred_username`. +- `basic` is every user's default. `pm` and `lead` are granted/revoked + *automatically* by triggers: becoming a project lead grants `pm`; becoming + an AoE team supervisor grants `lead`. `power` is granted manually via + `auth.grant_power` (or the employees UI's `is_power` flag). +- Role management uses SECURITY DEFINER functions in `auth` + (`create_user`, `del_user`, `grant_priv`, `revoke_priv`, `grant_power`, + `revoke_power`) — executable only by `power` (except `create_user`-via- + `api.onboard`, see below). + +**Row-level security** (see [014.schema.policies.sql](../dtrack/db/sql/002.postgres_user_app_admin.postgres_db_app/014.schema.policies.sql)) +enforces, roughly: + +- `time_trackings`: you see/write your own rows, plus rows of members you lead. +- `projects`: visible to members, their leads, and lead-of-member chains; + writable by project leads (PMs); deletable only by `power`. +- `users`: you see yourself plus people you lead/PM (and inverses); you update + only yourself. +- `activities` / `areas_of_work`: readable by all; only `power` creates/deletes; + per-project links managed by that project's leads. +- `areas_of_expertise` / `aoe_hierarchies`: visible to their leads and members; + leads manage their own team rows. +- `faqs`: everyone reads published rows; only `power` writes/deletes. +- `power` has `BYPASSRLS` — sees and can do everything. + +## Request lifecycle + +1. SPA sends a request with `Authorization: Bearer `. +2. PostgREST validates the JWT (`PGRST_JWT_SECRET`; in production a JWKS file + fetched from Azure by `rotate-keys.sh`, in debug mode a shared HS256 secret). +3. PostgREST executes `SET LOCAL ROLE ` and calls + `pre.request()`, which stores the user's id in `cyanaudit.uid` so audit rows + are attributed. +4. The query runs against `api.*` as that role; RLS and grants apply; INSTEAD + OF triggers translate writes; cyanaudit records column-level changes. + +### Onboarding + +A first-time user has a valid Azure JWT but no matching database role, so +PostgREST returns a "role does not exist" error. The SPA then calls +`POST /rpc/onboard` (executable by `anon`) with the **id token**; +`api.onboard` validates the token *inside the database* — plpython3u + +`guardpost` against the Azure tenant's JWKS — and calls `auth.create_user`, +which creates the role, grants `basic`, and inserts into `auth.users`. +The SPA retries and proceeds. Note: onboarding therefore requires real Azure +tokens and does not work in debug mode; debug users must already exist +(the initial power user, or users created via the Employees screen). + +## Audit — cyanaudit + +[Cyanaudit](https://bitbucket.org/neadwerx/cyanaudit) records every column +change (who/what/when/old/new) in the `cyanaudit` schema. It is compiled into +the postgres image, installed by `001.../000.install.cyanaudit.sh`, and +activated for all schemas by `098.activate.cyanaudit.sql`. `admin.*` functions +expose transaction history. Backups dump cyanaudit separately +(`cyanaudit_dump.pl` / `cyanaudit_restore.pl`). + +## Database bootstrap (initdb.sh) + +There are no migrations. [initdb.sh](../initdb.sh) executes every `.sql`/`.sh` +file under `dtrack/db/sql/` in lexicographic order, inside the running +`postgres` container. Directory names encode the executing identity: + +``` +dtrack/db/sql/../ + 000.postgres_user.postgres_db/ # as superuser on the default DB: create role + DB + 001.postgres_user.postgres_db_app/ # as superuser on dtrack: extensions, roles, cyanaudit + 002.postgres_user_app_admin.postgres_db_app/ # as app admin on dtrack: everything else +``` + +Files are piped through `envsubst`, so `$POSTGRES_USER_APP`-style variables in +SQL resolve from `.env`. Schema changes are deployed by dump → recreate → +restore (see [runbooks/backup-restore.md](runbooks/backup-restore.md)). + +## Front end + +`dtrack/ui` is a Vite + TypeScript react-admin app. One `` per api +view ([App.tsx](../dtrack/ui/src/App.tsx)); CRUD screens live in +`src/sections/`; the dashboard renders D3 sunburst and calendar charts from +`api.timetracking_summary` / `api.duration_calendar`. + +Two wiring modes, switched at build time by `VITE_ENVIRONMENT`: + +- **Production/dev**: MSAL (`ra-auth-msal`) against Azure AD; data provider + `@raphiniert/ra-data-postgrest` attaches the MSAL id token. +- **Debug**: a local login form signs an HS256 JWT in the browser + (`jsrsasign`) with `VITE_JWT_SECRET` — the same secret PostgREST is given in + [debug.yaml](../config/docker-compose.overrides/debug.yaml). Any existing + user's email can be impersonated. Never deploy this mode. + +The UI's permission checks (e.g. hiding the Employee column for non-leads) +are cosmetic; real enforcement is RLS in the database. + +## Infrastructure + +- **Terraform** ([main.tf](../main.tf)): AWS VPC + EC2 per workspace + (default = staging, `prod`), Cloudflare DNS, GitHub deploy keys. State in S3 + (`backend.tfconf`). +- **Ansible** ([config/ansible/base.playbook.yml](../config/ansible/base.playbook.yml)): + system updates, Docker, clone repo, render `.env` and + `docker-compose.override.yaml` (Traefik + TLS via the `staging`/`prod` + overrides), start stack. +- **GitHub Actions** (deployed-instance operations, not CI): + [backup.yml](../.github/workflows/backup.yml) — bi-hourly backup with tiered + retention + automatic test-restore; [rotate-keys.yml](../.github/workflows/rotate-keys.yml) + — refreshes the Azure JWKS used as PostgREST's JWT secret. + +## Trust boundaries worth knowing about + +These flow from the design and are useful context when writing tests +(they are *not* an invitation to change application code right now): + +- `auth.del_user` only accepts emails matching the hardcoded + `@hellodnk8n.onmicrosoft.com` domain; `internal.employees_upsert` likewise + defaults new emails to that domain. +- `internal.aoe_teams_grant_or_revoke_privs` is SECURITY DEFINER and executable + by `basic` (flagged `TODO: potentially insecure` in the source). +- The FAQ RLS policy hides unpublished rows from everyone, including their + authors (`power` bypasses RLS, so in practice only `power` manages FAQs). +- Debug mode trusts any bearer of the shared secret; it exists for local + development and tests only. diff --git a/docs/deployment.md b/docs/deployment.md new file mode 100644 index 0000000..10eacc0 --- /dev/null +++ b/docs/deployment.md @@ -0,0 +1,61 @@ +# Provisioning & deployment + +Staging and production each run the same compose stack on a single AWS EC2 +instance behind Cloudflare, with Traefik (from the `staging`/`prod` compose +overrides) terminating TLS. + +## Prerequisites + +- AWS account + IAM user (EC2 access) configured via AWS CLI +- Terraform, Ansible, GitHub CLI (or a GitHub API token) +- Cloudflare account with the target domain as a zone (API token) +- Azure AD app registration (tenant + client id) for MSAL login + +## First-time setup + +```sh +# Key pairs (or obtain existing ones securely) +mkdir -p private/keys/instance private/keys/deploy +ssh-keygen -t ed25519 -C "you@example.com" -f private/keys/instance/id +ssh-keygen -t ed25519 -C "you@example.com" -f private/keys/deploy/id + +cp terraform.tfvars.tpl terraform.tfvars # fill in Cloudflare/GitHub tokens, env config +cp backend.tfconf.tpl backend.tfconf # S3 state backend +make infra-init +``` + +## Provision infrastructure + +```sh +make infra # staging (default Terraform workspace) +make infra env=prod # production (separate workspace) +``` + +Terraform creates the VPC/EC2/DNS records and registers deploy keys with +GitHub; it renders the Ansible inventory from +[config/ansible/templates/hosts.ini.tpl](../config/ansible/templates/hosts.ini.tpl). + +## Configure & deploy with Ansible + +```sh +make play # all environments +make play env=staging # or one of staging | prod +``` + +The [base playbook](../config/ansible/base.playbook.yml) updates the system, +installs Docker, clones the repo with the deploy key, renders `.env` and +`docker-compose.override.yaml` for the environment, and starts the stack. +Deploys of new code are the same playbook re-run (tag `deploy`). + +Schema changes require a dump → recreate → restore cycle — see the +[backup & restore runbook](runbooks/backup-restore.md). + +## Scheduled operations (GitHub Actions) + +| Workflow | Schedule | What | +| --- | --- | --- | +| [backup.yml](../.github/workflows/backup.yml) | every 2 h | pg_dump + cyanaudit dump from prod, uploaded as artifacts with tiered retention (bi-hourly/daily/weekly/monthly), then restored onto staging as a continuous restore test | +| [rotate-keys.yml](../.github/workflows/rotate-keys.yml) | (see workflow) | refreshes the Azure JWKS file PostgREST uses as `PGRST_JWT_SECRET` and HUPs PostgREST | + +Both need repo secrets (`PRIVATE_KEY`, `HOST_KEY`, `HOST_KEY_TEST`) and use +the playbooks in [.github/ansible/](../.github/ansible/). diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..af806f6 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,126 @@ +# Local development + +Everything runs in Docker Compose. The only host prerequisites are Docker with +the compose plugin and (optionally, for UI work outside containers) Node 18+ +with yarn. + +On macOS without Docker Desktop: + +```sh +brew install colima docker docker-compose +colima start --cpu 4 --memory 8 +# one-time: let the docker CLI find brew's compose plugin +# add to ~/.docker/config.json: "cliPluginsExtraDirs": ["/opt/homebrew/lib/docker/cli-plugins"] +``` + +## One-time setup + +```sh +cp .env.tpl .env +mkdir -p keys && printf 'Dummy5ecr3t4D3bug0n1yN0T4Pr0D123' > keys/jwt-secret +``` + +The `.env` template's dummy values work as-is for local debug mode. The +`keys/jwt-secret` file must exist **as a file** before the first +`docker compose up` — if compose creates the bind-mount path for you, it will +be a directory and PostgREST will fail to read it (delete it and recreate as a +file). In debug mode its content is unused (the debug compose override injects +the shared HS256 secret), but the mount must still resolve. + +## Two modes + +| | `make dev` | `make debug` | +| --- | --- | --- | +| Login | Real Azure AD via MSAL | Local form; browser signs an HS256 JWT itself | +| Requires | Azure tenant + app registration, real `AZURE_*` values in `.env`, JWKS in `keys/jwt-secret` | Nothing external | +| Onboarding (`/rpc/onboard`) | Works (validates Azure id tokens in-database) | **Does not work** — create users via the Employees screen instead | +| Use for | Verifying the real auth path | Day-to-day development and automated tests | + +Both modes use the `dev` compose override (published ports, verbose PostgREST). +Switching between them: rebuild (`make dev` / `make debug` handle this) **and +clear the browser's storage for localhost** — MSAL and the debug provider +cache different state and will confuse each other. + +## Start, bootstrap, use + +```sh +./initdb.sh # first time, or after `make clean` +make debug # or: make dev +``` + +`initdb.sh` builds and starts postgres if needed, waits for readiness, then +executes all of `dtrack/db/sql/` in order (see +[architecture.md](architecture.md#database-bootstrap-initdbsh)). Run it once +per fresh data volume. It is not idempotent — on an already-initialized +database many statements will error; for a clean slate use `make clean` first +(this **deletes the local database volume**). + +Bootstrap before `make debug`, or re-run `make debug` afterwards: `initdb.sh` +invokes plain `docker compose`, which recreates the postgres container +*without* the dev override (no published 5432 port) — running `make debug` +last leaves everything in the intended dev configuration. + +| URL | What | +| --- | --- | +| | UI (log in as `Dummy.User@example.com` in debug mode) | +| | PostgREST API | +| | Swagger UI | +| `localhost:5432` | Postgres (`psql -h localhost -U admin -d dtrack`, password from `.env`) | + +The initial power user is `$POSTGRES_USER_APP_POWER` from `.env` +(`Dummy.User@example.com` by default). Create further users through the +Employees screen while logged in as the power user, then log in as them — +debug mode can impersonate any *existing* user. + +## Everyday commands + +```sh +make down # stop (keeps data) +make clean # stop + delete data volume +docker compose logs -f postgrest # follow a service's logs +docker compose exec postgres psql -U admin -d dtrack # SQL shell as app admin +docker compose up -d --build react-admin # rebuild just the UI image +``` + +Handy inspection queries (activity, audit log, durations) live in +[dtrack/dev/queries/useful.sql](../dtrack/dev/queries/useful.sql). + +### Calling the API directly + +Any HS256 JWT signed with the debug secret works against PostgREST in debug +mode. For example, with [jwt-cli](https://github.com/mike-engel/jwt-cli) or any +JWT tool: + +```sh +TOKEN=$(jwt encode --secret 'Dummy5ecr3t4D3bug0n1yN0T4Pr0D123' '{"preferred_username":"Dummy.User@example.com"}') +curl -H "Authorization: Bearer $TOKEN" http://localhost:3000/current_employee +``` + +## UI development with hot reload + +The containerized UI is a static build; for iterative UI work run Vite on the +host against the containerized API: + +```sh +cd dtrack/ui +yarn install +VITE_API_BASE_URI=http://localhost:3000 VITE_APP_BASE_URI=http://localhost:5173 yarn debug +``` + +`yarn debug` starts the debug-mode stack (if not already up) and then runs +Vite locally on with the debug login. `yarn lint` +formats + lints (this repo pins prettier/eslint via the UI package; a +pre-commit hook runs lint on commit inside `dtrack/ui`). + +## Troubleshooting + +- **PostgREST restarts / "Password authentication failed"** — database not + initialized yet; run `./initdb.sh`. +- **UI login succeeds but everything 401s** — stale browser storage after + switching dev/debug modes; clear site data for localhost. +- **`role "…" does not exist` from PostgREST** — you logged in (debug mode) as + a user that was never created. Log in as the power user and create the + employee first. +- **`keys/jwt-secret` is a directory** — see one-time setup above. +- **initdb.sh errors mid-way** — it assumes a fresh volume. `make clean`, then + `make debug`, then re-run. diff --git a/dtrack/dev/deploy.md b/docs/runbooks/backup-restore.md similarity index 93% rename from dtrack/dev/deploy.md rename to docs/runbooks/backup-restore.md index 8212c2b..d6dfe42 100644 --- a/dtrack/dev/deploy.md +++ b/docs/runbooks/backup-restore.md @@ -1,3 +1,10 @@ +# Backup & restore runbook + +Operational procedure for backing up production, testing the restore locally, +and restoring onto a live server (used for schema-change deploys, since schema +is recreated rather than migrated). The automated equivalent runs bi-hourly via +[backup.yml](../../.github/workflows/backup.yml). + # 1. Backup ## Dev machine - `ssh ubuntu@dtrack.rethinkcode.org -i keys/prod/instance/id` diff --git a/dtrack/dev/debug.md b/dtrack/dev/debug.md deleted file mode 100644 index a7cd433..0000000 --- a/dtrack/dev/debug.md +++ /dev/null @@ -1,19 +0,0 @@ -# Debug Mode Guide - -Steps to enter and exit debug mode in dev environment - -**WARNING:** Never under any circumstance follow these instructions on a live -server when there is sensitive or production data in the database! - -## Prerequisites -- Navigate to the `dtrack/ui` directory -- Upon switching from dev to debug and visa versa, you will need to clear - browser cache (a browser extension is recommended) - -## Local Environment -1. `yarn debug` -1. `yarn dev` - -### Containerized Environment -1. `./run.sh debug` -1. `./run.sh dev`