Personal educational and portfolio project. Synthetic data only. No real patient information, PHI, payer credentials, production healthcare integrations, or employer/client proprietary information belongs in this project.
A local synthetic environment for future AI agents that perform multi-step healthcare workflows through real HTTP tools. The first planned use case is prior authorization.
Milestone 2 provides persistent synthetic cases and explicit manual simulator actions. Browse patients and documents, request authored eligibility fixtures, and record messages that are never delivered. There is no PA agent, autonomous workflow, clinical reasoning, model inference, evaluation engine, authentication system, or deployment. Eligibility fixtures are test data, not coverage determinations or PA approvals.
flowchart LR
Web["Next.js Cases UI"] -->|HTTP via server-only proxy| API["FastAPI orchestrator"]
API -->|cases| O[("orchestrator schema")]
API -->|HTTP| EHR["EHR simulator"] --> E[("ehr schema")]
API -->|HTTP| Docs["Document simulator"] --> D[("documents schema")]
API -->|HTTP| Payer["Payer simulator"] --> P[("payer schema")]
API -->|HTTP| Comms["Communications simulator"] --> C[("communications schema")]
API -. future reasoning .-> Gateway["Provider-neutral gateway / local mock"]
API -. future events .-> Analytics["Disabled Databricks / MLflow boundary"]
The five schemas share one PostgreSQL instance. Runtime services access only their own schema; the orchestrator gets simulator data through HTTP. Migration/fixture commands are the explicit administrative exception. Evaluation remains separate from execution.
| Area | Stack |
|---|---|
| Frontend | Next.js 16, React 19, TypeScript, Tailwind CSS 4 |
| Backend | Python 3.13, FastAPI, Pydantic 2, HTTPX |
| Database | PostgreSQL 17, psycopg 3, versioned SQL migrations |
| Development | Docker Compose, uv and npm workspaces with lockfiles |
| Quality | pytest, Playwright, Ruff, strict mypy/TypeScript, ESLint, Prettier |
| Future boundaries | Provider adapters, standalone evaluation, Databricks/Delta, MLflow |
See architecture, ADRs, and the complete repository tree.
- Docker Desktop for Apple Silicon, installed and running.
- Node.js 24 and its bundled npm, matching
.nvmrcandpackage.json. An existing compatible installation is sufficient; nvm is optional. - uv 0.12.14 for host Python development and tests.
uv synccan provision Python 3.13. - Free ports 3000, 5432, and 8000–8004, or configured alternatives.
No model-provider key, Databricks account, or other external account is required.
Run from the repository root:
cd "/Users/kaitakeuchi/Documents/ChatGPT/AI Eval"
# Create local configuration only if it does not exist.
if [ ! -f .env ]; then cp .env.example .env; fi
docker compose up --build --wait --wait-timeout 180
make migrate
make seed
make data-status
npm ci
npm run devOpen the dashboard or Cases.
Keep the frontend terminal running. After initialization, ordinary startup is
docker compose up -d --wait followed by npm run dev. Rebuild containers after backend edits.
| Service | Host address |
|---|---|
| Frontend | http://127.0.0.1:3000 |
| Orchestrator | http://127.0.0.1:8000 |
| Payer | http://127.0.0.1:8001 |
| EHR | http://127.0.0.1:8002 |
| Documents | http://127.0.0.1:8003 |
| Communications | http://127.0.0.1:8004 |
| PostgreSQL | 127.0.0.1:5432, database lab |
Each API exposes /health, /docs, and /openapi.json. /health proves process liveness;
/system/status checks simulator reachability and database connectivity. It does not prove
migrations have been applied. Domain requests return a useful 503 if tables are missing.
Optional integrations are reported as disabled or unimplemented, never falsely connected.
If the browser cannot connect, inspect docker compose ps, docker compose logs, and the
frontend terminal. Verify curl http://127.0.0.1:8000/system/status. Avoid starting a duplicate
frontend when port 3000 already belongs to this project.
Versioned definitions live in fixtures/synthetic-v1.json.
All six cases begin in draft at a fixed UTC timestamp. Patient IDs end in 001–006;
case UUIDs end in the corresponding digit.
| Patient | Documents | Eligibility |
|---|---|---|
Demo Alpha / syn-patient-001 |
Referral and summary | eligible |
Demo Bravo / syn-patient-002 |
Empty, registered patient | eligible |
Demo Charlie / syn-patient-003 |
One document | ineligible |
Demo Delta / syn-patient-004 |
One document | authored unknown |
Demo Echo / syn-patient-005 |
Empty, registered patient | unknown because no fixture exists |
Demo Foxtrot / syn-patient-006 |
Two documents | eligible |
The baseline contains six patients, six document registrations, six documents, five
eligibility fixtures, six cases, and zero messages. All fixture records carry synthetic=true
and fixture_version=synthetic-v1.
make migrate # Apply unapplied SQL files, in filename order within each schema.
make seed # Insert missing baseline rows; identical rows are left alone.
make data-status # Counts and number of developer-created/modified records; no message bodies.Migrations and seed run transactionally with an administrative lock. Applied migrations have stored checksums; changing an applied file is rejected. Add a new migration instead. Seed preserves unrelated rows and aborts the entire seed if an existing fixture ID has different values. It never silently overwrites an edited fixture. Run migration before seed; tables are not created by ordinary HTTP requests. See ADR 0005.
Reset removes sandbox-created records and restores edited or missing fixtures. Inspect first:
make data-status
make reset RESET_FLAGS=--confirm-resetIf created or modified records exist, reset refuses. Only after intentionally choosing to delete those records, use:
make reset RESET_FLAGS='--confirm-reset --allow-user-records'Without flags, make reset fails. Reset checks the local host, database name, and project marker,
locks the owned tables before inspecting records, and truncates/reseeds them in one transaction.
It refuses arbitrary remote targets, unmarked databases, and names other than lab or generated
lab_test_<hex> names. It never uses CASCADE, removes unrelated schemas, or deletes volumes.
Custom database names can support runtime/migrations but are intentionally ineligible for reset.
See ADR 0006.
Stop the frontend with Ctrl+C and use docker compose down for the backend. The named
PostgreSQL volume remains. API restarts and container rebuilds also preserve stored data.
docker compose down --volumes deletes the database volume; it is not the guarded reset command.
- Open Cases, then Demo Alpha.
- Inspect its stored patient, draft status, fixture version, referral, and summary.
- Select Check synthetic eligibility. The authored
eligibleresult appears; case status remains draft. No model or clinical rule runs. - Enter invented content under Synthetic message body, keeping recipient
syn-provider-001. - Select Record synthetic message. The receipt displays Recorded · Not delivered.
- Save the receipt ID to retrieve it later. A page reload does not reload the last receipt; the API retains it. Use the retrieval route below.
- Inspect Bravo for an empty-document example and Charlie for an ineligible fixture.
The UI prevents pending duplicate submissions. Retrying an unchanged message in the same mounted page reuses its request UUID. Reloading the page starts a new request; API callers should retain their request UUID across uncertain outcomes. Editing after success permits a new message.
The orchestrator's interactive contract is at API docs.
| Operation | Orchestrator route | Behavior |
|---|---|---|
| List | GET /cases?limit=20&offset=0 |
Bounded 1–50, offset 0–100000, patient summaries |
| Create | POST /cases |
{ "patient_id": "syn-patient-001" }; checks EHR through HTTP, returns 201 |
| Retrieve | GET /cases/{case_id} |
Stored case or 404 |
| Context | GET /cases/{case_id}/context |
Patient and documents through HTTP; partial failure states |
| Eligibility | POST /cases/{case_id}/eligibility |
Fixture result; no request body or case mutation |
| Record | POST /cases/{case_id}/messages |
Required request UUID, synthetic recipient, body; returns 202 |
| Receipt | GET /cases/{case_id}/messages/{message_id} |
Stored message, checked against the requested case |
Direct simulator contracts remain GET /patients/{patient_id} on EHR,
GET /patients/{patient_id}/documents on Documents, POST /eligibility/check on Payer,
and POST /messages plus GET /messages/{message_id} on Communications.
EHR returns 404 for unknown patients. Documents returns 404 for an unregistered patient and
an empty list for a registered patient without documents. Payer returns unknown with a null
fixture version when no authored fixture exists. This response never implies a clinical decision.
Example recording request (use a fresh UUID for a different message, and reuse it for a retry):
curl --fail-with-body -X POST \
http://127.0.0.1:8000/cases/10000000-0000-4000-8000-000000000001/messages \
-H 'Content-Type: application/json' \
-d '{"request_id":"20000000-0000-4000-8000-000000000001","recipient_id":"syn-provider-001","body":"Invented walkthrough message."}'Replace RECEIPT_ID with the returned message_id:
curl --fail-with-body \
http://127.0.0.1:8000/cases/10000000-0000-4000-8000-000000000001/messages/RECEIPT_IDMessage bodies accept 1–2000 characters, recipients use the bounded syn- convention, and the
only accepted channel is synthetic. Duplicate request UUIDs return the existing receipt if
content matches; different content returns 409. delivered is always false, including a database
constraint. GET requests cannot create messages. Nothing logs or sends message bodies externally.
Validation errors return 422, missing records 404, unavailable dependencies 503, timeouts 504,
and invalid simulator responses 502. Context responses distinguish available, not_found, and
unavailable; failed documents use null, never a fabricated empty list. Outbound calls are bounded
and are not automatically retried. Case timestamps are timezone-aware.
Keep .env ignored and preserve its existing values. Compose uses Docker DNS names; the frontend
loads root configuration server-side. No credentials use NEXT_PUBLIC_ variables.
| Variables | Purpose |
|---|---|
DATABASE_URL |
Standard Postgres DSN for each service's repository |
PAYER_SERVICE_URL, EHR_SERVICE_URL, DOCUMENT_SERVICE_URL, COMMUNICATIONS_SERVICE_URL |
Orchestrator HTTP targets |
ORCHESTRATOR_URL |
Server-only frontend target |
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, *_PORT |
Local Compose settings |
AI_PROVIDER, AI_MODEL |
Default mock / mock-v1; no inference during case operations |
| Provider keys | Reserved and unused; leave blank |
DATABRICKS_HOST, DATABRICKS_TOKEN, MLFLOW_TRACKING_URI |
Optional boundary only; no export or connection |
Postgres initialization variables apply to a fresh volume. Changing a password in .env does
not change a password already stored in Postgres. Hosted Postgres can later replace the DSN with
appropriate TLS parameters; destructive reset remains local-only.
For optional host-native Python development, with local PostgreSQL available:
uv sync --locked
export DATABASE_URL='postgresql://lab:lab_local_only@127.0.0.1:5432/lab'
uv run --locked python -m scripts.manage_data migrate
uv run --locked python -m scripts.manage_data seed
uv run --locked python scripts/dev_services.pyDo not run native APIs on ports occupied by Compose. Native defaults use loopback simulator URLs;
do not export the root Docker service URLs unchanged. Admin commands accept LAB_DATABASE_URL
as an explicit override before DATABASE_URL, then use the default local lab DSN. They do not
implicitly load .env; make migrate/seed/reset use the environment inside the orchestrator.
Parallel checkouts need distinct Compose project names, published ports, and frontend ports.
Use docker compose -p lab-feature ..., configure matching .env ports and ORCHESTRATOR_URL,
and run npm run dev -- --port 3001. For make's Compose commands, set COMPOSE_PROJECT_NAME
consistently. Never share a development database between isolated test runs.
uv sync --locked
npm ci
make check
make test
# PostgreSQL must be available; tests create and remove their own databases.
make integration
npx playwright install chromium
make browser-test
# Standalone production build (stop a dev server using the same output directory first).
npm run buildmake browser-test builds into .next-verification, starts five real APIs and a separate
production frontend on temporary ports, and exercises Chromium against a generated test database.
It preserves the regular .next development output. Screenshots and failure traces go to
output/playwright/ and are ignored by Git.
Integration tests connect through TEST_DATABASE_ADMIN_URL (default: local lab on port 5432)
only to create a uniquely named lab_test_<uuid> database. All domain writes and destructive tests
use that generated database; cleanup drops only that name. The role needs CREATE DATABASE permission.
Changing DATABASE_URL does not redirect tests into the developer database. Native test commands
do not automatically load .env; export a host-compatible test admin DSN for nondefault ports.
CI runs unit and quality checks, contract drift detection, production builds, Compose image builds, repeated migration/seed initialization in its disposable environment, packaged API context checks, isolated backend integration tests, and browser tests. CI volume removal is restricted to its own runner. See tests and validation evidence.
After changing API contracts:
uv run --locked python scripts/export_openapi.py
npm run types:generate --workspace @lab/webCommit the generated apps/web/src/lib/schema.d.ts. Format with uv run ruff format . and
npm run format; optional hooks use uv run pre-commit install.
The gateway, execution-event boundary, evaluation directories, and proposed analytical schemas remain available for future work. The analytics plan is raw events → Bronze → normalized traces in Silver → independent evaluations and Gold metrics → comparisons/dashboards. It is not active.
The next milestone may introduce explicitly authorized controlled agent runs, tool-call records, and durable human review. Model adapters, tracing, evaluation, MLflow and Databricks follow in later steps. Runs, Evaluations, and Governance remain placeholders. See roadmap.
The current schemas share a development database role; ownership is enforced by the runtime code, not separate database grants. There is no authentication, PHI detection, or compliance certification. Keep the application local. See security and privacy.