A semiconductor characterization and manufacturing-execution platform. Combines a Python/FastAPI backend, a Next.js 15 / React 19 frontend, PostgreSQL persistence, and a Celery+Redis async layer into one repo. Covers metrology (electrical, optical, structural, chemical), process control (CVD, PVD, CMP, Diffusion, Oxidation, Ion Implant, RTP), SPC, ML/virtual-metrology, and a LIMS/ELN.
Project status (2026-08-04): production-ready. The remediation pass, the production-completion plan (Phases 0–6), and the account-lifecycle phase are all complete and merged. Release v0.1.0 exists with all four service images built, Trivy-scanned to zero HIGH/CRITICAL findings, and pushed to ghcr by the tag-triggered CD pipeline; the Kubernetes manifests deploy every workload (services, celery workers, MinIO, Postgres, redis, backups) and are render-validated in CI. What stands between this repo and a live deployment is provisioning only — a cluster, a domain, and secret values — with the exact sequence in
docs/deployment/PRODUCTION_RUNBOOK.mdand one-command helpers inscripts/provision/.
┌─────────────────────────────────────────────────────────────────┐
│ apps/web Next.js 15 frontend (port 3012) │
└────────────────────────────┬────────────────────────────────────┘
│ REST + WebSocket
┌────────────────────────────┴────────────────────────────────────┐
│ services/ │
│ analysis FastAPI: CVD, Diffusion, Oxidation, PVD, │
│ CMP, SPC, Calibration, ML (port 8001) │
│ lims FastAPI: samples, recipes, SOPs, ELN │
│ (port 8002) │
│ process_control FastAPI: Ion Implant, RTP, Jobs, │
│ WebSocket telemetry (port 8003) │
│ shared SQLAlchemy models, JWT/RBAC, SPC engine │
└────────────────────────────┬────────────────────────────────────┘
│
┌────────────────────────────┴────────────────────────────────────┐
│ PostgreSQL 15 (5435) Redis 7 (6381) MinIO Prometheus │
│ Alembic migrations Celery worker + beat + Flower (5555) │
└─────────────────────────────────────────────────────────────────┘
Detailed reference: docs/architecture/, docs/REPOSITORY_STRUCTURE.md.
- Docker and Docker Compose
- (Optional, for non-Docker dev) Python 3.11+, Node.js 20+
git clone https://github.com/alovladi007/SPECTRA-Lab.git
cd SPECTRA-Lab
export DATABASE_URL="postgresql+psycopg://spectra:spectra@localhost:5435/spectra"
make smoke-test-buildmake smoke-test-build builds all images, starts db, redis, analysis, lims, and web, and probes each /health endpoint. On success it prints the URLs below.
| Service | URL |
|---|---|
| Web (Next.js) | http://localhost:3012 |
| Analysis API + Swagger | http://localhost:8001/docs |
| LIMS API + Swagger | http://localhost:8002/docs |
| Process Control API + Swagger | http://localhost:8003/docs |
| PostgreSQL | localhost:5435 (spectra / spectra) |
| Redis | localhost:6381 |
| Flower (Celery monitor) | http://localhost:5555 |
| Prometheus | http://localhost:9090 |
| Grafana | http://localhost:3001 (admin / admin) |
make help # list every Makefile target
make dev-up # docker compose up -d (everything)
make dev-down # docker compose down (volumes preserved)
make dev-logs # tail logs
make migrate # apply Alembic migrations
make seed-db # populate demo data
make test-collect # show what pytest collects (debugging)For the canonical local-dev environment (ports, env vars, known issues), see TROUBLESHOOTING.md.
If Docker is broken or you want faster inner-loop iteration, the repo ships a complete native (Homebrew-based) dev path: brew bundle install && make dev-native-up && make venv-analysis && make serve-analysis. Native uses Postgres on 5432 and Redis on 6379 so it doesn't collide with the Docker stack. Full story: docs/dev-native.md.
apps/web/ Next.js 15 + React 19 + Tailwind + shadcn/ui
src/app/ Routes: cvd, dashboard, data, experiments,
login, process-control, results, samples,
system
services/
analysis/app/ CVD, Diffusion, Oxidation, SPC, calibration,
predictive maintenance, ML (AutoML, A/B,
explainability, monitoring); plus modules/
for PVD and CMP. (port 8001)
lims/app/ samples, recipes, SOPs, ELN (port 8002)
process_control/app/ Ion Implant, RTP, Jobs, WebSocket telemetry,
SPC, Virtual Metrology (port 8003)
shared/ SQLAlchemy models, JWT/RBAC, SPC engine,
test fixtures
alembic/versions/ 24 migration files; chain head =
20260803_1000_0024 (password reset tokens)
infra/ Prometheus + Grafana + Alertmanager + Keycloak
(dev realm) + MinIO lifecycle provisioning
k8s/ Kustomize base + production overlay: all four
services, celery worker/beat, pc-worker, MinIO,
Postgres, redis, backup CronJobs, ingress + TLS;
render-validated in CI on every PR
docs/ Architecture, API specs, build plan, archived
session reports
scripts/ Operational scripts (deploy, generators)
tests/ Cross-service integration tests
| Doc | What it covers |
|---|---|
docs/BUILD_PLAN.md |
37-session phased plan from current state to production. Authoritative. |
TROUBLESHOOTING.md |
Local-dev URLs, ports, DATABASE_URL semantics, smoke-test gaps |
docs/architecture/ |
Architecture decisions and design notes |
docs/ROADMAP.md |
High-level roadmap |
docs/DATA_MODEL_SPECIFICATION.md |
Database schema reference |
docs/api/ |
OpenAPI specs and per-service API references |
docs/deployment/PRODUCTION_RUNBOOK.md |
Go-live sequence: cluster → add-ons → DNS/TLS → secrets → deploy → first admin |
docs/deployment/ |
SECRETS (Sealed Secrets flow), AUTH (SSO), MIGRATIONS, ALERTING runbooks |
docs/sessions/ |
Per-session implementation notes |
docs/archive/sessions/ |
Historical status reports (not authoritative) |
The platform's domain coverage. Depth varies by area; details and current implementation state live in the docs above.
Metrology
- Electrical: 4PP, Hall, I-V, C-V, BJT, MOSFET, Solar Cell, DLTS, EBIC, PCD
- Optical: UV-Vis-NIR, FTIR, Ellipsometry, Photoluminescence, Raman
- Structural: XRD, SEM, TEM, AFM, optical microscopy
- Chemical: XPS, XRF (surface); SIMS, RBS, NAA, etch (bulk)
Process control & MES (each has both an engineering "Process Control" UI and a production "MES" dashboard)
- CVD (LPCVD, PECVD, MOCVD, AACVD)
- PVD (sputter, evaporation, PLD)
- CMP
- Diffusion (B, P, As, Sb)
- Oxidation
- Ion Implantation
- RTP
Cross-cutting
- SPC: X-bar/R, I-MR, EWMA, CUSUM, Western Electric rules, Cp/Cpk/Pp/Ppk, drift detection
- ML/Virtual Metrology: AutoML (Optuna), forecasting (Prophet), explainability (SHAP/LIME), drift, A/B testing, model registry, online learning
- LIMS/ELN: samples, recipes (with PI approval workflow), SOPs, ELN with e-signatures, exports
- Auth & accounts: JWT (HS256) + optional Keycloak OIDC/SSO (RS256), 5-role RBAC, Postgres row-level security on all 91 org-scoped tables, password reset + change-password + admin user invitations over env-configured SMTP (dev mode logs the emails)
- Real-time: WebSocket telemetry; Celery + Redis for background jobs
- Observability: Prometheus metrics on every service, Grafana dashboards, Alertmanager rules, OpenTelemetry tracing
Three major passes, every step a CI-green PR on a single main branch:
- Build-out + foundation re-audit (2026-03 → 2026-05) — the original
37-session build plan, then Phase A which hardened what the plan had
left as stubs: audit-log writers, Postgres RLS, schema/model drift,
OpenTelemetry. History:
docs/BUILD_PLAN.md. - Remediation pass (2026-05 → 2026-06, PRs #46–#104) — a
four-dimension audit found a convincing shell (auth couldn't complete,
~47 pages rendered fabricated data, "looks-real" endpoints returned
hardcoded numbers). Every fake was replaced with real physics, real
queries, or an honest error. End state:
docs/REMEDIATION_STATUS.md. - Production-completion plan (2026-06 → 2026-08, PRs #130–#174) —
security lockdown, broken execution paths, data-layer integrity
(RLS on all 91 org-scoped tables, CI-audited), frontend
deployability (runtime-env API proxy, standalone image), testing
depth (Playwright smoke + 26-page sweep in CI), deployability
(tag-triggered CD: build → Trivy gate → ghcr push → SBOM → release
→
kubectl apply), product completion (global search, notifications, localization, OIDC SSO), HIL-simulator physics reconciliation (virtual-clock rework), and the account lifecycle (password reset, change password, admin invitations over SMTP).
- Release: v0.1.0 — four images on ghcr (
analysis,lims,process-control,web), each scanned to zero HIGH/CRITICAL CVEs (runtime images ship without pip/npm), SPDX SBOMs attached. - CI: 17 required checks per PR — lint, typecheck ratchet, unit matrix (3.10/3.11/3.12), integration, LIMS suite, migration round-trip, Docker builds, security + secrets guardrails, k8s manifest rendering, Playwright e2e smoke against the real compose stack.
- Tests: ~950 backend tests across the four suites (analysis 304, lims 90, process-control 466 incl. accelerated-time soak, shared 88) plus 4-test e2e smoke and a 26-page dashboard sweep.
- Deploy path:
git tag v0.1.x && git push --tagsdoes everything throughkubectl rollout statusonce the oneKUBE_CONFIGrepo secret is set. First-admin bootstrap:bootstrap_admin.py(refuses demo seeding in production). - Remaining before a live URL: provisioning only — cluster, domain/DNS, TLS, sealed secret values, optional SMTP relay. Exact sequence:
docs/deployment/PRODUCTION_RUNBOOK.md; automation:scripts/provision/.
Demo login after make seed-db: admin@demo.lab / admin123 (also pi/engineer/tech/viewer @demo.lab).
This is a private project; there is currently no CONTRIBUTING.md. If you have access and want to land work, the cross-cutting conventions in docs/BUILD_PLAN.md apply: read first, plan-then-execute on >5-file changes, one concern per commit, tests with every behavior change, no new TODOs, docs updated in the same PR.
All rights reserved. (No LICENSE file is published; treat the contents as proprietary.)