Close the audit-trail gaps: enforced privilege separation, system-scope review, complete exports

[tgerke]

Closes the gaps found while assessing edc-core against FDA audit-trail requirements. Every regulatory claim below was checked against the source mirror (21 CFR Part 11 rev. 2024-04-01; FDA Electronic Systems Q&A, Oct 2024; ICH E6(R3) final). Worth stating up front: 11.10(e) requires the trail to exist, retain, and be copyable — it contains no review obligation, and FDA guidance says only that review "may be helpful." The review expectation the docs cite comes from ICH E6(R3) §4.2.3.

Privilege separation (migration 0024)

ADR-0002 claimed the application role "has no privilege to alter or drop these triggers," but nothing enforced it: the app connected as the table owner, which can ALTER TABLE ... DISABLE TRIGGER or TRUNCATE around the append-only guards. Now:

• The API connects as edc_app, which owns nothing and holds no TRIGGER, TRUNCATE, or REFERENCES privilege anywhere; UPDATE/DELETE are additionally revoked on the seven append-only tables (signatures keep UPDATE for the trigger-constrained one-way invalidation).
• Migrations run as the owning role via MIGRATE_DATABASE_URL (falls back to DATABASE_URL for single-role setups, so tests and scripts are unchanged).
• Fresh compose volumes create the login via infra/initdb/; existing databases need one line, documented in the deployment guide: ALTER ROLE edc_app LOGIN PASSWORD '...'.
• The migration also hands ownership of pre-split ducklake_% catalog schemas to edc_app so in-place catalog upgrades keep working. Serial-owned sequences are skipped — they cannot change owner independently and follow their table; this failed on a synthetic pre-split schema until fixed, so it is worth a close look.
• New integration tests prove the runtime role cannot disable or drop the triggers, TRUNCATE, or UPDATE/DELETE the trail, and can still read and append. Also verified live in psql against the compose stack.

System-level audit review (/admin/audit)

Events written with no study — logins, lockouts, account lifecycle, cross-study role changes — were recorded but unreviewable: no endpoint, no UI, absent from the archive. GET /admin/audit (system-administration gated, mirroring /admin/access-log) plus a System audit page now cover them.

Decision to review: these events deliberately stay out of the per-study archive. They span studies, and bundling the global trail into one study's inspection copy would leak activity from the others. The /admin/audit CSV is their inspection copy.

Complete CSV exports

The route exports silently truncated at 10,000 rows — a truncated inspection copy with no warning. Exports now stream the full trail with keyset pagination. The cursor is the id alone: a JS Date cursor loses Postgres's microsecond precision and silently drops rows (caught by a test crossing the batch boundary), so the export orders by id rather than the UI's timestamp sort. Two smaller landmines fixed along the way: Readable.from() defaults to objectMode, which breaks Fastify replies, and raw Date params in drizzle sql fragments miss the driver's type mapping.

Timestamps, UI, and docs

• The review UI rendered toLocaleString() with no timezone label, so two reviewers in different zones read different strings for the same event (E6(R3) 4.2.2(d) wants unambiguous times). It now renders labeled UTC, and exposes the from/to filters the API already had — compliance.md claimed time filtering that the UI didn't have.
• Both audit pages share one AuditTrail component (studyId null selects the system scope; Biome's useHookAtTopLevel pushed the design away from hook-as-prop).
• deployment.md gains Database roles and Clock synchronization sections plus checklist items — host NTP was an unaddressed dependency of the "time-stamped" claim.
• Traceability matrix: P11-01/P11-02/E6-03 mechanisms updated to match what now exists; stale .qmd evidence paths from the Astro migration fixed.

Verification

• Validation pack regenerated: 485/485, no problems.
• pnpm check green (lint, typecheck, all package suites; API at 273).
• Compose stack rebuilt on split credentials: boots, logs in, writes audit events as edc_app; all four bypass attempts rejected in psql.
• Browser-verified both audit pages, date filters, and both CSV endpoints.

Dev-environment note: your local cluster already has the ALTER ROLE applied, and demo-admin was set is_system_admin = true in the dev DB to exercise the admin pages. Outside the repo, the two 21 CFR files are now hashed into the source mirror's MANIFEST.sha256 (full check passes).