[P2][QA] Establish a test architecture with coverage, property, contract, migration, and security gates

[andreazedda]

Problem

The repository contains a growing Django test suite and several focused safety/regression tests, but quality evidence is fragmented across commands and historical documents. A prior release document incorrectly treated “30/32 tests passing” as “94% coverage,” demonstrating the need to separate execution success from measured code/branch coverage.

Scientific software also needs properties and invariants beyond example-based route tests.

Objective

Create an explicit test taxonomy and enforce risk-based gates.

Test layers

1. Unit tests

Pure functions for:

• PK/PD and ODE derivatives;
• exposure schedule expansion;
• toxicity signals;
• observation mappings;
• hashing/canonical serialization;
• dataset semantic identity;
• model comparability;
• uncertainty calculations.

2. Property-based tests

Use Hypothesis or equivalent for invariants such as:

• no negative cell populations where the model forbids them;
• zero dose does not create positive drug exposure;
• canonical hash independent of JSON key order;
• importer second run is semantically idempotent;
• identical seed/config/input produces identical output within declared tolerance;
• unauthorized object IDs never disclose data;
• solver results converge under step refinement within tolerance.

3. Contract tests

Validate:

• dataset JSON schema;
• Twin input/state/lineage schemas;
• run manifests;
• API/OpenAPI responses;
• artifact manifests;
• epistemic output labels.

4. Integration tests

Cover:

• dataset → DB import;
• patient → Twin initialization;
• calibration parent/child lineage;
• simulation → artifact generation;
• permission/role matrices;
• Celery job lifecycle;
• PostgreSQL behavior.

5. Migration tests

For high-risk migrations:

• apply from last supported schema;
• verify cardinality/constraints;
• test reversal where supported;
• test historical JSON payload compatibility;
• assert no unintended data rewrite.

6. Security tests

• IDOR/object authorization;
• CSRF/session behavior;
• upload/path traversal;
• artifact authorization;
• rate-limit bypass through aliases;
• CSP/iframe;
• log redaction;
• banned identifier scans.

7. UI/end-to-end tests

Use browser automation for critical flows:

• login and role access;
• synthetic patient navigation;
• simulator run;
• Simple Research View/Cockpit;
• import/audit workflows where exposed;
• artifact view/export;
• denial behavior.

No real patient data in E2E fixtures.

Coverage policy

Report statement and branch coverage by package. Do not use a single percentage as proof of scientific validity.

Set initial gates based on current baseline, then ratchet upward. New code should meet a higher threshold than legacy code. Critical policies, importers, lineage, security and numerical contracts require near-complete branch coverage.

Numerical regression policy

Golden outputs must include model/data/config versions and tolerances. Update only with an output-diff explanation and model-version decision.

Test environment

• locked dependencies;
• temporary DB/media/logs;
• deterministic seeds;
• no network unless explicitly mocked/integration-tagged;
• PostgreSQL CI job;
• optional chemistry dependency matrix.

Acceptance criteria

• Test taxonomy and commands are documented.
• CI reports statement and branch coverage accurately.
• Critical modules have explicit branch-coverage gates.
• Property tests enforce scientific and privacy invariants.
• API/dataset/run schemas have contract tests.
• High-risk migrations have forward/backward evidence.
• E2E tests cover the primary synthetic research flow.
• Golden numerical changes require versioned diff evidence.
• Flaky-test ownership and quarantine policy exist.

Milestone suggestion

M3 — Scientific Validation Layer.

Dependencies

Builds on CI, dependency locking, object authorization, API schemas and model registry.