Skip to content

Detect private cross-site model drift #2877

Description

@maziyarpanahi

Summary

A global model can pass aggregate gates while one site's language, specialty, or critical identifier performance deteriorates.

Scope

  • Compare coarse signed site metrics against local baselines using minimum-cell and privacy-noise rules.
  • Trigger hold, review, or rollback without exposing a site's exact slice values to other participants.

Acceptance criteria

  • The implementation is deterministic and performs no mandatory network call.
  • Logs, exceptions, reports, and fixtures contain no raw sensitive values.
  • .venv/bin/python -m pytest tests/unit/eval/test_cross_site_drift.py -q

Out of scope

  • A compliance certification or autonomous clinical decision guarantee.
  • Bundling restricted datasets, credentials, or proprietary services.
  • Unrelated changes to release publishing or existing public APIs.

Files

  • openmed/eval/cross_site_drift.py
  • tests/unit/eval/test_cross_site_drift.py
  • docs/evaluation/cross-site-drift.md

Task: OM-177d · Milestone: v3.3 · Priority: P0 · Size: L
Depends on: — · Blocks: —
Roadmap: Audited V3 horizon expansion, 2026-08-13: OPENMED_V3_HORIZON_BRIEF.md sections 3 and 9; OPENMED_BENCHMARK_PORTFOLIO_AND_TRUST.md section 8
Spec: PLANS/V2/EXECUTION/tasks/OM-177d.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0CriticalfeatureNew capabilityroadmap-v2OpenMed V2 roadmap backlog

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions