Skip to content

feat: add clinical domain coverage eval - #2227

Merged
maziyarpanahi merged 2 commits into
batch/maziyarpanahi-v22-clinical-eval-2026-08-13from
feature/om-508-clinical-domain-coverage-eval
Aug 13, 2026
Merged

feat: add clinical domain coverage eval#2227
maziyarpanahi merged 2 commits into
batch/maziyarpanahi-v22-clinical-eval-2026-08-13from
feature/om-508-clinical-domain-coverage-eval

Conversation

@maziyarpanahi

Copy link
Copy Markdown
Owner

Pull Request

Description

Adds an offline aggregate coverage gate for the shipped clinical label maps and synthetic span fixtures.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code refactoring
  • Performance improvement
  • Test addition/improvement

Changes Made

  • Added per-domain and per-label coverage reporting with offsets and counts only.
  • Added missing-fixture, orphan-label, invalid-offset, and no-span failure checks.
  • Registered the suite and benchmark CLI command, added synthetic fixtures, docs, and a CI gate.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • Focused evaluator, CLI, documentation, and label-map tests pass locally
  • I have tested this change with different models/inputs

Documentation

  • I have updated the documentation accordingly
  • I have added docstrings to new functions/classes
  • I have updated the CHANGELOG.md

Code Quality

  • Changed-file Ruff lint and format checks pass
  • I have performed a self-review of my own code
  • My changes generate no new warnings

Dependencies

  • I have not added any new dependencies

Related Issues

Closes #866

Screenshots/Examples

Not applicable.

@maziyarpanahi maziyarpanahi added roadmap-v2 OpenMed V2 roadmap backlog feature New capability P1 High labels Aug 5, 2026
@maziyarpanahi
maziyarpanahi changed the base branch from master to batch/maziyarpanahi-v22-clinical-eval-2026-08-13 August 13, 2026 14:05
@maziyarpanahi
maziyarpanahi merged commit dd4e29a into batch/maziyarpanahi-v22-clinical-eval-2026-08-13 Aug 13, 2026
21 of 23 checks passed
maziyarpanahi added a commit that referenced this pull request Aug 13, 2026
* Add medication reconciliation across mentions within a document (#2152)

Co-authored-by: Maziyar Panahi <5762953+maziyarpanahi@users.noreply.github.com>

* Add allergy and intolerance domain to NER catalog and label maps (#2220)

Co-authored-by: Maziyar Panahi <5762953+maziyarpanahi@users.noreply.github.com>

* feat: add synthetic section-label dataset builder (#2225)

* feat: add synthetic section label dataset builder

* fix: sync synthetic section labels with canonical sets

---------

Co-authored-by: Maziyar Panahi <5762953+maziyarpanahi@users.noreply.github.com>

* feat: add clinical event timeline assembler (#2226)

Co-authored-by: Maziyar Panahi <5762953+maziyarpanahi@users.noreply.github.com>

* feat: add clinical domain coverage eval (#2227)

Co-authored-by: Maziyar Panahi <5762953+maziyarpanahi@users.noreply.github.com>

* integration: cover allergy labels in policy profiles

---------

Co-authored-by: Maziyar Panahi <5762953+maziyarpanahi@users.noreply.github.com>
@maziyarpanahi

Copy link
Copy Markdown
Owner Author

Integrated via #2693 after exact-head validation. The combined clinical/evaluation batch passed 468 focused tests, Ruff checks and format verification, and a strict documentation build; every intermediate and final Git tree matched the rehearsal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New capability P1 High roadmap-v2 OpenMed V2 roadmap backlog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a clinical-domain label-map coverage eval suite over the synthetic fixtures

1 participant