OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition

Clinical named entity recognition (NER) has become a crucial technique for extracting structured medical information from electronic health records (EHRs). However, traditional supervised models such as CRF and BioClinicalBERT rely on costly manual annotations.

To address these limitations, we propose OEMA, a novel zero-shot clinical NER framework based on ontology-enhanced multi-agent collaboration. OEMA utilizes three core agents:

1. Self-annotator: Autonomously generates candidate examples from unlabeled data.
2. Discriminator: Leverages the SNOMED CT ontology to filter token-level examples based on clinical relevance.
3. Predictor: Incorporates entity-type descriptions to enhance inference consistency and accuracy.

This repository contains the code and data sets used in our paper "OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition". Experimental results demonstrate that OEMA consistently outperforms existing zero-shot baselines and performs comparably to supervised models under relaxed-match criteria.

If you use this code in your scientific publications kindly cite the paper below:

"OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition" (2026-03-26)

DOI: 10.1093/jamiaopen/ooag049

If you plan to use the data

We evaluate OEMA on three benchmark datasets: MTSamples, VAERS, and 12B2-2010. If you plan to use the real-world i2b2-2010 data: Please follow the instructions "here" to obtain the necessary data use agreements.

Initial setup

To use OEMA, you need Python installed, along with the required libraries. Since OEMA relies on large language models, you will also need valid API keys (e.g., OpenAI API for GPT-3.5-turbo/GPT-4.1, or Google API for Gemini-2.5-flash).

# Clone the repository
git clone [https://github.com/XinliTao/OEMA.git](https://github.com/XinliTao/OEMA.git)
cd OEMA

# Install dependencies
pip install -r requirements.txt

Note: Please ensure you set your API keys in the environment variables or configuration files before running the framework.

Using Code for Zero-Shot Clinical Concept Extraction

The fastest way to use OEMA for clinical NER is to run the inference script using your preferred LLM backbone:

python run_oema.py --model gpt-3.5-turbo --input test_sentences.txt --output predictions.json

The input file should contain unstructured clinical text. The output will be a structured JSON list containing the identified entities and their corresponding labels based on the predefined medical ontology.

Run the full Self-Improvement Pipeline

To run the complete multi-agent pipeline (Self-annotator -> Discriminator -> Predictor) on an unlabeled corpus to build your own self-annotated few-shot examples:

cd OEMA
# Use text-embedding-ada-002 to obtain vector representations of the dataset
python src/generate_embs_GPT.py --dataname MTSamples --datamode train/test
# Start building the self-annotated corpus (sourced from the training set) after configuring the model API key and base URL
scripts/self_annotate_TSMV.sh
# Extract medical ontology information from the test and training sets
python src/self_consistent_annotation/AskGPT_ontology.py
# Construct prompts based on the Diverse K-Nearest Neighbors (KNN) algorithm combined with medical ontology information
python src/self_consistent_annotation/GeneratePrompts_ontology.py
# Run the data in the test set to view entity recognition results and statistical metrics
scripts/test_inference.sh

The script will automatically retrieve candidates via cosine similarity, filter them using the SNOMED CT ontology discriminator, and execute the final predictions.

To see all configurable parameters (such as backbone model, helpfulness score threshold, top-k values, etc.), simply run:

cat -n scripts/*.sh