MedHandoff AI — Agentic Clinical Signout Powered by MedGemma (Tracks: Main Track + Agentic Workflow Prize)
- Olivia Rutler — Medical student in clinical clerkships. Defines real-world handoff constraints and validates that the workflow fits how clinical teams actually operate.
- Vivian Peng — Product-minded software engineer. Architects the multi-agent system and delivers the full-stack clinician-facing application.
- Leo Jin — PhD candidate (Columbia, Neurobiology). Leads dataset curation, model evaluation, and reproducibility.
The problem. Clinical handoffs — when one physician ends a shift and another takes over — are a leading source of preventable harm. Communication failures during handoffs contribute to roughly 80% of serious medical errors. Structured interventions like I-PASS have been shown to reduce errors by 30%, yet handoffs today remain largely manual and inconsistent.
At every shift change, the receiving resident must reconstruct complex clinical narratives from fragmented EHR data: notes, labs, vitals, medications, imaging, and orders. This takes 15–30 minutes per patient and requires extracting trends ("what changed?"), resolving inconsistencies ("stable" note vs. worsening labs), and identifying action items. Errors and omissions are common, and the time spent on synthesis is time not spent with patients.
The user. Our primary user is the resident receiving a night or cross-cover handoff. Their current workflow:
signout → manual chart review → mental synthesis → risk of missed information
MedHandoff changes this to:
signout → one click → structured I-PASS brief with trends, conflicts, action items, contingencies, and linked source evidence
This shifts effort from information retrieval to verification and clinical decision-making.
Why AI is required. This task involves synthesizing high-dimensional, longitudinal clinical data with temporal reasoning and cross-source consistency checks. Rule-based systems cannot generalize across heterogeneous records, and generic summarization models lack grounding. Our agentic architecture combines deterministic signal extraction with MedGemma-based clinical reasoning. Every claim is grounded to source data.
Impact. We evaluated a 20-case MIMIC-IV cohort:
| Metric | Baseline (human notes) | MedHandoff |
|---|---|---|
| I-PASS checklist coverage | 0% | 50% |
| High-risk misses / patient | 2.0 | 1.0 |
| Evidence grounding | — | 100% |
| Time saved / patient | — | ~16 min |
For a 20-patient shift, this returns 5 hours of clinician time ($450/shift at $90/hr blended cost), excluding downstream safety benefits from fewer missed deteriorations.
We use MedGemma 27B (text) through the Google Agent Development Kit (ADK) as the clinical reasoning backbone across a four-agent modular pipeline. Each agent receives a focused task, calls MedGemma independently, and passes structured output to the next stage. MedGemma is used exactly where it adds value beyond deterministic logic — clinical interpretation, evidence routing, and knowledge synthesis.
Agent 1 — Writer Agent. Loads patient data (labs, vitals, medications, notes, history) and passes it to MedGemma with a structured I-PASS prompt. MedGemma interprets multi-signal clinical context — for example, recognizing that hemoglobin dropping while a patient is on anticoagulants raises a bleeding concern, not just an anemia flag — and produces a structured draft: illness severity, patient summary, trend narrative with specific values, clinical "why" analysis, and prioritized action items. Rule-based templates can list values; MedGemma can reason about their clinical significance and generate fluent, handoff-ready language. A deterministic signal extractor (trend analyzer + conflict detector) provides pre-computed lab/vital trends and medication-lab conflicts as advisory input, but MedGemma decides how to weight and present them.
Agent 2 — Provenance Router. Takes the Writer's draft and the same patient record. MedGemma classifies each clinical claim and routes it to the correct FHIR-style evidence source (Observation/hemoglobin, MedicationRequest/med-2, DocumentReference/note-1). This enables clickable claim-to-source hyperlinks in the UI — every assertion in the handoff traces back to the specific lab value, medication order, or note line that supports it. Regex-based matching cannot handle the semantic flexibility of clinical language ("renal function worsening" → Observation/creatinine); MedGemma can.
Agent 3 — Evidence Companion. Identifies the top clinical issues from the Writer's output and retrieves external guidelines and literature. MedGemma summarizes the retrieved evidence in clinical context, producing short evidence summaries with source links for the most actionable findings.
Agent 4 — Orchestrator + Verifier. Combines outputs from all agents — the Writer's draft, the Provenance Router's citation map, a deterministic consistency check (note text vs. structured chart data), and external evidence — into the final I-PASS handoff brief. The Verifier enforces safety constraints: every claim must have a source citation or be marked uncertain, no prescription language is permitted, and the output must pass schema validation. If validation fails, MedGemma runs a repair pass.
Why MedGemma is the right model. MedGemma's medical domain training enables it to (1) interpret clinical significance rather than just detect numerical changes, (2) generate structured clinical language that fits handoff conventions, and (3) route clinical claims to the correct evidence types. Because MedGemma is open-weight, hospitals can run it on-premises or in a secured VPC — keeping PHI within institutional boundaries without sacrificing reasoning quality.
Stack. Python 3.10+, Google ADK (Agent Development Kit), MedGemma 27B text (deployed via Vertex AI Model Garden), FastAPI, React + TypeScript + Vite.
Modular agent pipeline. The orchestrator (orchestrator_agent.py) composes four MedGemma-powered agents as sequential ADK tool calls, each emitting auditable artifacts:
[1. Writer Agent] Load patient .pkl → MedGemma draft (I-PASS JSON)
↓
[2. Provenance Router] Draft + patient record → MedGemma FHIR citation mapping
↓
[3. Evidence Companion] Top issues → retrieve guidelines → MedGemma summary
↓
[4. Orchestrator] Merge all outputs + consistency check + verify → final I-PASS brief
Between agents, deterministic tools provide guardrails: the Signal Extractor runs TrendAnalyzer (linear regression on longitudinal lab/vital values with clinical thresholds) and ConflictDetector (rule-based medication-lab interaction checks per validated guidelines). The Consistency Checker compares note claims against structured data facts. These are not LLM calls — they are auditable, reproducible computations.
MedGemma integration. MedGemma 27B is deployed as a dedicated Vertex AI endpoint. The Writer Agent calls it via the Vertex Prediction API with structured chat formatting (<start_of_turn>user...model), temperature 0.1, and stop sequences to enforce JSON-only output. The same endpoint serves the Provenance Router (512-token responses for FHIR mapping) and the Verifier's repair pass. A self-repair loop detects prompt-echo or sparse outputs and triggers a second MedGemma call with the draft + patient data for revision.
Deployment modes. Environment toggles (MEDGEMMA_VERTEX_ENDPOINT, USE_HF_MEDGEMMA) allow the same codebase to run:
- Production: MedGemma 27B on a Vertex AI dedicated endpoint (or any secured VPC)
- Development: MedGemma via Hugging Face Transformers locally
No code changes required between modes. PHI never leaves the deployment boundary.
Evaluation harness. eval_mimic.py replays a 20-case MIMIC-IV cohort and logs: checklist coverage (I-PASS slot hits / 10), high-risk miss rate, evidence grounding rate (claims with source citations), runtime, and verifier pass/fail. Results are written to results.csv and results.summary.json for regression tracking. This doubles as a production QA harness — if a model or prompt change regresses coverage or grounding, it is caught immediately.
User-facing delivery. Clinicians access the handoff via three interfaces, all calling the same orchestrator:
- Dashboard (FastAPI + React): patient selector, I-PASS layout with clickable citations, role-based views (Resident vs. Nurse), external evidence panel, and chat for follow-up questions
- CLI (
run_handoff.py): for scripting and batch processing - ADK chat agent (
adk run handoff_agent): interactive clinical Q&A
Safety guardrails.
- Verifier gate blocks outputs with missing citations or unsupported statements
- Scope control: outputs are framed as documentation aids; conflicts use "verify/check" language, never prescriptive treatment instructions
- Full execution trace (step timings, model inputs/outputs) is logged per run for auditability
Deployment challenges.
- Latency: The full pipeline runs in ~37 seconds per patient. For shift handoffs covering 20 patients, this is acceptable (total < 15 min vs. hours of manual work). We are exploring parallel agent execution to reduce this further.
- Cost: MedGemma 27B on a dedicated Vertex endpoint costs approximately $3–5/hour. At ~37 seconds per patient, a 20-patient shift costs under $2 in compute — vs. ~$450 in clinician time saved.
- Reliability: The self-repair loop, deterministic fallbacks, and verifier gate ensure that even when MedGemma produces suboptimal output, the system degrades gracefully to clinically useful deterministic summaries rather than failing silently.