MedHandoff-AI/Writeup.html at main · vivianpengdev/MedHandoff-AI · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="pandoc" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
  <title>MedHandoff</title>
  <style>

html {
color: #1a1a1a;
background-color: #fdfdfd;
}
body {
margin: 0 auto;
max-width: 36em;
padding-left: 50px;
padding-right: 50px;
padding-top: 50px;
padding-bottom: 50px;
hyphens: auto;
overflow-wrap: break-word;
text-rendering: optimizeLegibility;
font-kerning: normal;
}
@media (max-width: 600px) {
body {
font-size: 0.9em;
padding: 12px;
}
h1 {
font-size: 1.8em;
}
}
@media print {
html {
background-color: white;
}
body {
background-color: transparent;
color: black;
font-size: 12pt;
}
p, h2, h3 {
orphans: 3;
widows: 3;
}
h2, h3, h4 {
page-break-after: avoid;
}
}
p {
margin: 1em 0;
}
a {
color: #1a1a1a;
}
a:visited {
color: #1a1a1a;
}
img {
max-width: 100%;
}
svg {
height: auto;
max-width: 100%;
}
h1, h2, h3, h4, h5, h6 {
margin-top: 1.4em;
}
h5, h6 {
font-size: 1em;
font-style: italic;
}
h6 {
font-weight: normal;
}
ol, ul {
padding-left: 1.7em;
margin-top: 1em;
}
li > ol, li > ul {
margin-top: 0;
}
blockquote {
margin: 1em 0 1em 1.7em;
padding-left: 1em;
border-left: 2px solid #e6e6e6;
color: #606060;
}
code {
font-family: Menlo, Monaco, Consolas, 'Lucida Console', monospace;
font-size: 85%;
margin: 0;
hyphens: manual;
}
pre {
margin: 1em 0;
overflow: auto;
}
pre code {
padding: 0;
overflow: visible;
overflow-wrap: normal;
}
.sourceCode {
background-color: transparent;
overflow: visible;
}
hr {
border: none;
border-top: 1px solid #1a1a1a;
height: 1px;
margin: 1em 0;
}
table {
margin: 1em 0;
border-collapse: collapse;
width: 100%;
overflow-x: auto;
display: block;
font-variant-numeric: lining-nums tabular-nums;
}
table caption {
margin-bottom: 0.75em;
}
tbody {
margin-top: 0.5em;
border-top: 1px solid #1a1a1a;
border-bottom: 1px solid #1a1a1a;
}
th {
border-top: 1px solid #1a1a1a;
padding: 0.25em 0.5em 0.25em 0.5em;
}
td {
padding: 0.125em 0.5em 0.25em 0.5em;
}
header {
margin-bottom: 4em;
text-align: center;
}
#TOC li {
list-style: none;
}
#TOC ul {
padding-left: 1.3em;
}
#TOC > ul {
padding-left: 0;
}
#TOC a:not(:hover) {
text-decoration: none;
}
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: min(4vw, 1.5em);}
div.column{flex: auto; overflow-x: auto;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}

ul.task-list[class]{list-style: none;}
ul.task-list li input[type="checkbox"] {
font-size: inherit;
width: 0.8em;
margin: 0 0.8em 0.2em -1.6em;
vertical-align: middle;
}
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
</style>
</head>
<body>
<header id="title-block-header">
<h1 class="title">MedHandoff</h1>
</header>
<h3 id="project-name">Project name</h3>
<p><strong>MedHandoff AI</strong> — Agentic Clinical Signout Powered by
MedGemma (Tracks: Main Track + Agentic Workflow Prize)</p>
<h3 id="our-team">Our Team</h3>
<ul>
<li><strong>Olivia Rutler</strong> — Medical student in clinical
clerkships. Defines real-world handoff constraints and validates that
the workflow fits how clinical teams actually operate.</li>
<li><strong>Vivian Peng</strong> — Product-minded software engineer.
Architects the multi-agent system and delivers the full-stack
clinician-facing application.</li>
<li><strong>Leo Jin</strong> — PhD candidate (Columbia, Neurobiology).
Leads dataset curation, model evaluation, and reproducibility.</li>
</ul>
<h3 id="problem-statement">Problem statement</h3>
<p><strong>The problem.</strong> Clinical handoffs — when one physician
ends a shift and another takes over — are a leading source of
preventable harm. Communication failures during handoffs contribute to
roughly 80% of serious medical errors. Structured interventions like
I-PASS have been shown to reduce errors by 30%, yet handoffs today
remain largely manual and inconsistent.</p>
<p>At every shift change, the receiving resident must reconstruct
complex clinical narratives from fragmented EHR data: notes, labs,
vitals, medications, imaging, and orders. This takes 15–30 minutes per
patient and requires extracting trends (“what changed?”), resolving
inconsistencies (“stable” note vs. worsening labs), and identifying
action items. Errors and omissions are common, and the time spent on
synthesis is time not spent with patients.</p>
<p><strong>The user.</strong> Our primary user is the resident receiving
a night or cross-cover handoff. Their current workflow:</p>
<blockquote>
<p>signout → manual chart review → mental synthesis → risk of missed
information</p>
</blockquote>
<p>MedHandoff changes this to:</p>
<blockquote>
<p>signout → one click → structured I-PASS brief with trends, conflicts,
action items, contingencies, and linked source evidence</p>
</blockquote>
<p>This shifts effort from information retrieval to verification and
clinical decision-making.</p>
<p><strong>Why AI is required.</strong> This task involves synthesizing
high-dimensional, longitudinal clinical data with temporal reasoning and
cross-source consistency checks. Rule-based systems cannot generalize
across heterogeneous records, and generic summarization models lack
grounding. Our agentic architecture combines deterministic signal
extraction with MedGemma-based clinical reasoning. Every claim is
grounded to source data.</p>
<p><strong>Impact.</strong> We evaluated a 20-case MIMIC-IV cohort:</p>
<table>
<thead>
<tr>
<th>Metric</th>
<th>Baseline (human notes)</th>
<th>MedHandoff</th>
</tr>
</thead>
<tbody>
<tr>
<td>I-PASS checklist coverage</td>
<td>0%</td>
<td>50%</td>
</tr>
<tr>
<td>High-risk misses / patient</td>
<td>2.0</td>
<td>1.0</td>
</tr>
<tr>
<td>Evidence grounding</td>
<td>—</td>
<td>100%</td>
</tr>
<tr>
<td>Time saved / patient</td>
<td>—</td>
<td>~16 min</td>
</tr>
</tbody>
</table>
<p>For a 20-patient shift, this returns ~5 hours of clinician time
(~$450/shift at $90/hr blended cost), excluding downstream safety
benefits from fewer missed deteriorations.</p>
<h3 id="overall-solution">Overall solution</h3>
<p>We use <strong>MedGemma 27B (text)</strong> through the Google Agent
Development Kit (ADK) as the clinical reasoning backbone across a
four-agent modular pipeline. Each agent receives a focused task, calls
MedGemma independently, and passes structured output to the next stage.
MedGemma is used exactly where it adds value beyond deterministic logic
— clinical interpretation, evidence routing, and knowledge
synthesis.</p>
<p><strong>Agent 1 — Writer Agent.</strong> Loads patient data (labs,
vitals, medications, notes, history) and passes it to MedGemma with a
structured I-PASS prompt. MedGemma interprets multi-signal clinical
context — for example, recognizing that hemoglobin dropping while a
patient is on anticoagulants raises a bleeding concern, not just an
anemia flag — and produces a structured draft: illness severity, patient
summary, trend narrative with specific values, clinical “why” analysis,
and prioritized action items. Rule-based templates can list values;
MedGemma can reason about their clinical significance and generate
fluent, handoff-ready language. A deterministic signal extractor (trend
analyzer + conflict detector) provides pre-computed lab/vital trends and
medication-lab conflicts as advisory input, but MedGemma decides how to
weight and present them.</p>
<p><strong>Agent 2 — Provenance Router.</strong> Takes the Writer’s
draft and the same patient record. MedGemma classifies each clinical
claim and routes it to the correct FHIR-style evidence source
(Observation/hemoglobin, MedicationRequest/med-2,
DocumentReference/note-1). This enables clickable claim-to-source
hyperlinks in the UI — every assertion in the handoff traces back to the
specific lab value, medication order, or note line that supports it.
Regex-based matching cannot handle the semantic flexibility of clinical
language (“renal function worsening” → Observation/creatinine); MedGemma
can.</p>
<p><strong>Agent 3 — Evidence Companion.</strong> Identifies the top
clinical issues from the Writer’s output and retrieves external
guidelines and literature. MedGemma summarizes the retrieved evidence in
clinical context, producing short evidence summaries with source links
for the most actionable findings.</p>
<p><strong>Agent 4 — Orchestrator + Verifier.</strong> Combines outputs
from all agents — the Writer’s draft, the Provenance Router’s citation
map, a deterministic consistency check (note text vs. structured chart
data), and external evidence — into the final I-PASS handoff brief. The
Verifier enforces safety constraints: every claim must have a source
citation or be marked uncertain, no prescription language is permitted,
and the output must pass schema validation. If validation fails,
MedGemma runs a repair pass.</p>
<p><strong>Why MedGemma is the right model.</strong> MedGemma’s medical
domain training enables it to (1) interpret clinical significance rather
than just detect numerical changes, (2) generate structured clinical
language that fits handoff conventions, and (3) route clinical claims to
the correct evidence types. Because MedGemma is open-weight, hospitals
can run it on-premises or in a secured VPC — keeping PHI within
institutional boundaries without sacrificing reasoning quality.</p>
<h3 id="technical-details">Technical details</h3>
<p><strong>Stack.</strong> Python 3.10+, Google ADK (Agent Development
Kit), MedGemma 27B text (deployed via Vertex AI Model Garden), FastAPI,
React + TypeScript + Vite.</p>
<p><strong>Modular agent pipeline.</strong> The orchestrator
(<code>orchestrator_agent.py</code>) composes four MedGemma-powered
agents as sequential ADK tool calls, each emitting auditable
artifacts:</p>
<pre><code>[1. Writer Agent]       Load patient .pkl → MedGemma draft (I-PASS JSON)
        ↓
[2. Provenance Router]  Draft + patient record → MedGemma FHIR citation mapping
        ↓
[3. Evidence Companion] Top issues → retrieve guidelines → MedGemma summary
        ↓
[4. Orchestrator]       Merge all outputs + consistency check + verify → final I-PASS brief</code></pre>
<p>Between agents, deterministic tools provide guardrails: the Signal
Extractor runs TrendAnalyzer (linear regression on longitudinal
lab/vital values with clinical thresholds) and ConflictDetector
(rule-based medication-lab interaction checks per validated guidelines).
The Consistency Checker compares note claims against structured data
facts. These are not LLM calls — they are auditable, reproducible
computations.</p>
<p><strong>MedGemma integration.</strong> MedGemma 27B is deployed as a
dedicated Vertex AI endpoint. The Writer Agent calls it via the Vertex
Prediction API with structured chat formatting
(<code>&lt;start_of_turn&gt;user...model</code>), temperature 0.1, and
stop sequences to enforce JSON-only output. The same endpoint serves the
Provenance Router (512-token responses for FHIR mapping) and the
Verifier’s repair pass. A self-repair loop detects prompt-echo or sparse
outputs and triggers a second MedGemma call with the draft + patient
data for revision.</p>
<p><strong>Deployment modes.</strong> Environment toggles
(<code>MEDGEMMA_VERTEX_ENDPOINT</code>, <code>USE_HF_MEDGEMMA</code>)
allow the same codebase to run: - <strong>Production:</strong> MedGemma
27B on a Vertex AI dedicated endpoint (or any secured VPC) -
<strong>Development:</strong> MedGemma via Hugging Face Transformers
locally</p>
<p>No code changes required between modes. PHI never leaves the
deployment boundary.</p>
<p><strong>Evaluation harness.</strong> <code>eval_mimic.py</code>
replays a 20-case MIMIC-IV cohort and logs: checklist coverage (I-PASS
slot hits / 10), high-risk miss rate, evidence grounding rate (claims
with source citations), runtime, and verifier pass/fail. Results are
written to <code>results.csv</code> and
<code>results.summary.json</code> for regression tracking. This doubles
as a production QA harness — if a model or prompt change regresses
coverage or grounding, it is caught immediately.</p>
<p><strong>User-facing delivery.</strong> Clinicians access the handoff
via three interfaces, all calling the same orchestrator: -
<strong>Dashboard</strong> (FastAPI + React): patient selector, I-PASS
layout with clickable citations, role-based views (Resident vs. Nurse),
external evidence panel, and chat for follow-up questions -
<strong>CLI</strong> (<code>run_handoff.py</code>): for scripting and
batch processing - <strong>ADK chat agent</strong>
(<code>adk run handoff_agent</code>): interactive clinical Q&amp;A</p>
<p><strong>Safety guardrails.</strong> - Verifier gate blocks outputs
with missing citations or unsupported statements - Scope control:
outputs are framed as documentation aids; conflicts use “verify/check”
language, never prescriptive treatment instructions - Full execution
trace (step timings, model inputs/outputs) is logged per run for
auditability</p>
<p><strong>Deployment challenges.</strong> - <em>Latency:</em> The full
pipeline runs in ~37 seconds per patient. For shift handoffs covering 20
patients, this is acceptable (total &lt; 15 min vs. hours of manual
work). We are exploring parallel agent execution to reduce this further.
- <em>Cost:</em> MedGemma 27B on a dedicated Vertex endpoint costs
approximately $3–5/hour. At ~37 seconds per patient, a 20-patient shift
costs under $2 in compute — vs. ~$450 in clinician time saved. -
<em>Reliability:</em> The self-repair loop, deterministic fallbacks, and
verifier gate ensure that even when MedGemma produces suboptimal output,
the system degrades gracefully to clinically useful deterministic
summaries rather than failing silently.</p>
</body>
</html>