Discharge Summary Agent — README

Overview

An agentic AI system that reads patient source-note PDFs and produces structured, clinically safe discharge summary drafts for clinician review.

---

Quick Start

pip install pdfplumber
python main.py

Outputs appear in ./outputs/.

---

Agent Loop Design

The agent uses a Plan → Act → Observe → Re-plan loop:

PLAN (what do I need?)
  ↓
TOOL CALL (pdf_extract / drug_check / escalation / conflict_detect)
  ↓
OBSERVE (what did I get? is it complete?)
  ↓
RE-PLAN (what's still missing? any conflicts?)
  ↓
COMPILE (structured summary with all flags)

Hard Constraints

• MAX_STEPS = 30 — agent cannot run forever
• Every step emits: reasoning → action → inputs → result → next_plan
• Tools can fail — agent retries, falls back, or reports — never crashes

Tools

Tool	Purpose	Agent decides when
PDFExtractor	Extract text from PDFs (pdfplumber + OCR fallback)	Always first
DrugInteractionChecker	Check med interactions from curated DB	After medication extraction
EscalationTool	Log safety-critical items for clinician	When drug interaction/conflict found
ConflictDetector	Final pass for cross-field conflicts	Before compilation

---

No-Fabrication Guardrail

This is the core safety constraint. Implementation:

1. Every field initialised to MISSING_MARKER = "[MISSING — FLAG FOR CLINICIAN REVIEW]"
2. Fields are only overwritten when sourced from a document
3. Pending results: PENDING_MARKER = "[PENDING — RESULT NOT YET AVAILABLE]"
4. No inference, no "reasonable defaults", no hallucination
5. The guardrail is hard-coded — not part of the learnable policy (Part 2)

---

Failure & Conflict Handling

• Tool failure: caught in _step(), field left as MISSING, flag raised
• Missing data: explicit marker + flag for clinician
• Conflicting information: both values preserved, conflict logged, clinician flagged
  • Example (Patient 2): Admission record final dx ≠ consultation sheets dx
  • Example (Patient 2): ABG sodium (114) ≠ Biochemistry sodium (127) on same day
  • Example (Patient 2): Urine C/S (no bacteriuria) ≠ CT KUB (bilateral pyelonephritis)
• DAMA: both patients flagged as discharged against advice/on request
• Pending results: explicitly listed in Section 9

---

Part 2: Learning Loop

Reward Signal

reward = 0.4 × (1 - normalized_edit_distance) + 0.6 × section_accuracy

• normalized_edit_distance: Levenshtein distance / max_length ∈ [0,1]
• section_accuracy: fraction of sections with NED < 0.2
• Higher reward = less editing needed

Simulated Reviewer (Hidden Policy)

The SimulatedDoctorReviewer applies a consistent, hidden editing policy:

1. Replace vague dose entries ("Not specified") with standard phrasing
2. Truncate hospital course to concise clinical narrative (≤10 sentences)
3. Keep only top 5 critical flags
4. Add discharge sodium annotation if missing
5. Standardise DAMA documentation
6. Normalise dates to DD-MMM-YYYY
7. Append clinician sign-off line

Learning Mechanism: Correction-Memory Injection

• After each doctor edit, structured lessons are extracted
• Lessons stored in correction_memory.json
• Next agent run injects top-K lessons into planning context
• No GPU/fine-tuning required; immediately measurable; transparent

Before/After Metrics (from learning_metrics.json)

See outputs/learning_loop_report.txt for full curve.

Limitations

1. Cold Start Problem With 2 synthetic patients, the correction memory is sparse. First iterations have no lessons. Mitigation: Seed with curated clinician-reviewed corrections.

2. Gaming Risk An agent could lower edit distance by producing vaguer/shorter summaries. Mitigation: Section accuracy (0.6 weight) penalises missing/incomplete sections regardless of length.

3. Safety Preservation The correction memory only affects formatting/style — never the no-fabrication guardrail. Even after learning: MISSING markers are preserved, conflicts are still flagged, no lab value or medication dose is ever invented.

4. Distribution Shift Simulated doctor has a fixed policy. Real doctors vary. Mitigation in production: Per-clinician correction memories.

5. Limited Data 2 patients → minimal signal. Real improvement needs 50+ pairs. The mechanism is correct; data volume is the constraint.

---

File Structure

discharge_agent/
├── main.py                        # Entry point — runs all patients + learning loop
├── agents/
│   ├── discharge_agent.py         # Patient 2 agent (DKA + full chart)
│   ├── patient1_agent.py          # Patient 1 agent (GE + UTI)
│   ├── report_formatter.py        # Formats summaries, traces, JSON
│   └── learning_loop.py           # Part 2: simulated reviewer + correction memory
├── tools/
│   ├── pdf_extractor.py           # PDF text extraction (pdfplumber + OCR)
│   ├── drug_interaction.py        # Drug interaction checker (curated DB)
│   ├── escalation.py              # Escalation logger
│   └── conflict_detector.py       # Cross-field conflict detector
└── outputs/                       # All generated outputs
    ├── Patient_1_*_discharge_summary.txt
    ├── Patient_1_*_step_trace.txt
    ├── Patient_1_*_full_result.json
    ├── Patient_2_*_discharge_summary.txt
    ├── Patient_2_*_step_trace.txt
    ├── Patient_2_*_full_result.json
    ├── learning_loop_report.txt
    ├── learning_metrics.json
    └── correction_memory.json

---

What I Would Do With More Time

1. OCR pipeline: Integrate Tesseract/Google Vision for handwritten page extraction
2. LLM extraction layer: Use Claude API to extract structured fields from raw OCR text
3. Vector similarity: Use embeddings to match lessons to relevant sections (vs. keyword)
4. Multi-patient evaluation: Run on 20+ patients to show meaningful improvement curve
5. Per-section reward: Track which sections improve fastest to focus learning
6. DPO fine-tuning: Collect 100+ (draft, edited) pairs and fine-tune a small model
7. Clinician UI: Web interface for reviewing, editing, and auto-submitting corrections back
8. Real drug interaction API: Integrate DrugBank/OpenFDA instead of curated mock DB

---

Clinical Safety Notes

• All output is explicitly marked DRAFT FOR CLINICIAN REVIEW
• No output should be used as a finalised clinical document without clinician sign-off
• DAMA (Discharge Against Medical Advice) events are prominently flagged
• Pending results are listed separately so clinicians cannot miss them
• Conflicting information is preserved — never silently resolved

---

Built for the Dscribe AI Engineer Take-Home Assignment. All patient data is synthetic.