diff --git a/docs/adr/ADR-003-on-device-tflite-inference.md b/docs/adr/ADR-003-on-device-tflite-inference.md new file mode 100644 index 0000000..aa98f3a --- /dev/null +++ b/docs/adr/ADR-003-on-device-tflite-inference.md @@ -0,0 +1,137 @@ +# ADR-003 — On-Device TFLite Inference for Sepsis Risk Detection + +| Field | Value | +|---|---| +| **ADR ID** | ADR-003 | +| **Title** | On-Device TFLite Inference for Sepsis Risk Detection | +| **Status** | Accepted | +| **Date** | 2026-05-13 | +| **Deciders** | MedTech R&D, Systems Architect, Clinical Informatics Lead | +| **Affected Repos** | `medtech-edge-analytics`, `medtech-device-os` | + +--- + +## Context + +The MedTech sepsis detection system must continuously evaluate incoming vitals and produce a risk score. The fundamental architectural question is: **where should inference execute?** + +Three options were considered: + +### Option A: Cloud Inference + +Vitals are forwarded to a cloud-hosted ML inference endpoint (e.g., AWS SageMaker, Azure ML, or a custom FastAPI inference service). The risk score is returned to the device and displayed on the clinician dashboard. + +### Option B: On-Device Full TensorFlow Inference + +The full TensorFlow (non-Lite) runtime is deployed on the edge device. The model is loaded and inference runs locally without any cloud dependency. + +### Option C: On-Device TensorFlow Lite (TFLite) Inference + +A quantized TFLite model is deployed on the edge device. Inference runs locally using the TFLite interpreter, which is optimized for ARM architecture and runs within a minimal memory footprint. + +--- + +## Decision + +**On-device TensorFlow Lite inference (Option C) is selected.** + +The sepsis risk model is compiled to `.tflite` format, packaged as `/app/models/sepsis_model.tflite`, and invoked by the `medtech-edge-analytics` service using the TFLite Python interpreter. Inference never leaves the device boundary. Results are published locally over MQTT. The cloud backend receives prediction events asynchronously, but the alarm path is entirely local. + +--- + +## Rationale + +### 1. HIPAA Privacy-by-Design + +Patient vitals are Protected Health Information (PHI). Transmitting vitals to a cloud inference endpoint over a potentially shared hospital network — even encrypted — creates a data-at-rest risk on cloud infrastructure, a de-identification failure risk if cloud logging captures payloads, and a HIPAA BAA obligation with every cloud provider in the inference path. + +On-device inference eliminates all three risks architecturally. No vitals leave the device boundary. Privacy is not a policy control or a contractual guarantee — it is a physical constraint. + +> **Note:** The current platform uses entirely synthetic data. In a production deployment, this architectural decision becomes a patient-safety and regulatory compliance control, not merely a design preference. + +### 2. Deterministic Sub-100ms Latency + +IEC 60601-1-8 §6.3 requires that physiological alarm conditions be presented to the alarm system within a bounded time from the triggering condition. Cloud inference introduces three latency variables that cannot be deterministically bounded: network round-trip time, cloud endpoint queue depth, and cold-start latency. + +TFLite inference on ARM64 (NXP i.MX8MP class hardware) completes within 10–40 ms under validated test conditions. End-to-end (vitals received → risk score published) is consistently < 100 ms. This bound is enforceable and testable in CI using QEMU ARM64 emulation. + +### 3. Offline Resilience + +Clinical networks are unreliable. Planned maintenance windows, VLAN misconfigurations, and network hardware failures are routine. A cloud-inference architecture silences the sepsis alarm system during every network outage. On-device inference means the alarm system continues to function in complete network isolation — even air-gapped environments. + +This directly addresses the ISO 14971 hazard: **"cloud network unavailability causes missed sepsis alarm."** On-device inference eliminates this hazard class entirely rather than mitigating it. + +### 4. TFLite vs. Full TensorFlow + +Full TensorFlow cannot run within the memory budget of a constrained ARM64 device (target: < 256 MB RSS for the full `medtech-edge-analytics` service). TFLite's quantized INT8 model format reduces the sepsis model size by approximately 4× versus FP32, enabling it to run within Yocto image constraints without a dedicated ML accelerator. TFLite also provides ARM-optimized XNNPACK delegate support for NEON SIMD acceleration on i.MX8MP. + +### 5. Model Swappability + +The `MODEL_PATH` environment variable enables model updates without code changes. A new `.tflite` artifact can be deployed via OTA update to the Yocto image, passing the same CI validation gates, without modifying the inference service binary. This is aligned with the FDA's Predetermined Change Control Plan framework for AI/ML-based SaMD. + +--- + +## Consequences + +### Positive + +- Zero PHI leaves the device during inference (privacy-by-design, not policy-dependent) +- Deterministic < 100 ms inference latency, bounded and testable in CI +- Alarm system operates fully offline — zero dependency on cloud availability +- Model updates via `MODEL_PATH` without code change +- ARM-optimized TFLite XNNPACK delegate provides NEON SIMD acceleration on production hardware +- Small footprint enables deployment within Yocto image memory budget + +### Negative + +- Model accuracy is bounded by the training data and the quantization loss from FP32 → INT8 conversion (~1–2% accuracy reduction, acceptable for screening use case) +- Model retraining and deployment requires a formal OTA workflow (planned for v3.x) +- TFLite does not support all TensorFlow ops natively; model architecture is constrained to ops in the TFLite op subset + +### Neutral + +- The cloud backend (`medtech-telemetry-cloud`) receives prediction events asynchronously for population analytics; it is not in the alarm path + +--- + +## Alternatives Considered + +| Alternative | Reason Rejected | +|---|---| +| Cloud inference (AWS SageMaker / custom endpoint) | Network-latency non-determinism incompatible with IEC 60601-1-8 alarm timing; PHI transmission risk; offline failure mode unacceptable | +| Full TensorFlow on-device | Memory footprint exceeds Yocto image budget on constrained ARM64; startup time excessive | +| ONNX Runtime on-device | Viable alternative, but TFLite provides better ARM64 optimization and broader MedTech edge deployment precedent | +| Rule-based threshold alerting (no ML) | Insufficient sensitivity for early sepsis onset; misses multi-variate patterns not captured by simple thresholds; not aligned with SOFA/NEWS2 scoring clinical evidence base | + +--- + +## Model Governance Considerations + +| Governance Area | Current State | Production Requirement | +|---|---|---| +| Model training data | Synthetic (Synthea-modeled) | Retrospective clinical data with IRB approval | +| Model validation | CI regression tests on labeled fixtures | Clinical validation study (sensitivity/specificity on holdout set) | +| Model version tracking | Logged at startup; `model_latency_ms` in every payload | `model_version` field in every prediction payload; fleet version audit | +| Model update mechanism | Manual `MODEL_PATH` update + image rebuild | OTA signed model artifact delivery | +| Regulatory classification | Not submitted | FDA 510(k) or De Novo for AI/ML-based SaMD (Class II) | + +--- + +## Standards References + +| Standard | Relationship to This Decision | +|---|---| +| **IEC 60601-1-8:2006+AMD1:2012 §6.3** | Alarm response time requirement drives the < 100 ms inference latency target. On-device inference is the only architecture that satisfies this deterministically. | +| **HIPAA §164.312 (Technical Safeguards)** | On-device inference eliminates the transmission security and access control obligations associated with cloud PHI processing. | +| **ISO 14971:2019 §7** | On-device inference eliminates the hazard "cloud unavailability causes missed sepsis alarm." This is a hazard elimination, not a risk mitigation — the highest-priority risk control. | +| **FDA AI/ML-Based SaMD Action Plan (2021)** | `model_latency_ms` in every prediction payload provides the monitoring data stream required by the Predetermined Change Control Plan. Model version logging at startup supports the audit trail. | +| **IEC 62304:2015 §5.5** | `MODEL_PATH` configurability constitutes a software configurable item (SCI) subject to change control. TFLite runtime version is a software item under configuration management. | + +--- + +## Review Date + +This decision should be revisited if: +- NXP i.MX8MP NPU (Neural Processing Unit) drivers become stable in Yocto — enabling hardware-accelerated inference that may warrant a runtime switch from TFLite CPU to NPU delegate +- The model complexity grows beyond the TFLite op subset (e.g., transformer architectures), requiring ONNX Runtime or full TF +- A federated learning requirement is introduced, necessitating a cloud-coordinated inference architecture diff --git a/docs/prd/PRD-003-edge-analytics.md b/docs/prd/PRD-003-edge-analytics.md new file mode 100644 index 0000000..b4be1f6 --- /dev/null +++ b/docs/prd/PRD-003-edge-analytics.md @@ -0,0 +1,204 @@ +# PRD-003 — MedTech Edge Analytics (On-Device Sepsis Inference) + +| Field | Value | +|---|---| +| **Document ID** | PRD-003 | +| **Product** | MedTech Edge Analytics | +| **Repo** | `chaithubk/medtech-edge-analytics` | +| **Author** | MedTech R&D | +| **Status** | Active | +| **Service Version** | 2.2.0 | +| **Last Updated** | 2026-05-13 | + +> **Zero PHI Declaration:** All inference inputs are synthetic vitals generated by the MedTech Vitals Publisher using Synthea-modeled profiles. No real patient data, PHI, or PII is processed at any point. This service is an educational R&D prototype only. + +--- + +## 1. Opportunity + +Sepsis is a time-critical condition. Every hour of delayed treatment increases mortality by approximately 7%. Traditional hospital workflows rely on clinicians to manually recognize early sepsis indicators — rising lactate, falling SpO₂, tachycardia — from multiple disconnected monitoring systems. This cognitive load, combined with the reality of busy ICU and ward environments, creates a window of preventable harm. + +**Automated on-device inference** closes this gap. By running a trained TensorFlow Lite sepsis risk model directly on the bedside device — without a round-trip to the cloud — the system can: +1. Produce a risk score within 15-minute vitals windows +2. Trigger an alarm condition in < 100 ms from inference invocation +3. Operate fully offline (air-gapped ward, network failure, cloud outage) +4. Protect patient data from leaving the device (HIPAA privacy-by-design) + +The **MedTech Edge Analytics** service is that on-device inference engine. It subscribes to the MQTT vitals topic, accumulates a feature window, invokes the TFLite model, and publishes a structured prediction payload on `medtech/predictions/sepsis`. Downstream consumers — the clinician dashboard and the cloud backend — act on this prediction without needing access to the raw vitals. + +### Business & Clinical Value + +| Outcome | Metric | +|---|---| +| Earlier sepsis detection | Target: alarm triggered ≥ 1 hour before clinical deterioration threshold | +| Reduced ICU readmissions | Industry benchmark: 20–30% reduction with automated EWS | +| Offline capability | 100% availability during network partition | +| Privacy-by-design | Zero cloud PHI exposure for inference | +| Deterministic latency | < 100 ms inference regardless of network state | + +--- + +## 2. Target Audience + +### Primary Users (Indirect — Clinical) + +| Persona | Need | +|---|---| +| **Bedside Nurse / Rapid Response Team** | A reliable, timely sepsis risk alert on the clinical dashboard without needing to manually interpret vitals trends | +| **ICU Attending Physician** | A risk score with confidence level and model latency to calibrate trust in the automated alert | + +### Primary Users (Direct — Technical) + +| Persona | Need | +|---|---| +| **Edge Analytics Engineer** | A lightweight, deployable TFLite inference service that runs within the Yocto device OS memory budget (< 256 MB RSS) | +| **ML Engineer** | A model-swappable architecture: update `MODEL_PATH` to deploy a retrained `.tflite` without code changes | +| **QA / Validation Engineer** | A deterministic inference pipeline that produces bit-identical outputs for identical inputs across device builds | + +### Secondary Users + +| Persona | Need | +|---|---| +| **Regulatory Affairs** | Evidence that model inference is contained on-device, model version is tracked, and latency is deterministically bounded | +| **Clinical Informatics** | Prediction payload consumable by HL7 FHIR Clinical Decision Support (CDS) Hooks | + +--- + +## 3. Product Vision + +> Deliver a deterministic, offline-capable, sub-100ms sepsis risk inference engine that runs entirely on the bedside device using TensorFlow Lite, so that clinicians receive timely, privacy-preserving early-warning alerts regardless of network state. + +--- + +## 4. Success Metrics + +| Metric | Target | Measurement Method | +|---|---|---| +| Model inference latency (p99) | **< 100 ms** | Timed integration test on QEMU ARM64 | +| Prediction publish latency (end-to-end from vitals receipt) | **< 500 ms** | Integration test timestamp delta | +| Sepsis sensitivity (true positive rate on `sepsis_onset` scenario) | **> 90%** | CI regression test vs labeled fixtures | +| Healthy specificity (true negative rate on `healthy` scenario) | **> 95%** | CI regression test vs labeled fixtures | +| Service memory footprint | **< 256 MB RSS** | Docker stats / QEMU process monitor | +| Schema compliance of prediction payloads | **100%** | Runtime validator + CI | +| Service uptime | **≥ 99.9%** in 24-hour integration test | Compose healthcheck + test harness | + +--- + +## 5. Scope + +### In Scope (v2.x) + +- MQTT subscriber on topic `medtech/vitals/latest` +- Sliding vitals window (configurable, default: 15-minute equivalent at 10s publish interval = 90 samples) +- TFLite model inference using `MODEL_PATH` (default: `/app/models/sepsis_model.tflite`) +- Prediction payload publication on `medtech/predictions/sepsis` +- Runtime schema validation of incoming vitals against vendored contract +- Contract vendoring from `medtech-telemetry-contract` +- GHCR image publishing with pinned release tags + +### Out of Scope + +- Model training (model is pre-trained and packaged as a `.tflite` artifact) +- Real-time waveform data (operates on spot-check vitals at publish interval) +- Multi-patient concurrent inference (single patient stream per container) +- Cloud feedback loop (model updates are offline; see `medtech-device-os` OTA roadmap) + +--- + +## 6. Functional Requirements + +### FR-001: Vitals Subscription + +The service MUST subscribe to `medtech/vitals/latest` via MQTT on startup and validate each received payload against the vendored telemetry contract schema before adding it to the inference window. + +### FR-002: Feature Window Management + +The service MUST maintain a rolling vitals window (minimum 1 sample, default window configurable via `WINDOW_SIZE`). Inference MUST be invoked on every new vitals sample after the window is populated. + +### FR-003: TFLite Inference + +The service MUST load `MODEL_PATH` at startup, fail fast if the model file is missing or corrupt, and invoke inference with < 100 ms wall-clock time on the target ARM64 hardware profile. + +### FR-004: Prediction Payload Publication + +Every inference run MUST publish a schema-compliant payload on `medtech/predictions/sepsis`: +```json +{ + "timestamp": "", + "risk_score": , + "risk_level": "", + "confidence": , + "model_latency_ms": +} +``` + +### FR-005: Risk Level Thresholds + +Risk level MUST be derived from risk_score according to: +- `low`: score < 0.30 +- `medium`: score 0.30–0.59 +- `high`: score 0.60–0.84 +- `critical`: score ≥ 0.85 + +### FR-006: Contract Vendoring + +The service MUST vendor the telemetry contract at build time and validate incoming vitals at runtime. A CI drift-check workflow MUST open a PR when the upstream contract releases a new version. + +--- + +## 7. Non-Functional Requirements + +| ID | Requirement | Standard Reference | +|---|---|---| +| NFR-001 | Inference latency MUST be deterministically bounded at < 100 ms (p99) | IEC 60601-1-8 §6.3 (alarm response time) | +| NFR-002 | Service MUST operate fully offline — no outbound network calls during inference | HIPAA §164.312 (access control, transmission security) | +| NFR-003 | Model version MUST be logged at startup and included in prediction metadata | ISO 14971 §10 (test and verification traceability) | +| NFR-004 | Memory footprint MUST stay within device OS resource budget (< 256 MB RSS) | Yocto/QEMU NXP i.MX8MP memory allocation | +| NFR-005 | Service MUST fail fast on missing schema or model file, not silently degrade | IEC 62304 §5.5 (software safety requirements) | +| NFR-006 | Prediction `confidence` field MUST be populated to support alarm suppression logic | IEC 60601-1-8 alarm condition management | + +--- + +## 8. Regulatory & Standards Alignment + +| Standard | Relevance to This Product | +|---|---| +| **IEC 60601-1-8:2006+AMD1:2012** | This service is the physiological alarm source. The `risk_level` field maps directly to alarm priority categories (low/medium/high/critical → advisory/caution/warning/crisis). Inference latency bounds are driven by §6.3 alarm response time requirements. | +| **HL7 FHIR R4 — ClinicalImpression / RiskAssessment** | The prediction payload (`risk_score`, `risk_level`, `confidence`) is structurally aligned with the FHIR `RiskAssessment` resource to enable downstream EHR integration via CDS Hooks. | +| **ISO 14971:2019 §7 (Risk Control)** | On-device inference is the primary risk control for the hazard "delayed sepsis recognition." The `confidence` field is a secondary risk control mechanism (enables alarm suppression below a confidence threshold to reduce nuisance alarms). | +| **HIPAA § 164.312** | Inference executes entirely on-device. No vitals or predictions leave the device network boundary during inference. Privacy is architectural, not policy-dependent. | +| **IEC 62304:2015 §5.5** | The model file path, window size, and MQTT topics are software configurable items (SCIs) subject to change control. Updates require CI validation gate passage. | +| **FDA AI/ML-Based SaMD Action Plan (2021)** | `model_latency_ms` in every prediction payload constitutes performance monitoring data. Model version logging at startup constitutes the audit trail required by the Predetermined Change Control Plan framework. | + +--- + +## 9. Risks & Mitigations (ISO 14971 Format) + +| Risk | Likelihood | Severity | Risk Control | +|---|---|---|---| +| False negative: high-risk patient scored `low` | Low | Critical (patient harm) | Regression test suite against labeled `sepsis_onset` fixture; > 90% sensitivity gate | +| False positive: healthy patient alarms `critical` | Medium | Medium (alarm fatigue, care disruption) | Specificity gate ≥ 95% on `healthy` fixture in CI | +| Model file corrupted or missing at runtime | Low | High (no inference) | Fail-fast startup check; systemd `Restart=on-failure` in device OS | +| Inference latency spike under memory pressure | Low | High (missed time-critical alarm) | p99 latency test in CI on QEMU ARM64; resource limit in Docker compose | +| Contract drift causing vitals parse failure | Medium | High (inference starved silently) | Runtime schema validator; drift-check CI workflow | + +--- + +## 10. Dependencies + +| Dependency | Repo | Note | +|---|---|---| +| Telemetry Contract | `medtech-telemetry-contract` | Vitals schema vendored at build time | +| Vitals Publisher | `medtech-vitals-publisher` | MQTT source of vitals; must be `service_healthy` before this service starts | +| Device OS | `medtech-device-os` | Deploys this service as a systemd unit on the Yocto image | +| Platform | `medtech-platform` | Pinned image tag in `docker-compose.yml` | +| Clinician UI | `medtech-clinician-ui` | Subscribes to prediction topic output by this service | +| Telemetry Cloud | `medtech-telemetry-cloud` | Bridges prediction MQTT → cloud ingestion | + +--- + +## 11. Open Questions + +1. Should `model_version` be included in every prediction payload (not just logged at startup) to support multi-model fleet management? +2. Should the risk thresholds (low/medium/high/critical) be externally configurable to support hospital-specific alarm tuning without code changes? +3. Should the service support a `EXPLAIN_MODE` that emits top-feature contributions alongside the risk score for clinical interpretability?