Graduation Project — Mansoura University An AI-powered healthcare ecosystem seamlessly connecting patients, doctors, secretaries, and hospital managers.
medAI is a comprehensive, production-grade medical platform designed to streamline clinical and administrative hospital workflows. The system integrates advanced AI capabilities, including:
- Egyptian Arabic Medical Voice-to-Report Transcription: Tailored pipeline utilizing fine-tuned Whisper and Llama-3 models.
- AI Lab Report Analysis: Document text parsing, OCR, and medical summarization.
- Cross-Platform Mobile App: A dedicated Kotlin Multiplatform (Compose Multiplatform) client targeting Android & iOS for patients and doctors.
- Role-Based Web Dashboard: Dashboards for patients, doctors, secretaries, and managers built with Next.js.hboards.
| Layer | Technology |
|---|---|
| Frontend | Next.js 16 (React 19), Tailwind CSS v4, Radix UI, TanStack Query v5, react-hook-form + Zod v4, framer-motion, date-fns |
| Backend | NestJS 11 (Node.js), TypeScript, TypeORM, PostgreSQL 16 |
| Auth | Passport.js (Local + JWT), Argon2 hashing, httpOnly cookies, email verification |
| Queue | BullMQ v5 (Redis-backed async job processing) |
| File Storage | Azure Blob Storage (avatars, scan images, reports, audio) |
| Brevo SMTP via Nodemailer | |
| AI | Fine-tuned Whisper Large-V3 + Llama-3 8B (4-bit) for Egyptian Arabic medical voice-to-report |
| Mobile | Kotlin Multiplatform (Compose Multiplatform) — Android + iOS |
| Infra | Docker Compose (local/staging/prod), Terraform, GitHub Actions CI/CD |
┌─────────────┐ ┌──────────────────────────────────────┐ ┌───────────┐
│ Next.js 16 │────▶│ NestJS 11 API │────▶│ PostgreSQL│
│ (Frontend) │ │ ┌─────┐ ┌──────┐ ┌──────────────┐ │ └───────────┘
└─────────────┘ │ │Auth │ │Users│ │ Patients │ │
│ ├─────┤ ├──────┤ ├──────────────┤ │ ┌───────────┐
┌─────────────┐ │ │Docs │ │Sched │ │Appointments │ │────▶│ Redis │
│ Kotlin App │────▶│ ├─────┤ ├──────┤ ├──────────────┤ │ └───────────┘
│ (Mobile) │ │ │Scans│ │Report│ │Voice Reports │ │ ┌───────────┐
└─────────────┘ │ ├─────┤ └──────┘ └──────────────┘ │────▶│Azure Blob │
│ │Diagnosis │ BullMQ Workers │ └───────────┘
│ │ (AI Pipe)│ ◀──────────────────────│────▶│ AI Server│
└──────────────────────────────────────┘ └───────────┘
- Email/password registration with role selection (patient, doctor, secretary, manager)
- JWT authentication (access + refresh tokens as httpOnly cookies, auto-rotation)
- Email verification (token-based, Brevo SMTP)
- Forgot/reset password flow
- Email change with confirmation
- Profile management: name, phone, avatar upload (Azure Blob), bio, gender, birth date
Four roles with separate dashboards and permissions:
| Role | Capabilities |
|---|---|
| Patient | Book appointments, view medical records, upload scans, view diagnoses, track AI reports |
| Doctor | Manage schedules, accept appointments, view patient scans, write diagnoses, record voice reports |
| Secretary | Manage appointments (approve/reject), manage patients & medical records, upload scans, manage doctor schedules & specialities |
| Manager | Approve/reject doctor & secretary registrations, manage all users, view patients |
- Hospital-grade workflow:
pending → confirmed/cancelled → completed - Patients browse doctors by name or speciality
- View real-time available time slots for selected date
- Book an appointment from an available slot
- Secretaries/managers approve or reject pending appointments
- Patients can rate and review completed appointments
- Blacklist support for problematic users
- Schedule Templates: Define recurring weekly patterns (e.g., Mon–Wed 9:00–17:00)
- Apply Template: Generate concrete date slots from a template for a date range
- Schedule Slots: CRUD for individual date/time slots (e.g., 2026-07-14 09:00–09:30)
- Slots visible to patients for booking
Complete medical history management:
- Allergies (name, description)
- Chronic diseases (name, description, diagnosis date)
- Family history (relation, condition, notes)
- Surgeries (name, date, description)
- Emergency contacts (name, relation, phone, email, address)
- Patient vitals: height, weight, blood type, marital status
- Upload scan images (linked to patient & optional appointment)
- Upload medical reports (PDF/images)
- View scan images directly in browser (served from Azure Blob)
- All files stored securely in Azure Blob Storage
- Role-based access: patients see their own, doctors see assigned patients, secretaries see all
- Upload a medical report (image/PDF)
- Async analysis via BullMQ queue → external AI server
- Poll for analysis status (processing, completed, failed)
- Analysis results stored as JSONB on the report
- Doctors record spoken notes after an appointment
- Audio uploaded → BullMQ job triggers pipeline:
- FFmpeg converts to 16kHz mono WAV
- Uploads to Azure Blob
- Sends to AI server (fine-tuned Whisper Large-V3 + Llama-3 8B)
- Results: transcription + structured clinical JSON
- Supports Egyptian Arabic medical dialect
- Doctors write diagnoses linked to patient & appointment
- Structured fields: symptoms + summary
- Viewable in patient medical records
- Editable (symptoms and full diagnosis)
- Patient Dashboard: Upcoming appointments, book appointment, medical records
- Doctor Dashboard: Appointment list with details, working hours management, availability viewer
- Secretary Dashboard: All appointments overview, doctor & patient management, specialities CRUD
- Manager Dashboard: User approval workflow, patient list, user role management
- Shared Compose UI: Declarative layouts shared between Android and iOS.
- Unified Presentation: Unidirectional MVI architecture using Voyager ScreenModels.
- Native Dictation: Low-latency recording (MPEG-4 AAC) via platform hardware APIs.
- Silent Token Rotation: Ktor client auth interceptor providing automated session refresh.
- Lab Analysis Telemetry: Cold Flow-based background status polling with real-time UI feedback.
- Upload Guards: Client-side multipart upload validation restricted to 10MB limits.
- Custom Design System: Consistent colors, custom forms, and widgets across platform views.
MedAI/
├── backend/ # NestJS API server
│ ├── src/
│ │ ├── auth/ # Authentication module
│ │ ├── users/ # User management module
│ │ ├── patients/ # Patient medical records module
│ │ ├── doctors/ # Doctor profiles & specialities module
│ │ ├── schedules/ # Schedule templates & slots module
│ │ ├── appointments/ # Appointment booking module
│ │ ├── scans/ # Medical scan & report uploads module
│ │ ├── diagnosis/ # Doctor diagnosis module
│ │ ├── voice-reports/ # Voice report + async AI pipeline module
│ │ ├── report-analysis/ # Lab report AI analysis module
│ │ ├── mail/ # Brevo SMTP email module
│ │ ├── database/ # TypeORM config, migrations (25+), seeds
│ │ └── shared/ # Shared entities, guards, decorators, services
│ ├── tests/ # E2E tests
│ └── docker/ # Dockerfiles (dev, test, prod)
│
├── frontend-next/ # Next.js 16 frontend
│ ├── src/
│ │ ├── app/
│ │ │ ├── page.js # Landing page
│ │ │ ├── auth/ # Login, signup, forgot-password
│ │ │ ├── (dashboard)/
│ │ │ │ ├── patient/ # Patient dashboard pages
│ │ │ │ ├── doctor/ # Doctor dashboard pages
│ │ │ │ ├── secretary/ # Secretary dashboard pages
│ │ │ │ ├── manager/ # Manager dashboard pages
│ │ │ │ └── profile/ # User profile page
│ │ │ ├── reset-password/
│ │ │ └── download/
│ │ ├── components/
│ │ │ ├── ui/ # 60+ shadcn-inspired UI components
│ │ │ ├── landing/ # Landing page sections
│ │ │ ├── auth/ # Authentication components
│ │ │ ├── doctor/ # Doctor-related components (18)
│ │ │ ├── patient/ # Patient-related components (25)
│ │ │ ├── secretary/ # Secretary components
│ │ │ ├── manager/ # Manager components
│ │ │ ├── schedule/ # Schedule management components (22)
│ │ │ ├── appointments/ # Appointment components
│ │ │ ├── scans/ # Scan/report components
│ │ │ ├── diagnosis/ # Diagnosis components
│ │ │ └── voice-reports/ # Voice report components
│ │ ├── contexts/ # Auth, Sidebar, DoctorInfo contexts
│ │ ├── hooks/ # Custom React hooks
│ │ ├── services/ # API client services (server + client)
│ │ └── lib/ # Utilities, Zod schemas, API fetch helpers
│ ├── public/
│ └── docker/
│
├── ai/ # AI pipeline (Python)
│ └── egyptian-voice-medical-analyzer/
│ ├── Llama3_Egyptian_Medic_Final/ # Fine-tuned model
│ ├── models/ # Model weights
│ ├── main_test.py # Inference runner
│ └── requirements.txt
│
├── mobile/ # Mobile Application
│ └── MedAI/ # Kotlin Multiplatform Project
│ ├── composeApp/ # Shared Compose UI + platform code
│ │ └── src/commonMain/kotlin/org/example/project/
│ │ ├── core/ # Base MVI framework classes
│ │ ├── data/ # Remote Ktor client, DTOs & repositories
│ │ ├── domain/ # Domain models, use cases & native audio contracts
│ │ ├── presentation/# View layer: screens (Voyager) & viewmodels (MVI)
│ │ └── design_system/# Palette, typography, spacing and forms
│ └── iosApp/ # iOS Xcode entry point and app delegate
│
├── infra/ # Terraform (prod/staging environments)
├── docker-compose/ # Docker Compose (local/staging/prod)
└── Makefile # Dev commands
- Docker & Docker Compose (with BuildKit enabled)
- Node.js 22+ (for running migrations/seeders locally)
- pnpm
- Java Development Kit (JDK) 17+ (for compiling the Mobile app)
- Android Studio / Xcode (for Android / iOS simulator builds)
# 1. Clone and enter the repo
git clone <repo-url> && cd MedAI
# 2. Configure environment
cp backend/.env.example backend/.env
# Edit backend/.env with your values (JWT secrets, Azure keys, Brevo SMTP, etc.)
# 3. Start all services
make runThis starts: NestJS API (port 8000), Next.js frontend (port 3000), PostgreSQL (5432), Redis (6379), runs migrations and seeds.
make run # Full local environment
make stop # Stop all containers
make migrate # Run pending TypeORM migrations
make generate-migration MIGRATION_NAME=name # Generate migration from entities
make seed # Run database seeders
make lint # Lint both frontend and backend
make test-backend # Run backend tests with coverage
make format # Format all code with Prettier
make clean-db # Wipe PostgreSQL data volumeKey variables (see backend/.env.example for full list):
| Variable | Description |
|---|---|
JWT_SECRET / REFRESH_TOKEN_SECRET |
Token signing secrets |
BREVO_SMTP_* |
Brevo email credentials (verification, password reset) |
AZURE_STORAGE_CONNECTION_STRING |
Azure Blob Storage (avatars, scans, reports) |
REDIS_HOST / REDIS_PORT |
Redis for BullMQ queues |
AI_SERVER_URL |
AI voice-to-report pipeline endpoint |
LAB_AI_SERVER_URL |
Lab report analysis AI endpoint |
The mobile app codebase is located at mobile/MedAI/.
Create/edit local.properties in mobile/MedAI/ and add the backend base URL:
# Android Emulator loopback pointing to host machine localhost
medai.base_url=http://10.0.2.2:8000/api/
# For iOS Simulator (localhost works directly)
# medai.base_url=http://localhost:8000/api/Ensure you have an Android virtual device running or a physical device connected:
- macOS / Linux:
cd mobile/MedAI ./gradlew :composeApp:installDebug - Windows:
cd mobile\MedAI .\gradlew.bat :composeApp:installDebug
Open the Xcode workspace inside mobile/MedAI/iosApp:
cd mobile/MedAI/iosApp
open iosApp.xcodeprojSelect your target simulator (e.g. iPhone 15) and click Run (Cmd + R) inside Xcode.
The NestJS backend exposes a REST API at /api/*. Full Swagger documentation is available at /api/docs when the server is running.
Auth: POST /api/auth/login, POST /api/auth/refresh-token, POST /api/auth/forgot-password, POST /api/auth/reset-password, POST /api/auth/change-email
Users: POST /api/users (register), GET /api/users/me, PATCH /api/users/:id, POST /api/users/:id/avatar
Patients: GET /api/patients, GET /api/patients/:id, CRUD for allergies, chronic diseases, family history, surgeries, emergency contacts
Doctors: GET /api/doctors, POST /api/doctors/search/speciality, POST /api/doctors/search/name, GET /api/doctors/top-rated, CRUD specialities
Schedules: CRUD schedule-templates, schedule-slots, POST apply-template (per doctor)
Appointments: POST /api/appointments, GET /api/appointments/me, PATCH /api/appointments/:id/status, PATCH /api/appointments/:id/review
Voice Reports: POST /api/appointments/:id/voice-reports, GET /api/appointments/:id/voice-reports/:reportId
Scans & Reports: POST /api/scans/:id, POST /api/scans/:id/:scanId/reports, GET /api/scans/images/:id/:imageId/file
Diagnoses: POST /api/diagnosis/:id, PATCH /api/diagnosis/:id/:diagnosisId
- Doctor records audio on the frontend
- Audio uploaded to backend → stored in Azure Blob → BullMQ job queued
- FFmpeg converts to 16kHz mono WAV
- Sent to AI server running fine-tuned Whisper Large-V3 (Egyptian Arabic STT)
- Transcription fed into fine-tuned Llama-3 8B (4-bit quantized) for medical NLU
- Structured clinical JSON returned and stored
- Secretary uploads report (image/PDF) for a patient
- BullMQ job triggers analysis on
LAB_AI_SERVER_URL - Frontend polls for status until complete
# Backend unit + integration tests
make test-backend
# E2E tests
pnpm --prefix backend run test:e2e
# Linting
make lint
# Run Mobile KMP shared tests
cd mobile/MedAI
./gradlew :composeApp:testDebugUnitTestThree Docker Compose environments:
| File | Purpose |
|---|---|
docker-compose/local.yaml |
Local development (hot-reload, debug) |
docker-compose/staging.yaml |
Staging environment |
docker-compose/prod.yaml |
Production (standalone Next.js build) |
Terraform configurations under infra/ for cloud infrastructure provisioning.