Dengue is a critical seasonal public health crisis in Sri Lanka. This repository presents the data preprocessing, cleaning, scaling, and dimensional reduction pipeline for classifying weekly district-wise dengue risk levels as "High Risk" or "Low Risk" using combined clinical case files and meteorological predictors.
The goal is to predict high-risk transmission weeks for individual Sri Lankan districts to enable proactive vector controls and healthcare allocation strategies.
The baseline dataset merges multi-modal streams covering 4,367 rows from 2022 to August 8, 2025 across all 26 clinical districts of Sri Lanka.
| Parameter | Specification | Details / Source |
|---|---|---|
| Weekly Cases | Clinical dengue caseload | PDF reports from the National Dengue Control Unit, Ministry of Health, Sri Lanka |
| Meteorological Vectors | Temperature, rainfall, wind speed, solar indexes | Downloaded as CSV streams from the Open-Meteo Historical Weather API |
| Temporal Granularity | Weekly indexing | Centered on ISO Monday week start dates |
Target Variable (risk_level) |
Binary Classification |
1 (High Risk 0 (Low Risk below baseline) |
- Caseload:
Cases(weekly count) - Thermal:
temperature_mean(°C),temperature_max(°C),temperature_min(°C) - Hydrological:
precipitation_sum(mm),rain_sum(mm),et0_fao_evapotranspiration(mm) - Solar/Wind:
sunshine_duration(s),daylight_duration(s),wind_speed_10m_max(km/h)
The preprocessing and dimensionality reduction pipeline is split into 6 distinct stages, representing individual researcher roles for Progress Review I:
graph TD
A[Raw PDF Case Files & Weather CSVs] --> B[Stage 1: Extraction & Imputation]
B --> C[Stage 2: Winsorization / Outlier Capping]
C --> D[Stage 3: Feature Engineering & Lags]
D --> E[Stage 4: Target Encoding]
E --> F[Stage 5: Standard Z-Score Scaling]
F --> G[Stage 6: Principal Component Analysis]
G --> H[Final Processed Dataset: PC1 - PC7 & Target]
| Member ID | Researcher Name | Assigned Preprocessing Specialization | Notebook |
|---|---|---|---|
| IT24101873 | Jesmeen M.B.A | Data Extraction, Merging, & Missing Values | IT24101873_extract_combine.ipynb |
| IT24101952 | Senevirathna U.K.J. | Outlier Capping (Winsorization) | IT24101952_Outlier_capping.ipynb |
| IT24101927 | Liyanage J.L.K.L. | Feature Engineering & Lags | IT24101927_feature_engineering.ipynb |
| IT24101972 | Nethsara K.P.S. | Categorical & Target Encoding | IT24101972_Encoding.ipynb |
| IT24101829 | Ranasinghe R.P.V.K. | Standard Scaling & Normalization | IT24101829_Scaling_Normalization.ipynb |
| IT24103815 | Fernando W.P.S. | Dimension Reduction (PCA) | IT24103815_feature_selection.ipynb |
We have designed a modern single-page Interactive Research Portal inside this repository at dashboard/index.html.
✅ This repository is ready for GitHub Pages. Configure Pages to deploy from the main branch and /(root). The root index.html redirects to the dashboard at ./dashboard/.
- I. Interactive Pipeline Timeline: A step-by-step accordion detailing each member's operational steps, equations, and actual Jupyter-tested python scripts.
- II. Interactive EDA Lightbox Gallery: A high-fidelity grid displaying all 13 exploratory plots (PCA Variance Screes, Heatmaps, Capping Histograms) with smooth overlay modal zooms and tech specs.
- III. Climate Risk Simulator: A live machine learning playground where users adjust temperature, rainfall, and solar sliders to see real-time vector probability calculations and alert notifications.
Simply open the local HTML file in any standard web browser. No compilation, external scripts, or servers required:
# Double-click the file or open it via terminal:
open dashboard/index.htmlEnsure you have Python 3.9+ and pip dependencies installed:
# 1. Clone the repository
git clone <repository-url>
cd DengueRisk
# 2. Place raw clinical datasets in the data folder
# 3. Launch Jupyter environment
jupyter notebook
# 4. Run the master pipeline to generate the processed dataset
# Opens and executes: group_pipeline.ipynb