DeepFlow-IDS is an end-to-end deep learning framework built for network intrusion detection using the benchmark CIC-IDS2017 dataset. It offers a complete pipeline from data preprocessing to model training and evaluation, supporting three powerful neural network architectures:
- 1D-CNN — Fast and efficient feature extraction
- RNN (LSTM) — Captures sequential dependencies in network traffic
- Hybrid (CNN-LSTM) — Combines spatial and temporal feature learning for superior performance
- Automated Data Pipeline — Cleaning, balancing (stratified downsampling), and normalization
- Multiple Deep Learning Models — CNN, LSTM, and Hybrid architectures
- Production-Ready — CLI support, training callbacks, and evaluation suite
- Comprehensive Metrics — Accuracy, Precision, Recall, F1-Score, and Confusion Matrices
- Visualization — Benchmark comparisons and confusion matrix heatmaps
- Class Imbalance Handling — Stratified downsampling to handle 15-class imbalance
flowchart TD
A[Raw CIC-IDS2017 CSV Files] --> B[Data Ingestion & Cleaning]
B -->|Drop NaN / Inf & Whitespaces| C[Stratified Class Downsampler]
C -->|Balance Benign & Attack Classes| D[MinMax Normalization]
D --> E[Reshape Tensor: Batch, 78, 1]
E --> F[1D-CNN Model]
E --> G[RNN LSTM Model]
E --> H[Hybrid CNN-LSTM Model]
subgraph CNN_Branch [1D-CNN Architecture]
F --> F1[Conv1D + BatchNorm + MaxPool]
F1 --> F2[Conv1D + BatchNorm + MaxPool]
F2 --> F3[Dense Layer + Dropout]
F3 --> F4[Softmax Classifier]
end
subgraph RNN_Branch [RNN-LSTM Architecture]
G --> G1[LSTM Layer 64 units]
G1 --> G2[Dropout 0.2]
G2 --> G3[LSTM Layer 64 units]
G3 --> G4[Dense Layer + Softmax]
end
subgraph Hybrid_Branch [CNN-LSTM Architecture]
H --> H1[2x Conv1D + MaxPool Blocks]
H1 --> H2[LSTM Layer 64 units]
H2 --> H3[Dropout 0.3 + Dense Layer]
H3 --> H4[Softmax Classifier]
end
F4 --> I[Evaluation & Benchmark Engine]
G4 --> I
H4 --> I
I --> J[Normalized Confusion Matrices]
I --> K[Classification Reports: F1, Precision, Recall]
sequenceDiagram
autonumber
actor User
participant CLI as main.py
participant Loader as data_loader.py
participant Models as models.py
participant Engine as train.py
participant Eval as evaluate.py
User->>CLI: Execute python main.py
CLI->>Loader: load_and_preprocess_data()
Loader-->>CLI: Return X_train, X_test, y_train, y_test
CLI->>Models: Instantiate CNN, RNN, and Hybrid
Models-->>CLI: Model objects compiled
loop For Each Model Architecture
CLI->>Engine: Train with EarlyStopping & Validation
Engine-->>CLI: Trained Model Artifacts
CLI->>Eval: evaluate_model(model, X_test, y_test)
Eval-->>User: Render Metrics & Plot Confusion Matrix
end
deep-flow-ids/
├── data/
│ ├── raw/ # Raw CIC-IDS2017 CSV files
│ └── processed/ # Serialized/preprocessed arrays
├── notebooks/
│ ├── 01_eda.ipynb # Exploratory Data Analysis & Class Distribution
│ └── 02_benchmarks.ipynb # Comparative Metric Analysis
├── src/
│ ├── __init__.py
│ ├── data_loader.py # Cleaning, balancing, and scaling pipeline
│ ├── models.py # 1D-CNN, LSTM, and Hybrid architectures
│ ├── train.py # Training loops and callbacks
│ └── evaluate.py # Evaluation and confusion matrix generator
├── assets/
│ ├── 1D_CNN.keras # Saved CNN model
│ ├── RNN_LSTM.keras # Saved LSTM model
│ ├── Hybrid_CNN_LSTM.keras # Saved Hybrid model
│ ├── 1D_CNN_cm.png # CNN confusion matrix
│ ├── RNN_LSTM_cm.png # LSTM confusion matrix
│ └── Hybrid_CNN_LSTM_cm.png # Hybrid confusion matrix
├── requirements.txt # Production dependencies
├── main.py # CLI entry-point
├── LICENSE # MIT License
└── README.md
- Download the CIC-IDS2017 Dataset (Generated by the Canadian Institute for Cybersecurity).
- Place the CSV files inside
data/raw/:
mkdir -p data/raw
# Place the downloaded CSV files inside data/raw/** Fallback Mode:** If no files are placed in
data/raw/, the built-in loader generates a mock tabular dataset matching CIC-IDS2017 dimensions to test the pipeline.
# Clone the repository
git clone https://github.com/yuvraj333/deep-flow-ids.git
cd deep-flow-ids
# Create and activate a virtual environment
python -m venv venv
source venv/bin/activate # On Windows use: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txttensorflow>=2.10.0
numpy>=1.22.0
pandas>=1.4.0
scikit-learn>=1.0.0
matplotlib>=3.5.0
seaborn>=0.11.0Run the entire pipeline (data cleaning, class balancing, training, and evaluation):
python main.py --data_path "data/raw/*.csv" --samples_per_class 10000 --epochs 15 --batch_size 256| Argument | Type | Default | Description |
|---|---|---|---|
--data_path |
str |
data/raw/*.csv |
Path or glob pattern for raw dataset files |
--samples_per_class |
int |
10000 |
Sample cap per class to handle majority imbalance |
--epochs |
int |
15 |
Maximum training epochs per model |
--batch_size |
int |
256 |
Training mini-batch size |
--val_split |
float |
0.15 |
Validation split ratio |
The dataset contains 15 attack categories with severe class imbalance:
| Class | Original Count | After Balancing |
|---|---|---|
| BENIGN | 2,271,320 | 10,000 |
| DoS Hulk | 230,124 | 10,000 |
| PortScan | 158,804 | 10,000 |
| DDoS | 128,025 | 10,000 |
| DoS GoldenEye | 10,293 | 10,000 |
| FTP-Patator | 7,935 | 7,935 |
| SSH-Patator | 5,897 | 5,897 |
| DoS slowloris | 5,796 | 5,796 |
| DoS Slowhttptest | 5,499 | 5,499 |
| Bot | 1,956 | 1,956 |
| Web Attack – Brute Force | 1,507 | 1,507 |
| Web Attack – XSS | 652 | 652 |
| Infiltration | 36 | 36 |
| Web Attack – Sql Injection | 21 | 21 |
| Heartbleed | 11 | 11 |
Total Samples: ~2.8M → ~70K (after stratified downsampling)
Performance benchmark evaluated across stratified test sets on CIC-IDS2017:
| Model Architecture | Accuracy | Precision | Recall | F1-Score | Training Time |
|---|---|---|---|---|---|
| 1D-CNN | 99% | 0.99 | 0.99 | 0.98 | ~20 sec/epoch |
| RNN (LSTM) | 98% | 0.97 | 0.98 | 0.97 | ~68 sec/epoch |
| Hybrid (CNN-LSTM) | 70% | 0.76 | 0.70 | 0.67 | ~28 sec/epoch |
Best Overall: 1D-CNN
- Achieves 99% accuracy with excellent performance across all classes
- Fastest training time (~20 sec/epoch)
- Handles rare attack classes well (e.g., Heartbleed, Infiltration)
RNN-LSTM:
- Solid 98% accuracy with good temporal pattern recognition
- Moderate training time (~68 sec/epoch)
- Struggles with very rare classes (Heartbleed, Infiltration)
Hybrid CNN-LSTM:
- Currently achieving 70% accuracy - requires further optimization
- Shows potential but needs hyperparameter tuning
- Early stopping triggered at epoch 10
| Class | Precision | Recall | F1-Score | Support |
|---|---|---|---|---|
| BENIGN | 0.99 | 0.97 | 0.98 | 2,000 |
| DDoS | 1.00 | 1.00 | 1.00 | 2,000 |
| DoS GoldenEye | 1.00 | 1.00 | 1.00 | 2,000 |
| DoS Hulk | 1.00 | 1.00 | 1.00 | 2,000 |
| PortScan | 1.00 | 1.00 | 1.00 | 2,000 |
| FTP-Patator | 1.00 | 1.00 | 1.00 | 1,587 |
| SSH-Patator | 0.99 | 1.00 | 1.00 | 1,180 |
| DoS Slowhttptest | 0.99 | 0.99 | 0.99 | 1,100 |
| DoS slowloris | 0.99 | 0.99 | 0.99 | 1,159 |
| Bot | 0.90 | 0.98 | 0.94 | 391 |
| Web Attack – Brute Force | 0.71 | 1.00 | 0.83 | 301 |
| Web Attack – XSS | 1.00 | 0.02 | 0.03 | 131 |
| Infiltration | 1.00 | 0.57 | 0.73 | 7 |
| Heartbleed | 1.00 | 1.00 | 1.00 | 2 |
| Web Attack – Sql Injection | 0.00 | 0.00 | 0.00 | 4 |
- 1D-CNN is the most reliable model for this dataset with 99% accuracy
- Class imbalance remains challenging for minority classes like Web Attack – XSS
- Hybrid model needs architectural improvements (currently at 70%)
- Real-time detection possible with CNN (fastest training and inference)
Distributed under the MIT License. See LICENSE for more information.
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'Add some AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Yuvraj Kumar Mahato — @yuvraj333
Project Link: https://github.com/yuvraj333/deep-flow-ids