ns3-miot-simulation

Performance Evaluation of Cluster-Based WSN-AODV and MQTT-SN Architectures for Medical IoT Using ns-3

A statistically validated ns-3 simulation study comparing two cluster-based Medical IoT communication architectures across 270 independent simulation runs spanning five experimental sets (10 seeds per configuration; Welch t-test and Mann-Whitney U). Reproduces all results in Baseet & Bütün, IEEE Sensors Journal 2026 (submitted).

Architecture A (WSN): Cluster-based AODV routing with fixed-rate UDP transmission
Architecture B (MQTT-SN): Priority-aware publish-subscribe with QoS differentiation

Paper: Performance Evaluation of Cluster-Based WSN-AODV and MQTT-SN Architectures for Medical IoT Using ns-3 — submitted to IEEE Sensors Journal

SAVTEK 2026: Savunma Saha Sağlık Ağlarında Küme Tabanlı KAA-AODV ve MQTT-SN Mimarilerinin ns-3 ile Başarım Değerlendirmesi — early findings submitted for presentation

⚠️ Reproducibility Warning — ns-3 Version

This code reproduces the paper's results only on ns-3 v3.40.

ns-3 v3.41 introduced a regression that causes PDR = 0 % in ad-hoc AODV+UDP mode (affects cluster-aodv-nosink in particular). Do not use v3.41, v3.42, or v3.43 until the upstream fix is verified against the CSVs in this repository.
cd ~/ns-3-dev && git describe --tags --exact-match   # should print: ns-3.40

Architecture

Architecture A — Traditional WSN (AODV + UDP):
  [Sensor] --OnOff 4kbps UDP--> [CH: UdpForwarder] --forward--> [Sink]

Architecture B — MQTT-SN Publish-Subscribe:
  [ECG  250ms/128B/QoS2]---+
  [HR   1s/64B/QoS1    ]---+--> [CH: MqttSnGateway] --MQTT-SN--> [MqttSnBroker]
  [Temp 5s/32B/QoS0    ]---+    (priority queue)
  [Emergency 0.5% prob ]---+

Shared Protocol Stack

Layer	Protocol
Application	WSN: OnOffApplication / MQTT-SN: Publish-Subscribe
Transport	UDP
Network	IPv4 + AODV (HelloInterval=2s, RreqRetries=5, ActiveRouteTimeout=15s)
MAC / PHY	IEEE 802.11b Ad-Hoc (11 Mbps, 20 dBm, ~300m range)
Propagation	LogDistance (exponent = 2.5)

MQTT-SN Sensor Types

Type	Interval	Payload	QoS	Priority
ECG	250 ms	128 B	2	HIGH
Heart Rate	1 s	64 B	1	MEDIUM
Temperature	5 s	32 B	0	LOW
Emergency	0.5% per publish	128 B	2	CRITICAL

Experimental Results (10 seeds; Welch t-test and Mann-Whitney U)

All results are mean ± std over 10 independent random seeds. Statistical significance: * p<0.05, ** p<0.01, *** p<0.001, ns p≥0.05. The 200-node comparisons are additionally corroborated by a two-sided Mann-Whitney U test (Set 2 p=0.5368, Set 3 p=0.0017, Set 3b p=0.7679); it agrees with Welch's verdict in every case.

Set 1 — No Broker (Application-Layer Isolation)

Nodes	WSN PDR	MQTT-SN PDR	p(PDR)	WSN Delay	MQTT-SN Delay	p(Delay)
50	84.1±11.3%	81.3±11.9%	0.5877 ns	50.7±19.5 ms	32.7±11.5 ms	0.0245 *
100	99.2±0.9%	98.4±1.5%	0.1989 ns	77.8±13.2 ms	71.1±13.0 ms	0.2712 ns
150	98.4±0.7%	99.2±1.1%	0.0715 ns	103.3±10.7 ms	84.4±21.6 ms	0.0278 *
200	96.9±1.8%	99.2±0.6%	0.0028 **	118.1±12.2 ms	141.2±23.1 ms	0.0147 *

Set 2 — With Broker (Full End-to-End Architecture)

Nodes	WSN PDR	MQTT-SN PDR	p(PDR)	WSN Delay	MQTT-SN Delay	p(Delay)
50	84.6±11.2%	82.2±12.3%	0.6657 ns	48.0±14.3 ms	35.1±16.7 ms	0.0822 ns
100	94.9±4.3%	92.5±5.9%	0.3133 ns	61.7±24.7 ms	75.9±23.1 ms	0.1994 ns
150	82.0±2.8%	79.3±3.6%	0.0782 ns	125.4±22.6 ms	153.6±28.8 ms	0.0261 *
200 †	73.5±2.1%	72.8±2.0%	0.5878 ns	190.2±20.6 ms	196.3±16.9 ms	0.6066 ns

† First batch only (WSN n=6, MQTT-SN n=5). The 200-node brokered runs were collected in two batches on two hosts; the second batch, executed on a different build, transmitted 4–5× too few packets and is excluded (paper Sec. V-D). Full-sample values are 67.2±8.3% / 65.6±7.7%; the exclusion narrows the spread but leaves the Set 2 200-node tie unchanged.

Set 3 — Equal Load, No Broker (Protocol Mechanism Isolation)

WSN sensors adopt the MQTT-SN heterogeneous traffic profile (ECG 4kbps, HR 512bps, Temp 51bps). MQTT-SN data is identical to Set 1.

Nodes	WSN-Hetero PDR	MQTT-SN PDR	p(PDR)
50	84.3±11.9%	81.3±11.9%	0.5781 ns
100	98.9±1.0%	98.4±1.5%	0.3867 ns
150	99.0±0.5%	99.2±1.1%	0.6026 ns
200	94.5±5.4%	99.2±0.6%	0.0223 *

Set 3b — Equal Load, With Broker (Gateway Overhead Under Congestion)

Same heterogeneous traffic as Set 3, broker active. MQTT-SN+broker data is identical to Set 2 (first batch, n=5, at 200 nodes).

Nodes	WSN-Hetero PDR	MQTT-SN PDR	p(PDR)
50	84.0±11.6%	82.2±12.3%	0.7472 ns
100	93.8±5.1%	92.5±5.9%	0.6099 ns
150	80.3±3.0%	79.3±3.6%	0.5122 ns
200	72.3±2.1%	72.8±2.0%	0.6920 ns

Set 5 — Multi-Broker Hierarchy (200 nodes)

Two-tier broker architecture: N local tier-1 brokers + 1 root broker. The LB=1 baseline is the first-batch Set 2 MQTT-SN data (n=5). The LB=2/4/5 runs were executed on the second host (paper Sec. V-D), so the comparison is cross-host and PDR-deficit magnitudes are upper bounds; the robust, host-independent finding is that no depth recovers PDR.

Configuration	PDR	p(vs LB=1)	Delay (ms)	p(Delay)
LB=1 (baseline, n=5)	72.8±2.0%	—	196.3±16.9	—
LB=2	59.4±1.1%	<0.0001 ***	217.8±12.8	0.0438 *
LB=4	63.9±1.3%	0.0001 ***	160.3±9.0	0.0063 **
LB=5	63.5±1.7%	<0.0001 ***	156.1±13.0	0.0028 **

Finding: No broker count recovers PDR beyond the single-broker baseline — shared IEEE 802.11b channel saturation, not broker architecture, is the binding constraint at 200 nodes. Delay improves at LB≥4, suggesting distributed aggregation relieves the single-broker queueing bottleneck even when it cannot recover the channel.

Methodology note. All PDR and delay values are computed on sensor-origin flows only (traffic originating at sensor nodes, excluding cluster-head/broker forwarding and QoS acknowledgement flows), matching the paper. Run python3 scripts/analyze_comparison.py to regenerate every value above from the raw CSVs.

Key Findings

MQTT-SN wins delivery without a broker at scale — significantly higher PDR at 200 nodes (p=0.0028), and significantly lower delay at 50 and 150 nodes (p=0.0245, p=0.0278); at 200 nodes its gateway overhead makes it significantly slower (p=0.0147).
A central broker collapses delivery for both — PDR falls below 74% for each architecture at 200 nodes, with no significant difference between them at any density (Set 2 @200: 73.5% vs 72.8%, p=0.5878 ns).
MQTT-SN's protocol mechanisms are real — even under equal traffic load (Set 3), MQTT-SN still wins at 200 nodes (p=0.0223, both tests). The PDR advantage is not purely lower traffic volume — the study's most robust result.
The broker erases, not reverses, the advantage — under equal traffic with a broker (Set 3b), the significant no-broker MQTT-SN advantage at 200 nodes is erased; the two architectures deliver equivalently (72.3% vs 72.8%, p=0.6920 ns). Gateway overhead is the mechanism.
Single-seed simulation is insufficient — preliminary single-seed runs showed MQTT-SN winning in Set 2; the 10-seed analysis overturned that conclusion. Multi-seed validation with per-run offered-load checks is essential.
Broker distribution does not recover PDR — a two-tier hierarchy (LB=2/4/5) recovers no PDR at any depth; channel saturation is the binding constraint. Delay does improve at LB≥4.

Quick Start

Need the full end-to-end recipe (fresh Ubuntu → compiled ns-3 → all 270 runs → all figures)? See REPRODUCE.md.

Requirements

ns-3 v3.40 (important: v3.41 causes PDR=0% bug in ad-hoc mode)
Python 3.10+ with dependencies in requirements.txt:
```
python3 -m pip install -r requirements.txt
```

# Verify ns-3 version before running
cd ~/ns-3-dev && git log --oneline -1

Build

mkdir -p ~/ns-3-dev/scratch/cluster-aodv-mqtt
mkdir -p ~/ns-3-dev/scratch/cluster-aodv-nosink

cp src/mqtt-sn/* ~/ns-3-dev/scratch/cluster-aodv-mqtt/
cp src/wsn/cluster-aodv-nosink.cc ~/ns-3-dev/scratch/cluster-aodv-nosink/

cd ~/ns-3-dev
./ns3 build

Run Single Simulations

cd ~/ns-3-dev

# WSN — no sink
./ns3 run "cluster-aodv-nosink --numRegular=200 --numCH=20 --useSink=false --simTime=300"

# WSN — with sink
./ns3 run "cluster-aodv-nosink --numRegular=200 --numCH=20 --useSink=true --simTime=300"

# WSN — heterogeneous traffic (equal load)
./ns3 run "cluster-aodv-nosink --numRegular=200 --numCH=20 --useSink=false --hetero=true --simTime=300"

# MQTT-SN — no broker
./ns3 run "cluster-aodv-mqtt --nSensors=200 --nCH=20 --numLocalBrokers=0 --simTime=300 --anim=false"

# MQTT-SN — single broker
./ns3 run "cluster-aodv-mqtt --nSensors=200 --nCH=20 --numLocalBrokers=1 --simTime=300 --anim=false"

# MQTT-SN — two-tier hierarchy (4 local brokers + 1 root)
./ns3 run "cluster-aodv-mqtt --nSensors=200 --nCH=20 --numLocalBrokers=4 --simTime=300 --anim=false"

Run All Statistical Experiments

cd ~/ns-3-dev
mkdir -p experiments/statistical experiments/hier

# Full 5-set comparison (270 runs, ~10h on 4-core machine)
tmux new-session -d -s experiments "bash ~/ns3-miot-simulation/scripts/run-comparison.sh"
tmux attach -t experiments

# Multi-broker hierarchy sweep (Set 5, 30 runs; LB=1 reuses Set 2)
tmux new-session -d -s set5 "bash ~/ns3-miot-simulation/scripts/run-comparison.sh --set5-only"

Analyze Results

cd ~/ns-3-dev
python3 ~/ns3-miot-simulation/scripts/analyze_comparison.py

Generates 28 graphs (TR+EN): 24 in graphs/comparison/ and 4 in graphs/hier/, plus full statistical output with Welch t-test and Mann-Whitney U p-values.

Command-Line Parameters

cluster-aodv-mqtt (MQTT-SN)

Parameter	Default	Description
--nSensors	200	Number of sensor nodes
--nCH	20	Number of cluster heads
--numLocalBrokers	1	0=no broker, 1=single broker, N≥2=two-tier hierarchy
--run	1	Random seed index
--simTime	300	Simulation time (s)
--anim	false	Generate NetAnim XML
--csv	auto	Output CSV filename

cluster-aodv-nosink (WSN)

Parameter	Default	Description
--numRegular	200	Number of sensor nodes
--numCH	20	Number of cluster heads
--useSink	false	Enable sink with forwarding
--hetero	false	Heterogeneous traffic rates (equal-load mode)
--run	1	Random seed index
--simTime	300	Simulation time (s)
--csv	auto	Output CSV filename

CSV Output Schema

Each simulation writes a per-flow CSV (one row per IPv4 flow, from FlowMonitor):

#	Column	Unit	Description
1	FlowID	int	FlowMonitor flow identifier
2	SrcAddr	IPv4	Source address
3	DstAddr	IPv4	Destination address
4	TxPackets	count	Packets transmitted in this flow
5	RxPackets	count	Packets received in this flow
6	LostPackets	count	Tx − Rx
7	PDR_pct	%	Per-flow delivery ratio
8	Throughput_kbps	kbps	Mean throughput
9	AvgDelay_ms	ms	Mean end-to-end delay
10	AvgJitter_ms	ms	Mean inter-arrival jitter

MQTT-SN runs additionally write <csv-name>.csv.energy.csv with columns: NodeID, Type, InitialEnergy_J, RemainingEnergy_J, ConsumedEnergy_J, ConsumedPct.

Sanity check — reproduce a paper cell from a single CSV

Aggregate PDR for one seed (Set 2, 200-node WSN, seed 1):

awk -F',' 'NR>1 {tx+=$4; rx+=$5} END {printf "PDR = %.2f%%\n", 100*rx/tx}' \
    experiments/statistical/exp2_wsn_200_r1.csv

Individual seeds vary (~55–76 %). This raw all-flows average over all 10 seeds gives ≈ 67.35 %. The paper instead reports 73.5 % for Set 2, 200-node WSN — it counts only sensor-origin flows and excludes the contaminated second batch (first batch, n=6; see Sec. V-D). Run python3 scripts/analyze_comparison.py to reproduce the paper values exactly:

for r in 1 2 3 4 5 6 7 8 9 10; do
  awk -F',' 'NR>1{tx+=$4;rx+=$5} END{if(tx>0) printf "%.2f\n", 100*rx/tx}' \
      experiments/statistical/exp2_wsn_200_r${r}.csv
done | awk '{s+=$1; ss+=$1*$1; n++} END{m=s/n; printf "mean=%.2f%%  std=%.2f%%  N=%d\n", m, sqrt(ss/n - m*m), n}'
# Expected: mean ≈ 67.35 %  std ≈ 7.88 %  N=10

If you see PDR = 0 % on a working seed, you are running ns-3.41. See the version warning at the top of this README.

Project Structure

ns3-miot-simulation/
│
├── src/
│   ├── wsn/
│   │   └── cluster-aodv-nosink.cc       WSN: AODV + UDP + UdpForwarder + --hetero
│   └── mqtt-sn/
│       ├── mqtt-sn-header.h/cc          Packet format + 4-level priority
│       ├── mqtt-sn-publisher.h/cc       Sensor app + emergency detection
│       ├── mqtt-sn-gateway.h/cc         Gateway + priority queue + 1ms delay
│       ├── mqtt-sn-broker.h/cc          Broker + two-tier hierarchy support
│       └── cluster-aodv-mqtt.cc         Main simulation + --numLocalBrokers
│
├── experiments/
│   ├── statistical/                     240 CSVs (Sets 1, 2, 3, 3b)
│   │   ├── exp1_wsn_{50,100,150,200}_r{1-10}.csv
│   │   ├── exp1_mqtt_{50,100,150,200}_r{1-10}.csv
│   │   ├── exp2_wsn_{50,100,150,200}_r{1-10}.csv
│   │   ├── exp2_mqtt_{50,100,150,200}_r{1-10}.csv
│   │   ├── exp3_wsn_hetero_{50,100,150,200}_r{1-10}.csv
│   │   └── exp3b_wsn_hetero_broker_{50,100,150,200}_r{1-10}.csv
│   └── hier/                            30 CSVs (Set 5; LB=1 reuses Set 2)
│       └── exp_hier_lb{2,4,5}_200_r{1-10}.csv
│
├── graphs/
│   ├── comparison/                      24 analysis graphs (TR+EN)
│   └── hier/                            4 multi-broker hierarchy graphs (TR+EN)
│
├── scripts/
│   ├── analyze_comparison.py            Main analysis: Welch + Mann-Whitney U tests + 28 graphs
│   └── run-comparison.sh                Full experiment runner (Sets 1-3b + Set 5)
│
└── README.md

Simulation Parameters

Parameter	Value
Simulation area	1000 m × 1000 m
Wireless standard	IEEE 802.11b Ad-Hoc, 11 Mbps
Propagation model	LogDistance (exponent = 2.5)
Transmit power	20 dBm (~300 m range)
Routing	AODV (HelloInterval=2s, RreqRetries=5, ActiveRouteTimeout=15s)
Simulation duration	300 s
Node start (warmup)	Uniform 30–45 s per node
Random seeds	10 independent seeds per configuration
Node counts	50, 100, 150, 200 sensors
CH ratio	10% (fixed grid placement)
Statistical test	Welch's t-test (equal_var=False) + Mann-Whitney U (two-sided)
Energy model	BasicEnergySource: sensors 2.0J, CHs 5.0J
Radio currents	Tx=0.346A, Rx=0.285A, Idle=0.248A (both architectures)

Roadmap

Citation

If you use this code or data in your research, please cite the paper:

@article{baseet2026sensors,
  title     = {Performance Evaluation of Cluster-Based WSN-AODV and MQTT-SN
               Architectures for Medical IoT Using ns-3},
  author    = {Baseet, Amro and B{\"u}t{\"u}n, {\.I}smail},
  journal   = {IEEE Sensors Journal},
  year      = {2026},
  note      = {Under review},
  url       = {https://github.com/amirbaseet/ns3-miot-simulation}
}

…and the archived code/dataset snapshot (replace ZENODO-ID once Zenodo mints the DOI for release v1.0):

@software{baseet2026ns3miot,
  author    = {Baseet, Amro and B{\"u}t{\"u}n, {\.I}smail},
  title     = {{ns3-miot-simulation}: Cluster-based {WSN-AODV} vs.\ {MQTT-SN}
               for {Medical IoT} in ns-3},
  year      = {2026},
  version   = {v1.0},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.ZENODO-ID},
  url       = {https://github.com/amirbaseet/ns3-miot-simulation}
}

Author

Amro Baseet MSc Computer Engineering, Sakarya University Supervisor: Assoc. Prof. Dr. İsmail Bütün

References

C. Perkins et al., "Ad Hoc On-Demand Distance Vector (AODV) Routing," IETF RFC 3561, 2003
A. Stanford-Clark and H. L. Truong, "MQTT for Sensor Networks (MQTT-SN) v1.2," IBM, 2013
G. F. Riley and T. R. Henderson, "The ns-3 Network Simulator," Springer, 2010
OASIS, "MQTT Version 5.0," 2019

License

This project is released under the MIT License. See LICENSE for details. Simulation data in experiments/ and figures in graphs/ are released under CC BY 4.0.

Navigation Menu

Repository files navigation

ns3-miot-simulation

Performance Evaluation of Cluster-Based WSN-AODV and MQTT-SN Architectures for Medical IoT Using ns-3

⚠️ Reproducibility Warning — ns-3 Version

Architecture

Shared Protocol Stack

MQTT-SN Sensor Types

Experimental Results (10 seeds; Welch t-test and Mann-Whitney U)

Set 1 — No Broker (Application-Layer Isolation)

Set 2 — With Broker (Full End-to-End Architecture)

Set 3 — Equal Load, No Broker (Protocol Mechanism Isolation)

Set 3b — Equal Load, With Broker (Gateway Overhead Under Congestion)

Set 5 — Multi-Broker Hierarchy (200 nodes)

Key Findings

Quick Start

Requirements

Build

Run Single Simulations

Run All Statistical Experiments

Analyze Results

Command-Line Parameters

cluster-aodv-mqtt (MQTT-SN)

cluster-aodv-nosink (WSN)

CSV Output Schema

Sanity check — reproduce a paper cell from a single CSV

Project Structure

Simulation Parameters

Roadmap

Citation

Author

References

License

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 75 Commits
archive		archive
diagrams		diagrams
experiments		experiments
graphs		graphs
scripts		scripts
src		src
.gitignore		.gitignore
CITATION.cff		CITATION.cff
LICENSE		LICENSE
README.md		README.md
REPRODUCE.md		REPRODUCE.md
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

ns3-miot-simulation

Performance Evaluation of Cluster-Based WSN-AODV and MQTT-SN Architectures for Medical IoT Using ns-3

⚠️ Reproducibility Warning — ns-3 Version

Architecture

Shared Protocol Stack

MQTT-SN Sensor Types

Experimental Results (10 seeds; Welch t-test and Mann-Whitney U)

Set 1 — No Broker (Application-Layer Isolation)

Set 2 — With Broker (Full End-to-End Architecture)

Set 3 — Equal Load, No Broker (Protocol Mechanism Isolation)

Set 3b — Equal Load, With Broker (Gateway Overhead Under Congestion)

Set 5 — Multi-Broker Hierarchy (200 nodes)

Key Findings

Quick Start

Requirements

Build

Run Single Simulations

Run All Statistical Experiments

Analyze Results

Command-Line Parameters

cluster-aodv-mqtt (MQTT-SN)

cluster-aodv-nosink (WSN)

CSV Output Schema

Sanity check — reproduce a paper cell from a single CSV

Project Structure

Simulation Parameters

Roadmap

Citation

Author

References

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages