FL Trilemma

Empirical study of the Privacy-Robustness-Performance trilemma in Federated Learning (FL): how much do Differential Privacy, Byzantine-robust aggregation, and communication-efficient compression cost each other when combined, versus used alone?

The three mechanisms studied:

• Privacy - DP-SGD via Opacus, giving each client a per-round privacy budget (epsilon).
• Robustness - FLTrust, a server-side aggregation strategy that scores each client update's trust against a reference update trained on a small clean root dataset, to defend against Byzantine (malicious) clients.
• Performance/efficiency - Top-k gradient sparsification, keeping only the largest-magnitude update entries to reduce communication cost.

All three (and every combination of them) are run as 8 configurations across two datasets/models - MNIST (small CNN) and CIFAR-10 (ResNet-20, BatchNorm swapped for GroupNorm so DP-SGD's per-sample gradients stay valid) - simulated with Flower (flwr) on top of Ray, under a label-flip or random-gradient Byzantine attack.

Contributions

• An open-source, end-to-end FL framework that integrates DP-SGD, FLTrust, and Top-k sparsification in one modular pipeline, for reproducible experiments on how they interact.
• An empirical finding that combining DP-SGD with FLTrust severely degrades accuracy under full local-epoch training, but reducing the number of local SGD steps per round largely mitigates this - refining the theoretical worst-case bound with a practical mitigation.
• An analysis of how FLTrust's per-client trust scores evolve over training, showing the behavioral divergence between honest and malicious clients.

Repository layout

code/
  src/            federated learning code - see code/src/README.md
    mechanisms/   DP, FLTrust, TopK, and the Byzantine attacks - see code/src/mechanisms/README.md
    models/       MNIST CNN and CIFAR-10 ResNet-20 architectures
    scripts/      experiment grid runner + results visualizer - see code/src/scripts/README.md
    results/      JSON result files from experiment runs
  data/           downloaded datasets (MNIST, CIFAR-10) - see code/data/README.md
  pyproject.toml  package definition and dependencies

Cloning and setup

git clone git@github.com:Snyuki/fl-trilemma.git
cd fl-trilemma/code
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Requires Python >=3.11. Dependencies (Flower, Opacus, PyTorch/torchvision, scikit-learn, pandas, matplotlib, etc.) are declared in code/pyproject.toml and installed automatically via the command above. MNIST/CIFAR-10 are downloaded automatically on first use - nothing to fetch by hand.

Running it

From code/, after installing:

# Sanity checks
python -m src.train_mnist                    # plain, non-federated MNIST training loop
python -m src.test_compatibility_mnist        # checks MnistCNN is Opacus/DP-SGD compatible
python -m src.test_compatibility_cifar10      # checks Cifar10ResNet20 is Opacus/DP-SGD compatible

# One ad-hoc FL simulation using server.py's own default globals
python -m src.server

# The actual experiment grid (8 configs x their parameter variants)
python -m src.scripts.run_configurations --config 1     # just config 1 (baseline) and its variants
python -m src.scripts.run_configurations --all           # every config
python -m src.scripts.run_configurations --config 1 --no-attack   # clean (no-Byzantine) baseline

# Visualize a results folder
python -m src.scripts.visualize_results src/results/<run_timestamp>/

The 8 configurations (see code/src/scripts/run_configurations.py):

#	DP	FLTrust	TopK
1	-	-	-
2	x	-	-
3	-	x	-
4	-	-	x
5	x	x	-
6	x	-	x
7	-	x	x
8	x	x	x

Future work

• TopK doesn't reduce actual bytes on the wire yet. fit() always returns a dense, full-shaped parameter vector (Flower's FedAvg-style aggregation needs every client's arrays to line up by position); mechanisms/topk.py's update_size_bytes() only computes what a real sparse (index, value) encoding would cost. Making TopK actually shrink what's transmitted is still open.
• Clean up server.py's leftover module-level globals (NUM_CLIENTS, USE_DP, etc.) - these are overridden by every real experiment run via ExperimentConfig and only remain for the python -m src.server ad-hoc entry point; worth relocating or removing.
• Deduplicate the flat-vector reshape logic in client.py/mechanisms/attacks.py into the unflatten() helper mechanisms/topk.py already provides.
• CIFAR10 deeper investigation into the CIFAR10 dataset and optimization of the model.
• And more :)