ESST-eval: eBPF-Based System Security Tools Evaluation Set

This is an evaluation set to support the reproducibility guarantee and expansion of the paper "An In-Depth Analysis of eBPF-Based System Security Tools in Cloud-Native Environments"

Type of Experiment

There are four types of evaluations:

I) Microbenchmark - Process

ii) Microbenchmark - Network

iii) Policy Enforcement Benchmark

iv) Log Loss Benchmark

Test Results

Microbenchmark - Process

without policy:

ESST	Default	KubeArmor	Tetragon	Falco	Tracee
#1	4.9546279	5.884083	5.29466	5.32804242	5.45508866
#2	4.99669445	5.85285226	5.27523491	5.31152793	5.41237749
#3	4.94440715	5.80859747	5.30988963	5.34787037	5.41883315
#4	4.94896562	5.81991029	5.27711935	5.35604799	5.40608776
#5	4.91936588	5.82914561	5.24667667	5.3761733	5.3776263
#6	4.93692131	5.83131865	5.27454857	5.36462793	5.37809519
#7	4.89521857	5.82953357	5.26141843	5.36093241	5.44926667
#8	4.93907795	5.85411104	5.28040272	5.35737035	5.46158744
#9	4.92011812	5.85158206	5.2588743	5.37134664	5.46026351
#10	4.89535763	5.81645298	5.26701554	5.36487302	5.45504472

with policy:

ESST	KubeArmor	Tetragon	Falco	Tracee
#1	5.864683998	5.434441259	5.455088663	6.78024249
#2	5.89490546	5.396746904	5.412377491	6.76667327
#3	5.86622289	5.412324396	5.418833154	6.719877885
#4	5.839614717	5.434468692	5.406087764	6.780599546
#5	5.851240285	5.41572771	5.377626304	6.775699062
#6	5.813149078	5.424959486	5.378095194	6.820738264
#7	5.835470297	5.456274724	5.44926667	6.855191617
#8	5.815764466	5.353248727	5.461587435	6.775040962
#9	5.748165138	5.404906642	5.460263513	6.827484696
#10	5.71680199	5.447916813	5.455044716	6.803393761

Microbenchmark - Network

without policy:

ESST	Default	Tetragon	KubeArmor	Falco	Tracee
#1	911565758	980952841	1579325196	1092520422	914031612
#2	791854524	864247898	1574744383	959094342	802394873
#3	801044351	712086300	1551366565	981143339	805227747
#4	816065203	769298762	1552224796	961273956	809579426
#5	813907377	905623654	1607570915	941121128	799572999
#6	844102908	862437498	1547471361	1074715932	826175744
#7	888660483	943065623	1546436201	987505162	884975693
#8	808515693	891411595	1611855450	996925550	865109472
#9	809158617	813617686	1532497655	975673495	898661617
#10	841977078	845333590	1529342227	1016028050	844705609

with policy:

ESST	Tetragon	KubeArmor	Falco	Tracee
#1	1573097367	1569590505	1138944383	1514338057
#2	1381840549	1536923069	1000183864	1347081330
#3	1379944310	1556327384	1023741229	1391171302
#4	1444440121	1591000463	992499020	1498975537
#5	1532673331	1574107893	975817867	1510037274
#6	1403913922	1572257617	1196679695	1363375191
#7	1367545661	1558812382	982011779	1351925439
#8	1438301239	1579163099	1029921988	1454435721
#9	1476760079	1577179906	1053440978	1486243649
#10	1400373072	1587914474	996458747	1362776223

Policy Enforcement Benchmark

ESST	KubeArmor	Tetragon	Falco	Tracee
#1	0.34539482	3.59379425	17.8702008	11.8775833
#2	0.50570934	4.82516413	18.0638405	11.7425662
#3	0.55288487	3.59782367	18.0638405	12.084489
#4	0.6004429	4.23034553	16.5723303	11.5343157
#5	0.55288487	5.34684555	17.2325589	12.6610075
#6	0.5766698	4.37272476	18.1763435	12.1359271
#7	0.56405348	3.91951119	17.8205914	12.053461
#8	0.47282188	4.66795495	19.507852	11.5543147
#9	0.54287068	4.88813695	19.8582698	11.3218037
#10	0.49941367	4.38228932	18.345009	11.4469633

Log Loss Benchmark

Throughput	9.8	19.3	49.01	96.42	193.42	486.85	968.05	1874.41	2480.15
KubeArmor	100	11	2.2	2.2	1.1	0.22	0.11	0.02	0.02
Tetragon	100	96	97	86	86	54.47	49.3	30.83	14.81
Falco	100	100	100	100	100	100	100	100	83
Tracee	100	100	100	100	100	100	58	48	10