haproxy_req_fp

HAProxy request fingerprint module that computes a compact, structured fingerprint string for every HTTP request/response pair and exposes it as a transaction variable.

It is provided in three forms:

• haproxy_req_fp.lua — an in-process Lua action that sets txn.req_fp.
• haproxy_req_fp_spoa.py — a standalone SPOE agent that sets txn.reqfp.req_fp.
• haproxy_req_fp_spoa_rust/ — a standalone Rust SPOE agent that sets txn.reqfp.req_fp with significantly lower overhead than the Python agent.

---

Overview

haproxy_req_fp is the HAProxy counterpart of nginx_req_fp. It generates the same 17-field fingerprint format. The Lua variant runs inside HAProxy as a native action, while the SPOE variants offload the work to external Python or Rust agents. The Rust agent is the fastest external option; see the Benchmarks section for numbers.

5FX6Li0LMETnpsF7YNY_ge_11_02_aae_ssz_4-7-25_0000_07_aacchuu_enus_n_c_s_r_200_712

---

Why not existing tools?

Tool	Limitation
hfinger	Offline only — consumes pcap files after the fact; no live path
JA3 / JARM	TLS layer only — sees the handshake, not HTTP structure or parameters
ModSecurity + CRS	Rule-based anomaly scoring, not fingerprinting; heavyweight; not designed for ML consumption
Zeek / Bro	Separate network-tap infrastructure required; operates on raw packets
Akamai HTTP/2 FP	Closed research; HTTP/2 SETTINGS only; doesn't capture application-layer signals

This module sits inside HAProxy, requires no separate infrastructure, captures both network-layer signals (HTTP version, header patterns) and application-layer signals (parameter shapes, content types, auth presence) in a single pass, and produces structured output that needs zero further parsing to feed an ML pipeline.

---

Why not store full requests and responses?

Storage cost — A full request with headers, body, and response can be kilobytes to megabytes. A fingerprint is ~100–200 bytes. At 10 000 req/s that is the difference between ~10 GB/day and ~150 MB/day of log data.

Privacy and compliance — Full requests contain passwords, session tokens, PII, health data, and payment information. The fingerprint captures shape (param count, value types, value lengths) without capturing content. You can retain fingerprints indefinitely under most data-handling policies; you cannot do that with raw requests.

ML readiness — Full HTTP logs require complex parsing, tokenization, and normalization before any model can consume them. Fingerprint fields are already structured, typed, and bounded — they load directly into a feature matrix.

Noise reduction — Most bytes in a raw request are irrelevant to anomaly detection (the actual values of normal parameters, response HTML, etc.). The fingerprint discards that noise at source.

---

Goal

Identify requests that fall outside the expected behavior patterns of a specific application.

A given web application has a stable fingerprint distribution: specific paths receive specific methods, specific parameter shapes, specific header sets, and specific content types. Legitimate browsers and API clients produce consistent, predictable fingerprints. Attackers, scanners, misconfigured clients, and bots do not.

By collecting fingerprints over time you build a behavioral baseline per endpoint and flag anything that deviates: missing headers, unexpected auth methods, wrong content types for the method, unusual path depths, parameters that are suddenly objects instead of strings.

The module is designed to be always on in production. The Lua action runs once per response and adds negligible overhead.

---

Fingerprint format

{path_b62}_{method2}_{http_ver}_{path_depth}_{param_keys}_{param_types}_{param_lens}_{req_ctype}_{hdr_count}_{hdr_list}_{accept_lang}_{auth_type}_{cookie}_{cookie_fields}_{referer}_{status}_{body_bytes}

17 underscore-separated fields, grouped by purpose:

Request line

#	Field	Description
1	path_b62	URI path bytes as a big-endian integer, base62-encoded
2	method2	First 2 chars of HTTP method, lowercased (ge, po, pu, de, …)
3	http_ver	2-digit protocol version: 09 10 11 20 30
4	path_depth	Number of / chars in URI, zero-padded 2 digits, max 99

Parameters (query string; see Body params for POST/PUT/PATCH)

#	Field	Description
5	param_keys	First char of each param name, sorted; nil if none
6	param_types	Type code per value (same order); nil if none
7	param_lens	Decoded value lengths, dash-separated; 0 if none
8	req_ctype	First 4 alpha chars of request Content-Type subtype; 0000 if absent (e.g. json, html, xwww, form)

Request headers

#	Field	Description
9	hdr_count	Total request header count, zero-padded 2 digits, max 99
10	hdr_list	Sorted first-char initials of all header names (limit 32); nil if none

Client context

#	Field	Description
11	accept_lang	First 4 lowercase alpha chars of primary Accept-Language tag (hyphens/digits stripped); 0000 if absent
12	auth_type	n none · b Basic · t Bearer/token · d Digest · o other
13	cookie	c if Cookie header present, n if absent
14	cookie_fields	Sorted first-char initials of cookie field names (limit 32); nil if none
15	referer	n no Referer · s same-domain · x cross-domain

Response

#	Field	Description
16	status	HTTP response status code
17	body_bytes	Response Content-Length; 0 if header absent

Value type codes

Code	Type	Example values
i	integer	42, -7
f	float	3.14, -0.5
s	string	hello, foo-bar
c	char (single char)	x
b	bool	true, false
t	time	14:30:00
d	date	2026-02-19
z	datetime with timezone	2026-02-19T22:54:13+05:00
e	empty	`` (zero-length value)
o	object	{…} (starts with {)
l	list / array	[…] (starts with [)

---

SPOE application

Standalone SPOE agents offload fingerprint generation to an external process. The Python agent (haproxy_req_fp_spoa.py) is easy to customize, while the Rust agent (haproxy_req_fp_spoa_rust/) is significantly faster.

Requirements

• HAProxy 2.0 or newer with SPOE support
• Python 3.8 or newer

python3 -m pip install -r requirements.txt

Run the agent

python3 haproxy_req_fp_spoa.py [--host 0.0.0.0] [--port 9002]

HAProxy configuration

Add the SPOE filter to your frontend and point HAProxy at the SPOE config file (haproxy_req_fp.spoe.conf):

frontend http-in
    bind *:80
    filter spoe engine req-fp config /etc/haproxy/haproxy_req_fp.spoe.conf
    log-format "%ci:%cp [%tQ] %[var(txn.reqfp.req_fp)]"
    default_backend app

The backend used by the SPOE engine must be declared in haproxy.cfg. See haproxy.cfg.example for a complete setup.

The agent exposes the fingerprint as txn.reqfp.req_fp.

Rust SPOA

A Rust implementation of the SPOE agent lives in haproxy_req_fp_spoa_rust/. It supports the same SPOE arguments and dynamic fields as the Python agent, plus an optional batched HTTP sender for JSON records.

Build

cd haproxy_req_fp_spoa_rust
cargo build --release

Run

./target/release/haproxy_req_fp_spoa --host 0.0.0.0 --port 9003

With an HTTP logging endpoint:

./target/release/haproxy_req_fp_spoa \
    --host 0.0.0.0 --port 9003 \
    --http-endpoint https://log.example.com/reqfp

See haproxy_req_fp_spoa_rust/README.md for all CLI options and configuration file usage.

---

Requirements (Lua module)

• HAProxy 2.0 or newer
• Compiled with Lua 5.3 or LuaJIT support

haproxy -vv | grep -i lua
# Expected: Built with Lua version = Lua 5.3.x  or  LuaJIT ...

---

Installation (Lua module)

cp haproxy_req_fp.lua /etc/haproxy/haproxy_req_fp.lua

---

Configuration

SPOE minimal setup

1. Start the Python agent:

   python3 haproxy_req_fp_spoa.py

   Or start the Rust agent:

   ./haproxy_req_fp_spoa_rust/target/release/haproxy_req_fp_spoa --port 9003

2. Reference haproxy_req_fp.spoe.conf from your HAProxy frontend:

   frontend http-in
       bind *:80
       filter spoe engine req-fp config /etc/haproxy/haproxy_req_fp.spoe.conf
       log-format "%ci [%tQ] %[var(txn.reqfp.req_fp)]"
       default_backend app

   See haproxy.cfg.example for the matching backend req-fp-backend.

SPOE JSON log

log-format '{"ts":"%tQ","client":"%ci","method":"%HM","path":"%HP","status":%ST,"fp":"%[var(txn.reqfp.req_fp)]"}'

SPOE inject header

filter spoe engine req-fp config /etc/haproxy/haproxy_req_fp.spoe.conf
http-response set-header X-Req-FP %[var(txn.reqfp.req_fp)]

Lua minimal setup

global
    lua-load /etc/haproxy/haproxy_req_fp.lua

frontend http-in
    bind *:80
    mode http

    http-response lua.req_fp
    log-format "%ci [%tQ] %[var(txn.req_fp)]"

    default_backend app

---

Body params

The default action runs in the http-res phase. By that point the request body has been forwarded to the backend, so application/x-www-form-urlencoded body parameters are not included in param_keys / param_types / param_lens.

To include body params, use the two-phase pattern:

1. Enable request body buffering:

   option http-buffer-request

2. Register a companion http-req action in your Lua script that reads txn.req:dup(), parses the body params, and stores them in a txn variable (e.g. txn.req_fp_body_params).

3. In the existing build_fingerprint function, read that variable and merge the body params before sorting.

A reference implementation of this extension is planned; see the issue tracker.

---

Differences from nginx_req_fp

Aspect	nginx_req_fp	haproxy_req_fp (Python SPOA)	haproxy_req_fp (Rust SPOA)	haproxy_req_fp (Lua)
Language	C (compiled module)	Python (external SPOA)	Rust (compiled binary)	Lua (runtime script)
Reload	Requires nginx rebuild	Restart agent process	Restart agent process	lua-load — edit and reload HAProxy
Body params	Via nginx body filter	Requires option http-buffer-request + separate SPOE message	Requires option http-buffer-request + separate SPOE message	Requires option http-buffer-request + two-phase pattern
body_bytes source	Filter-chain byte counter	Response Content-Length header	Response Content-Length header	Response Content-Length header
Variable name	$req_fp	txn.reqfp.req_fp	txn.reqfp.req_fp	txn.req_fp
Access in config	$req_fp anywhere	%[var(txn.reqfp.req_fp)]	%[var(txn.reqfp.req_fp)]	%[var(txn.req_fp)] in log-format / ACLs

The fingerprint string format and all field definitions are identical, so the same parsing code and ML models work against logs from both proxies.

---

Benchmarks

A k6 benchmark harness in benchmark/ compares the cost of computing the fingerprint in HAProxy Lua, the Python SPOA, and the Rust SPOA.

cd benchmark
./run.sh              # single-thread
THREADS=8 ./run.sh    # multi-thread

Requirements: k6, jq, python3, and a HAProxy binary built with Lua support. The included benchmark/haproxy was built from HAProxy 3.4.2 with Lua 5.4 for this workspace; see benchmark/gen-configs.py for the generated configs.

Results on macOS (Apple Silicon), HAProxy 3.4.2, Python 3.14, Rust 1.97:

Single-thread (nbthread 1)

Test	reqs/sec	% baseline	avg (ms)	p95 (ms)	p99 (ms)
Baseline	67 815	100	0.06	0.09	0.14
Lua	66 309	97	0.05	0.09	0.14
Python	12 425	18	0.30	0.43	0.71
Rust	39 027	57	0.10	0.14	0.22

Multi-thread (nbthread 8)

Test	reqs/sec	% baseline	avg (ms)	p95 (ms)	p99 (ms)
Baseline	63 389	100	0.07	0.10	0.14
Lua	65 157	102	0.08	0.10	0.14
Python	12 189	19	0.31	0.42	0.66
Rust	44 992	70	0.10	0.15	0.21

Observations:

• The Lua action runs in-process and adds essentially no overhead.
• The Python SPOA drops throughput to ~18-19% of baseline.
• The Rust SPOA is roughly 3.2x faster than the Python SPOA in single-thread and 3.7x faster with 8 threads, recovering 57-70% of baseline throughput.
• Rust scales noticeably better with nbthread, while Python is limited by the GIL.

---

License

Licensed under the Apache License 2.0 with the Commons Clause condition.

• You may use, copy, modify, and distribute this software freely under the terms of the Apache 2.0 license.
• The Commons Clause restricts selling the software or offering it as a hosted/managed service for a fee.

See LICENSE for the full text.