Skip to content

TESTING.md

Latest commit

 

History

History
225 lines (174 loc) · 6.75 KB

TESTING.md

File metadata and controls

225 lines (174 loc) · 6.75 KB
Raw

pftest: Pre-deployment verification for OpenBSD PF rulesets

A Python simulator that parses real pf.conf syntax and evaluates packet flow against the full ruleset — before you deploy to hardware.

Closes the gap between pfctl -nf (syntax check) and tcpdump (runtime debug). Tests your firewall logic offline, catches rule ordering bugs, verifies VLAN isolation, and simulates attack chains.

What it tests

  • Last-match-wins semantics (no quick = keep evaluating)
  • quick short-circuit rules
  • Anchor ordering and evaluation
  • Tag assignment via match (last match wins) and tag carry through NAT
  • Antispoof expansion into real block rules
  • rdr-to destination rewriting with egress re-evaluation
  • State table simulation with max-src-conn rate limit enforcement
  • NAT tracking (nat-to, rdr-to)
  • route-to for policy routing (VPN kill-switches)
  • Interface groups (egress resolves to WAN)
  • Tables (<tablename> with persist/const members)
  • Macros (full recursive resolution)

What it doesn't test

  • Scrub / TCP reassembly — needs real packets and a TCP stack
  • pfctl syntax edge cases — the parser uses regex, not PF's grammar. Always run pfctl -nf /etc/pf.conf on the router before deploying.

Quick start

your-repo/
  pf.conf              # your main config
  pf.d/                # anchor files
    trust.conf
    iot.conf
    ...
  pftest.py      # the simulator engine
  test_your_rules.py   # your tests (see below)

1. Drop in your config

Place your pf.conf and pf.d/ directory alongside pftest.py. The engine auto-detects your topology:

  • Pairs *_if macros with *_net macros by prefix (trust_if/trust_net, vpn_if/vpn_net, etc.)
  • Detects WAN from wan macro or NAT rules
  • Scans match ... tag rules for any remaining interface-to-network bindings
  • Resolves egress interface group to your WAN

No configuration file needed.

2. Run the built-in tests

python3 pftest.py

This runs the engine's self-tests (rule parsing, evaluation, antispoof, rdr-to rewriting, and state table rate limits) against your config.

3. Write your own tests

from pftest import Packet, load_all_rules, evaluate

rules = load_all_rules()

# test a single packet — use YOUR segment IPs and interfaces
result = evaluate(
    Packet(src="192.0.2.50", dst="198.51.100.1", proto="tcp",
           dport=443, iface="vlan10", direction="in"),
    rules,
)
assert result.action == "pass"
assert result.tag == "MY_TAG"

You can also use the class API directly for isolation:

from pftest import PFSimulator, Packet

sim = PFSimulator(pf_conf="/path/to/pf.conf", pf_d="/path/to/pf.d")
rules = sim.load()
result = sim.evaluate(Packet(src="192.0.2.50", dst="198.51.100.1",
                             proto="tcp", dport=443,
                             iface="vlan10", direction="in"))

Packet fields

Field Type Description
src str Source IP
dst str Destination IP
proto str tcp, udp, or icmp
dport int Destination port (0 = any)
sport int Source port (0 = any)
iface str Interface the packet arrives on / exits from
direction str in or out
tag str Pre-assigned tag (for egress tests)
icmp_type str echoreq, unreach, timex

FlowResult fields

Field Type Description
action str pass or block
rule Rule The matching rule object
tag str Tag assigned during evaluation
nat_applied bool NAT was applied
rdr_applied bool rdr-to rewrote the destination
rdr_dst str Rewritten destination IP
rdr_dport str Rewritten destination port
route_to str Policy routing target
trace list [(rule, action_str)] evaluation trace

Test patterns

Device simulation (ingress + egress)

Test a full connection path — ingress on one interface, tag assignment, then egress on another. Use your own IPs and tags:

# ingress — packet enters on an internal interface
ingress = evaluate(
    Packet(src="192.0.2.10", dst="198.51.100.1", proto="tcp",
           dport=8883, iface="vlan30", direction="in"),
    rules,
)
assert ingress.action == "pass"
assert ingress.tag == "IOT"  # your tag name

# egress — carry tag from ingress to WAN
egress = evaluate(
    Packet(src="192.0.2.10", dst="198.51.100.1", proto="tcp",
           dport=8883, iface="igc1", direction="out",
           tag=ingress.tag),
    rules,
)
assert egress.action == "pass"

Rate limit testing

from pftest import enable_state_tracking, disable_state_tracking

enable_state_tracking()

# send more connections than max-src-conn allows
for i in range(55):
    result = evaluate(
        Packet(src="192.0.2.20", dst="198.51.100.1", proto="tcp",
               dport=443, iface="vlan30", direction="in"),
        rules,
    )
    if i < 50:
        assert result.action == "pass"
    else:
        assert result.action == "block"  # max-src-conn exceeded

disable_state_tracking()

Attack chain simulation

Model multi-step adversary paths:

# Step 1: compromised device phones home (allowed — blends with cloud traffic)
c2 = evaluate(
    Packet(src="192.0.2.50", dst="203.0.113.5", proto="tcp",
           dport=8883, iface="vlan30", direction="in"),
    rules,
)
assert c2.action == "pass"  # MQTT is in allowlist

# Step 2: attacker pivots to another VLAN (blocked — isolation)
pivot = evaluate(
    Packet(src="192.0.2.50", dst="198.51.100.211", proto="tcp",
           dport=445, iface="vlan30", direction="in"),
    rules,
)
assert pivot.action == "block"  # chain stopped

Writing tests

Import PFSimulator and Packet, point at your config, build packets that match your topology, and assert the results. See the test patterns above for examples.

Auto-detected topology

The engine reads your pf.conf and builds a network map automatically:

$ python3 -c "from pftest import *; load_all_rules(); \
  [print(f'  {k:10s} -> {v}') for k,v in sorted(IFACE_NETWORKS.items())]"

This drives antispoof expansion and interface group resolution. No hardcoded values in the engine — it derives everything from your macros and match rules.

Realism: 9/10

The simulator faithfully models PF's rule evaluation semantics. The one gap (scrub/TCP reassembly) requires real packets and a kernel TCP stack. For everything else — rule ordering, tag propagation, antispoof, rdr-to rewriting, rate limits — the simulator matches PF behavior.

Always run pfctl -nf /etc/pf.conf on your actual router before deploying. The simulator catches logic bugs; pfctl catches syntax bugs.

License

BSD 2-Clause (see LICENSE)