test: PFCP/N4 integration test suite for the UPF - #5
Open
tariromukute wants to merge 8 commits into
Open
Conversation
Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
Machinery for driving a UPF over N4/PFCP and asserting on the result.
It ships no scenarios and knows nothing about any particular UPF, so a
project supplies only what is specific to its own.
pfcp/ socket, sequence correlation, retries, heartbeat
responder; spec dataclasses <-> scapy IEs; the
PfcpClient facade
models.py PdrSpec / FarSpec / QerSpec / SessionSpec /
ModificationSpec as frozen dataclasses. Lookups such
as spec.qer(id) raise when absent, so a stale rule id
is an immediate error rather than a request that
quietly does nothing
presets.py ready-made session specs
scenarios/ PfcpScenario: a Template Method lifecycle whose
execute() always tears down, never raises, and skips
act() when a precondition fails; plus the registry
and discovery
capabilities.py the Capability enum and the RuleState / QosState /
LogSource protocols -- the seam that lets a scenario
name the question it asks rather than the storage
that answers it
inspect/ generic mechanics: a CommandRunner (the only
subprocess use in the tree) and parsers for bpftool,
tc and windowed logs, with the map catalogue injected
by the caller
report.py soft checks, so one failed assertion never hides the
rest of a scenario
Includes 118 unit tests that pass with no UPF, no root, and no adapter
importable -- the executable form of the claim that this package is
reusable.
Fixtures are output captured from a live UPF rather than invented. That
matters most for tc: -j is honoured for `qdisc show` but silently
ignored for `class show` on iproute2 5.15, so the text parser is the
path actually used.
Assisted-by: Claude:claude-5-opus
Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
pfcpkit/README.md covers the scenario lifecycle, the capability protocols, how to build an adapter for a different UPF, and the conventions worth adopting -- each of which exists because its absence caused a real problem while this suite was built. pfcpkit/examples/ is the same walkthrough as code: an adapter implementing all three protocols over an in-memory dict, a project base class narrowing the context to concrete adapter types, and three scenarios in the styles a reader is likely to need -- PFCP-only, state-asserting, and log-asserting. The examples are deliberately not registered. Nothing carries @register, so importing them adds nothing to the registry and a real run never picks them up; unit/test_boundary.py asserts that by behaviour rather than by inspection. unit/library/test_examples.py drives the examples against a fake client that behaves as a correct UPF and asserts they pass, so a library change that breaks the documented pattern fails the build instead of leaving the guide quietly wrong. It also compiles every Python block in the README. Assisted-by: Claude:claude-5-opus Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
Implements pfcpkit's three capability protocols for this UPF.
Everything true only of the OAI implementation lives here and nowhere
else.
layouts.py the BPF map catalogue and key/value struct layouts
class_ids.py HTB class-id arithmetic mirroring generate_minor_id()
logs.py this UPF's log strings over pfcpkit's windowing
inspector.py OaiRuleState / OaiQosState, and the CAPABILITIES set
the runner compares against each scenario's needs
settings.py Deployment: container, interfaces, bpftool path --
separate from pfcpkit's Settings, which holds only
PFCP addressing and timing
harness.py build_context(): the single place a context is wired
Two facts in layouts.py make an absence assertion pass for the wrong
reason when they drift, so both are pinned by unit tests against a live
dump: the kernel truncates map names to 15 characters
(session_by_ue_ip_map -> session_by_ue_i), and {u16 pdr_id; u64 seid}
is 16 bytes rather than 10.
The class id in class_ids.py is a pure hash of (SEID, QFI) with no
counter, which is why a rebuild targets the classid the previous setup
already created.
build_context() being the only wiring point is what makes the execution
split explicit: bpftool and tc run inside the container, because no BPF
map is pinned and tc is namespace-scoped, while docker logs runs on the
host.
unit/test_boundary.py enforces the rule the type system cannot express:
pfcpkit must never import upf_test, and must never carry a @register of
its own. Checked by parsing every library module rather than by
importing, so an import inside a rarely-taken branch is caught too.
Assisted-by: Claude:claude-5-opus
Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
One registered class per file, named NN-topic.py. The number is the run
order rather than a label: discovery walks the package sorted, so
16-repeated_modification.py runs last because it can leave the UPF
degraded.
01..03 smoke and negative: establishment, CH=1 F-TEID allocation,
duplicate F-SEID
04,05 rules pruned when a modification removes them, and when the
session is deleted
06,14 an Update QER changes the enforced rate; and the tc
operations behind it
07,15 what establishment alone installs, and what it merely records
08 one QER removed while the session continues
09 shaping state released on deletion, shared root left alone
10 UE-IP ownership after re-establishing on a recycled address
11 a Create PDR referencing a FAR the session already has
12,13 how many times one modification reprograms the data plane
16 stability across repeated modification
Each states a requirement and never names a defect, a fix number, or a
source file, so it stays correct once a defect is fixed and simply
starts passing. Every module opens with a Scenario / Expected behaviour
/ Expected output docstring, the last of which says what a passing run
prints -- so a reader can tell "the UPF is wrong" from "I am reading the
wrong scenario".
Three conventions are enforced by unit/test_scenario_conventions.py
rather than documented, because breaking any of them is silent:
* a unique ue_index per scenario. Two scenarios sharing a UE address
make the first one's leftovers the second one's failure, and because
this UPF does not clear its UE-address state on deletion the symptom
looks like a bug in the second scenario. Not hypothetical -- it cost
real debugging time here.
* one registered class per module, with unique and contiguous
numbering
* every capability a scenario reads is declared in `requires`, so a
gap is a reported skip rather than a mid-run failure that reads like
a UPF defect
Against a freshly restarted UPF, 12 of the 16 fail: 22 failing checks
reproducing 9 documented defects. That is the suite working as
intended; `--tag smoke` exits 0.
Assisted-by: Claude:claude-5-opus
Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
run_scenarios.py needs only Python 3.10+ and scapy, so the suite runs in a bare container with no pytest. It offers --list, --scenario, --tag, --all and --junit-xml. Exit codes distinguish outcomes CI should treat differently: 0 every scenario passed, 1 a scenario failed, 2 the suite could not run (bad config, port conflict, unreachable UPF), 130 interrupted. Before running anything it compares each scenario's declared capabilities against the adapter's, and reports every skip by name and missing capability. A suite that silently drops what it cannot observe looks identical to one that passed those scenarios. conftest.py and test_scenarios.py make each discovered scenario a pytest case as well, for -k filtering and JUnit XML, without making pytest a dependency of the scenarios themselves. The generate-tests hook skips a test that parametrises `scenario_cls` itself; otherwise a unit test picking that obvious argument name collides with "duplicate parametrization". Assisted-by: Claude:claude-5-opus Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
docker-compose.yaml the UPF with NET_ADMIN / SYS_ADMIN /
SYS_RESOURCE and three networks named
n4/n3/n6 via Compose's interface_name
(needs Compose >= 2.30), a gnb-sim holding
the N3 gateway address so the UPF's ARP
resolution succeeds, and an optional
containerised test runner
conf/make_test_config.py generates the UPF config from
etc/config.yaml: BPF datapath and QoS on, NF
registration off, debug logging
setup_env.sh brings the stack up and fails fast with a
diagnosis rather than a traceback
Dockerfile.tests the optional runner image
The config generator remaps every interface_name under nfs.upf, not
just n3/n4/n6, because upf::validate() also validates sbi and loops over
all of them -- otherwise the UPF exits at startup probing a device that
does not exist.
Two traps the preflight handles, both of which produced misleading
output before being fixed: `set -o pipefail` with `grep -q` turns a
successful match into a failure via SIGPIPE, which had the port check
cheerfully reporting UDP 8805 free while it was occupied; and waiting on
container state rather than on the readiness banner reported success
moments before the UPF exited, blaming the wrong thing.
Assisted-by: Claude:claude-5-opus
Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
README.md bring-up, teardown, running, configuration,
how to inspect datapath state by hand, how to
add a scenario, and troubleshooting for the
failures this environment actually produces
architecture.drawio five pages: the shape of the suite; how
observing works, with the shell command that
answers each assertion and its real output;
how to read a scenario, quoting execute() and
two real ones; how to extend it; and a module
reference
docs/ARCHITECTURE.md a guided read of the diagram, one section per
page, saying what each shows and which file
to open next
docs/export-diagrams.sh regenerates the PNGs under the filenames the
doc expects
Every file:line citation on the diagram was checked against the source,
and every command and sample output on it was captured from a live run
rather than composed.
unit/test_docs.py keeps the three artefacts in agreement: one embedded
image per diagram page, a section naming each page, export indices
0..n-1 in order, and the script's filenames matching the doc's table.
Adding a page without documenting it fails the build.
It deliberately does not require the PNGs to be present -- the .drawio
is the source of truth, and a fresh clone should not fail its tests for
want of a generated artefact.
Assisted-by: Claude:claude-5-opus
Signed-off-by: Tariro Mukute <tariro.mukute@phine.tech>
Author
Sample run output |
|
CI Build: #11 | Passed |
Contributor
|
I would like to address some points here:
|
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an integration test suite that drives the UPF over N4/PFCP and asserts on what the
data plane actually did. It exists because the UPF had no test infrastructure of any kind —
no
tests/directory, noenable_testing(), andhasUnitTests: falsein the Jenkinsfile —and because a set of PFCP Session Modification and QoS defects fixed pre-refactor had
regressed without anything noticing.
No production code changes. The only file touched outside
ci-scripts/tests/is.dockerignore. Nothing here fixes a defect; the suite makes the defects reproducible ondemand so a fix can be shown to work.
Why
Everything under
src/upf_app/is currently verified by hand or by the external eBPFtutorial job. That makes two things impossible: proving that a fix works, and noticing when
a refactor silently drops one. This suite closes both.
What is in it
pfcpkit/bpftool/tc/log inspectors, and a runnable worked example. Ships no scenarios.upf_test/unit/docs/How to run it
sudois only for the hostFORWARDpermit; without it that step is skipped with a warningand everything else still works.
Restart the UPF before comparing numbers between runs. Deleted sessions leak their HTB
classes, and the class id is a hash of (SEID, QFI), so whether a later run collides with
leftovers depends on where the SEID counter reached. A dirty UPF has been observed to let
qos_rate_changepass, taking the total from 22 failing checks to 20.What it currently reports
map_pruning,map_pruning_on_deleteqos_rate_changeqos_rebuild_tc_failurestcoperations while applying the new rate — PFCP response still an acceptqos_at_establishmentqer_removal_prunes_classqos_class_cleanupstatic_ip_reattachcreate_pdr_existing_farMANDATORY_IE_MISSING; accepted once a Create FAR is addedrebuild_oncerebuild_once_update_onlyrepeated_modification_stabilityReviewing this
Start with
ci-scripts/tests/docs/ARCHITECTURE.md— a five-page diagram with a writtenwalkthrough. Page 1 is the shape of it; page 2 traces each assertion down to the shell
command that answers it, so any verdict can be checked by hand.