Skip to content

fix(pppoe): bind the session interface unnumbered on ha restore - #510

Merged
BSpendlove merged 1 commit into
mainfrom
fix/pppoe-ha-restore-unnumbered
Aug 29, 2026
Merged

fix(pppoe): bind the session interface unnumbered on ha restore#510
BSpendlove merged 1 commit into
mainfrom
fix/pppoe-ha-restore-unnumbered

Conversation

@BSpendlove

Copy link
Copy Markdown
Member

Problem

A PPPoE subscriber restored onto a promoted node gets a session interface with no IPv4 enabled. restoreFromHASync creates the interface and re-applies the MSS clamp, but never binds it unnumbered to the subscriber gateway loopback. VPP therefore keeps its ip4-not-enabled guard on that interface's ip4-unicast arc and drops the subscriber's traffic at the end of the arc.

Caught on the promoted node during a failover:

ip4-unicast on pppoe_session0:
  ip4-sv-reassembly-feature
  tcp-mss-clamping-ip4-in
  cgnat-in2out
  ip4-not-enabled          <- blackholes whatever reaches it

pppoe_session0 had no address and no unnumbered binding; loop100 had IPv4 enabled. Counters read cgnat-in2out: 8 translated, cgnat-in2out-slowpath: 1 translated, null-node: 9 blackholed — every translated packet destroyed.

Every other path that builds a session interface binds it: fresh bring-up, the opdb replay in setupSessionRestore, and IPoE's own restoreFromHASync (internal/ipoe/restore.go). Only the PPPoE HA path was missing it, so this restores the rule rather than inventing one.

Why it surfaced now

CGNAT was hiding it. cgnat-in2out used to leave the feature arc by jumping straight to ip4-lookup, which skipped the arc end and therefore skipped ip4-not-enabled: translated traffic was being routed off an interface VPP had been told not to route on. That bypass was veesix-networks/osvbng-vpp#25, fixed in veesix-networks/osvbng-vpp#35. With the plugin corrected to stay on the arc, the guard does what it says and the latent gap became a traffic failure.

This is why the fix is urgent rather than cosmetic: dataplane-latest already carries that plugin, so main's own nightly will fail suite 15 until this lands.

Verification

Suite 15 on the rig, three ways, same lab:

plugin this fix result
pre-#35 26/26
post-#35 no 25/26Verify Traffic Recovers After Failover, 0/10 flows
post-#35 yes 26/26

Suite 14 (14-ha-failover-ipoe) also passes 26/26 with the new plugin and this fix. It failed once in CI, which was the flake it looked like: IPoE already had the unnumbered call, so nothing here changes its behaviour.

go build ./..., go test ./internal/pppoe/... and gofmt are clean.

Not fixed here

The same function does not re-apply the IPv6 address or the delegated prefix either, which setupSessionRestore does — so IPv6 is likely unbound after a PPPoE failover as well. No rig suite covers IPv6 after failover, so I have left it rather than ship a change nothing would catch if it were wrong. Happy to do it with a suite that proves it.

A PPPoE subscriber restored onto a promoted node gets a session
interface with no IPv4 enabled. restoreFromHASync creates the interface
and re-applies the MSS clamp, but never binds it unnumbered to the
subscriber gateway loopback, so VPP keeps ip4-not-enabled on that
interface's ip4-unicast arc and drops the subscriber's traffic at the
end of the arc.

Every other path that builds a session interface does bind it: fresh
bring-up, the opdb replay in setupSessionRestore, and IPoE's own
restoreFromHASync. Only the PPPoE HA path was missing it, so this is
the odd one out rather than a new rule.

It went unnoticed because CGNAT was hiding it. cgnat-in2out used to
leave the feature arc by jumping straight to ip4-lookup, which skipped
the arc end and therefore skipped ip4-not-enabled: translated traffic
was routed off an interface VPP had been told not to route on. Once the
plugin was corrected to stay on the arc
(veesix-networks/osvbng-vpp#35), the guard did what it says and suite 15
started failing on the traffic-recovery check.

Verified on the rig, suite 15 three ways: 26 of 26 with the old
plugin, 25 of 26 with the new plugin and no fix (Verify Traffic
Recovers After Failover, 0 of 10 flows, with cgnat-in2out counting 8
translated and null-node 9 blackholed), and 26 of 26 with the new
plugin and this fix. Confirmed directly on the promoted node that
pppoe_session0 carried ip4-not-enabled on its arc and had no address or
unnumbered binding while loop100 had IPv4 enabled.

Not fixed here: the same function does not re-apply the IPv6 address or
the delegated prefix either, which setupSessionRestore does. No rig
suite covers IPv6 after a PPPoE failover, so that is left rather than
shipped unverified.
@BSpendlove
BSpendlove deployed to integration-rig August 29, 2026 08:52 — with GitHub Actions Active
@BSpendlove
BSpendlove deployed to integration-rig August 29, 2026 11:56 — with GitHub Actions Active
@BSpendlove
BSpendlove merged commit 1e7b0f7 into main Aug 29, 2026
40 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant