feat: primary mode — AXFR zone transfers with automatic NOTIFY and TSIG - #5
Open
jleal52 wants to merge 35 commits into
Open
feat: primary mode — AXFR zone transfers with automatic NOTIFY and TSIG#5jleal52 wants to merge 35 commits into
jleal52 wants to merge 35 commits into
Conversation
Scope analysis for making pdns-etcd3 act as an authoritative primary serving AXFR to an external secondary: gap analysis of remote-backend methods, functional requirements (list/getUpdatedMasters/setNotified/ getTSIGKey), serial monotonicity, TSIG, and DNSSEC presigned interplay. Three open design decisions captured in §10.
- Snapshot: RLock the zone subtree during the in-memory list walk (TCP transfer happens in PowerDNS afterwards, no backend lock held). - Serial: project zoneRev() to uint32 (RFC 1982 monotone), keep the X-PE3-MINIMUM-SERIAL floor; X-PE3-FIXED-SERIAL keeps precedence. - TSIG keys: global named store under reserved -tsig-/<name> pseudo-prefix.
Bite-sized TDD task breakdown across 4 phases (F1 AXFR core, F2 NOTIFY, F3 TSIG, F4 DNSSEC presigned) plus transversal versioning/docs, grounded in exact file/line references and the repo's unit/integration test idioms. Refines design decision R4: notified_serial is kept in-memory (not in etcd) to avoid a NOTIFY feedback loop; primary mode requires standalone. Design doc updated accordingly.
End-to-end AXFR proof: seeds an unsigned zone into etcd, starts PowerDNS with the pe3 remote backend in primary mode (allow-axfr-ips + master/ primary), and verifies a full zone transfer via a miekg/dns dns.Transfer client (SOA brackets the transfer; NS and A records present). Named TestPDNSAXFR so CI's `-run PDNS` matrix job (PDNS 34..51) executes it. Cannot be run locally in this environment (host VPN occupies pe3's port 8053, which also fails the pre-existing TestWithPDNS); validated to compile, vet, and follow the harness/etcd-seeding conventions. CI validates the run.
…change) A TSIG key mutation under the watched prefix otherwise resolved to dataRoot and scheduled an all-zones reload. TSIG keys are read on demand and never cached in the tree, so the watch path now ignores -tsig- events entirely, matching reload's early skip.
walkZoneRecordsAuth's delegation predicate checked dn.records["NS"][""], but NS records are stored keyed by non-empty id (NS#1, NS#first, ...), so real multi-NS delegations were not detected and their NS/glue/subtree were wrongly emitted as auth=true. Detect via len(dn.records["NS"]) > 0.
Named TestPDNSAXFRTSIG so CI's -run PDNS matrix executes it. Cannot run locally (host VPN occupies pe3 port 8053); validated to compile, vet, and follow the harness conventions. CI validates the run.
Review found the doc implied a reserved metadata key that doesn't exist in code. The notified serial is exposed via the notified_serial JSON field of getDomainInfo/getUpdatedMasters/getAllDomains, not an etcd metadata key.
Named TestPDNSAXFRPresigned so CI's -run PDNS matrix executes it. Cannot run locally (host VPN occupies pe3 port 8053); validated to compile, vet, and follow harness conventions. CI validates the run.
The recursive walks read dn.children/records without a lock while zone reloads rebuild them under a write lock — a concurrent-map panic that primary-mode getUpdatedMasters polling now exercises regularly. Each node now self-RLocks during its walk (parent-before-child, matching getChild). getDomainInfo now reports getQname() like the bulk methods.
…iver reload() stored metadata entries on dn (the reload receiver) instead of itemData (the node the entry belongs to), unlike values/defaults/options. A freshly-created zone reloads via the root (handleEvents -> dataRoot), so a zone's metadata landed on the root node and was invisible to the zone: - soaSerial() never saw X-PE3-FIXED-SERIAL -> wrong (auto) SOA serial, - PowerDNS got empty PRESIGNED -> dropped DNSKEY/RRSIG from presigned AXFR, - PowerDNS got empty TSIG-ALLOW-AXFR -> TSIG-gated AXFR unauthorized. Store on itemData (for an apex-level reload itemData == dn, so the common path is unchanged). Found by the AXFR integration tests; regression-guarded by a reload-on-root unit test.
- primaryModeSetting(): set master=yes only for PDNS <4.5 and primary=yes for >=4.5 (mutually exclusive). The previous "set both" approach FATALed on PDNS 5.0, which removed the deprecated "master" alias — every TestPDNS* AXFR test died at PDNS startup. TestPDNSAXFR and TestPDNSAXFRPresigned now pass end-to-end. - TestPDNSAXFRTSIG: t.Skip as WIP. With metadata fixed, TSIG-ALLOW-AXFR is served correctly, but PowerDNS never calls getTSIGKey/getTSIGKeys on the remote backend, so a signed AXFR is denied (NOTAUTH). Needs investigation of PowerDNS remote-backend TSIG key retrieval. Root cause documented inline.
Enabling remote-dnssec=yes in PowerDNS (required to unlock getTSIGKey for TSIG-secured AXFR) makes PowerDNS enumerate per-zone DNSSEC keys via getDomainKeys. pe3 manages none (zones are plain or pre-signed), so report an empty set instead of returning an "unimplemented" error.
Root cause of the previously-skipped TestPDNSAXFRTSIG: PowerDNS's remote backend gates getTSIGKey/getTSIGKeys behind its dnssec flag (remotebackend.cc: `if (!d_dnssec) return false;`), so without remote-dnssec=yes PowerDNS never queries pe3 for the TSIG key and denies the signed transfer with NOTAUTH. Setting remote-dnssec=yes makes the signed and unsigned subtests pass end-to-end (PDNS 5.0). Un-skip the test and document the required PowerDNS setting in README + doc/ETCD-structure.md.
Spin up an actual secondary DNS server (ISC BIND9) in its own container on a shared docker network and verify the full primary->secondary flow against PowerDNS + the pe3 remote backend: - BIND transfers example.net. from the primary via AXFR on startup and serves it (www.example.net. A 192.0.2.1); - after the zone changes in etcd (serial bumps), `pdns_control notify-host` makes PowerDNS NOTIFY the secondary, which re-transfers and serves the new record (www2.example.net.). A short SOA refresh is the fallback. startPDNS gains an optional trailing networks arg (variadic, existing callers unchanged) to join the shared network. This also covers the previously deferred end-to-end NOTIFY test. Runs/passes locally and in CI (-run PDNS); pulls internetsystemsconsortium/bind9:9.20 at runtime via testcontainers.
These were working artifacts (design analysis + TDD implementation plan); they are not project documentation and are kept out of the upstream PR. The user-facing docs live in README.md and doc/ETCD-structure.md.
The notified serial was kept in process memory, so automatic NOTIFY only worked in standalone mode (in pipe mode PowerDNS spawns a process per request thread and the state was not shared). Persist it instead in a global -notified-/<id> etcd pseudo-entry: - It lives OUTSIDE any zone prefix, so writing it never enters a zone's zoneRev()/serial (no NOTIFY feedback loop) and is skipped by reload and handleEvents (like -tsig-). - It is keyed by a now-DETERMINISTIC domain_id (31-bit FNV-1a of the zone name) instead of an in-memory counter, so every process computes the same id and setNotified — which only receives the id — needs no reverse lookup. - getUpdatedMasters/getDomainInfo/getAllDomains read it on demand; setNotified writes it. Being in etcd it is shared across processes → NOTIFY now works in any run mode (pipe and standalone). Replaces the in-memory zoneRegistry with domainID() + etcd helpers. Verified by unit tests (deterministic id, update filter) and integration tests (notified serial persists/reads back via etcd; the real BIND9 secondary still gets NOTIFY-driven updates; TestWithPDNS unaffected).
Reverses the earlier "automatic NOTIFY requires standalone mode" note now that the notified serial is persisted in a global -notified-/<id> etcd entry (shared across processes), so NOTIFY works in pipe mode too. Documents the new pseudo-entry in README and doc/ETCD-structure.md.
Adds TestPDNSAXFRSecondaryPipe: PowerDNS runs the pe3 BINARY in PIPE mode (remote-connection-string=pipe:command=/pdns-etcd3,...; distributor-threads=1), spawning a separate process per request — the operator's real deployment shape. It builds a static pe3 binary, runs etcd + PowerDNS(pipe) + a real ISC BIND9 secondary on a shared docker network, then verifies the secondary AXFRs the zone and picks up a later change via NOTIFY. This proves primary operation — and the now etcd-persisted notified serial, shared across the separate spawned processes — works in pipe mode, not only standalone. Skipped for PDNS < 4.4 (older/non-default protocol; the pipe protocol itself is covered by TestPipeRequests). startETCD gains an optional networks arg (variadic; existing callers unchanged).
AXFR-out via the remote-backend list method is not supported on the legacy PowerDNS 3.4 protocol (it returns only the bracketing SOA), so the CI matrix job for PDNS 34 failed. Skip the AXFR tests below PDNS 4.0 (they pass on 4.0+) and note the requirement in the README.
jleal52
force-pushed
the
feat/zone-transfer-axfr
branch
from
June 17, 2026 03:58
24b963c to
9168d37
Compare
Owner
|
Thank you for the PR, I will work through it as soon as I can. |
Two cases where the parser rejected keys that exist in production data: - Owner names with a single leading underscore per label (RFC 8552 underscored node names, e.g. _domainkey for delegated DKIM CNAMEs) were dropped on load for hostname-ish record types, so the records were not served (NXDOMAIN). Interior underscores are still rejected. - metaRegex required the KIND#id form, but the zone-revision guard in handleEvents writes the -metadata-/X-PE3-MINIMUM-SERIAL key without an id. Watchers logged '(metadata) invalid key' and skipped the event, and on reload the key's revision was lost, defeating the serial floor. The id part is now optional (empty id), matching what the guard and setDomainMetadata write. No on-etcd shape change: this makes the reader accept what the writers already produce, so dataVersion stays at 2.1.
The standalone HTTP listener binds a fixed 0.0.0.0:8053, which fails on hosts where that port is taken outside the test's view (e.g. WSL2 mirrored networking with a Windows-side DNS proxy). PE3_TEST_HTTP_PORT overrides the port for such hosts; the default stays 8053.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pdns-etcd3 can now act as an authoritative primary: PowerDNS serves outbound AXFR zone
transfers of any zone held in ETCD, sends NOTIFY to secondaries when a zone changes, and
(optionally) authenticates transfers with TSIG. Every zone is reported to PowerDNS as
MASTERautomatically. An external secondary (BIND, Knot, NSD, PowerDNS, …) can replicate yourETCD-backed zones over the standard DNS transfer protocol — in any run mode (pipe or standalone).
Motivation
Until now the backend could only answer direct queries: a zone living in ETCD could not be
replicated to DNS servers outside the cluster. Production setups want redundancy and geographic
spread without every node talking to ETCD — a primary close to the data and ordinary secondaries
everywhere else, replicating via AXFR/NOTIFY/TSIG like any other zone. Speaking the standard
primary side of zone transfer is the missing piece that lets pdns-etcd3 fit into real
authoritative DNS topologies.
What changed
src/lookup.go— newlistmethod (AXFR-OUT): a zone-subtree walk emitting every record,stopping at child zones, with correct
authflags for delegationNS/glue (pre-signed DNSSECzones transfer intact).
src/metadata.go/src/data.go—getDomainInfo/getAllDomainsreportkind=MASTER,a
domain_idandnotified_serial;getUpdatedMasters/getUpdatedPrimaries+setNotifieddrive automatic NOTIFY.
src/zoneid.go(new) — a deterministicdomain_id(31-bit FNV-1a of the zone name) and thenotified-serial persistence. The notified serial is stored in ETCD under a global
-notified-/<id>pseudo-entry (outside any zone's prefix, so recording it never bumps a zone'sserial → no NOTIFY feedback loop; shared across processes → automatic NOTIFY works in pipe mode
too, not only standalone). Keying by the deterministic id means
setNotifiedneeds no reverselookup.
src/rr.go— the SOA serial is projected touint32for the wire (RFC 1982-safe);X-PE3-FIXED-SERIALkeeps precedence.src/tsig.go(new) +src/const.go/src/data.go/src/pdns-etcd3.go— TSIG keysstored as a global
-tsig-/<keyname>pseudo-entry, read on demand bygetTSIGKey/getTSIGKeys; per-zone ACL viaTSIG-ALLOW-AXFRmetadata.getDomainKeysreturns[].src/data.go(bug fix) — reloaded metadata is now stored on the entry's own node instead ofthe reload receiver. A freshly-created zone reloads via the root, so its metadata previously
landed on the root node and was invisible — silently breaking
X-PE3-FIXED-SERIAL,PRESIGNED(incl. the existing pre-signed feature) and
TSIG-ALLOW-AXFRfor newly-created zones.README.md/doc/ETCD-structure.md— new "Primary mode (AXFR zone transfer)" and "TSIGkeys" sections; Features / support matrix updated;
dataVersion2.0 → 2.1 (new on-ETCD shapes:the global
-tsig-and-notified-pseudo-entries).Operational notes
persisted in ETCD and shared across the per-request processes PowerDNS spawns in pipe mode.
remote-dnssec=yesin PowerDNS, whose remote backend gatesgetTSIGKeybehindthe
dnssecflag (hencegetDomainKeysreturning[]).Out of scope (deliberate)
Tests
make unit-tests— serial projection, deterministicdomain_id, the NOTIFY update filter, thezone walk + delegation auth,
-tsig-parsing, TSIG value parsing, and a metadata-placementregression.
make integration-tests—TestPDNS*against real PowerDNS containers: AXFR of an unsignedzone; a pre-signed DNSSEC zone (DNSKEY/RRSIG transferred, serial pinned); TSIG-secured AXFR
(signed accepted, unsigned refused); the notified serial persists in ETCD; and a real ISC
BIND9 secondary that AXFRs the zone and picks up a later change via NOTIFY — in both
standalone (HTTP connector) and pipe (PowerDNS spawning the pe3 binary per request) modes.
Backwards compatibility
Additive: new remote-backend methods only; existing record paths and queries are unchanged. No new
required configuration for plain AXFR (primary NOTIFY and TSIG are opt-in via PowerDNS settings).
New on-ETCD key shapes are the global
-tsig-and-notified-pseudo-entries;dataVersionbumped 2.0 → 2.1.