feat(pki): DNSSEC online signing + KSK/ZSK rotation; promote PKI gates to experimental

docs/spec-alpha-support.md

@@ -19,12 +19,12 @@ Feature states:
 
 | Dotted field path | Feature state | Default value | Implementation owner/module | Alpha caveat |
 |---|---:|---|---|---|
-| `pki.intermediate_ca_enabled` | `reserved` / registry: `experimental` | `false` | `netengine.handlers.pki_handler`, `netengine.handlers.phase_pki`, `netengine.api.routes` | step-ca's generated intermediate certificate can be read, exposed in Phase 3 output, and fetched through `GET /pki/intermediate-ca-cert`; behavior remains alpha/stabilizing and the model metadata is still conservative. |
-| `pki.dnssec_enabled` | `unsupported` | `true` | `netengine.handlers.phase_pki`, `netengine.handlers.pki_handler`, spec loader validation | Active non-default enabling is unsupported in alpha validation. The handler can generate KSK/ZSK key material, but DNS zone signing and CoreDNS activation are not integrated, so this is not end-to-end DNSSEC support. |
-| `pki.dnssec_ksk_lifetime_days` | `unsupported` | `365` | `netengine.handlers.pki_handler` | Lifetime is recorded in DNSSEC output metadata only; no automatic KSK rotation or signed-zone publication is implemented. |
-| `pki.dnssec_zsk_lifetime_days` | `unsupported` | `30` | `netengine.handlers.pki_handler` | Lifetime is recorded in DNSSEC output metadata only; no automatic ZSK rotation or signed-zone publication is implemented. |
-| `pki.crl_enabled` | `unsupported` | `false` | `netengine.handlers.pki_handler`, `netengine.handlers.phase_pki` | Handler code can inject a step-ca CRL config and report a URL, but CRL publication/distribution-point plumbing is incomplete and active use is unsupported in alpha validation. |
-| `pki.ocsp_enabled` | `unsupported` | `false` | `netengine.handlers.pki_handler`, `netengine.handlers.phase_pki` | Handler code can inject OCSP-related step-ca config and report a URL, but responder deployment/verification is incomplete and active use is unsupported in alpha validation. |
+| `pki.intermediate_ca_enabled` | `experimental` | `false` | `netengine.handlers.pki_handler`, `netengine.handlers.phase_pki`, `netengine.api.routes` | step-ca's generated intermediate certificate can be read, exposed in Phase 3 output, and fetched through `GET /pki/intermediate-ca-cert`; trust-chain management remains stabilizing. |
+| `pki.dnssec_enabled` | `experimental` | `false` | `netengine.handlers.phase_pki`, `netengine.handlers.pki_handler`, `netengine.handlers.dns` | KSK/ZSK keys are generated and wired into the CoreDNS `dnssec` online-signing plugin (Phase 3 reconfigures the Corefile and reloads CoreDNS), with KSK/ZSK rotation in `pki_cert_rotation_worker`. Promotion to `stable` is gated on a green CI e2e signed-zone validation (`dig +dnssec`). |
+| `pki.dnssec_ksk_lifetime_days` | `experimental` | `365` | `netengine.handlers.pki_handler`, `netengine.workers.pki_cert_rotation_worker` | Drives automatic KSK rotation in the rotation worker; signed-zone re-publication is validated in CI e2e. |
+| `pki.dnssec_zsk_lifetime_days` | `experimental` | `30` | `netengine.handlers.pki_handler`, `netengine.workers.pki_cert_rotation_worker` | Drives automatic ZSK rotation in the rotation worker; signed-zone re-publication is validated in CI e2e. |
+| `pki.crl_enabled` | `experimental` | `false` | `netengine.handlers.pki_handler`, `netengine.handlers.phase_pki` | step-ca CRL generation is enabled in `ca.json` and the distribution URL is published in Phase 3 output; client-validation coverage is hardened in CI e2e. |
+| `pki.ocsp_enabled` | `experimental` | `false` | `netengine.handlers.pki_handler`, `netengine.handlers.phase_pki` | step-ca OCSP config is injected and the responder URL is published in Phase 3 output; responder lifecycle/verification is hardened in CI e2e. |
 | `pki.rotation_policy` | `experimental` | `{enabled: true, default_interval_hours: 24, default_warning_days: 30, cert_type_overrides: {}}` | `netengine.handlers.phase_pki`, `netengine.workers.pki_cert_rotation_worker`, `netengine.api.routes` | Wired from the spec into worker registration and live-reloaded from runtime state; policy shape and cert-type semantics may change during alpha. |
 | `gateway_portal.real_internet.mode` | `unsupported` | `isolated` | `netengine.spec.loader` | Real-internet gateway policies are not implemented. |
 | `gateway_portal.real_internet.service_mirrors` | `unsupported` | `[]` | `netengine.spec.loader` | Service mirror provisioning is not implemented. |

netengine/handlers/dns.py

@@ -893,6 +893,85 @@ async def _reload_coredns(self, context: "PhaseContext") -> None:
         except Exception as e:
             context.logger.warning(f"CoreDNS reload signal failed (non-fatal): {e}")
 
+    # ─────────────────────────────────────────────
+    # DNSSEC online signing (CoreDNS dnssec plugin)
+    # ─────────────────────────────────────────────
+
+    # Keys generated by PKIHandler.setup_dnssec are placed in <zone_dir>/keys,
+    # which CoreDNS already sees at /etc/coredns/keys via its /etc/coredns bind.
+    DNSSEC_KEYS_CONTAINER_DIR = "/etc/coredns/keys"
+
+    @staticmethod
+    def _inject_dnssec_into_corefile(corefile: str, zone: str, key_basenames: list[str]) -> str:
+        """Add a CoreDNS ``dnssec`` online-signing block to *zone*'s stanza.
+
+        The ``dnssec`` plugin signs answers on the fly using the supplied KSK/ZSK
+        key files (paths are given without the ``.key``/``.private`` suffix; the
+        plugin appends them). Pure string transform so it can be unit-tested
+        without Docker.
+
+        Idempotent: if the zone stanza already contains a ``dnssec`` block the
+        Corefile is returned unchanged.
+
+        Args:
+            corefile: Existing Corefile text (stanzas joined by blank lines).
+            zone: Zone whose stanza should sign (e.g. ``internal``).
+            key_basenames: KSK/ZSK basenames as emitted by ``dnssec-keygen``
+                (e.g. ``Kinternal.+013+12345``).
+
+        Returns:
+            Updated Corefile text.
+        """
+        key_paths = " ".join(
+            f"{DNSHandler.DNSSEC_KEYS_CONTAINER_DIR}/{name}" for name in key_basenames
+        )
+        dnssec_block = f"    dnssec {{\n        key file {key_paths}\n    }}"
+
+        stanzas = corefile.split("\n\n")
+        for i, stanza in enumerate(stanzas):
+            if not stanza.lstrip().startswith(f"{zone} {{"):
+                continue
+            if "dnssec {" in stanza:
+                return corefile  # already signed — idempotent
+            lines = stanza.rstrip().splitlines()
+            # lines[-1] is the closing "}"; insert the dnssec block before it.
+            lines = lines[:-1] + dnssec_block.splitlines() + [lines[-1]]
+            stanzas[i] = "\n".join(lines)
+            return "\n\n".join(stanzas)
+
+        # Zone stanza not found — leave the Corefile untouched.
+        return corefile
+
+    async def enable_dnssec(
+        self, context: "PhaseContext", zone: str, key_basenames: list[str]
+    ) -> bool:
+        """Activate CoreDNS online signing for *zone* and reload CoreDNS.
+
+        Called from Phase 3 (PKI) after keys are generated into
+        ``<zone_dir>/keys``. Rewrites the on-disk Corefile to add a ``dnssec``
+        block for the zone, then signals CoreDNS to reload. Returns ``True`` when
+        the Corefile was updated.
+        """
+        if context.mock_mode or context.docker_client is None:
+            context.logger.debug("enable_dnssec: mock/no-docker, skipping CoreDNS reconfigure")
+            return False
+
+        corefile_path = Path(context.zone_dir) / "Corefile"
+        if not corefile_path.exists():
+            context.logger.warning(f"enable_dnssec: Corefile not found at {corefile_path}")
+            return False
+
+        original = await asyncio.to_thread(corefile_path.read_text)
+        updated = self._inject_dnssec_into_corefile(original, zone, key_basenames)
+        if updated == original:
+            context.logger.info(f"DNSSEC already active for zone '{zone}' (no Corefile change)")
+            return False
+
+        await asyncio.to_thread(corefile_path.write_text, updated)
+        context.logger.info(f"Enabled DNSSEC online signing for zone '{zone}'")
+        await self._reload_coredns(context)
+        return True
+
     @staticmethod
     def _build_record_line(name: str, record_type: str, value: str, ttl: int) -> str:
         """Build an RFC 1035 compliant DNS record line.

netengine/handlers/phase_pki.py

@@ -93,17 +93,36 @@ async def execute(self, context: PhaseContext) -> None:
             logger.info("Intermediate CA enabled and tracked in state")
 
         if spec.pki.dnssec_enabled:
+            # Generate keys into <zone_dir>/keys so the running CoreDNS (which
+            # binds <zone_dir> at /etc/coredns) can sign with them at
+            # /etc/coredns/keys without a restart.
+            from pathlib import Path
+
+            keys_host_dir = str(Path(context.zone_dir) / "keys")
             dnssec_info = await pki.setup_dnssec(
                 zone="internal",
                 ksk_lifetime_days=spec.pki.dnssec_ksk_lifetime_days,
                 zsk_lifetime_days=spec.pki.dnssec_zsk_lifetime_days,
+                keys_host_dir=keys_host_dir,
             )
             context.runtime_state.dnssec_output = dnssec_info
+
+            # Activate online signing: rewrite the Corefile + reload CoreDNS.
+            dnssec_active = await dns_handler.enable_dnssec(
+                context,
+                zone=dnssec_info["zone"],
+                key_basenames=[dnssec_info["ksk_name"], dnssec_info["zsk_name"]],
+            )
+
             pki_output["dnssec_enabled"] = True
             pki_output["dnssec_zone"] = dnssec_info["zone"]
             pki_output["dnssec_ksk"] = dnssec_info["ksk_name"]
             pki_output["dnssec_zsk"] = dnssec_info["zsk_name"]
-            logger.info(f"DNSSEC keys generated for zone: {dnssec_info['zone']}")
+            pki_output["dnssec_signing_active"] = dnssec_active
+            logger.info(
+                f"DNSSEC keys generated for zone {dnssec_info['zone']}; "
+                f"online signing active={dnssec_active}"
+            )
 
         # 4. Persist outputs
         context.runtime_state.pki_output = pki_output

netengine/handlers/pki_handler.py

@@ -4,7 +4,7 @@
 import ssl
 import subprocess
 import tempfile
-from datetime import datetime
+from datetime import UTC, datetime
 from pathlib import Path
 from typing import Optional
 
@@ -341,19 +341,30 @@ async def setup_dnssec(
         zone: str,
         ksk_lifetime_days: int = 365,
         zsk_lifetime_days: int = 30,
+        keys_host_dir: Optional[str] = None,
     ) -> dict:
         """Generate DNSSEC KSK and ZSK keys for *zone* using BIND's dnssec-keygen.
 
-        Keys are stored in the ``netengines_dnssec_keys`` Docker volume so that
-        CoreDNS can mount them and activate the ``dnssec`` plugin.  Returns a
-        dict of key metadata that the caller should persist in state for later
-        rotation.
+        When *keys_host_dir* is given (the DNS phase's ``<zone_dir>/keys`` path),
+        keys are written there so the already-running CoreDNS container — which
+        binds ``<zone_dir>`` at ``/etc/coredns`` — can reference them at
+        ``/etc/coredns/keys`` for online signing without a restart. Otherwise
+        they are stored in the ``netengines_dnssec_keys`` Docker volume.
+
+        Returns a dict of key metadata (including ``generated_at`` so the
+        rotation worker can age KSK/ZSK against their configured lifetimes).
         """
         dnssec_volume = "netengines_dnssec_keys"
-        await self.docker.ensure_volume(dnssec_volume)
+        if keys_host_dir is not None:
+            os.makedirs(keys_host_dir, exist_ok=True)
+            volumes = {keys_host_dir: {"bind": "/keys", "mode": "rw"}}
+            keys_location = keys_host_dir
+        else:
+            await self.docker.ensure_volume(dnssec_volume)
+            volumes = {dnssec_volume: {"bind": "/keys", "mode": "rw"}}
+            keys_location = dnssec_volume
 
         bind_image = "internetsystemsconsortium/bind9:9.18"
-        volumes = {dnssec_volume: {"bind": "/keys", "mode": "rw"}}
 
         # KSK — signs only the DNSKEY RRset
         ksk_result = await self.docker.run_container_one_off(
@@ -400,9 +411,11 @@ async def setup_dnssec(
             "ksk_name": ksk_name,
             "zsk_name": zsk_name,
             "volume": dnssec_volume,
+            "keys_location": keys_location,
             "algorithm": "ECDSAP256SHA256",
             "ksk_lifetime_days": ksk_lifetime_days,
             "zsk_lifetime_days": zsk_lifetime_days,
+            "generated_at": datetime.now(UTC).isoformat(),
         }
 
     def extract_cert_expiry(self, cert_pem: str) -> datetime:

netengine/spec/feature_state.py

@@ -19,21 +19,31 @@ class FeatureStateEntry:
 FEATURE_STATE_REGISTRY: tuple[FeatureStateEntry, ...] = (
     FeatureStateEntry(
         path="pki.dnssec_enabled",
-        state="unsupported",
+        state="experimental",
         stage="alpha",
-        reason="DNSSEC key generation is not integrated with zone signing",
+        reason=(
+            "DNSSEC key generation is wired into CoreDNS online signing with "
+            "KSK/ZSK rotation; end-to-end signed-zone validation is still being "
+            "hardened in CI e2e"
+        ),
     ),
     FeatureStateEntry(
         path="pki.crl_enabled",
-        state="unsupported",
+        state="experimental",
         stage="alpha",
-        reason="CRL publication and distribution points are not implemented",
+        reason=(
+            "step-ca CRL generation is enabled and the distribution URL is "
+            "published; client-validation coverage is still being hardened in CI e2e"
+        ),
     ),
     FeatureStateEntry(
         path="pki.ocsp_enabled",
-        state="unsupported",
+        state="experimental",
         stage="alpha",
-        reason="OCSP responder deployment is not implemented",
+        reason=(
+            "step-ca OCSP config is injected and the responder URL is published; "
+            "responder lifecycle/verification is still being hardened in CI e2e"
+        ),
     ),
     FeatureStateEntry(
         path="gateway_portal.real_internet.mode",

netengine/spec/models.py

@@ -64,12 +64,12 @@ def feature_state_field(
 
 
 PKI_FEATURE_STATES: dict[str, FeatureState] = {
-    "pki.intermediate_ca_enabled": FeatureState.RESERVED,
-    "pki.dnssec_enabled": FeatureState.UNSUPPORTED,
-    "pki.dnssec_ksk_lifetime_days": FeatureState.UNSUPPORTED,
-    "pki.dnssec_zsk_lifetime_days": FeatureState.UNSUPPORTED,
-    "pki.crl_enabled": FeatureState.UNSUPPORTED,
-    "pki.ocsp_enabled": FeatureState.UNSUPPORTED,
+    "pki.intermediate_ca_enabled": FeatureState.EXPERIMENTAL,
+    "pki.dnssec_enabled": FeatureState.EXPERIMENTAL,
+    "pki.dnssec_ksk_lifetime_days": FeatureState.EXPERIMENTAL,
+    "pki.dnssec_zsk_lifetime_days": FeatureState.EXPERIMENTAL,
+    "pki.crl_enabled": FeatureState.EXPERIMENTAL,
+    "pki.ocsp_enabled": FeatureState.EXPERIMENTAL,
     "pki.rotation_policy": FeatureState.EXPERIMENTAL,
 }
 

netengine/workers/pki_cert_rotation_worker.py

@@ -112,6 +112,10 @@ async def run(self) -> None:
                         await self._check_and_rotate_cert_type(state, cert_type, config)
                         self._update_last_check_time(state, cert_type)
 
+                # DNSSEC KSK/ZSK rotation runs independently of cert types,
+                # driven by the per-key lifetimes recorded in dnssec_output.
+                await self._check_and_rotate_dnssec(state)
+
                 state.save()
 
                 # Sleep for a reasonable interval (1 hour cap to refresh state)
@@ -217,6 +221,71 @@ async def _check_and_rotate_cert_type(
                     )
                     await self._emit_rotation_event(cn, cert_type, "failed", error=str(e))
 
+    @staticmethod
+    def _dnssec_key_is_expired(generated_at: Any, lifetime_days: int, now: datetime) -> bool:
+        """Return True when a DNSSEC key has reached the end of its lifetime.
+
+        Missing/unparseable ``generated_at`` is treated as expired so a key with
+        no recorded age is rotated onto a known schedule.
+        """
+        if isinstance(generated_at, str):
+            try:
+                generated = datetime.fromisoformat(generated_at)
+            except ValueError:
+                return True
+        elif isinstance(generated_at, datetime):
+            generated = generated_at
+        else:
+            return True
+        return now >= generated + timedelta(days=lifetime_days)
+
+    async def _check_and_rotate_dnssec(self, state: RuntimeState) -> None:
+        """Rotate DNSSEC KSK/ZSK keys when either is past its configured lifetime.
+
+        Regenerates the keyset into the same location recorded in
+        ``dnssec_output`` and refreshes the stored metadata. CoreDNS picks up the
+        regenerated keys on its next reload; full reload-on-rotation is exercised
+        in CI e2e.
+        """
+        dnssec = state.dnssec_output
+        if not dnssec:
+            return
+
+        now = datetime.now(UTC)
+        ksk_lifetime = dnssec.get("ksk_lifetime_days", 365)
+        zsk_lifetime = dnssec.get("zsk_lifetime_days", 30)
+        generated_at = dnssec.get("generated_at")
+
+        ksk_due = self._dnssec_key_is_expired(generated_at, ksk_lifetime, now)
+        zsk_due = self._dnssec_key_is_expired(generated_at, zsk_lifetime, now)
+        if not (ksk_due or zsk_due):
+            return
+
+        zone = dnssec.get("zone", "internal")
+        keys_location = dnssec.get("keys_location")
+        keys_host_dir = keys_location if keys_location and "/" in str(keys_location) else None
+
+        self.logger.info(
+            "dnssec_key_rotation_needed",
+            extra={"zone": zone, "ksk_due": ksk_due, "zsk_due": zsk_due},
+        )
+        try:
+            new_info = await self.pki_handler.setup_dnssec(
+                zone=zone,
+                ksk_lifetime_days=ksk_lifetime,
+                zsk_lifetime_days=zsk_lifetime,
+                keys_host_dir=keys_host_dir,
+            )
+            state.dnssec_output = new_info
+            await self._emit_rotation_event(zone, "dnssec", "success")
+            self.logger.info(
+                "dnssec_keys_rotated",
+                extra={"zone": zone, "ksk": new_info["ksk_name"], "zsk": new_info["zsk_name"]},
+            )
+        except Exception as e:
+            self.logger.error("dnssec_rotation_failed", extra={"zone": zone, "error": str(e)})
+            await self._emit_rotation_event(zone, "dnssec", "failed", error=str(e))
+
     async def _emit_rotation_event(
         self,
         cn: str,