ci(notice): a red DAST nightly notified nobody, and the issue body named the wrong workflow (BACKLOG #318)

#318 carries this as a tracked follow-up rather than deferred scope: `dast.yml` has NO
`pull_request` trigger, deliberately, and is not a required context, so every finding
from the authenticated authorization sweep surfaced in the Actions tab and nowhere else.
An authenticated security sweep reporting into the void is the exact shape
`nightly-notice.yml` exists to end -- it just was not watching it.

Widened `workflows:` to include DAST, which is the fix the item names.

ALSO FIXED, because widening made it worse rather than introducing it: the issue BODY
opened with a hardcoded "Nightly (scheduled) CI failed." whatever had actually run. The
TITLE was already derived from $WF_NAME; the body was not, so a red Security run has been
opening an issue whose first line names CI. With a third watched workflow that is a
reader deciding what broke from a wrong sentence. It now reads from the same $WF_NAME the
title does.

NEW GUARD, one level up from the item. A watched name that no workflow answers to, or one
whose workflow has no `schedule:` trigger, can never match: the notice job fires only when
the completed run's event was `schedule`, so such an entry sits in the list looking like
protection and matches nothing, forever, silently. That is the notice's own failure mode
turned on itself. `test_every_watched_workflow_exists_and_can_actually_fire` asserts both
arms for EVERY watched name, so a fourth workflow added later cannot be added wrongly. Its
scan carries a positive control, because an empty name map would make every assertion in
it vacuous.

The module docstring claimed it "pins the three ways" the notice could stop working; there
were already six tests. A count in prose has no checker and has to be maintained by
whoever adds the next test, so it now states the kind rather than the number (SDS-3.6).

Four mutants, all killed: drop DAST from the list; watch a name no workflow answers to;
watch a real workflow that has no cron; restore the hardcoded CI in the body. Every mutant
asserted a unique anchor and a changed file hash before scoring, and every restore was
byte-identical. Two share a test by design -- they are its two arms -- and their failure
messages were checked to differ, the typo case listing every name actually present.

Scope: the nightly-notice widening only. Increment 2 of #318 (schema-driven breadth, the
MLLP/TCP/X12 ingress fuzzing, the /ui plane, a TLS black-box target) is untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: wshallwshall

.github/workflows/nightly-notice.yml

@@ -20,14 +20,23 @@ name: Nightly failure notice
 # SCOPE: schedule-only, deliberately. A push/PR failure is already visible on the PR itself; alerting
 # there would be pure noise, and `workflow_run` fires for every CI completion regardless of trigger.
 #
-# TWO WORKFLOWS, ONE ISSUE PER WORKFLOW. `Security` was added because its daily cron carries jobs that
-# do not run on a PR -- `released-line-audit` most of all, whose whole subject (the LATEST RELEASE) no
-# PR can change. The issue title is DERIVED from the completed workflow's name, so a green nightly CI
-# cannot close an issue opened by a red Security run: a single shared title would let one signal close
-# the other, which is the same silence this workflow exists to end.
+# THREE WORKFLOWS, ONE ISSUE PER WORKFLOW. `Security` was added because its daily cron carries jobs
+# that do not run on a PR -- `released-line-audit` most of all, whose whole subject (the LATEST
+# RELEASE) no PR can change. `DAST` was added for a stronger version of the same reason (BACKLOG
+# #318): it has NO `pull_request` trigger AT ALL, deliberately, so before this every DAST finding
+# surfaced in the Actions tab and nowhere else -- an authenticated authorization sweep reporting into
+# the void, which is precisely the shape this workflow exists to end.
+#
+# A WATCHED WORKFLOW MUST HAVE A `schedule:` TRIGGER. The job below fires only when the completed
+# run's event was `schedule`, so watching a workflow with no cron adds a name that can never match:
+# dead config that reads as coverage. `tests/test_nightly_notice.py` pins this for every watched name.
+#
+# The issue title is DERIVED from the completed workflow's name, so a green nightly CI cannot close an
+# issue opened by a red Security or DAST run: a single shared title would let one signal close
+# another, which is the same silence this workflow exists to end.
 on:
   workflow_run:
-    workflows: ["CI", "Security"]
+    workflows: ["CI", "Security", "DAST"]
     types: [completed]
 
 # Read-only by default; the one job that writes escalates to `issues: write` and nothing else.
@@ -93,7 +102,7 @@ jobs:
             exit 0
           fi
 
-          BODY="Nightly (scheduled) CI failed.
+          BODY="Nightly (scheduled) $WF_NAME failed.
 
           - run: $RUN_URL
           - commit: \`$HEAD_SHA\`
@@ -102,7 +111,8 @@ jobs:
           A scheduled run is not a PR context, so this failure appears nowhere else. For CI that is the
           server-DB store, load/throughput and service-smoke legs the three required \`test\` legs skip;
           for Security it is the daily dependency audits and the released-line audit, whose subject is a
-          published release that no pull request can change.
+          published release that no pull request can change; for DAST it is the entire workflow, which
+          carries no \`pull_request\` trigger by design.
 
           This issue is opened once and commented on each subsequent failure; it closes itself when a
           nightly goes green again."

tests/test_nightly_notice.py

@@ -10,7 +10,7 @@
 suites (exactly what the three required ``test`` legs SKIP) could break invisibly.
 
 ``.github/workflows/nightly-notice.yml`` turns that silence into one deduplicated issue. This module
-pins the three ways it could quietly stop working.
+pins the structural ways it could quietly stop working -- at least the ones checkable before it ships.
 
 WHAT CANNOT BE TESTED HERE, stated rather than papered over. A ``workflow_run`` workflow only triggers
 from the **default branch**, so this one cannot fire on the PR that adds it — its end-to-end behaviour
@@ -70,6 +70,79 @@ def test_it_also_watches_the_security_workflow() -> None:
     )
 
 
+def test_it_also_watches_the_dast_workflow() -> None:
+    """DAST needs this more than either of the others (BACKLOG #318).
+
+    ``dast.yml`` has NO ``pull_request`` trigger at all -- deliberately -- so before this widening a
+    genuine authorization finding surfaced in the Actions tab and nowhere else. An authenticated
+    security sweep reporting into the void is the exact shape this notice exists to end.
+    """
+    watched = _on(_load(_NOTICE))["workflow_run"]["workflows"]
+    dast_name = _load(_WORKFLOWS / "dast.yml").get("name")
+    assert dast_name, "dast.yml has no `name:` -- workflow_run has nothing to key on"
+    assert dast_name in watched, (
+        f"nightly-notice.yml watches {watched} but dast.yml is named {dast_name!r}. Its findings "
+        "would then reach nobody, which is the gap BACKLOG #318 recorded."
+    )
+
+
+def test_every_watched_workflow_exists_and_can_actually_fire() -> None:
+    """A watched name that no workflow answers to, or that has no cron, is dead config reading as
+    coverage.
+
+    The notice job gates on ``workflow_run.event == 'schedule'``, so a watched workflow with no
+    ``schedule:`` trigger can never satisfy it -- the name sits in the list looking like protection
+    and matches nothing, forever, silently. That is the same failure the notice exists to fix, one
+    level up, so it is asserted for EVERY watched name rather than per workflow.
+    """
+    watched = _on(_load(_NOTICE))["workflow_run"]["workflows"]
+    assert watched, "the watch list is empty"
+
+    by_name: dict[str, Path] = {}
+    for path in sorted(_WORKFLOWS.glob("*.yml")):
+        name = _load(path).get("name")
+        if isinstance(name, str):
+            by_name.setdefault(name, path)
+    # Positive control: the scan must actually be reading workflows, or every assertion below would
+    # be vacuous against an empty map.
+    assert len(by_name) > 5, f"the workflow scan found only {len(by_name)} named files"
+
+    for name in watched:
+        path = by_name.get(name)
+        assert path is not None, (
+            f"nightly-notice.yml watches {name!r} but no workflow in {_WORKFLOWS.name}/ is named that. "
+            f"A workflow_run trigger matches on the NAME, so this entry can never fire. "
+            f"Names present: {sorted(by_name)}"
+        )
+        triggers = _on(_load(path))
+        assert "schedule" in triggers, (
+            f"nightly-notice.yml watches {name!r} ({path.name}) but that workflow has no `schedule:` "
+            "trigger. The notice job only fires when the completed run's event was `schedule`, so "
+            "this entry can never match -- dead config that reads as coverage."
+        )
+
+
+def test_the_issue_body_names_the_workflow_that_failed() -> None:
+    """The TITLE was always derived from the completed workflow; the BODY was not.
+
+    It opened with a hardcoded "CI failed" whatever had run, so a red Security run produced an issue
+    whose first line named the wrong workflow. Harmless-looking, and exactly the kind of thing a
+    reader uses to decide what broke. Widening the watch list to a third workflow made it worse
+    rather than introducing it.
+    """
+    body = "\n".join(
+        str(s.get("run", "")) for s in _load(_NOTICE)["jobs"]["notice"]["steps"] if "run" in s
+    )
+    assert body, "the notice job has no `run:` step to inspect"
+    assert "Nightly (scheduled) $WF_NAME failed." in body, (
+        "the issue body does not name the workflow that actually failed. It must read from $WF_NAME, "
+        "the same value the title is derived from, or it will assert the wrong workflow broke."
+    )
+    assert "Nightly (scheduled) CI failed." not in body, (
+        "the body still hardcodes CI, so a Security or DAST failure opens an issue naming CI."
+    )
+
+
 def test_it_only_reacts_to_scheduled_runs() -> None:
     """Without this gate every PR and push failure opens an issue.