fix(supply-chain): verify the web console's provenance at load, and retire the pre-claim assertions (BACKLOG #1193) (#812)

* docs(supply-chain): retire the pre-claim assertions about the console distribution (BACKLOG #1193)

The name messagefoundry-webconsole has been registered on PyPI since the first
webconsole-v* release on 2026-07-29, which is what the guard test's
_PUBLISHED_DISTRIBUTIONS already records. Five shipped artifacts still asserted
the opposite, so a reader who trusted them got the pre-claim world.

Corrected, located by content rather than by line number:
  - .github/workflows/release.yml: the release-webconsole comment said a PENDING
    Trusted Publisher "does NOT reserve the name" and that the first run of the
    job would create the project. That run has happened.
  - docs/WEBCONSOLE-PACKAGE.md: "Until the owner publishes, install by path".
    The remaining true half is that the ENGINE declares no webconsole extra.
  - docs/SERVICE.md: "The wheel is not published to an index yet".
  - packaging/messagefoundry-webconsole/RELEASE.md: "not automated", "no
    PyPI-publishing job for this wheel exists", "an unpublished dependency would
    break uv lock", and the stale claim that release-harness is the precedent to
    mirror. release-webconsole exists and fires on its own tag.
  - messagefoundry/api/app.py: the serve_ui ImportError comment said to point at
    the path install while the string beneath it already named the distribution.
    The comment is corrected, not the string.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(serve): verify the web console's provenance before importing it (BACKLOG #1193)

The engine mounts the web console IN-PROCESS, and `serve` gated that on
`find_spec("messagefoundry_webconsole") is not None` -- PRESENCE, not
provenance. Whatever occupied the import name was executed with the engine's
own privileges, and a wheel's payload runs AT IMPORT.

The check now runs in the else-branch of that same presence test, before
create_managed_app reaches the guarded import in api/app.py. It reads
importlib.metadata and accepts three things and nothing else:

  - the file about to be imported belongs to the installed
    messagefoundry-webconsole distribution, and no other distribution claims
    that import name;
  - it sits in a source checkout of this repository, proved by two markers
    (the packaging pyproject and the package itself). A checkout is the
    STRONGEST case, not the weakest: no index resolved anything;
  - it sits under the directory a PEP 610 editable install recorded, which is
    a provenance statement the installer made rather than one inferred here.

Anything else FAILS CLOSED with a message naming what was found and what was
expected, plus the named opt-out MEFOR_ALLOW_UNVERIFIED_WEBCONSOLE for a
packaging layout the check does not recognise.

Honest scope, recorded at the call site: it covers the documented `serve` /
NSSM path, not an embedder calling create_app itself; it cannot reach a
squatted SDIST, whose build backend runs during install before any engine
process exists; and it identifies the distribution, not its contents.

Negative controls, because a guard never seen to fire is not evidence. Seven
refusal branches are driven, including the shadowing case that a
packages_distributions()-only check would call green. The end-to-end one
plants a package that writes a marker file from its top level:
find_spec reports it present (the old check would have passed), the new check
refuses and names it, and the marker does not exist -- so the refusal happened
BEFORE the payload ran, which is the whole claim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(provenance): widen the install guard onto two blind trees and add the prose arm (BACKLOG #1193)

Two gaps let five artifacts assert an unclaimed name for weeks while every
assertion in this module stayed green.

FILE SET. The scan read zero files of two of the three distributions this repo
builds: messagefoundry_webconsole/ and harness/. Both ship code that raises
operator-facing errors, exactly like the api/app.py RuntimeError this module
exists because of. Both trees are now scanned, and
test_every_packaged_distribution_has_its_code_tree_scanned pins it per tree
rather than as a total, because a total stays healthy while one tree silently
drops to zero. The tree names are read from each packaging pyproject's
force-include target, so a list here cannot drift from the build.

PROSE ARM. Nothing read a factual CLAIM about publication state, only install
commands. test_no_tracked_text_asserts_a_published_distribution_is_unpublished
flags a blank-line block that asserts a distribution is unpublished while
_PUBLISHED_DISTRIBUTIONS classifies it published. Derived from that set, so it
inverts on its own if a name is ever reclassified.

Measured against the five sites at 46ea10a, all five fire. A single-block
rule found four and missed docs/SERVICE.md, whose claim and distribution name
sit one blank line apart, so the window is the claim's block and its two
neighbours. "unclaimed" was tried as a pattern and REMOVED on evidence: it
fired on the console README's sentence saying the opposite, and it caught none
of the five.

Also read: packaging/*/*.md and release.yml, which the install-command set
excludes on the stated ground that nobody pastes them. A factual claim is wrong
wherever it is written, and two of the five lived there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(serve): simplify the provenance helpers (BACKLOG #1193)

Quality pass over the code this branch added.

  - _normalized_distribution hand-rolled a PEP 503 normalizer to dodge an
    `import re`. checks._normalize_dist already states the same rule in one
    line, so this now matches it character for character and names it. It is
    still a local copy rather than an import: checks.py is the
    `messagefoundry check` gate, and importing it would put the whole gate on
    the serve startup path for one line.
  - _console_source_checkout returned its own argument or None, which the
    caller then filtered for truth. It is a predicate, so it is now
    _is_console_source_checkout returning bool and the caller is a plain
    comprehension.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(backlog): record what landed on #1193 and what stays open

Item stays OPEN. Its closing act is a scorecard re-score, which lives in the
vault and is the tracking seat's to perform.

Records the re-measurement at HEAD (the 2026-08-20 research held), two
corrections to its own proposal (the master-test-plan rows are unreachable from
an engine checkout under ADR 0160 D1; docs/SERVICE.md is a fifth site the list
did not name), what both limbs shipped, the negative-control evidence that the
load-path check fires before the payload runs, and the six subjects left open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: wshallwshall <mefordev@messagefoundry.org>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: 3 people

.github/workflows/release.yml

@@ -493,11 +493,12 @@ jobs:
     # so it fires on its OWN `webconsole-v*` tag and has NO `needs: release` — an engine release must not
     # drag the console along, and a console release must not wait on one.
     #
-    # ASVS 15.2.4 (dependency confusion). `messagefoundry-webconsole` is registered on PyPI as a PENDING
-    # Trusted Publisher against this workflow, which grants permission to publish but does NOT reserve the
-    # name: until a distribution is actually uploaded the name is claimable by anyone, and our own docs
-    # reference it. The FIRST successful run of this job creates the project and closes that exposure.
-    # Until then docs must point at the source-tree install (pinned by tests/test_install_instruction_provenance.py).
+    # ASVS 15.2.4 (dependency confusion). `messagefoundry-webconsole` is REGISTERED on PyPI -- the first
+    # `webconsole-v*` run of this job created the project on 2026-07-29 -- so this job now republishes a
+    # name the project owns, and shipped text may name it in a bare-name install. The pre-claim wording
+    # that stood here is retired rather than restated as a retraction (BACKLOG #1193).
+    # tests/test_install_instruction_provenance.py holds the classification (`_PUBLISHED_DISTRIBUTIONS`)
+    # and fails if any tracked text drifts back to the pre-claim claim.
     if: github.repository == 'MEFORORG/MessageFoundry' && startsWith(github.ref_name, 'webconsole-')
     runs-on: ubuntu-latest
     permissions:

docs/BACKLOG.md

@@ -10884,6 +10884,14 @@ Nothing here touches the TLS/FTPS context in the same module, which was already
 > **Filed 2026-08-08 - not started. RESEARCH item: the goal is an HONEST pass, and "cannot honestly reach pass" is a valid finding.** ASVS **15.2.4** (L3) currently scores **partial**. The pinned verb asks that third-party components and all transitive dependencies come from the expected repository and that there is no risk of a dependency confusion attack. What holds it short is the in-scope web console: shipped docs instruct a bare-name index install (`README.md:110`) and the engine loads whatever occupies that import name by presence, not provenance (`messagefoundry/__main__.py:1769`).
 
 > **SHIPPED-BUT-OPEN 2026-08-22 -- builder-2 round 3, same [ADR 0165](adr/0165-a-builder-pr-satisfies-the-ledger-gate-with-a-paired-commit-authored-by-the-dispatcher-or-lander.md) pairing.** The README's supply-chain note claimed signing coverage that two of three releases do not have; the claim is corrected and a derived guard added so the prose cannot drift from the artefacts again.
+>
+> **SHIPPED-BUT-OPEN 2026-09-03 -- the two limbs below landed; the item STAYS OPEN because its closing act is a scorecard re-score, which is the tracking seat's to perform and not a Builder's.** Re-measured at HEAD first, and the 2026-08-20 research held in both places it was checked: `tests/test_install_instruction_provenance.py` still carries an EMPTY `_UNPUBLISHED_DISTRIBUTIONS` with `messagefoundry-webconsole` in `_PUBLISHED_DISTRIBUTIONS`, so the guard remains factually right, and the 2026-08-22 README signing correction plus its derived guard are present and passing, so nothing there was re-fixed. **Two corrections to the research's own proposal, both measured:** the "two web-console test-plan rows" CANNOT be retired from this repository -- `docs/testing/` is excluded by `.gitignore:291` except `VERIFY.md`, the master test plan moved to the vault under [ADR 0160](adr/0160-public-repo-content-policy-operator-and-security-review-material-only.md) D1, and `git ls-files docs/testing` returns one file; and a FIFTH reachable site the list did not name, `docs/SERVICE.md`, said the wheel was not published to an index yet.
+>
+> **Limb A, five stale pre-claim assertions retired**, each located by content: the `release-webconsole` comment in `.github/workflows/release.yml` (a PENDING Trusted Publisher "does NOT reserve the name", whose own text said the first run of the job would create the project -- it has); `docs/WEBCONSOLE-PACKAGE.md` ("Until the owner publishes"), where the still-TRUE half is that the engine declares no `webconsole` extra; `docs/SERVICE.md`; `packaging/messagefoundry-webconsole/RELEASE.md` ("not automated", "no PyPI-publishing job for this wheel exists", and the stale claim that `release-harness` is the precedent to mirror); and the `messagefoundry/api/app.py` `serve_ui` ImportError comment, which told the reader to point at the path install while the string beneath it already named the distribution -- the COMMENT was corrected, not the string. The guard's file set scanned zero files of two of the three trees this repo builds (`messagefoundry_webconsole/`, `harness/`); both are in, pinned PER TREE by `test_every_packaged_distribution_has_its_code_tree_scanned` because a total stays healthy while one tree drops to zero, with the tree names read from each packaging pyproject's force-include target. The new PROSE arm flags a text block asserting a distribution is unpublished while the classification set says published; **measured against the five sites at `46ea10a78`, all five fire.** A single-block rule found four and missed `docs/SERVICE.md`, whose claim and distribution name sit one blank line apart, so the window is the claim's block and its two neighbours. `unclaimed` was tried as a pattern and REMOVED on evidence: it fired on the console README's sentence saying the opposite.
+>
+> **Limb B, the load-path check is BUILT, not declined.** `messagefoundry/__main__.py` mounted the console on `find_spec` presence; the else-branch of that same test now reads `importlib.metadata` and accepts three things -- the file belongs to the installed `messagefoundry-webconsole` and no other distribution claims that import name; it sits in a source checkout of this repository, proved by two markers; or it sits under the directory a PEP 610 editable install recorded. Anything else FAILS CLOSED naming what was found and what was expected, with the named opt-out `MEFOR_ALLOW_UNVERIFIED_WEBCONSOLE`. **Negative-control evidence, since a guard never seen to fire is not evidence:** a planted package writing a marker from its top level gives `find_spec ... is not None -> True` (the old check would have passed it), a refusal naming its path, and `payload executed? False` -- so the refusal precedes the import, which is the whole claim. Seven refusal branches are driven in `tests/test_webconsole_provenance.py`, including the shadowing case a `packages_distributions()`-only check would call green. Scope stated at the call site rather than implied: it covers the documented `serve`/NSSM path and not an embedder calling `create_app` itself, it cannot reach a squatted SDIST (whose build backend runs before any engine process exists), and it identifies the distribution rather than verifying its contents.
+>
+> **Explicitly still open, and NOT touched here:** extending signing, provenance attestation and the SBOM to the console and harness release jobs so the README's unscoped claim becomes true; publishing `requirements.lock` per release; removing the bare-name install form from shipped text as the whole-trigger-arm reading; the `docs/SUPPLY-CHAIN.md:37-38` correction (PEP 740 disclaims that the installer verifies attestations automatically); the two master-test-plan rows, which are unreachable from an engine checkout and belong to whoever holds the vault; the written scope reading for the container image, its dependency locks, the deploy-time supervisor download and the four hook repositories pinned by mutable tag; and the re-score itself.
 > Verdict: research
 > Closing-act: scorecard-rescore
 **Cluster:** Security / ASVS remediation research. **Priority:** P1. **Verdict:** research.

docs/SERVICE.md

@@ -394,10 +394,11 @@ icacls "C:\ProgramData\MessageFoundry\logs" /inheritance:r `
 
 This service is **headless**. Operators watch and run it from the **browser web console** served
 same-origin at `/ui` (not part of the service runtime — a separate, version-matched wheel the engine
-mounts in-process). The wheel is **not published to an index yet**, so install it by path:
+mounts in-process). It publishes as its **own** distribution, so install it into the engine venv:
 
 ```powershell
-pip install -e packaging/messagefoundry-webconsole   # into the engine venv
+pip install messagefoundry-webconsole                # into the engine venv
+pip install -e packaging/messagefoundry-webconsole   # or, from a source checkout
 ```
 
 No switch is needed: the console is **on by default** for a loopback bind

docs/WEBCONSOLE-PACKAGE.md

@@ -14,10 +14,13 @@ This is the authoritative doc for that package: its architecture, the injection
 package agree on, the **version-skew gate** that keeps the two in lockstep of *meaning* (not version),
 how to develop and test it, and its **honest scope** (what the extraction does and does not decouple).
 
-> **Publishing the second wheel is a separate, owner-only step.** The engine `[webconsole]` extra is
-> deliberately **not** wired yet (an unpublished dependency breaks `uv lock`). Until the owner publishes,
-> install the console by path (`pip install -e packaging/messagefoundry-webconsole`). The full publish
-> checklist lives in [`../packaging/messagefoundry-webconsole/RELEASE.md`](../packaging/messagefoundry-webconsole/RELEASE.md).
+> **The wheel is published; the engine-side extra is still not wired.** The distribution name
+> `messagefoundry-webconsole` was registered on PyPI on 2026-07-29 by the first `webconsole-v*` release,
+> so `pip install messagefoundry-webconsole` resolves to this project's own wheel. What is still missing
+> is the **engine** side: the engine `pyproject.toml` declares no `webconsole` key, so an install of the
+> engine's `webconsole` extra still fails. Install the console as its own distribution, or from a
+> checkout by path (`pip install -e packaging/messagefoundry-webconsole`). Wiring the extra is step 1 of
+> [`../packaging/messagefoundry-webconsole/RELEASE.md`](../packaging/messagefoundry-webconsole/RELEASE.md).
 
 ---