fix(store): a missing VIEW DEFINITION grant read as a missing claim proc

ADR 0114 §4 has always specified the startup gate's probe (a) as "OBJECT_ID of
both procs", but the implementation folded (a) into (b) and inferred absence from
a NULL OBJECT_DEFINITION.

MEASURED: a principal holding only EXECUTE on the proc gets a non-NULL OBJECT_ID
and a NULL OBJECT_DEFINITION, and the compat probe still passes. So a deployed,
working, correct procedure was reported as *missing* and the operator was sent to
grant CREATE PROCEDURE — neither the cause nor the cure. WITH ENCRYPTION produces
the identical NULL. This is not hypothetical: the sub-lever B design note in the
same module explicitly serves a fleet whose principal can never hold CREATE
PROCEDURE (DBA-provisioned procs + a least-privilege app principal), which is
exactly the posture that hits it.

The probe now returns OBJECT_ID beside the definition and the two conditions get
separate reasons; the actual cure, GRANT VIEW DEFINITION, is named. Both still
DEGRADE — the gate hashes the body and cannot pass on one it cannot read — so no
accept/reject behaviour changed, only the diagnosis.

The offline stub pins the probe SQL by exact match (a typo'd probe must fail
loudly rather than silently match); that pin moved with the SQL and stayed exact,
and now also asserts both placeholders bind the same object. New offline legs
cover both arms and fail without the fix. A live leg pins the premise no stub can
show — that OBJECT_ID and OBJECT_DEFINITION genuinely disagree on a real server —
using WITH ENCRYPTION, which needs no security principal; the permission half
stays deferred with AC-10's other permission scenarios.

Also adds the store-side half of AC-7's degraded gauge: a claim_proc_status()
accessor on the store protocol (None on every backend without the lever and when
the flag is off, so "not requested" stays distinguishable from "requested and
degraded"). The surfaces that read it land next.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: wshallwshall

messagefoundry/store/base.py

@@ -48,6 +48,7 @@
     AlertInstance,
     CapturedResponse,
     ClaimedHeads,
+    ClaimProcStatus,
     ConnectionEvent,
     ConnectionMetrics,
     DbStatus,
@@ -1294,6 +1295,17 @@ def pool_status(self) -> PoolStatus | None:
         on SQLite (no pool)."""
         ...
 
+    def claim_proc_status(self) -> ClaimProcStatus | None:
+        """The ADR 0114 sub-lever A stored-procedure-claim startup-gate verdict, or ``None`` when
+        this backend has no such lever (AC-6: SQL Server is the only one that reads its flag, whose
+        literal name this module therefore does not write) or that flag is off. AC-7's **degraded
+        gauge** — the surface an operator can actually see the degraded
+        state on (``/status``, ``/metrics``, the console store panel); before it existed the whole
+        signal was one WARNING at ``open()``. Synchronous + free (three attributes the gate recorded
+        once at open — no DB round-trip), read-only, and additive: the ``/status`` field defaults
+        ``None``, so an older client deserializes it unchanged."""
+        ...
+
     async def integrity_check(self) -> tuple[bool, str]: ...
 
     async def connection_metrics(

messagefoundry/store/postgres.py

@@ -108,6 +108,7 @@
     AlertInstance,
     CapturedResponse,
     ClaimedHeads,
+    ClaimProcStatus,
     ConnectionEvent,
     ConnectionMetrics,
     DbStatus,
@@ -1274,6 +1275,11 @@ def pool_status(self) -> PoolStatus | None:
             acquire_wait=self._acquire_wait.summary(),
         )
 
+    def claim_proc_status(self) -> ClaimProcStatus | None:
+        """``None``: the ADR 0114 sub-lever A stored-procedure claim path is SQL-Server-only (AC-6 —
+        this backend never reads its flag), so there is no gate verdict to report here."""
+        return None
+
     async def _fetchall(self, sql: str, *params: Any) -> list[Any]:
         return list(await self._pool.fetch(sql, *params))
 

messagefoundry/store/sqlserver.py

@@ -86,6 +86,7 @@
     AlertInstance,
     CapturedResponse,
     ClaimedHeads,
+    ClaimProcStatus,
     ConnectionEvent,
     ConnectionMetrics,
     DbStatus,
@@ -1757,6 +1758,14 @@ async def _gate_claim_proc(self) -> None:
         (c) compatibility_level >= 130 (OPENJSON). Any miss records the reason, logs a WARNING, and
         leaves ``_claim_proc_effective`` False — the shipped batch runs; NEVER a lane outage.
 
+        (a) and (b) are probed in ONE statement that returns ``OBJECT_ID`` beside the definition,
+        because a NULL body has two very different causes: the proc is absent, or it is deployed
+        and this principal simply cannot READ it (no ``VIEW DEFINITION``, or ``WITH ENCRYPTION``).
+        Both degrade — the gate cannot hash a body it cannot see — but they need opposite remedies,
+        and the second is the exact posture the sub-lever B design note above anticipates — a fleet
+        whose DB principal can never hold CREATE PROCEDURE, i.e. DBA-provisioned procs plus a
+        least-privilege app principal.
+
         The comparison is against the STORED forms, not against ``_claim_proc_body()`` directly:
         the engine rewrites the ``CREATE OR ALTER`` head when it stores the module, so comparing
         with the submitted text can never match (the defect that left this gate inert in every
@@ -1773,16 +1782,34 @@ async def _gate_claim_proc(self) -> None:
             else:
                 expected = _claim_proc_shipped_hashes()
                 for proc_name in (_CLAIM_PROC_CID, _CLAIM_PROC_DST):
+                    # OBJECT_ID rides along so a NULL body can be told apart from an ABSENT proc.
+                    # MEASURED: a principal holding only EXECUTE on the proc gets a non-NULL
+                    # OBJECT_ID and a NULL OBJECT_DEFINITION — the module is deployed and working,
+                    # and the compat probe above still passes. Without the id, that reads as
+                    # "missing" and sends the operator to grant CREATE PROCEDURE, which is not the
+                    # problem and does not fix it. WITH ENCRYPTION produces the identical NULL.
                     row = await self._fetchone(
-                        "SELECT OBJECT_DEFINITION(OBJECT_ID(?)) AS body", (f"dbo.{proc_name}",)
+                        "SELECT OBJECT_ID(?) AS oid, OBJECT_DEFINITION(OBJECT_ID(?)) AS body",
+                        (f"dbo.{proc_name}", f"dbo.{proc_name}"),
                     )
                     deployed = row["body"] if row else None
                     if not deployed:
-                        reason = (
-                            f"stored procedure dbo.{proc_name} is missing (guarded DDL skipped —"
-                            " CREATE PROCEDURE / ALTER-on-schema denied, or a pre-2016-SP1"
-                            " engine?)"
-                        )
+                        if (row["oid"] if row else None) is None:
+                            reason = (
+                                f"stored procedure dbo.{proc_name} is missing (guarded DDL skipped —"
+                                " CREATE PROCEDURE / ALTER-on-schema denied, or a pre-2016-SP1"
+                                " engine?)"
+                            )
+                        else:
+                            reason = (
+                                f"stored procedure dbo.{proc_name} is DEPLOYED but its definition is"
+                                " unreadable (OBJECT_ID resolves, OBJECT_DEFINITION is NULL) — the"
+                                " proc is not missing and CREATE PROCEDURE is not the fix. Either"
+                                " this principal lacks VIEW DEFINITION on it (GRANT VIEW DEFINITION"
+                                f" ON OBJECT::dbo.{proc_name} TO <the engine's principal>) or the"
+                                " module was created WITH ENCRYPTION. The gate compares the body"
+                                " hash, so it cannot pass on a body it cannot read"
+                            )
                         break
                     got = hashlib.sha256(_normalize_tsql(deployed).encode()).hexdigest()
                     matched = expected[proc_name].get(got)
@@ -2908,6 +2935,22 @@ def pool_status(self) -> PoolStatus | None:
             claim_pool=claim_pool,
         )
 
+    def claim_proc_status(self) -> ClaimProcStatus | None:
+        """The ADR 0114 sub-lever A startup-gate verdict — AC-7's **degraded gauge** (``/status``,
+        ``/metrics``, the console's store panel).
+
+        ``None`` when ``fifo_claim_proc`` is off, so "not requested" stays distinguishable from
+        "requested and degraded"; otherwise the gate's own recorded outcome. Synchronous and free —
+        it copies three attributes ``open()`` set once, no DB round-trip. Read-only: nothing here
+        feeds the claim path, and the accept/degrade decision is not re-evaluated."""
+        if not self._fifo_claim_proc:
+            return None
+        return ClaimProcStatus(
+            effective=self._claim_proc_effective,
+            degraded_reason=self._claim_proc_degraded_reason,
+            head_forms=dict(self._claim_proc_head_forms),
+        )
+
     @asynccontextmanager
     async def _cursor(self, conn: Any) -> AsyncIterator[Any]:
         """Yield a cursor that is ALWAYS closed before its connection returns to the pool (EF-6).

messagefoundry/store/store.py

@@ -651,6 +651,31 @@ class ConnectionMetrics:
     destinations: dict[tuple[str, str], DestinationMetrics]  # by (channel_id, destination_name)
 
 
+@dataclass(frozen=True)
+class ClaimProcStatus:
+    """The ADR 0114 sub-lever A stored-procedure-claim startup-gate verdict — AC-7's **degraded
+    gauge**, as an operator-readable snapshot.
+
+    ``None`` from :meth:`~messagefoundry.store.base.QueueStore.claim_proc_status` on any backend
+    without the lever and on SQL Server when its flag is off, so "not requested" is a distinct
+    state from "requested and degraded" rather than an indistinguishable ``False``. (The flag's
+    name is deliberately not written in this module — AC-6's sentinel proves the lever is a no-op
+    here by the absence of that literal.)
+
+    AC-7's compensating-control story assumes an operator can SEE the degraded state. Until this
+    existed the whole signal was one WARNING at ``open()``, in a log nobody was watching — which is
+    a load-bearing part of why the gate could degrade on every open, in every deployment, for the
+    entire life of the feature without anyone noticing.
+    """
+
+    effective: bool  # the gate passed and pooled claims run through the procs
+    degraded_reason: str | None  # why it fell back to the shipped batch; None when effective
+    # proc name -> which stored head form the deployed module matched ("rewritten" | "verbatim").
+    # Populated only when effective. "verbatim" means this engine does NOT rewrite CREATE OR ALTER —
+    # no engine measured to date does, so it is worth reporting (not a fault).
+    head_forms: Mapping[str, str] = field(default_factory=dict)
+
+
 @dataclass(frozen=True)
 class MessageSearchResult:
     """The outcome of a scan-and-decrypt content search (ADR 0046 #51). ``rows`` are matched message
@@ -8251,6 +8276,11 @@ def pool_status(self) -> PoolStatus | None:
         measures does not exist on this backend."""
         return None
 
+    def claim_proc_status(self) -> ClaimProcStatus | None:
+        """``None``: the ADR 0114 sub-lever A stored-procedure claim path is SQL-Server-only (AC-6 —
+        no other backend so much as reads its flag), so there is no gate verdict to report here."""
+        return None
+
     async def integrity_check(self) -> tuple[bool, str]:
         """Run ``PRAGMA quick_check`` (can be slow on a large DB — call on demand only). Runs on a
         pooled read-only connection so a long check never blocks the writer (lockfree-reads)."""

tests/test_adr0114_claim_proc.py

@@ -295,6 +295,8 @@ def _gate_rows(
     compat: int = 150,
     cid_body: str | None = "STORED",
     dst_body: str | None = "STORED",
+    cid_oid: int | None = None,
+    dst_oid: int | None = None,
 ) -> dict[str, dict[str, Any] | None]:
     """Build the _fetchone stub's answers.
 
@@ -307,6 +309,12 @@ def _gate_rows(
     identity function. Both sides of the gate's comparison were then the same function of the same
     argument, so the suite could not distinguish a working gate from a broken one — the defect that
     let ADR 0114 sub-lever A ship inert. Do not restore it.
+
+    ``*_oid`` is the ``OBJECT_ID`` the same probe returns. It DEFAULTS to "present iff the body is",
+    which is the server's behaviour for the two ordinary cases (deployed-and-readable, absent) and
+    keeps ``cid_body=None`` meaning "genuinely missing". Pass an id WITH a ``None`` body to model
+    the third case a real server produces: deployed, but its definition unreadable by this
+    principal (no VIEW DEFINITION, or WITH ENCRYPTION).
     """
 
     def resolve(value: str | None, proc: str, col: str) -> str | None:
@@ -316,14 +324,18 @@ def resolve(value: str | None, proc: str, col: str) -> str | None:
             return _as_object_definition(ss._claim_proc_body(proc, col))
         return value
 
+    def answer(value: str | None, oid: int | None, proc: str, col: str) -> dict[str, Any]:
+        body = resolve(value, proc, col)
+        return {"oid": oid if oid is not None else (917578307 if body else None), "body": body}
+
     return {
         "compat": {"compatibility_level": compat},
-        "dbo.mefor_claim_fifo_heads_cid_v1": {
-            "body": resolve(cid_body, "mefor_claim_fifo_heads_cid_v1", "channel_id")
-        },
-        "dbo.mefor_claim_fifo_heads_dst_v1": {
-            "body": resolve(dst_body, "mefor_claim_fifo_heads_dst_v1", "destination_name")
-        },
+        "dbo.mefor_claim_fifo_heads_cid_v1": answer(
+            cid_body, cid_oid, "mefor_claim_fifo_heads_cid_v1", "channel_id"
+        ),
+        "dbo.mefor_claim_fifo_heads_dst_v1": answer(
+            dst_body, dst_oid, "mefor_claim_fifo_heads_dst_v1", "destination_name"
+        ),
     }
 
 
@@ -333,7 +345,12 @@ def _stub_fetchone(store: SqlServerStore, answers: dict[str, dict[str, Any] | No
     async def fake_fetchone(sql: str, params: tuple[Any, ...] = ()) -> dict[str, Any] | None:
         if sql == "SELECT compatibility_level FROM sys.databases WHERE name = DB_NAME()":
             return answers["compat"]
-        assert sql == "SELECT OBJECT_DEFINITION(OBJECT_ID(?)) AS body", f"unexpected probe: {sql}"
+        assert sql == "SELECT OBJECT_ID(?) AS oid, OBJECT_DEFINITION(OBJECT_ID(?)) AS body", (
+            f"unexpected probe: {sql}"
+        )
+        # Both placeholders bind the SAME name — a probe that bound two different objects would
+        # report one proc's id against another's body.
+        assert params[0] == params[1], f"the probe must bind one object: {params!r}"
         return answers[params[0]]
 
     store._fetchone = fake_fetchone  # type: ignore[method-assign]
@@ -510,6 +527,51 @@ async def test_ac7_gate_degrades_loudly(
     assert ops[0][1].startswith("SET NOCOUNT ON;")
 
 
+@pytest.mark.parametrize(
+    "answers_kw",
+    [{"cid_body": None, "cid_oid": 917578307}, {"dst_body": None, "dst_oid": 917578307}],
+)
+async def test_ac7_gate_names_view_definition_when_the_proc_is_deployed_but_unreadable(
+    answers_kw: dict[str, Any],
+    monkeypatch: pytest.MonkeyPatch,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """MEASURED: a principal holding only EXECUTE on the proc gets a non-NULL ``OBJECT_ID`` and a
+    NULL ``OBJECT_DEFINITION`` — the module is deployed and working. Before the id rode along on
+    the probe, that fired the MISSING arm and sent the operator to grant CREATE PROCEDURE, which is
+    neither the cause nor the cure (the cure is GRANT VIEW DEFINITION; WITH ENCRYPTION produces the
+    identical NULL). This is not a hypothetical posture: the sub-lever B design comment in the same
+    module explicitly designs for a fleet whose principal can never hold CREATE PROCEDURE.
+
+    RED without the fix — the old arm keys on the body alone, so it cannot see the id at all."""
+    with caplog.at_level(logging.WARNING, logger="messagefoundry.store.sqlserver"):
+        store = await _gate(_gate_rows(**answers_kw), monkeypatch)
+    assert store.claim_proc_effective is False
+    reason = store.claim_proc_degraded_reason or ""
+    assert "VIEW DEFINITION" in reason, "the reason must name the grant that actually fixes it"
+    assert "WITH ENCRYPTION" in reason, "the other cause of the identical NULL"
+    assert "is missing" not in reason, "a deployed proc must not be reported as absent"
+    assert any("DEGRADED to the shipped ad-hoc batch" in r.getMessage() for r in caplog.records)
+    # Still a degrade, not an outage: the claim runs on the shipped batch.
+    ops, _, _ = await _drive_proc("ingress", ["lane-0"], store=store)
+    assert ops[0][1].startswith("SET NOCOUNT ON;")
+
+
+async def test_ac7_gate_still_reports_a_genuinely_absent_proc_as_missing(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """The other side of the split. OBJECT_ID NULL is a real absence and must keep pointing at the
+    DDL/permission cause — the fix above must not relabel every NULL body as a readability problem.
+    """
+    store = await _gate(
+        _gate_rows(cid_body=None), monkeypatch
+    )  # oid defaults to None with the body
+    reason = store.claim_proc_degraded_reason or ""
+    assert "is missing" in reason
+    assert "CREATE PROCEDURE" in reason
+    assert "VIEW DEFINITION" not in reason
+
+
 async def test_ac7_no_error_2812_handling_on_the_hot_path() -> None:
     # The hot path carries no missing-proc (error 2812) handling: the gate decides at open, the
     # claim never falls back mid-flight.

tests/test_adr0114_claim_proc_live.py

@@ -89,6 +89,33 @@ async def test_open_deploys_procs_and_gate_passes(proc_store: SqlServerStore) ->
         assert row is not None and row["body"], f"dbo.{proc} not deployed"
 
 
+async def test_a_deployed_proc_can_return_a_null_definition(proc_store: SqlServerStore) -> None:
+    """The premise the gate's missing-vs-unreadable split rests on, and the one thing no offline
+    stub can show: on a real engine ``OBJECT_ID`` and ``OBJECT_DEFINITION`` genuinely disagree — a
+    procedure can be DEPLOYED and its definition still come back NULL.
+
+    Before the gate probed the id it read that NULL as "the proc is missing" and sent the operator
+    to grant ``CREATE PROCEDURE`` — neither the cause nor the cure.
+
+    ``WITH ENCRYPTION`` is used because it needs no security principal, so this leg creates no login
+    or user and impersonates nobody; it drops its own proc. The OTHER cause the reason string names
+    — a principal with ``EXECUTE`` but no ``VIEW DEFINITION`` — produces the byte-identical NULL and
+    is deferred with AC-10's other permission scenarios to a purpose-configured server.
+    """
+    name = "mefor_gate_null_definition_probe"
+    await proc_store._execute(f"CREATE PROCEDURE dbo.{name} WITH ENCRYPTION AS SELECT 1;")  # noqa: S608
+    try:
+        probe = await proc_store._fetchone(
+            "SELECT OBJECT_ID(?) AS oid, OBJECT_DEFINITION(OBJECT_ID(?)) AS body",
+            (f"dbo.{name}", f"dbo.{name}"),
+        )
+        assert probe is not None
+        assert probe["oid"] is not None, "the proc is deployed — this is the PRESENT half"
+        assert probe["body"] is None, "and its definition is unreadable — the NULL half"
+    finally:
+        await proc_store._execute(f"DROP PROCEDURE IF EXISTS dbo.{name};")  # noqa: S608
+
+
 async def test_ac8_trancount_on_exit_equals_entry(proc_store: SqlServerStore) -> None:
     # Execute the proc inside an open transaction and read @@TRANCOUNT before/after: the proc
     # must not BEGIN/COMMIT/ROLLBACK (it runs inside the client's autocommit=False txn).