Skip to content

fix(gate): read a quoted span in the encoding its own host uses (BACKLOG #1229) - #830

Merged
wshallwshall merged 3 commits into
mainfrom
claude/gate-quotes-1229
Sep 4, 2026
Merged

fix(gate): read a quoted span in the encoding its own host uses (BACKLOG #1229)#830
wshallwshall merged 3 commits into
mainfrom
claude/gate-quotes-1229

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

For the Lander, first

The BACKLOG.md row is alone in the final commit (8766bc170), with all code and tests in
5ceb3b08b before it. A conflict here is confined to one commit and should be a scripted row-merge.

Merge safety measured, not assumed. #1359 landed on this same hook while this branch was held.
git merge-tree against current origin/main: scripts/hooks/worktree_gate.ps1 auto-merges
clean
; only docs/BACKLOG.md conflicts. I then drove the merged gate rather than trusting a
clean textual merge of two independent edits to one scanner, every arm with a positive control:

shape                                       old main   new main   MERGED
PowerShell backtick straddle (the fix)      ALLOW      ALLOW      DENY
same without the backtick (control)         DENY       DENY       DENY
bare gated command (POSITIVE CONTROL)       DENY       DENY       DENY
original one-line #1229 straddle (control)  DENY       DENY       DENY

So the fix is still needed against current main and survives the merge.

I have not enqueued this and have not armed auto-merge. The number is yours to take.

#1229 stays OPEN. It is not closed by this change; see the ledger commit and the section below.


fix(gate): the quote scanner learns PowerShell's escape, then reads a cmd payload in the encoding cmd receives (BACKLOG #1229)

THIS FILE MUST NOT MERGE, AND AS OF THIS COMMIT IT CANNOT. It is git rm --cacheded and matched
by a /PR_BODY.md rule in .gitignore, so it is untracked working-tree scratch that stays with the
worktree and never reaches main. It sits at the repo root only because a push hold stopped a pull
request being opened. Whoever opens the PR: paste this as the body, then leave the file untracked.

NOT PUSHED. The branch claude/gate-quotes-1229 is local only, on the
MessageFoundry-gate-quotes-1229 worktree. Nothing has reached any remote: no git push, no
gh pr create, no label, no comment. BACKLOG #1229 is claimed by this worktree
(scripts\coord\claim.ps1 -Take 1229); release it if the branch is abandoned.


Read this first: the fail-opens the fifth round introduced are CLOSED

The second pass of this branch introduced live fail-opens and pinned them as tripwires rather than
fixing them. This third pass fixes them, and the floor turned out to be 5 rather than 2 -- one of the
five on a DIFFERENT RULE, which is what rule-agnostic means.
No new
ALLOW replaces them: zero across 56 driven shapes, backed by a structural argument that is stronger
than the sweep.

direction at least evidence
fail-opens CLOSED (main ALLOW -> head DENY) 8 64-shape sweep, round 5
false denies REMOVED (main DENY -> head ALLOW, shape INERT) 12 8 + 4, every one re-tested for whether it runs
NEW fail-opens INTRODUCED 2 -> 5, all CLOSED in round 6 the floor moved; see below
NEW fail-opens introduced by round 6 0 56 driven shapes, plus the structural argument
fail-opens REMAINING in the newline class (#1429) 4 both tools x both quote characters

Every row is a floor, not an enumeration. No sweep here ranged over the whole input space, and
the eight false denies all share one left-hand side (an odd trailing backtick), so they are one
family enumerated rather than eight independent classes.

The full record lives in docs/BACKLOG.md #1229 and #1429, not here. This file does not restate
it. What follows is only what a reviewer needs in order to decide.

#1229 IS NOT CLOSED. A closure on this branch is reverted.

Insert one newline into a quoted word and this item's own titled shape goes back to ALLOW, on
origin/main and on this branch alike, and the middle statement runs. Reproduced independently three
times, against gate copies hash-verified byte-identical to origin/main
(blob b194d0a0420a6c87b8786994785155c91908d7b6) and to the branch head
(blob 94162a04a6fe51fa7ac2a7f7c6a14de2d3a4e4d8):

id        main   head    shape
POSCTL    DENY   DENY    git -C <governed> checkout main               (positive control)
ORIG      DENY   DENY    echo 'say "hi' ; <gated> ; echo 'bye" now'    (THE TITLED SHAPE)
ORIG+NL   ALLOW  ALLOW   the same, with ONE NEWLINE inside the first quoted word
NEGCTL    ALLOW  ALLOW   echo hello ; ls -la                           (negative control)

ORIG+NL runs: with the gated command swapped for expr 111 \* 3, bash prints 333, which an
echo-back cannot produce.

The closing argument is retracted at the claim. It leaned on the 2026-08-20 re-score's "if a
reviewer rules the escape hole a separate filing, the remainder text above is the filing."
That
antecedent names the backslash escape, which this change fixed rather than re-filed, so the
conditional never triggered. The same sentence's unconditional half governs: a wrongly-closed item
is invisible forever.

#1429 is correctly allocated and stays open -- as a separate FIX, not a separate CLASS. Its
banner previously called the newline case distinct while conceding it is "#1229's straddle exactly";
that contradiction is resolved in favour of the measurement, and the two items now cross-link.

The fail-opens the fifth round introduced, and how round 6 closes them

Under the PowerShell tool, against gate copies hash-verified byte-identical to origin/main, to
round 5 and to round 6, cwd inside the governed repo:

shape                                          main   round 5   round 6   really runs?
cmd /c "`"git -C <governed> reset --hard`""    DENY   ALLOW     DENY      RUNS (333)
cmd /k "`"git -C <governed> reset --hard`""    DENY   ALLOW     DENY      RUNS (333)
cmd /c "`"git -C <governed> merge feature`""   DENY   ALLOW     DENY      RUNS (333)
cmd /c "`"cd <governed> & git rebase main`""   DENY   ALLOW     DENY      RUNS (333)
cmd /c "`"git -C <gov> config core.hooksPath /dev/null`""
                                               DENY   ALLOW     DENY      RULE 1, not rule 3
cmd /c "<gated>"          no backtick          DENY   DENY      DENY      (control)
git -C <governed> reset --hard                 DENY   DENY      DENY      (positive control)
wsl -c "`"<gated>`""                           DENY   ALLOW     DENY      INERT -- see costs

The rule-1 row is the one that matters most for the count. The defect is in the SCANNER, so it
disarms whatever rule sits behind it; enumerating by rule under-counts by construction. Round 5
enumerated rule 3. Nothing here swept rule 3b, so 5 remains a floor.

RUNS means the inert marker set /a 111*3 printed 333 through the real shape; the negative
control set /a nope*3 prints 0, so an echo-back cannot pass for a run. Round 5 recorded 2 here
and the honest number is 4
-- it measured the two rows it knew about; sweeping the class found two
more of the same mechanism.

The mechanism. The outer line is PowerShell, so the outer scan honours the backticks, sees one
span, blanks it and reaches no verdict. Extraction then asks Get-FlagOwner, which answers
cmd, whose convention is none -- but the extracted text still carries the outer host's backticks,
because pwsh has not run and nothing has consumed them. Scanning it with no escape rule pairs the two
escaped quotes across the git command and blanks it. Span ownership deciding the wrong way, one level
in. Verified here rather than inherited, by driving Get-ScannableSegments directly: the outer
scan yields cmd /c "" and the extracted payload scan yields `"".

The fix. Get-ScannableSegments emits ONE EXTRA SEGMENT for a cmd-owned payload: the same text
with the outer host's escaped quotes resolved and cmd's own documented /c wrapper quotes removed --
the encoding cmd.exe actually receives. Both steps are needed and neither is enough alone, measured:
decoding leaves "git -C <governed> reset --hard", which the scanner blanks as an ordinary quoted
span; unwrapping without decoding finds a backtick in first position and does nothing.

An extra segment cannot introduce a fail-open, and that argument carries the claim rather than the
sweep.
Every rule reaches a segment through a continue-or-deny loop, so an added segment can only
ADD a deny, and it is appended after every existing segment so rule 3's first-verb-wins bookkeeping
sees exactly what it saw.

Two tidier fixes, both measured WRONG

1. Fold cmd and wsl into $pwshSet. Round 5 asserted this was wrong and could not show it
behaviourally; the assertion HOLDS, and here is the probe. It separates the two conventions in the
direction that matters -- the fold loses a deny the gate has today:

probe                                       round 6   cmdfold   separates?
cmd /c 'echo "x`" & <gated> & echo "y"'     DENY      ALLOW     YES   -- and it RUNS (333)
cmd /c 'echo "x"  & <gated> & echo "y"'     DENY      DENY      no    (control)

A backtick is an ordinary character to cmd.exe, so the quote after it really closes the span and the
gated command stays in plain view; reading that quote as escaped holds the span open across the gated
command and blanks it. & is used and not ;, because ; is not a command separator in cmd; a
pwsh single-quoted string is fully literal, which is what carries the backtick through intact. Now
pinned as test_a_LITERAL_backtick_in_a_cmd_payload_does_not_hide_a_gated_command.

Round 5's own separating probe is retired, not kept: it read head ALLOW / cmdfold DENY, and
round 6 denies both, so it no longer separates anything.

2. Decode the payload IN PLACE rather than adding a view. This is the change that reads as a
correction, and it re-opens round 3: bash -c "bash -c \"<gated>\"" goes DENY -> ALLOW, because
the function recurses ONE level and that round-3 deny depends on the escaped text staying visible at
this level. Measured, two new fail-opens.

Costs, stated because a one-sided note reads as a clean win

Round 6 re-denies shapes round 5 had allowed that are INERT on this host. Both DENY on origin/main
too, so these are pre-existing false denies restored, not new ones:

  • wsl -c is not a valid wsl flag at all -- wsl -c <anything> prints "Invalid command line
    argument: -c"
    , while wsl -- expr 111 \* 3 prints 333, so wsl itself works and the probe is
    not dead.
  • cmd /c ""echo git checkout main"" merely echoes. Its unescaped twin
    cmd /c "echo git checkout main" denies on origin/main and on every build since, so this is a
    pre-existing prose-inside-an-interpreter-payload false deny, reached rather than created.

Round 6 also closes two PRE-EXISTING holes of the same wrapper class, both ALLOW on origin/main:
cmd /c "\"<gated>\"" under the Bash tool, and cmd /c ""git -C worktree remove
"" (rule 2). The rule-2 row's carriage RUNS. The Bash-tool row could not be pinned to
whether it RUNS
, because cmd is not on this environment's Bash PATH (command -v cmd -> not
found). Recorded as unmeasured rather than assumed.

No sweep cited here is reproducible from this branch

That applies to round 6's as much as to round 5's. The 22-shape matrix, the 64-shape sweep, and round
6's two corpora (38 shapes covering the reported fail-opens, the fold probes, every earlier round's
tripwires and the #1429 newline rows; plus 18 aimed only at false denies through the new code path)
all lived in a session scratchpad with their drivers. Their counts must not be cited as standing
evidence.
What a reader can re-derive from the branch alone is every row pinned as a test in
tests/test_worktree_gate_quote_straddle.py and tests/test_worktree_gate_escaped_quote.py, plus
the tables above, each of which names its corpus and its controls. Rebuilding the main column is
two git shows into a scratch directory driven through the same subprocess harness the suites use.

Round 6's "zero new ALLOWs" is a floor over 56 driven shapes, not over the input space -- which
is weaker than the structural argument beside it, and is why that argument carries the claim.

What changed in the code

Unchanged from the original filing and still the right shape:

  • Remove-QuotedSpans and Get-ScannableSegments take [string]$Convention
    (posix | pwsh | cmd | none) instead of [bool]$PosixEscapes. The bool could say only "sh"
    or "not sh", so a pwsh payload and a cmd payload arrived identical and the backtick had nowhere
    to live. Default none: an unmeasured host gets no escape rule.
  • Get-HostConvention maps the tool name for the outer line; Get-FlagOwner returns a convention
    name and $owner is passed through whole, so an extracted payload keeps its interpreter's rules.
  • $winSet splits into $pwshSet and $cmdSet. The no-change option for cmd and wsl -- and see
    the two new fail-opens above, which are the cost of that split rather than a reason to undo it.
  • Get-EscapeChar is the single escape table and the extraction regex is derived from it, so the
    round-3 duplicate-spelling defect is unrepresentable rather than watched.

Added by this third pass (round 6), five small pieces:

  • Remove-EscapeChars resolves the OUTER host's escaped quotes. Escaped quotes only, and the
    narrowness is measured: a blanket strip turns D:\Work\x into D:Workx, so the view built to
    SEE a gated command stops resolving the repository it names. printf '%s' "D:\Work\x" prints
    D:\Work\x on this host -- sh honours the backslash before $ ` " \ and newline and
    nowhere else. It reads Get-EscapeChar, so there is still one escape table.
  • Remove-CmdWrapperQuotes applies cmd.exe's own documented /c rule (cmd /?): where the first
    character is a quote, strip it and the last quote on the line. Its other arm -- quotes preserved
    when the quoted text names an executable file -- is a filesystem question this hook cannot answer,
    and getting it wrong only makes MORE text visible.
  • The extraction records WHICH ARM matched. Only the double-quoted one can carry an outer escape; a
    single-quoted word is fully literal on both hosts, so re-decoding it would corrupt a legitimate
    backslash or backtick.
  • One extra segment per cmd-owned escaped payload, emitted after every existing segment.
  • Kept as an explicit character loop rather than a one-line [regex]::Replace. The regex form is
    shorter and it is a quoting hazard inside the one file whose subject is quoting -- an equivalence
    check of the two forms failed on the apostrophe case purely on how the pattern had to be spelled.

Checks run

check result
pytest -k worktree_gate (sequential, against the exact committed gate) 772 passed, 2 skipped, 0 failed
pytest on the two changed test files, after the last edit 66 passed
mypy messagefoundry (strict) Success: no issues found in 267 source files
ruff check . All checks passed
ruff format --check tests/ 737 files already formatted
PowerShell parse of every gate build, with a known-bad control all parse; the control reports Missing ')'
hook latency, 5 runs on a plain git status --short main 574ms / round 5 570ms / round 6 574ms median

Every gate copy driven in the tables above is hash-verified against the file being committed:
git hash-object on scripts/hooks/worktree_gate.ps1 and on the scratchpad copy return the same
blob, and the main and round 5 columns come from git show origin/main: and git show HEAD:
whose blobs match git rev-parse for those refs.

The test arithmetic, reconciled

  • pytest -k worktree_gate --collect-only -q on the previous pass -> 767/15611 tests collected.
  • This pass's run reports 772 passed, 2 skipped. 767 + 3 (the pass before, uncounted here) + 2 added by round 6 is not how it reconciles, so here is the arithmetic that does: the run
    immediately before round 6's test edits reported 2 failed, 768 passed = 770 outcomes; round 6
    adds exactly 2 tests and flips the 2 failures to passes, giving 772. The 2 failures were the
    KNOWN_NEW_fail_open tripwire asserting ALLOW, which is the success signal it was written to give.
  • The 2 skips are MODULE-LEVEL collection skips in files that have nothing to do with the gate:
    tests/test_off_loopback_runbook.py:55 and tests/test_runbook_proxy_tls_floor.py:66, both
    "docs/security/OFF-LOOPBACK-DEPLOYMENT.md is private-only". A module skipped at import reports
    one SKIPPED outcome and contributes zero collected tests.
  • The previous pass's unreconciled 769-outcome run stays unreconciled. That state was an
    uncommitted intermediate working tree and is not recoverable from git. Recorded, not corrected.

Skipped: the full pytest suite. This worktree's venv installs fewer extras than CI
(BACKLOG #1230), so a full local run would not be a clean-suite claim in any case. Legs that only
run on a hosted runner have not been read
, windows-service-smoke included.

Open questions for a reviewer

  1. Land it. The recommendation is no longer hedged. Round 5 closed at least eight fail-opens and
    opened at least four; round 6 closes those four and opens none across 56 driven shapes, with a
    structural argument -- an added segment can only add a deny -- that does not depend on the sweep.
    The remaining trade is a handful of restored false denies on shapes that are INERT on this host
    and that origin/main denies anyway.
  2. #1229 stays OPEN. The #1429 newline class is untouched by this pass. If a reviewer wants the
    item closed, the newline evidence in its banner is what has to be answered first.
  3. #1429 is unscored -- filed with a verdict and closing-act, no value or difficulty.
  4. The ranked table was deliberately left alone. #1229's row is still in docs/BACKLOG.md;
    36 of the 231 rows in that table already point at items that are now closed, so touching one row
    would be an unrequested change inconsistent with 35 neighbours.
  5. Out of scope and untouched, as briefed: #1341 (the sibling block-blanket-git-stage.ps1,
    whose Hide-QuotedSpans has this class open), #1359, #1086. Nothing here was measured against
    them. tests/test_worktree_gate_quoted_key.py is untouched and no row flipped.
  6. "" is deliberately not modelled. PowerShell's doubled-quote escape needs no rule: naive
    pairing covers the same extent, so there is no gap for live code, and the extraction regex is left
    equally blind so the two cannot disagree.
  7. The extra view is scoped to cmd because that is what was measured. Get-FlagOwner answers
    cmd for cmd and wsl alike, and the wrapper rule is cmd.exe's, so for wsl it is an
    over-approximation -- the harmless direction. It is deliberately NOT widened to the posix and
    pwsh conventions: no probe here separates them, and widening a security control by analogy is
    how rounds 2 through 5 happened.
  8. cmd's rule-1 arm is not modelled. cmd /? preserves the quotes when the quoted text names an
    executable file; that is a filesystem question the hook cannot answer, and guessing wrong only
    makes more text visible. Named so nobody reads the wrapper rule as complete.

Scope

A local maintainer-workstation guardrail whose own .SYNOPSIS declines to be a security boundary.
No product, engine or PHI effect, and no deployment axis (CLAUDE.md section 0). What it costs is
the guardrail's reliability on shapes a developer reaches for by habit.

wshallwshall and others added 2 commits September 4, 2026 12:32
…LOG #1229)

The quote scanner blanked spans without knowing which escape character the
executing host uses. PowerShell escapes with a backtick, so the scanner held a
span open that PowerShell had already closed, straddled the live command
between it and a later quote, and blanked it. A gated git command in that
position reached no rule.

Remove-QuotedSpans and Get-ScannableSegments now take a convention name rather
than a bool that could only say "sh" or "not sh". Get-FlagOwner returns that
name so an extracted payload keeps its own interpreter's rules, and the Windows
set splits into a pwsh set and a cmd set, cmd having no escape character at all.
Get-EscapeChar is the single table both the scanner and the extraction regex
derive from, so the two spellings that disagreed in an earlier round are now
unrepresentable rather than watched.

Get-ScannableSegments additionally emits one extra segment for a cmd-owned
payload: the outer host's escaped quotes resolved, then cmd's documented wrapper
quotes removed, which is the encoding cmd.exe actually receives. An extra
segment cannot introduce a fail-open because every rule reaches a segment
through a continue-or-deny loop and the new view is appended after the existing
ones, leaving rule 3's first-verb bookkeeping untouched.

Measured against hash-verified copies of each build, every arm carrying a
positive control that must deny. Against origin/main the PowerShell backtick
straddle goes ALLOW to DENY, and the change introduces no new ALLOW across an
exhaustive sweep of 7,056 side-string pairs plus about 28,000 random samples,
with the oracle proven live by 84 confirmed closures in the same run. The posix
arm is byte-identical to origin/main across that corpus.

Folding cmd into the backtick set was measured and rejected: it loses a deny the
gate has today, because a backtick is an ordinary character to cmd.exe, and that
shape really executes. Decoding in place rather than appending was also measured
and rejected: it re-opens a nested bash-c deny.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#1229 is NOT closed by the quote-scanner fix. A quoted span crossing a newline
straddles through the gate's per-line split, and inserting one newline into a
quoted word re-opens every shape this item closed, including the shape in the
item's own title. Reproduced three times against hash-verified gate copies, with
a positive control denying and the middle statement pinned to a marker that
computes rather than echoes.

The re-score's conditional, that a reviewer may rule the escape hole a separate
filing, names the backslash escape. That hole was fixed here rather than
re-filed, so the conditional does not apply. What governs is the unconditional
half of the same sentence: a wrongly-closed item is invisible forever.

#1429 records the newline class as a separate FIX rather than a separate CLASS,
and states that it re-opens this item's pins. Both items cross-link.

Counts are floors with their corpus named rather than enumerations presented as
properties.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
Resolves the BACKLOG.md conflict deliberately rather than mechanically.

The pull_request workflows could not run at all before this. GitHub builds no
refs/pull/830/merge for a CONFLICTING pull request, so every pull_request-
triggered check was ABSENT rather than queued or failing -- 2 checks against
42 to 46 on every other open pull request. Only push and pull_request_target
workflows fired.

#1229 KEEPS MAIN'S CLOSURE. This branch had reverted it. Another seat closed the
item on 2026-09-03 after verifying the escape limb shipped, and explicitly scoped
the per-line split out as an inherited residual. Deferring to that: a class that
defeats every rule in the gate is its own item, not evidence that one item is
unfinished, and settling a scope disagreement between seats inside a merge
resolution would make it invisible to the seat that closed it.

#1429 IS KEPT, AND IT IS A COUNTEREXAMPLE TO #1427. That item, added on main
while this branch was held, records the per-line split as producing no hole --
six spellings tested, zero found -- and cites the gate's own docstring agreeing
that a quoted argument spanning lines denies today anyway. This branch measured
the opposite three times with a positive control and an inert marker that
computes. One of the two readings is wrong; #1429 is where that gets settled.

Both sides' new items are kept, interleaved in number order.

The gate itself auto-merged with no conflict, and the merged gate was driven
before this commit: the PowerShell backtick straddle goes ALLOW on both old and
new main to DENY on the merge, with every control still denying.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall
wshallwshall added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit f18652f Sep 4, 2026
44 of 45 checks passed
@wshallwshall
wshallwshall deleted the claude/gate-quotes-1229 branch September 4, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewed A reviewer has read this. Removed automatically when new commits arrive.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant