fix(macos): the sweep reopened its path list once per changed file

README.md

@@ -345,8 +345,8 @@ that changed: at every attach (`up`/`claude`/`shell --macos`), and again every *
 the VM runs, which is the only thing that makes a host edit reach an agent that is *already running*.
 See [`docs/decisions/0016-shared-file-cache-refresh.md`](docs/decisions/0016-shared-file-cache-refresh.md).
 
-The cost scales with the number of **changed** files (~0.36 ms each, host + guest, paid serially), and
-nothing caps that number — past roughly 14,000 changed files one sweep outlasts the 5 s interval and
+The cost scales with the number of **changed** files (~0.29 ms each, host + guest, paid serially), and
+nothing caps that number — past roughly 17,000 changed files one sweep outlasts the 5 s interval and
 the loop starts running most of the time on one core. An `npm install`, a big `git checkout` or a full
 build can get there. Two run-scoped flags, written after the command:
 

augur

@@ -1026,7 +1026,7 @@ require_safe_workspace() {
 #
 # One file per project, for the settings whose right value is a property of THE REPO rather than of
 # the run — today the two macOS share-refresh knobs, whose cost scales with the project's
-# changed-file count (~0.36 ms each, so past roughly 14,000 changed files one sweep outlasts the
+# changed-file count (~0.29 ms each, so past roughly 17,000 changed files one sweep outlasts the
 # default 5 s interval; see _MACOS_REFRESH_MODE). Without this, an operator whose repo is big enough
 # to need `--share-refresh attach` has to type it on every command, forever.
 #
@@ -3581,14 +3581,21 @@ _MACOS_SWEEP_NOWORK=4    # nothing had changed since the last sweep, so the gues
 # WHY THERE IS A DIAL AT ALL. The "no cap" paragraph above is right about the SWEEP and wrong about
 # the whole. It measures the guest-side invalidation (0.23 ms/file) and the host-side `find` (0.31 s
 # for 12,626 files) and concludes "bounded at seconds" — but it never measured the host-side loop
-# that ACCUMULATES the list (see _refresh_macos_shares_locked, which appends once per changed file
-# and reopens the list file every iteration). The 2026-07-28 security snapshot did: ~130 µs per
-# changed file, i.e. ~99% of the host-side cost. Added to the 0.23 ms the guest pays for the same
-# file, and both are paid serially, one sweep costs ~0.36 ms per changed file end to end. Past
-# roughly 14,000 changed files a single sweep therefore exceeds the default 5 s interval, and the
-# loop — which sleeps THEN sweeps — degrades into a rising duty cycle on one host core for the
-# lifetime of the VM. 14,000 needs no adversary: `npm install`, a large `git checkout` or a full
-# build reach it, and augur's own repo is 12,626 files.
+# that ACCUMULATES the list. The 2026-07-28 security snapshot did: ~130 µs per changed file, i.e.
+# ~99% of the host-side cost — because that loop reopened the list file once per changed file. IT NO
+# LONGER DOES: the redirect now sits on the loop rather than on the printf (see
+# _refresh_macos_shares_locked), and the host-side term re-measured through that same function is
+# 64.7 µs per changed file, down from 122.0. Added to the 0.23 ms the guest pays for the same file,
+# both still paid serially, one sweep now costs ~0.29 ms per changed file end to end, down from
+# ~0.35 ms. Past roughly 17,000 changed files a single sweep therefore exceeds the default 5 s
+# interval — it was ~14,200 before that fix — and the loop, which sleeps THEN sweeps, degrades into
+# a rising duty cycle on one host core for the lifetime of the VM.
+#
+# THE FIX DID NOT RETIRE THIS DIAL and the numbers are quoted above so that stays checkable: the
+# guest's 0.23 ms per file is untouched by anything host-side and is now ~78% of the total, so
+# halving the host half bought ~20% on the crossing point and nothing more. 17,000 still needs no
+# adversary: `npm install`, a large `git checkout` or a full build reach it, and augur's own repo is
+# 12,626 files.
 #
 # THREE MECHANISMS, and an operator turning the cost down has to be able to say WHICH:
 #
@@ -3851,11 +3858,32 @@ _refresh_macos_shares_locked() {
         roots=$((roots+1))
         [[ -d "$root" ]] || continue
         scanned=$((scanned+1))
+        # THE REDIRECT IS ON THE LOOP, NOT ON THE `printf`, so `$list` is opened once per ROOT
+        # instead of once per CHANGED FILE. That reopen was most of the host-side cost and it is what
+        # ADR-0016 §Cost's "~130 us per changed file" measured. Re-measured through this very function
+        # (ssh_macos stubbed, so it is the host half only), 12,628 files across four roots, bash 3.2:
+        # 122.0 -> 64.7 us per changed file, 1541 -> 817 ms for a full blind sweep of a repo this size.
+        #
+        # It does NOT retire the dial, and saying so here is the point: the guest's own 0.23 ms per
+        # file is untouched by this and is ~78% of what is left, so the crossing point where one sweep
+        # outlasts the 5 s interval moves only ~14,200 -> ~17,000 changed files. ADR-0016 §Cost carries
+        # the arithmetic and the caveat that this reading was taken in a guest, not on a real host.
+        #
+        # NOT on the OUTER loop, and that is not a style choice. `warn` writes to STDOUT (augur:123),
+        # so a redirect spanning the outer body would put any warning added there INTO the NUL path
+        # list, and the guest would be handed a path built from a sentence — the exact trap
+        # macos_share_roots documents at its own `error`-not-`warn` line. This body is one
+        # assignment, one printf and one increment, and can be checked by eye to emit nothing else.
+        #
+        # Still no subshell: a redirection on a compound command does not fork, so `n`, `roots` and
+        # `scanned` keep accumulating in the CURRENT shell, which is what the comment above relies on.
+        # Byte-equivalence with the per-printf form was checked against paths containing spaces, tabs,
+        # embedded newlines, quotes, non-ASCII and a leading dash: identical bytes, identical NUL count.
         while IFS= read -r -d '' f; do
             rel="${f#"$root"/}"
-            printf '/Volumes/My Shared Files/%s/%s\0' "$name" "$rel" >> "$list"
+            printf '/Volumes/My Shared Files/%s/%s\0' "$name" "$rel"
             n=$((n+1))
-        done < <(find "$root" -type f ${find_age[@]+"${find_age[@]}"} -print0 2>/dev/null)
+        done < <(find "$root" -type f ${find_age[@]+"${find_age[@]}"} -print0 2>/dev/null) >> "$list"
     done < <(macos_share_roots "$vm")
 
     # `n == 0` used to mean two different things, and only one of them is good news:
@@ -6631,7 +6659,7 @@ cmd_help() {
     echo -e "    • macOS shared-file refresh: a macOS guest's virtiofs client serves ${BOLD}stale file data${RESET} after a"
     echo -e "      host edit, with no timeout (issues #124/#135; still present on macOS 26.6). augur invalidates"
     echo -e "      the changed files at each attach and every ${CYAN}5${RESET}s while the VM runs. Cost scales with the"
-    echo -e "      CHANGED-file count (~0.36 ms each, host + guest), so a repo where one sweep outlasts the"
+    echo -e "      CHANGED-file count (~0.29 ms each, host + guest), so a repo where one sweep outlasts the"
     echo -e "      interval should use ${CYAN}--share-refresh attach${RESET} (sweep on attach, no loop) rather than a longer"
     echo -e "      interval. ${CYAN}--share-refresh off${RESET} disables the mitigation entirely. See ADR-0016."
     echo -e "    • ${BOLD}augur refresh --macos${RESET} sweeps once, on demand, against the VM that is already running —"

docs/decisions/0016-shared-file-cache-refresh.md

@@ -97,7 +97,7 @@ implementation shipped the bug until a test caught it.
 | | Measured |
 |---|---|
 | Host-side detection | 0.31 s (`find -newer`, 12,626 files) |
-| Host-side list accumulation | ~130 µs per **changed** file (2026-07-28 snapshot, item 37) |
+| Host-side list accumulation | ~130 µs per **changed** file (2026-07-28 snapshot, item 37) — **superseded 2026-08-05, see the amendment at the end of this section** |
 | Guest-side invalidation | 0.23 ms per file |
 | Full blind sweep, all five shares | 2.87 s for 12,600 files, zero failures |
 | Realistic incremental sweep | ~10 ms for a few dozen files |
@@ -106,7 +106,10 @@ There is deliberately **no cap** on the changed set. A legitimate one can be lar
 a branch switch), and silently truncating the list would be the same class of failure the surrounding
 work exists to remove.
 
-The second row was **added after the fact** and it corrects this section. The original four rows did
+The second row was **added after the fact** and it corrects this section. *(This paragraph and the one
+after it are kept as written; the reopen they describe was removed on 2026-08-05 and their figures are
+superseded by the amendment at the end of this section. They are the reasoning the amendment reversed,
+and are the reason the dial exists at all.)* The original four rows did
 not measure the shell loop that builds the path list, which appends once per changed file and reopens
 the list file each time — ~99 % of the host-side cost. Adding it to the guest's 0.23 ms, both paid
 serially, one sweep costs **~0.36 ms per changed file end to end**, so past roughly **14,000** changed
@@ -127,6 +130,39 @@ figure of 30–40k is a third thing again: it is the host-side term *alone*, wit
 excluded. Nothing here rests on the exact crossing; what it rests on is that a crossing exists at a
 count no adversary is needed to reach, which all three readings agree on.
 
+**Amendment (2026-08-05): the accumulation row was a defect, not a floor.** The loop that builds the
+path list held its `>> "$list"` redirect on the `printf`, so it reopened the list file once per
+changed file. Moving the redirect onto the enclosing `while … done` — one open per **root** instead
+of one per **file** — is the entire change. Re-measured through `_refresh_macos_shares_locked` itself
+with `ssh_macos` stubbed, so the reading is the host half alone; 12,628 files across four roots,
+bash 3.2, best of three runs:
+
+| | before | after |
+|---|---|---|
+| Host-side term (`find` + accumulation) | 122.0 µs per changed file | **64.7 µs** |
+| Full blind sweep of a 12,628-file tree | 1541 ms | **817 ms** |
+| End to end, with the guest's 0.23 ms | ~0.35 ms per changed file | **~0.29 ms** |
+| Crossing point against the 5 s interval | ~14,200 changed files | **~17,000** |
+
+**It does not retire the dial**, and that is the part worth keeping in view: the guest's 0.23 ms is
+untouched by any host-side change and is now ~78% of the total, so halving the host half bought ~20%
+on the crossing point and nothing more. §4's `--share-refresh` argument stands unchanged. It also
+does not help the common case — a realistic incremental sweep is a few dozen files, where the saving
+is well under a millisecond. What it helps is the large sweep: a branch switch, a build, and the
+first blind sweep on a fresh marker.
+
+The redirect sits on the **inner** loop, not the outer one, and that is load-bearing rather than
+stylistic: `warn` writes to **stdout**, so a redirect spanning the outer body would put any warning
+added there into the NUL path list and hand the guest a path built from a sentence. That is the same
+trap `macos_share_roots` documents at its own `error`-not-`warn` line. Byte-equivalence with the
+per-`printf` form was checked against paths containing spaces, tabs, embedded newlines, quotes,
+non-ASCII and a leading dash — identical bytes, identical NUL count.
+
+**Measured in a Virtualization.framework guest (`hw.model VirtualMac2,1`) on its local APFS, not on a
+real host.** That is the one caveat on the absolute figures; the *ratio* is a property of the shell
+loop and does not depend on it. The 122.0 µs "before" reading corroborates the snapshot's ~130 µs
+from a different machine and a different method, which is what makes the comparison usable at all.
+
 ## 3. Placement, and why the order is load-bearing
 
 The refresh runs **before** the `ensure_macos_*` wiring, not after. `ensure_macos_claude_profile`

tests/41_macos_share_refresh.sh

@@ -903,4 +903,35 @@ else fail "…without printing the whole help" "say what to type; the command li
 if grep -q 'augur ${CYAN}refresh${RESET} --macos' "$AUGUR"; then ok "…and \`--help\` lists it among the macOS commands"
 else fail "…and \`--help\` lists it" "a command nobody can discover does not close the gap it was written for"; fi
 
+section "the path list is opened once per ROOT, not once per changed FILE"
+
+# WHY THIS IS PINNED STRUCTURALLY, when nothing else in this file is pinned on cost. A revert is
+# INVISIBLE to every other arm here: the output is byte-for-byte identical (checked against paths
+# containing spaces, tabs, embedded newlines, quotes, non-ASCII and a leading dash — same bytes, same
+# NUL count), so the whole suite stays green while the sweep silently costs twice what it should.
+# What a revert costs, re-measured through _refresh_macos_shares_locked itself with ssh_macos stubbed:
+# 122.0 us per changed file against 64.7, i.e. 1541 ms against 817 ms for a full blind sweep of a
+# 12,628-file tree. ADR-0016 §Cost carries the amendment and the caveat on the absolute numbers.
+read -r _rs _re <<<"$(fn_range _refresh_macos_shares_locked)"
+_pf="$(first_in "$_rs" "$_re" "printf '/Volumes/My Shared Files/%s/%s")"
+_inner="$(first_in "$_rs" "$_re" 'done < <(find "$root" -type f')"
+_outer="$(first_in "$_rs" "$_re" 'done < <(macos_share_roots "$vm")')"
+if [[ -n "$_pf" && -n "$_inner" && -n "$_outer" ]]; then
+    ok "the accumulation loop is locatable (printf $_pf, inner done $_inner, outer done $_outer)"
+else fail "the accumulation loop is locatable" "printf='$_pf' inner='$_inner' outer='$_outer' — the three arms below would assert nothing"; fi
+
+if ! sed -n "${_pf}p" "$AUGUR" | grep -qF '>>'; then ok "the \`printf\` carries no redirect of its own"
+else fail "the \`printf\` carries no redirect of its own" "it reopens \$list once per CHANGED FILE, which is ~2x the host-side cost of every sweep (ADR-0016 §Cost)"; fi
+
+if sed -n "${_inner}p" "$AUGUR" | grep -qF '>> "$list"'; then ok "…because the redirect is on the inner loop, so \$list opens once per root"
+else fail "…because the redirect is on the inner loop" "$(sed -n "${_inner}p" "$AUGUR")"; fi
+
+# THE NEGATIVE CONTROL, and it guards something worse than cost. `warn` writes to STDOUT (augur:123),
+# so hoisting the redirect one level further — onto the loop over the SHARE ROOTS — would put any
+# warning added to that body INTO the NUL path list, and the guest would be handed a path built from
+# a sentence. macos_share_roots documents the same trap at its own `error`-not-`warn` line. One open
+# per root is already almost all of the win, so the remaining three opens do not buy that risk.
+if ! sed -n "${_outer}p" "$AUGUR" | grep -qF '>>'; then ok "…and NOT on the outer loop, where a \`warn\` would land in the NUL list"
+else fail "…and NOT on the outer loop" "warn writes to stdout, so a warning in that body becomes a path the guest is told to msync"; fi
+
 finish

tests/43_macos_share_refresher.sh

@@ -155,8 +155,8 @@ eval "$_saved_refresh"
 section "the mode gate on the loop — --share-refresh attach|off"
 
 # The loop is the UNATTENDED, repeated cost: it sweeps for the whole lifetime of the VM with nobody
-# watching, and its price scales with a changed-file count nobody caps (~0.36 ms per changed file,
-# host + guest, both paid serially — past ~14,000 one sweep outlasts the default 5 s interval). So it
+# watching, and its price scales with a changed-file count nobody caps (~0.29 ms per changed file,
+# host + guest, both paid serially — past ~17,000 one sweep outlasts the default 5 s interval). So it
 # is the mechanism an operator turning the refresh down is actually paying for, and both non-default
 # modes stop it. The attach-time sweep is tests/41's subject, not this file's.
 _saved_mode="$_MACOS_REFRESH_MODE"