chore(bench): move iOS snapshot benchmark evidence to the evidence/ios-snapshot branch

[thymikee]

Summary

The three hash-named raw results under scripts/ios-snapshot-benchmark/evidence/ (18,411 lines) are measurement output, not fixtures. They now live on the orphan branch evidence/ios-snapshot (commit 2d4baf461aa8897d49c6d4683cd16d8f43588ae8: the files plus a README with each sha256 and the fetch recipe; verified with git ls-remote), pinned by the annotated tag evidence/ios-snapshot/71fb2483f — the durable ref, since the branch tip is mutable.

In tree:

• git rm the three JSON files; evidence/README.md records each sha256 and git fetch origin refs/tags/evidence/ios-snapshot/71fb2483f && git show 2d4baf461aa8897d49c6d4683cd16d8f43588ae8:<file> > <dest>; fetched copies are gitignored there.
• evidence.ts + pnpm bench:ios-snapshot:evidence -- [--evidence-dir <dir>] validates a directory against the raw-result schema and the published hashes. The default directory must hold the complete PUBLISHED_EVIDENCE filename set and names the missing file(s) if not; an explicit --evidence-dir stays permissive by design (a fresh --out location, a partial re-check).
• evidence.test.ts runs on evidence-fixture.v1.json, a two-cell excerpt of the warm/relaunch result, and skips the corpus check with the fetch command when the directory holds no evidence.

Gross diff 19,065 lines: 18,419 deletions (18,411 the moved JSON), 646 additions.

Validation

• npx vitest run --project unit-core scripts/ios-snapshot-benchmark: 17 files, 42 passed, 1 skipped.
• pnpm bench:ios-snapshot:evidence: empty directory exits 1 with the fetch hint; the full corpus fetched at the tag + commit reports three valid files, hashes match.
• Planted red 1: tampered hash in evidence/README.md → README does not cite the sha256 of ios-snapshot-cold-local-71fb2483f.json.
• Planted red 2: byte-altered corpus file in the default directory → ios-snapshot-cold-local-71fb2483f.json differs from the published hash.
• Planted red 3: default directory with only ios-snapshot-cold-local-71fb2483f.json present (fetched from the pinned tag, hash-matched) → ... is missing published evidence file(s): ios-snapshot-warm-relaunch-local-71fb2483f.json, ios-snapshot-proxy-71fb2483f.json. Also covered by a unit test on checkEvidenceCorpus.
• All three published sha256 lines re-verified against files fetched from tag evidence/ios-snapshot/71fb2483f at commit 2d4baf461aa8897d49c6d4683cd16d8f43588ae8.
• pnpm check:quick, oxfmt --check, fallow audit, layering check: clean.
• Full affected gate green at head 806dfb522856397f25232b8c10082940bb49eec0: CI run.
• iOS Smoke: initial run failed smoke:automation-input with an accessibility-capture-truncation timing flake in the live simulator harness, unrelated to this evidence-relocation change. Rerun on the same head passed clean, confirming the flake.

Tradeoffs / follow-ups

Both durability gaps from review are closed at head 806dfb522856397f25232b8c10082940bb49eec0. The tag pins the corpus to an immutable ref; two applied repository rulesets now protect it: protect-evidence-tags blocks deletion/update of refs/tags/evidence/*, protect-evidence-ios-snapshot-branch blocks deletion/force-push of refs/heads/evidence/ios-snapshot. Both active, no bypass actors.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-03 15:18 UTC

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	2.58 MB	2.58 MB	0 B
JS gzip	869.6 kB	869.6 kB	0 B
npm bundled raw	2.58 MB	2.58 MB	0 B
npm bundled gzip	869.6 kB	869.6 kB	0 B
npm tarball	1.02 MB	1.02 MB	+6 B
npm unpacked	3.46 MB	3.46 MB	+113 B
npm clean-installed	3.46 MB	3.46 MB	+113 B

npm unpacked components

Component	Base	Current	Diff
JS / dist source	2.74 MB	2.74 MB	0 B
Apple runner source/project	544.4 kB	544.4 kB	0 B
Apple snapshot presentation source	33.8 kB	33.8 kB	0 B
macOS helper source	54.8 kB	54.8 kB	0 B
Android helper artifacts	43.3 kB	43.3 kB	0 B
Other package files	46.3 kB	46.4 kB	+113 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	30.8 ms	31.1 ms	+0.3 ms
CLI --help	79.9 ms	81.1 ms	+1.2 ms

Top changed chunks: no changes in the largest emitted chunks.

Top changed packed files

Packed file	Base	Current	Diff
package.json	19.9 kB	20.0 kB	+113 B

[thymikee]

BLOCKED

1. runEvidenceReport accepts any nonempty schema-valid directory and does not require the complete three-file PUBLISHED_EVIDENCE corpus; the default verifier can therefore succeed with one file. Require the full published filename set for the default directory and add a missing-file planted-red test, while custom dirs may remain permissive if intentional.

2. The only raw corpus source is mutable, unprotected branch evidence/ios-snapshot (currently 2d4baf461aa8897d49c6d4683cd16d8f43588ae8); there is no immutable ref and the README fetches the moving branch. Protect or pin a durable immutable evidence ref and link that exact ref before merge.

Exact-head CI is green, but these are data-integrity and durability blockers.

[thymikee]

Addressed both BLOCKED findings.

1. runEvidenceReport now requires the default evidence directory to hold the complete PUBLISHED_EVIDENCE filename set, naming any missing file(s) instead of succeeding on a partial corpus. An explicit --evidence-dir stays permissive by design (documented at the call site). Planted red: default dir with only the cold-local file (fetched, hash-matched) fails with ... is missing published evidence file(s): ios-snapshot-warm-relaunch-local-71fb2483f.json, ios-snapshot-proxy-71fb2483f.json.; covered by a new unit test on checkEvidenceCorpus.
2. Pushed the annotated tag evidence/ios-snapshot/71fb2483f at commit 2d4baf461aa8897d49c6d4683cd16d8f43588ae8 as the immutable ref. The README and every fetch command now pin to that tag and full SHA instead of FETCH_HEAD; all three sha256 lines re-verified against files fetched from it. A protected-ref ruleset is a maintainer action — exact gh api shape in the PR body's Tradeoffs.

Head SHA 287dbf50186f2b7dc83aa1d5872813d90148e67c. Full affected gate result follows from the gate stage.

[thymikee]

Fixed the Fallow "Unused exports" finding on scripts/ios-snapshot-benchmark/evidence.ts:

• EVIDENCE_TAG, EVIDENCE_COMMIT: only used inside fetchEvidenceCommand in the same file — dropped export.
• missingPublishedEvidence: only used inside checkEvidenceCorpus in the same file — dropped export.
• runEvidenceReport: the CLI entry point, but it's invoked through the file's own node evidence.ts argv guard, not an import — matches the sibling runDeepButtonRule pattern in deep-button.ts, which is unexported. Dropped export for consistency; no fallow-ignore needed since there's no real external consumer.

No baseline edits, no new tests (no behavior change, just visibility).

Head SHA: 806dfb522856397f25232b8c10082940bb49eec.

Separately, the red Test App Build Cache / Android Release job (run 33728318738, Java heap space in :app:compileReleaseArtProfile) is a CI lane defect unrelated to this change, owned by another PR in this program (F2b); left untouched.

The full affected-gate result at this head follows from the serialized gate stage.

[thymikee]

BLOCKED at 806dfb522856397f25232b8c10082940bb49eec0. Full default-corpus enforcement is fixed, but durability is not: the annotated evidence tag and source branch are unprotected. Repository rulesets contain only the default-branch rule, so either ref can be updated/deleted. Protect the immutable ref or use a durable store, then obtain a passing iOS Smoke rerun or same-head/main baseline-control evidence.

[thymikee]

Both closed at 806dfb522856397f25232b8c10082940bb49eec0.

1. Durability: added two repository rulesets (repo admin action) instead of leaving it as a follow-up — protect-evidence-tags (target tag, blocks deletion/update of refs/tags/evidence/*) and protect-evidence-ios-snapshot-branch (target branch, blocks deletion/force-push of refs/heads/evidence/ios-snapshot). Both enforcement: active, no bypass actors, verified via gh api repos/callstack/agent-device/rulesets.
2. iOS Smoke: the initial run failed smoke:automation-input with is absent could not prove absence for selector id="automation-close-sheet": capture was truncated — an accessibility-capture timing flake in the live simulator harness (the API's own hint: "Retry after the accessibility capture is complete"). No runner/Swift/simulator files are touched by this diff, so this can't be a regression from it. Reran the same head and it passed clean in 19m34s, confirming the flake.

PR body updated with both. Full affected gate is green at this head (CI run).