Ship it: merge the configuration, review a real PR, judge the result

[simonoppowa]

Part of #1023

Question

Ship it, review a real PR, and judge the result.

The one ticket on this map that carries execution rather than a decision. Everything upstream is settled by the time this runs; this is where it meets a real contributor's code.

Do:

• Land whatever configuration the map decided on — steering file, trigger, scope rule — as a PR, conventional-commit titled, following the repo's own conventions. No AI co-author trailer, no assistant named in the subject or body.
• Point it at a real, currently-open outside PR: #1010 (AzazelSensei, a README change) or #995 (ibrahim-iqbal, a real Flutter change touching search-state UI — the better test, since it should trip the Semantics(identifier:) convention if anything does).
• Read the review as the contributor would read it, not as its author.

Then judge, and write the verdict down:

• Did it catch the conventions? Specifically the ones no generic reviewer would know. If it missed them, the steering is wrong and that is a finding, not a failure.
• How much of it was noise? Count the findings worth acting on against the total.
• Would this have helped or discouraged the contributor?
• What did it cost in quota, against what What Google AI Pro actually buys in Jules capacity #1026 established.

Verdict is keep / tune / drop, stated plainly. "Tune" must name what to tune. Reaching a verdict completes the map — including a verdict of "drop", which is a real outcome and not a wasted effort.

[simonoppowa]

Two prerequisites surfaced while resolving #1029 — both are execution work that belongs with the ship, not separate efforts.

1. .github/PULL_REQUEST_TEMPLATE.md:35 contradicts what Codex will say. It reads "Localization updated in ARBs and lib/generated/ when user-facing strings change" — but lib/generated/ is gitignored (.gitignore:42) and CONTRIBUTING.md forbids hand-editing it. The new review rules explicitly tell the reviewer that the absence of generated Dart is correct. Ship both together or a contributor gets a checklist and a bot review that disagree on the same PR. Issue #578's acceptance criteria carry the same stale instruction.

2. The byte plan from #1029 must land as part of this. In order: move AGENTS.md:224-277 to a new tools/adb/README.md (2,783 B) with a one-line pointer left behind; move AGENTS.md:35-102 into the doc comments already on lib/core/utils/demo/demo_seeder.dart (3,496 B); insert ## Code Review Rules after ## Project Overview; add the check_agents_md recipe to the justfile and a step to linux-checks. Verify with wc -c AGENTS.md — the target is 27,337 bytes.

A follow-up worth considering rather than doing here: extend just check_l10n to fail when a key added in the current diff is byte-identical to English in every non-English ARB, with an allowlist for the seven legitimate cases (appTitle, notificationsDailyReminderTitle, the two OFF contribute links, settingsAccentSubtitleMaterialYou, and the two placeholder-only {hour} strings). That converts this review's single highest-value finding — the one that justified automating at all — from advisory prose into a red X, which is strictly better. It would also let rule 1 shrink.

[simonoppowa]

Execution gate carried over from #1032.

#1032 is closed with a decision — the map's Notes put execution here, so its checklist is unexecuted work that belongs to this ticket. Verified just now: main still has 0 required approving reviews, all four environments still report protection_rules: [], all 26 secrets are still repo-level, and every environment holds 0 secrets of its own.

Codex must not be installed before this lands. The App requests workflows: write, and the whole point of #1032's finding is that scoping without a required reviewer buys nothing — an actor holding workflows: write can add environment: X to a job it writes. The reviewer is the control, precisely because a GitHub App cannot approve a deployment gated on a human.

Order of operations for this ticket:

1. Protect the Actions secret store before installing any write-capable app #1032's checklist first — create release-build (branch policy main, no reviewer) and release-ship (branch policy main, required reviewer simonoppowa), move the four ANDROID_* secrets into the former and the APP_STORE_CONNECT_* / MATCH_* / GOOGLE_PLAY_SERVICE_ACCOUNT_JSON into the latter, then add environment: to android-package, ios-package, ios-deploy, android-deploy. Each secret must be re-entered from its original source — GitHub never reveals a stored value.
2. Verify before deleting anything. A PR still passes (it uses the stubs from .github/actions/write-env-file and touches no environment); a push to main pauses for approval before ios-package. Only after a release run is green end-to-end, delete the 11 orphaned repo-level secrets.
3. Then the What the review checks, how loud it is, and what fits in 3,087 bytes #1029 byte plan and the PR-template fix noted above.
4. Then install Codex.

Recorded residual risk, not closed by any of this: the 11 orphaned secrets (AWS_*, CLOUDFLARE_*, R2_*, TF_*, CATALOG_ACCESS_TOKEN) stay readable by any write-capable actor until step 2 completes, and main keeps 0 required reviews.

[simonoppowa]

Partially executed — branch chore/codex-review-setup, three commits, nothing pushed

Everything that needs no credentials is done and verified locally. What remains needs a Google/OpenAI account or secret values, so it is yours.

Done

commit	what
267fc366	## Code Review Rules into AGENTS.md at byte 653; ADB/device prose → tools/adb/README.md; demo-seeder prose → docs/demo-data.md; check_agents_md recipe + linux-checks step
b5c9ae98	groups: on all four Dependabot ecosystems
e0fd3ce4	PR template no longer asks contributors to edit gitignored lib/generated/

AGENTS.md: 29,681 → 27,776 bytes. 4,992 under the 32 KiB cap, 3,224 under the 31,000 guard.

A correction to #1029's plan, caught before it shipped. The trim range was recorded as lines 35–102 for the demo block. Lines 91–101 are the flutter test <file> and flutter analyze commands, which belong to ## Commands and are not demo content — cutting to 102 would have deleted them. The real block ends at line 89 (3,362 bytes, not 3,496). The arithmetic still lands comfortably.

One deviation from the plan, deliberate. #1029 said the demo prose should go into demo_seeder.dart's doc comments. It is 3,362 bytes of prose with tables and fenced blocks; forcing that into Dart /// comments would read badly. It went to docs/demo-data.md instead, matching the existing docs/ convention (ai-architecture.md, export-format.md, supabase-self-hosting.md).

The guard was tested on both failure paths, not just the happy one: it fails on an oversized file (31,776 > 31,000) and on a missing/renamed heading. Both YAML files re-parse.

Not done — needs you

1. Protect the Actions secret store before installing any write-capable app #1032's checklist first. Create release-build (branch policy main, no reviewer) and release-ship (branch policy main, required reviewer simonoppowa), re-enter the secrets from their original sources, then add environment: to android-package, ios-package, ios-deploy, android-deploy.

   I deliberately did not add the environment: keys. Adding them before the environments exist creates them implicitly with no protection — the guard would look present and do nothing. The order has to be create-with-reviewer, move secrets, then wire the jobs.

2. Install the Codex GitHub App, scoped to selected repositories → this repo only. Not before step 1: it requests contents/workflows/actions: write, which is the grant Protect the Actions secret store before installing any write-capable app #1032 exists to contain.

3. Settings: Automatic reviews on, trigger "On PR open" (not On every push), policy left on "Review all PRs" — see Codex reviews every PR including mine — is that acceptable, and what is the trigger surface? #1028 for why Review my PRs is a trap.

4. Backfill by hand. Enabling will not reach #1010 or #995 — automatic review is webhook-driven on opened/ready_for_review and both fired days ago. Post @codex review on each, on one line, exactly that form.

5. Worth 30 seconds while you are in there: check whether the code-review quota is displayed separately from the ChatGPT Plus coding budget (Codex Cloud auto code review: silent fail + dashboard shows quota available but review says limit reached openai/codex#15477). If it is, the entire cost concern behind Codex reviews every PR including mine — is that acceptable, and what is the trigger surface? #1028 dissolves.

Then judge

The bar from #1035: does it catch the conventions no generic reviewer knows, how much is noise, and would it have helped or discouraged the contributor. Verdict is keep / tune / drop, and "drop" is a real outcome — CodeRabbit is the recorded fallback in #1034 and needs no fresh research.

[simonoppowa]

Opened as #1045 against develop, rebased off the ci/ios-smoke-advisory-on-main branch it was cut from so it carries only its own three commits.

Still outstanding here, and all of it needs your account or the secret values: #1032's environment checklist, installing the App scoped to this repo, the trigger settings, and the @codex review backfill on #1010 and #995. See the status comment above.

[simonoppowa]

Repo-side work is merged

#1045 landed on develop as 74464cb9, and #1047 as 24c25dd4. Verified on develop, not just assumed from a green merge:

AGENTS.md	28,140 bytes — 2,860 under the guard, 4,628 under Codex's 32 KiB cap
## Code Review Rules	byte 653, well inside the head where truncation cannot reach it
tools/adb/README.md	3,183 bytes
docs/demo-data.md	3,363 bytes
check_agents_md	in justfile (first in ci:) and in linux-checks
Dependabot	all 4 ecosystems grouped
"120-char" claim	0 occurrences across AGENTS.md, CONTRIBUTING.md, PR template

Copilot reviewed four times across the two PRs and found three real defects, all mine — a relative link broken by moving a file into docs/, a stale linux-checks command list that my own rules had gone out of date with inside the same PR, and an internal contradiction between the PR template and AGENTS.md that was only visible reading the branch as a whole rather than the diff. Worth noting against #1029: that last one is a class of finding my rules, written diff-scoped, would not produce.

Still outstanding — all of it needs your account or the secret values

1. Protect the Actions secret store before installing any write-capable app #1032's environment checklist. Verified unexecuted: main still has 0 required approving reviews, all four environments still report protection_rules: [], and every environment holds 0 secrets. Create release-build and release-ship, re-enter the secrets from source, then add environment: to android-package, ios-package, ios-deploy, android-deploy. This gates everything below — the Codex App requests contents/workflows/actions: write.
2. Install the Codex App, selected repositories → this repo only.
3. Settings: Automatic reviews on, trigger "On PR open", policy left on "Review all PRs" (see Codex reviews every PR including mine — is that acceptable, and what is the trigger surface? #1028 for why "Review my PRs" is a trap).
4. Backfill: @codex review on docs(readme): name every contact in What leaves your device #989 #1010 and feat: add scan and create-custom actions to no-results search state (#578) #995 — enabling reaches neither.
5. Worth 30 seconds: check whether the code-review quota displays separately from the ChatGPT Plus coding budget (Codex Cloud auto code review: silent fail + dashboard shows quota available but review says limit reached openai/codex#15477). If it does, the cost concern behind Codex reviews every PR including mine — is that acceptable, and what is the trigger surface? #1028 dissolves.

[simonoppowa]

Closed with the repo-side work shipped and the account-side work outstanding

Recording the split honestly rather than closing as "done", because this map's destination was "a real outside PR carries a review, and you've judged it keep / tune / drop" — and no Codex review has run yet.

Shipped and verified on develop

#1045 as 74464cb9, #1047 as 24c25dd4. Checked against what actually landed, not against a green merge:

AGENTS.md	28,140 bytes — 2,860 under the guard, 4,628 under the 32 KiB cap
## Code Review Rules	byte 653
tools/adb/README.md · docs/demo-data.md	3,183 · 3,363 bytes
check_agents_md	justfile (first in ci:) and linux-checks
Dependabot	4/4 ecosystems grouped
"120-char" claim	0 occurrences repo-wide

Not done — needs the maintainer's account or the secret values

1. #1032's environment checklist. Verified unexecuted at close: main still 0 required approving reviews, all four environments protection_rules: [], 0 environment-scoped secrets. This is the gate — the Codex App requests contents/workflows/actions: write, and Protect the Actions secret store before installing any write-capable app #1032's own finding is that scoping without a required reviewer buys nothing against that.
2. Install the Codex App, selected repositories → this repo only.
3. Settings: Automatic reviews on, trigger "On PR open", policy "Review all PRs".
4. Backfill @codex review on docs(readme): name every contact in What leaves your device #989 #1010 and feat: add scan and create-custom actions to no-results search state (#578) #995 — enabling reaches neither.
5. The verdict itself — keep / tune / drop, against the bar set in Prove the value by hand before installing anything: review PR #995 with /code-review #1035.

What this map produced

A decision trail rather than a shipped reviewer. The load-bearing findings, each verified against a primary source rather than inherited:

• Jules cannot review pull requests. Settled against an API discovery document generated the day before it was checked — not the changelog, which had been silent for six months. The original request is not achievable as asked.
• The GitHub-App-cannot-read-secrets premise is false in the way that matters: workflows: write is secret access, because the holder can author a workflow that echoes them.
• Only a vendor-side integration can serve fork PRs, and four independent parties refuse to run agents on fork code with secrets in scope.
• OpenAI's is the only consumer subscription that buys it — and it has no author filter, which measured out at 80.5% of PRs being ones already reviewed locally.

Loose ends worth knowing

• Copilot found three real defects across four reviews of this work. One — a contradiction between a changed file and an unchanged one — is a class the diff-scoped rules written in What the review checks, how loud it is, and what fits in 3,087 bytes #1029 cannot produce. Worth revisiting if Codex reviews the same way.
• A ruleset named "Copilot automatic code review" applies to ~ALL branches and enforces deletion and non_fast_forward, so force-pushes and branch deletions are blocked repo-wide. docs/line-width-is-80 is stale on the remote and cannot be deleted.
• gh pr edit --base fails here with the Projects-classic GraphQL error; gh api ... -X PATCH -f base=... works.