From 744b7508b105aace8989ddc4b255d928e0cc74cd Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 15:12:54 +0200 Subject: [PATCH 01/10] Record what a waiting_for_present robot timeout actually means Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index d127ac66f..e1e9d3939 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -175,3 +175,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **`Span::mixed_site()` does not shield generated `let` patterns from call-site `const` items** — pattern resolution treats a visible const as a const pattern, and item lookup for mixed-site tokens happens at the call site. `macro_rules!` has the identical hole (probe: a macro emitting `let value = 1u32` under a call-site `const value: u8` fails E0308 "interpreted as a constant"); only nightly `def_site` hygiene closes it. Mixed-site still isolates generated locals from user *locals* — worth doing — but a module const named exactly like a crate-internal generated binding is unfixable on stable. - Host-dependent dependency-budget trap: `just dep-budget` (and so `just budgets` / `just ci`) can be red on a Mac while the same tree is green in CI. The budget walks `cargo tree --duplicates`, which resolves for the host platform only, and the CI budgets job runs on Linux (`.github/workflows/rust.yml`, "architecture budgets (linux)"). Any Apple-only duplicate is therefore invisible to CI and unmissable locally — on 2026-08-27 it was `objc2`/`objc2-app-kit`/`objc2-foundation` (0.5.2 via `accesskit_macos`, plus 0.6.4). Before blaming your change, re-run the budget on a stashed tree; a duplicate that reproduces on a clean baseline is platform-scoped, not yours. - Doubled event-delivery diagnosis: when a composition-scoped collector reports every platform event exactly twice, count the *service registry's observers* before suspecting a double publish — two live registrations feeding one collector look identical to one publisher firing twice, and only the observer count separates them. `two_event_streams_in_one_composable_each_deliver_exactly_once` in `crates/cranpose-core/tests/effects_and_frames.rs` is the harness shape: a fake service that counts registrations, plus per-publish delivery assertions. +- **A robot `wait_for_idle` timeout with `waiting_for_present=true` is a host-load symptom, not a composition bug.** The budget is an iteration count, not wall clock, so on a machine under load (an Android emulator and Android Studio alone put this one at `load_1m=24/10`) the app can still be legitimately mid-present when the count runs out — `needs_update=false, has_animations=false` says composition already settled. Check `uptime` and re-run the one example (`just robot-one `) before believing it: the same test passed in isolation and the full 121-test suite passed on the second run, same commit. From 39cdb16112f4285972d968c8e7bfcf67ae183a4c Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 18:01:34 +0200 Subject: [PATCH 02/10] Record the hygiene gates a per-crate test run never reaches Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index e1e9d3939..430984cc2 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -176,3 +176,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - Host-dependent dependency-budget trap: `just dep-budget` (and so `just budgets` / `just ci`) can be red on a Mac while the same tree is green in CI. The budget walks `cargo tree --duplicates`, which resolves for the host platform only, and the CI budgets job runs on Linux (`.github/workflows/rust.yml`, "architecture budgets (linux)"). Any Apple-only duplicate is therefore invisible to CI and unmissable locally — on 2026-08-27 it was `objc2`/`objc2-app-kit`/`objc2-foundation` (0.5.2 via `accesskit_macos`, plus 0.6.4). Before blaming your change, re-run the budget on a stashed tree; a duplicate that reproduces on a clean baseline is platform-scoped, not yours. - Doubled event-delivery diagnosis: when a composition-scoped collector reports every platform event exactly twice, count the *service registry's observers* before suspecting a double publish — two live registrations feeding one collector look identical to one publisher firing twice, and only the observer count separates them. `two_event_streams_in_one_composable_each_deliver_exactly_once` in `crates/cranpose-core/tests/effects_and_frames.rs` is the harness shape: a fake service that counts registrations, plus per-publish delivery assertions. - **A robot `wait_for_idle` timeout with `waiting_for_present=true` is a host-load symptom, not a composition bug.** The budget is an iteration count, not wall clock, so on a machine under load (an Android emulator and Android Studio alone put this one at `load_1m=24/10`) the app can still be legitimately mid-present when the count runs out — `needs_update=false, has_animations=false` says composition already settled. Check `uptime` and re-run the one example (`just robot-one `) before believing it: the same test passed in isolation and the full 121-test suite passed on the second run, same commit. +- **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. From e951b360ee8d700dc748e9b292143db7f26f9a62 Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 18:03:08 +0200 Subject: [PATCH 03/10] Say what to do in the ten seconds before fixing a red gate Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index 430984cc2..865922e9c 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -176,4 +176,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - Host-dependent dependency-budget trap: `just dep-budget` (and so `just budgets` / `just ci`) can be red on a Mac while the same tree is green in CI. The budget walks `cargo tree --duplicates`, which resolves for the host platform only, and the CI budgets job runs on Linux (`.github/workflows/rust.yml`, "architecture budgets (linux)"). Any Apple-only duplicate is therefore invisible to CI and unmissable locally — on 2026-08-27 it was `objc2`/`objc2-app-kit`/`objc2-foundation` (0.5.2 via `accesskit_macos`, plus 0.6.4). Before blaming your change, re-run the budget on a stashed tree; a duplicate that reproduces on a clean baseline is platform-scoped, not yours. - Doubled event-delivery diagnosis: when a composition-scoped collector reports every platform event exactly twice, count the *service registry's observers* before suspecting a double publish — two live registrations feeding one collector look identical to one publisher firing twice, and only the observer count separates them. `two_event_streams_in_one_composable_each_deliver_exactly_once` in `crates/cranpose-core/tests/effects_and_frames.rs` is the harness shape: a fake service that counts registrations, plus per-publish delivery assertions. - **A robot `wait_for_idle` timeout with `waiting_for_present=true` is a host-load symptom, not a composition bug.** The budget is an iteration count, not wall clock, so on a machine under load (an Android emulator and Android Studio alone put this one at `load_1m=24/10`) the app can still be legitimately mid-present when the count runs out — `needs_update=false, has_animations=false` says composition already settled. Check `uptime` and re-run the one example (`just robot-one `) before believing it: the same test passed in isolation and the full 121-test suite passed on the second run, same commit. -- **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. +- **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. And when a gate does go red, `gh pr list --search ""` before writing a patch: a break that is *in* main is not cleared by rebasing, and someone else's fix is usually already open. From d21f4f9e94d725f9ace4542f8056bd4e5f2976bb Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:25:41 +0200 Subject: [PATCH 04/10] Record the disk-full failure shapes that read as compile errors Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index 865922e9c..d2d8f32a2 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -177,3 +177,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - Doubled event-delivery diagnosis: when a composition-scoped collector reports every platform event exactly twice, count the *service registry's observers* before suspecting a double publish — two live registrations feeding one collector look identical to one publisher firing twice, and only the observer count separates them. `two_event_streams_in_one_composable_each_deliver_exactly_once` in `crates/cranpose-core/tests/effects_and_frames.rs` is the harness shape: a fake service that counts registrations, plus per-publish delivery assertions. - **A robot `wait_for_idle` timeout with `waiting_for_present=true` is a host-load symptom, not a composition bug.** The budget is an iteration count, not wall clock, so on a machine under load (an Android emulator and Android Studio alone put this one at `load_1m=24/10`) the app can still be legitimately mid-present when the count runs out — `needs_update=false, has_animations=false` says composition already settled. Check `uptime` and re-run the one example (`just robot-one `) before believing it: the same test passed in isolation and the full 121-test suite passed on the second run, same commit. - **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. And when a gate does go red, `gh pr list --search ""` before writing a patch: a break that is *in* main is not cleared by rebasing, and someone else's fix is usually already open. +- **A full disk reports itself as a compile or test failure, never as a disk error.** The signatures are `failed to build archive ...: failed to open object file (os error 2)`, `extern location for does not exist: .../deps/lib-*.rmeta`, and a test binary that `could not execute process ... (never executed)` — all of which read as a dependency or code problem. They are truncated artifacts from a build that was killed mid-write. Check `df -h /` first. Two follow-on traps: `cargo clean --profile

` can itself die partway on the corrupted tree and print `No such file or directory` while leaving it broken, so confirm the directory is actually gone or `mv` it aside and let the build recreate it; and at genuinely zero bytes every tool that writes a temp file fails, including the ones you would use to diagnose, so free space before anything else. On this workspace `target/` reaches ~200G across profiles and a review worktree adds ~60G more. From e14aad623627a28444a904176e1ce3b5adc42430 Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:28:40 +0200 Subject: [PATCH 05/10] Record the CI-only ANSI that defeats a cargo tree prefix guard Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index d2d8f32a2..5ecf5820b 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -178,3 +178,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **A robot `wait_for_idle` timeout with `waiting_for_present=true` is a host-load symptom, not a composition bug.** The budget is an iteration count, not wall clock, so on a machine under load (an Android emulator and Android Studio alone put this one at `load_1m=24/10`) the app can still be legitimately mid-present when the count runs out — `needs_update=false, has_animations=false` says composition already settled. Check `uptime` and re-run the one example (`just robot-one `) before believing it: the same test passed in isolation and the full 121-test suite passed on the second run, same commit. - **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. And when a gate does go red, `gh pr list --search ""` before writing a patch: a break that is *in* main is not cleared by rebasing, and someone else's fix is usually already open. - **A full disk reports itself as a compile or test failure, never as a disk error.** The signatures are `failed to build archive ...: failed to open object file (os error 2)`, `extern location for does not exist: .../deps/lib-*.rmeta`, and a test binary that `could not execute process ... (never executed)` — all of which read as a dependency or code problem. They are truncated artifacts from a build that was killed mid-write. Check `df -h /` first. Two follow-on traps: `cargo clean --profile

` can itself die partway on the corrupted tree and print `No such file or directory` while leaving it broken, so confirm the directory is actually gone or `mv` it aside and let the build recreate it; and at genuinely zero bytes every tool that writes a temp file fails, including the ones you would use to diagnose, so free space before anything else. On this workspace `target/` reaches ~200G across profiles and a review worktree adds ~60G more. +- **A parser that reads `cargo tree` output must strip ANSI, not just the box-drawing characters.** `.github/workflows/rust.yml` sets `CARGO_TERM_COLOR: always`, and cargo honours it even through a pipe, so a line that reads `│ └── name` locally arrives as `ESC[2m│ESC[0m ESC[2m└──ESC[0m name` in CI — a `starts_with('│')` guard never fires and the name comes out carrying the drawing. It is invisible on every dev machine because a pipe auto-disables colour: `cargo tree | od -c` shows no `033` bytes, `CARGO_TERM_COLOR=always cargo tree | od -c` shows `033 [ 2 m` ahead of every prefix. Reproduce any CI-only text-parsing difference with `CARGO_TERM_COLOR=always `, and fix it in both layers — strip ANSI in the parser so any colour source is handled, and pass `--color never` so it is never generated. From 6adbd80319b1ba3fc5f1bfd275d7472ac82980e6 Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:32:42 +0200 Subject: [PATCH 06/10] Fix the iteration-count inference, and record the waiter that ends on a gap Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index 5ecf5820b..b9f61b557 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -8,7 +8,7 @@ Signature → cause → what to do. One lesson per line, no incident history. - **Confirm a cause by REMOVING it and re-running, before writing the fix.** Binary-search by cutting half the code. - **A/B two revisions on ONE machine, back to back.** Byte-identical output (same SHA-256, empty `ImageChops.difference` bbox) ends the question. Two machines — or one machine hours apart — compares host state as much as code. - **"Fails on clean main too" ≠ environmental.** A shared harness mistake fails on clean main as well. -- **Host flake vs regression, from the log alone:** `wait_for_idle: timed out after 1 iterations` means the process was never scheduled (a real non-convergence burns thousands of iterations in the same wall clock); sum the gaps between `Running robot_...` lines excluding the suspect for a load index (the suite reproduces within a second — 843/843/844s — so a 970s run is unmistakable); then `gh workflow run heavy-selfhosted.yml --ref ` to re-run the identical commit. Do that dispatch first, not last. +- **Host flake vs regression, from the log alone:** read the diagnostic fields, not the iteration count. `wait_for_idle: timed out after 1 iterations` means the process was never scheduled, but the converse does not hold: a huge count rules starvation *out* and proves nothing else. `needs_update=false, has_animations=false, waiting_for_present=true` is a slow compositor no matter how many iterations ran — `robot_idle_fps_after_tab_walk` burned 1,280,577 of them with composition already settled, and passed in isolation and in a full 121-test suite on the same commit. Only fields that accuse the application justify blaming it. Then sum the gaps between `Running robot_...` lines excluding the suspect for a load index (the suite reproduces within a second — 843/843/844s — so a 970s run is unmistakable); then `gh workflow run heavy-selfhosted.yml --ref ` to re-run the identical commit. Do that dispatch first, not last. - **A failure one hair over its bound with a timing metric that moved with it is contention.** A real leak grows every cycle and clears the tolerance by a mile. Never widen the bound — for `robot_text_handle_cycle_stability` the accumulation ratio *is* the guard. - CI preserves full stdout on the runner (`Preserving robot result artifacts after status 1:

`) — complete, unlike the truncated Actions log. - Before acting on a proxy metric, read how the tool decides. `scripts/public_api_test_coverage.py` matched names as *substrings* (`with_timeout` "covered" by `exit_with_timeout`) and scanned `crates/` only, so the robot suite — the sole exercise most of the driver API gets — counted as untested. @@ -179,3 +179,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. And when a gate does go red, `gh pr list --search ""` before writing a patch: a break that is *in* main is not cleared by rebasing, and someone else's fix is usually already open. - **A full disk reports itself as a compile or test failure, never as a disk error.** The signatures are `failed to build archive ...: failed to open object file (os error 2)`, `extern location for does not exist: .../deps/lib-*.rmeta`, and a test binary that `could not execute process ... (never executed)` — all of which read as a dependency or code problem. They are truncated artifacts from a build that was killed mid-write. Check `df -h /` first. Two follow-on traps: `cargo clean --profile

` can itself die partway on the corrupted tree and print `No such file or directory` while leaving it broken, so confirm the directory is actually gone or `mv` it aside and let the build recreate it; and at genuinely zero bytes every tool that writes a temp file fails, including the ones you would use to diagnose, so free space before anything else. On this workspace `target/` reaches ~200G across profiles and a review worktree adds ~60G more. - **A parser that reads `cargo tree` output must strip ANSI, not just the box-drawing characters.** `.github/workflows/rust.yml` sets `CARGO_TERM_COLOR: always`, and cargo honours it even through a pipe, so a line that reads `│ └── name` locally arrives as `ESC[2m│ESC[0m ESC[2m└──ESC[0m name` in CI — a `starts_with('│')` guard never fires and the name comes out carrying the drawing. It is invisible on every dev machine because a pipe auto-disables colour: `cargo tree | od -c` shows no `033` bytes, `CARGO_TERM_COLOR=always cargo tree | od -c` shows `033 [ 2 m` ahead of every prefix. Reproduce any CI-only text-parsing difference with `CARGO_TERM_COLOR=always `, and fix it in both layers — strip ANSI in the parser so any colour source is handled, and pass `--color never` so it is never generated. +- **A CI waiter that greps for "pending" fires on an output gap.** `until ! gh pr checks 2>/dev/null | grep -q pending` reads "the word is absent" as "the run finished", so one transient gh failure ends the wait exactly like completion — and the `2>/dev/null` hides the error that would have shown it. `gh pr checks` also exits 8 while checks are still queued, so a waiter keyed on its status reports failure while its own banner says complete. Require all three: gh exited 0, output non-empty, and every check terminal via `[.statusCheckRollup[] | (.status // .state // "UNKNOWN")]` rejecting `QUEUED|IN_PROGRESS|PENDING|WAITING|REQUESTED|UNKNOWN`. That still only sees *registered* checks, so a partial list reads as complete — assert the expected set by name; a cranpose PR carries seven (Android release build, architecture budgets, fmt + tests + clippy, iOS build, robot e2e, robot external captures, wasm build). And check what the merge actually did rather than printing success after calling it: a `gh pr merge` refused for a conflict still returns to the next line. From 61d854dbf69343fc9c732fe3273ad6f4b7a9288b Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:35:07 +0200 Subject: [PATCH 07/10] Require pass over not-failed, and ancestry over MERGED state Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index b9f61b557..62defd887 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -179,4 +179,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **Targeted `-p ` test runs miss the source-hygiene gates, and they are the ones that take main red.** `apps/desktop-demo/tests/source_hygiene_aliases.rs` scans *every* crate's sources — `workspace_tests_do_not_default_to_tmpfs_paths` rejects a `/tmp/` literal anywhere, even in a pure in-memory fixture that never opens the path, and `no_cargo_config_enables_unstable_web_sys_bindings` guards the rustflags. A per-crate gate set can be entirely green while these fail, and a sandboxed reviewer's `just test` can die at the socket-bound `cranpose-services::peer` tests before ever reaching them: both happened at once, so a `/tmp/` literal reached main and four sessions then fixed the same line in parallel (#503, #504, #506, #507). Run `cargo test --profile ci -p desktop-app --test source_hygiene_aliases` alongside targeted runs, or the full `just test`, before pushing. And when a gate does go red, `gh pr list --search ""` before writing a patch: a break that is *in* main is not cleared by rebasing, and someone else's fix is usually already open. - **A full disk reports itself as a compile or test failure, never as a disk error.** The signatures are `failed to build archive ...: failed to open object file (os error 2)`, `extern location for does not exist: .../deps/lib-*.rmeta`, and a test binary that `could not execute process ... (never executed)` — all of which read as a dependency or code problem. They are truncated artifacts from a build that was killed mid-write. Check `df -h /` first. Two follow-on traps: `cargo clean --profile

` can itself die partway on the corrupted tree and print `No such file or directory` while leaving it broken, so confirm the directory is actually gone or `mv` it aside and let the build recreate it; and at genuinely zero bytes every tool that writes a temp file fails, including the ones you would use to diagnose, so free space before anything else. On this workspace `target/` reaches ~200G across profiles and a review worktree adds ~60G more. - **A parser that reads `cargo tree` output must strip ANSI, not just the box-drawing characters.** `.github/workflows/rust.yml` sets `CARGO_TERM_COLOR: always`, and cargo honours it even through a pipe, so a line that reads `│ └── name` locally arrives as `ESC[2m│ESC[0m ESC[2m└──ESC[0m name` in CI — a `starts_with('│')` guard never fires and the name comes out carrying the drawing. It is invisible on every dev machine because a pipe auto-disables colour: `cargo tree | od -c` shows no `033` bytes, `CARGO_TERM_COLOR=always cargo tree | od -c` shows `033 [ 2 m` ahead of every prefix. Reproduce any CI-only text-parsing difference with `CARGO_TERM_COLOR=always `, and fix it in both layers — strip ANSI in the parser so any colour source is handled, and pass `--color never` so it is never generated. -- **A CI waiter that greps for "pending" fires on an output gap.** `until ! gh pr checks 2>/dev/null | grep -q pending` reads "the word is absent" as "the run finished", so one transient gh failure ends the wait exactly like completion — and the `2>/dev/null` hides the error that would have shown it. `gh pr checks` also exits 8 while checks are still queued, so a waiter keyed on its status reports failure while its own banner says complete. Require all three: gh exited 0, output non-empty, and every check terminal via `[.statusCheckRollup[] | (.status // .state // "UNKNOWN")]` rejecting `QUEUED|IN_PROGRESS|PENDING|WAITING|REQUESTED|UNKNOWN`. That still only sees *registered* checks, so a partial list reads as complete — assert the expected set by name; a cranpose PR carries seven (Android release build, architecture budgets, fmt + tests + clippy, iOS build, robot e2e, robot external captures, wasm build). And check what the merge actually did rather than printing success after calling it: a `gh pr merge` refused for a conflict still returns to the next line. +- **A CI waiter that greps for "pending" fires on an output gap.** `until ! gh pr checks 2>/dev/null | grep -q pending` reads "the word is absent" as "the run finished", so one transient gh failure ends the wait exactly like completion — and the `2>/dev/null` hides the error that would have shown it. `gh pr checks` also exits 8 while checks are still queued, so a waiter keyed on its status reports failure while its own banner says complete. Require all three: gh exited 0, output non-empty, and every check terminal via `[.statusCheckRollup[] | (.status // .state // "UNKNOWN")]` rejecting `QUEUED|IN_PROGRESS|PENDING|WAITING|REQUESTED|UNKNOWN`. That still only sees *registered* checks, so a partial list reads as complete — assert the expected set by name; a cranpose PR carries seven (Android release build, architecture budgets, fmt + tests + clippy, iOS build, robot e2e, robot external captures, wasm build). Require *pass*, not "nothing failed" — a skipped or cancelled check is not green. Then check what the merge actually did rather than printing success after calling it: a `gh pr merge` refused for a conflict still returns to the next line, and its exit status must be read directly, never through a pipe (same trap as the `| tail` entry above). Finish by confirming the squash commit is an ancestor of `origin/main` with the expected subject — `state == MERGED` can be set by auto-merge, a merge queue or an admin action, so it never asserts that *your* merge is what landed. From 7bbeb4fcb1ca904d98f7dbff395bc71d51e2f331 Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:35:46 +0200 Subject: [PATCH 08/10] Record that an untested guard is not a guard Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index 62defd887..307486b02 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -180,3 +180,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **A full disk reports itself as a compile or test failure, never as a disk error.** The signatures are `failed to build archive ...: failed to open object file (os error 2)`, `extern location for does not exist: .../deps/lib-*.rmeta`, and a test binary that `could not execute process ... (never executed)` — all of which read as a dependency or code problem. They are truncated artifacts from a build that was killed mid-write. Check `df -h /` first. Two follow-on traps: `cargo clean --profile

` can itself die partway on the corrupted tree and print `No such file or directory` while leaving it broken, so confirm the directory is actually gone or `mv` it aside and let the build recreate it; and at genuinely zero bytes every tool that writes a temp file fails, including the ones you would use to diagnose, so free space before anything else. On this workspace `target/` reaches ~200G across profiles and a review worktree adds ~60G more. - **A parser that reads `cargo tree` output must strip ANSI, not just the box-drawing characters.** `.github/workflows/rust.yml` sets `CARGO_TERM_COLOR: always`, and cargo honours it even through a pipe, so a line that reads `│ └── name` locally arrives as `ESC[2m│ESC[0m ESC[2m└──ESC[0m name` in CI — a `starts_with('│')` guard never fires and the name comes out carrying the drawing. It is invisible on every dev machine because a pipe auto-disables colour: `cargo tree | od -c` shows no `033` bytes, `CARGO_TERM_COLOR=always cargo tree | od -c` shows `033 [ 2 m` ahead of every prefix. Reproduce any CI-only text-parsing difference with `CARGO_TERM_COLOR=always `, and fix it in both layers — strip ANSI in the parser so any colour source is handled, and pass `--color never` so it is never generated. - **A CI waiter that greps for "pending" fires on an output gap.** `until ! gh pr checks 2>/dev/null | grep -q pending` reads "the word is absent" as "the run finished", so one transient gh failure ends the wait exactly like completion — and the `2>/dev/null` hides the error that would have shown it. `gh pr checks` also exits 8 while checks are still queued, so a waiter keyed on its status reports failure while its own banner says complete. Require all three: gh exited 0, output non-empty, and every check terminal via `[.statusCheckRollup[] | (.status // .state // "UNKNOWN")]` rejecting `QUEUED|IN_PROGRESS|PENDING|WAITING|REQUESTED|UNKNOWN`. That still only sees *registered* checks, so a partial list reads as complete — assert the expected set by name; a cranpose PR carries seven (Android release build, architecture budgets, fmt + tests + clippy, iOS build, robot e2e, robot external captures, wasm build). Require *pass*, not "nothing failed" — a skipped or cancelled check is not green. Then check what the merge actually did rather than printing success after calling it: a `gh pr merge` refused for a conflict still returns to the next line, and its exit status must be read directly, never through a pipe (same trap as the `| tail` entry above). Finish by confirming the squash commit is an ancestor of `origin/main` with the expected subject — `state == MERGED` can be set by auto-merge, a merge queue or an admin action, so it never asserts that *your* merge is what landed. +- **Verify the verifier: a guard you have not watched reject a bad input is not a guard.** Every hardening step is itself a predicate nobody tested, and the test is the easy thing to get silently wrong. `timeout` does not exist on macOS, so `timeout 8 ./wait.sh && echo FIRED || echo held` prints "held" from `command not found` — a green result for a script that never ran, and AGENTS.md sends people to `macm3` over ssh where a `timeout`-based test copied from Linux fails exactly this way. Feed the guard the states it exists to catch and watch each one: for a CI gate that is empty rollup, partial-but-all-green, one QUEUED, one FAILURE, one CANCELLED, one SKIPPED, an unexpected extra check, and only then all-green. The partial case is not hypothetical — a PR was observed going "no checks reported" to 4 registered to 7, so a gate that trusts the registered list can pass before the slowest job is created. From 34cf6ad9830863a55db012ff47db8c0ab32c1119 Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:40:35 +0200 Subject: [PATCH 09/10] Record that background app work is billed to the framework Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index 307486b02..4eb86828a 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -181,3 +181,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **A parser that reads `cargo tree` output must strip ANSI, not just the box-drawing characters.** `.github/workflows/rust.yml` sets `CARGO_TERM_COLOR: always`, and cargo honours it even through a pipe, so a line that reads `│ └── name` locally arrives as `ESC[2m│ESC[0m ESC[2m└──ESC[0m name` in CI — a `starts_with('│')` guard never fires and the name comes out carrying the drawing. It is invisible on every dev machine because a pipe auto-disables colour: `cargo tree | od -c` shows no `033` bytes, `CARGO_TERM_COLOR=always cargo tree | od -c` shows `033 [ 2 m` ahead of every prefix. Reproduce any CI-only text-parsing difference with `CARGO_TERM_COLOR=always `, and fix it in both layers — strip ANSI in the parser so any colour source is handled, and pass `--color never` so it is never generated. - **A CI waiter that greps for "pending" fires on an output gap.** `until ! gh pr checks 2>/dev/null | grep -q pending` reads "the word is absent" as "the run finished", so one transient gh failure ends the wait exactly like completion — and the `2>/dev/null` hides the error that would have shown it. `gh pr checks` also exits 8 while checks are still queued, so a waiter keyed on its status reports failure while its own banner says complete. Require all three: gh exited 0, output non-empty, and every check terminal via `[.statusCheckRollup[] | (.status // .state // "UNKNOWN")]` rejecting `QUEUED|IN_PROGRESS|PENDING|WAITING|REQUESTED|UNKNOWN`. That still only sees *registered* checks, so a partial list reads as complete — assert the expected set by name; a cranpose PR carries seven (Android release build, architecture budgets, fmt + tests + clippy, iOS build, robot e2e, robot external captures, wasm build). Require *pass*, not "nothing failed" — a skipped or cancelled check is not green. Then check what the merge actually did rather than printing success after calling it: a `gh pr merge` refused for a conflict still returns to the next line, and its exit status must be read directly, never through a pipe (same trap as the `| tail` entry above). Finish by confirming the squash commit is an ancestor of `origin/main` with the expected subject — `state == MERGED` can be set by auto-merge, a merge queue or an admin action, so it never asserts that *your* merge is what landed. - **Verify the verifier: a guard you have not watched reject a bad input is not a guard.** Every hardening step is itself a predicate nobody tested, and the test is the easy thing to get silently wrong. `timeout` does not exist on macOS, so `timeout 8 ./wait.sh && echo FIRED || echo held` prints "held" from `command not found` — a green result for a script that never ran, and AGENTS.md sends people to `macm3` over ssh where a `timeout`-based test copied from Linux fails exactly this way. Feed the guard the states it exists to catch and watch each one: for a CI gate that is empty rollup, partial-but-all-green, one QUEUED, one FAILURE, one CANCELLED, one SKIPPED, an unexpected extra check, and only then all-green. The partial case is not hypothetical — a PR was observed going "no checks reported" to 4 registered to 7, so a gate that trusts the registered list can pass before the slowest job is created. +- **An app's own background work is measured as framework frame cost.** Scroll on a 2018-class Android device read as 17 fps with `present` p50 29-33 ms and was filed against the framework as swapchain-bound; the app was in fact still running on-device inference, settled at 442% CPU before a finger touched the screen. With that work off and the app at 0-3% CPU, the same page on the same device gave ~40 fps and `present` p50 5.1-7.3 ms — 4-5x cheaper, and a different diagnosis. Before trusting any on-device frame number, poll `adb shell top` for the package until its CPU settles and print what it settled at; a run that does not report its settled CPU is not a measurement, and the same applies to any app doing background indexing or sync. Never A/B across two package names either — separate installs carry separate app data and settings, so the arms differ in more than the build (theirs disagreed on `vsync_period_ms`, 15.841 vs 16.632, which is how the contamination surfaced). Install both arms over the same package, or flip them inside one binary with a `debug.cranpose.*` property override as `segment_surface` and `pipeline_disk_cache` already do. `idle_iters` tells you whether a telemetry window was a real scroll window or mostly idle. From 0a67941c418889502fd53a671652a60b690b07bc Mon Sep 17 00:00:00 2001 From: Dmitry Samoylenko Date: Thu, 27 Aug 2026 20:42:20 +0200 Subject: [PATCH 10/10] Correct the contamination magnitude and what it did not touch Co-Authored-By: Claude Fable 5 --- TIME_WASTERS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TIME_WASTERS.md b/TIME_WASTERS.md index 4eb86828a..dd3656549 100644 --- a/TIME_WASTERS.md +++ b/TIME_WASTERS.md @@ -181,4 +181,4 @@ Signature → cause → what to do. One lesson per line, no incident history. - **A parser that reads `cargo tree` output must strip ANSI, not just the box-drawing characters.** `.github/workflows/rust.yml` sets `CARGO_TERM_COLOR: always`, and cargo honours it even through a pipe, so a line that reads `│ └── name` locally arrives as `ESC[2m│ESC[0m ESC[2m└──ESC[0m name` in CI — a `starts_with('│')` guard never fires and the name comes out carrying the drawing. It is invisible on every dev machine because a pipe auto-disables colour: `cargo tree | od -c` shows no `033` bytes, `CARGO_TERM_COLOR=always cargo tree | od -c` shows `033 [ 2 m` ahead of every prefix. Reproduce any CI-only text-parsing difference with `CARGO_TERM_COLOR=always `, and fix it in both layers — strip ANSI in the parser so any colour source is handled, and pass `--color never` so it is never generated. - **A CI waiter that greps for "pending" fires on an output gap.** `until ! gh pr checks 2>/dev/null | grep -q pending` reads "the word is absent" as "the run finished", so one transient gh failure ends the wait exactly like completion — and the `2>/dev/null` hides the error that would have shown it. `gh pr checks` also exits 8 while checks are still queued, so a waiter keyed on its status reports failure while its own banner says complete. Require all three: gh exited 0, output non-empty, and every check terminal via `[.statusCheckRollup[] | (.status // .state // "UNKNOWN")]` rejecting `QUEUED|IN_PROGRESS|PENDING|WAITING|REQUESTED|UNKNOWN`. That still only sees *registered* checks, so a partial list reads as complete — assert the expected set by name; a cranpose PR carries seven (Android release build, architecture budgets, fmt + tests + clippy, iOS build, robot e2e, robot external captures, wasm build). Require *pass*, not "nothing failed" — a skipped or cancelled check is not green. Then check what the merge actually did rather than printing success after calling it: a `gh pr merge` refused for a conflict still returns to the next line, and its exit status must be read directly, never through a pipe (same trap as the `| tail` entry above). Finish by confirming the squash commit is an ancestor of `origin/main` with the expected subject — `state == MERGED` can be set by auto-merge, a merge queue or an admin action, so it never asserts that *your* merge is what landed. - **Verify the verifier: a guard you have not watched reject a bad input is not a guard.** Every hardening step is itself a predicate nobody tested, and the test is the easy thing to get silently wrong. `timeout` does not exist on macOS, so `timeout 8 ./wait.sh && echo FIRED || echo held` prints "held" from `command not found` — a green result for a script that never ran, and AGENTS.md sends people to `macm3` over ssh where a `timeout`-based test copied from Linux fails exactly this way. Feed the guard the states it exists to catch and watch each one: for a CI gate that is empty rollup, partial-but-all-green, one QUEUED, one FAILURE, one CANCELLED, one SKIPPED, an unexpected extra check, and only then all-green. The partial case is not hypothetical — a PR was observed going "no checks reported" to 4 registered to 7, so a gate that trusts the registered list can pass before the slowest job is created. -- **An app's own background work is measured as framework frame cost.** Scroll on a 2018-class Android device read as 17 fps with `present` p50 29-33 ms and was filed against the framework as swapchain-bound; the app was in fact still running on-device inference, settled at 442% CPU before a finger touched the screen. With that work off and the app at 0-3% CPU, the same page on the same device gave ~40 fps and `present` p50 5.1-7.3 ms — 4-5x cheaper, and a different diagnosis. Before trusting any on-device frame number, poll `adb shell top` for the package until its CPU settles and print what it settled at; a run that does not report its settled CPU is not a measurement, and the same applies to any app doing background indexing or sync. Never A/B across two package names either — separate installs carry separate app data and settings, so the arms differ in more than the build (theirs disagreed on `vsync_period_ms`, 15.841 vs 16.632, which is how the contamination surfaced). Install both arms over the same package, or flip them inside one binary with a `debug.cranpose.*` property override as `segment_surface` and `pipeline_disk_cache` already do. `idle_iters` tells you whether a telemetry window was a real scroll window or mostly idle. +- **An app's own background work is measured as framework frame cost.** Scroll on a 2018-class Android device read as 15-17 fps with `present` p50 29-33 ms; the app was in fact still running on-device inference, settled at 442% CPU before a finger touched the screen. Re-measured on the same scene with the app quiet at 0-3% CPU: 22-28 fps, `present` p50 13.5-17 ms — the headline numbers were inflated about 1.5x. Note what did *not* move: the ranked causes filed alongside them — render passes, isolated layers, and a 35% layer-cache hit rate re-measured at 34.6% with 1.78 MP of blurred offscreen per frame — reproduced on the quiet device, two of them worse than filed. Contamination inflates the headline figures you quote in the title; it does not necessarily touch the per-frame counters underneath, so re-verify rather than retracting wholesale. Before trusting any on-device frame number, poll `adb shell top` for the package until its CPU settles and print what it settled at; a run that does not report its settled CPU is not a measurement, and the same applies to any app doing background indexing or sync. Never A/B across two package names either — separate installs carry separate app data and settings, so the arms differ in more than the build (theirs disagreed on `vsync_period_ms`, 15.841 vs 16.632, which is how the contamination surfaced). Install both arms over the same package, or flip them inside one binary with a `debug.cranpose.*` property override as `segment_surface` and `pipeline_disk_cache` already do. `idle_iters` tells you whether a telemetry window was a real scroll window or mostly idle.