diff --git a/AGENTS.md b/AGENTS.md index 146260d..ff9d979 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -94,7 +94,7 @@ canonicalUrl: string **Publishing**: Raise a PR and review it. Then, per surface: - `data/blog/` - stop at review. Never merge, never enable automerge. A human publishes blog posts. -- `data/devlog/` - review and merge. The `4h devlog sweep` schedule owns this surface and merges its own PRs. +- `data/devlog/` - review and merge. Follow the editorial standards in `NOTES.md`. - Anything else - stop at review. **Styling**: Minimal industrial design system (flat hairlines, sharp corners, IBM Plex Mono labels). Follow the conventions in `.agents/skills/site-conventions/SKILL.md` for any page or component work. @@ -117,4 +117,4 @@ canonicalUrl: string - **Analytics**: Umami (configured in siteMetadata.js) - **Search**: kbar with local search index - **Discord**: Real-time presence via react-use-lanyard -- **GitHub**: Calendar integration via react-activity-calendar \ No newline at end of file +- **GitHub**: Calendar integration via react-activity-calendar diff --git a/NOTES.md b/NOTES.md index 6053b91..c51d3d4 100644 --- a/NOTES.md +++ b/NOTES.md @@ -1,6 +1,6 @@ -# Notes +# Devlog -The bar for `data/notes/`, the second content surface on this site. Longer first-person +The bar for `data/devlog/`, the second content surface on this site. Longer first-person opinion pieces live in `data/blog/`; see `AGENTS.md` and `.agents/skills/site-conventions/SKILL.md` for the shared formatting rules. @@ -9,33 +9,44 @@ opinion pieces live in `data/blog/`; see `AGENTS.md` and A note earns its place here if **a reader could act on it, or would otherwise have to rediscover the thing themselves.** That is the whole test. +Write about a specific day's work or learning in the author's voice. Make the +project, question or activity recognisable in the opening. Ground claims in +evidence from that work or learning. Distinguish documented behaviour from direct +experience, experiments from tested fixes, and proposed changes from shipped work. +Do not imply the author encountered or fixed something without support. + Write one if: - You hit a specific failure and worked out the cause and the fix. - There is a behaviour of a tool that surprised you and you verified why. - You found a pattern in a system that generalises past the one system. +- You investigated a concrete research question and can explain what the evidence + established, without presenting documented behaviour as a local reproduction. Do not write one if: -- It is a report of what you shipped. That is a work log, not a note. +- It is a list of shipped changes with no useful observation or explanation. - It is an opinion piece you could expand into a blog post. Put it in `data/blog/`. - It only makes sense to people who already know your context. - You could not state the symptom precisely. +- It only repeats a release announcement or another write-up without adding a + supported, useful lesson. ## Shape - Frontmatter: `title`, `summary`, `date`, `tags`, `draft`, `icon`. - `icon` must already be registered in `lib/post-icons.ts` (lucide icon names). The navbar, index rows and detail header all resolve from that one map. -- `date` is the date the note was written, not the date you fixed the thing. +- `date` identifies the day being described. When backfilling, use the verified + work or learning date; use a date without a time when the time is unknown. - Keep it short. If it runs past a few hundred lines, it is a blog post. - Footnotes (`[^1]`) for citations, never a `## References` heading. - No em dashes. No bold-as-speech. No AI slop openers or endings. - Link to the primary source when there is one. A note that cites a GitHub doc is worth ten notes that cite a forum. -## Notes are draft-free +## Drafts `draft: true` still works and hides the note from the index, the feed and the -static params. The seed notes are published. Use `draft` while a note is still -rough. +static params. Keep an entry in draft while it is rough or its claims remain +unverified. Correct and verify withdrawn entries before republishing them. diff --git a/data/devlog/app-bundled-clis-are-missing-from-daemon-path.mdx b/data/devlog/app-bundled-clis-are-missing-from-daemon-path.mdx deleted file mode 100644 index d0f283d..0000000 --- a/data/devlog/app-bundled-clis-are-missing-from-daemon-path.mdx +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: App-Bundled CLIs Are Missing From Daemon PATH -summary: | - A coding agent installed only inside ChatGPT.app or Codex.app is invisible to a daemon launched outside a terminal, because discovery walks PATH and a binary embedded in a .app bundle is not on it. Point the provider's command at the bundled binary and the agent shows up again. -date: '2026-10-15 21:15:00 +1000' -tags: - - macOS - - Configuration - - Gotcha -draft: false -icon: Bug ---- - -On macOS, a Codex CLI that only ships inside ChatGPT.app (`Contents/Resources/codex`) cannot be selected in Paseo Chat, even though the same binary works when called by hand. There is no error - the provider is simply not there. This is [getpaseo/paseo#6990](https://github.com/getpaseo/paseo/issues/6990). - -## Why it happens - -The daemon discovers the CLI by walking its own `PATH`, with a fallback that only covers the Microsoft Store on Windows. An app-bundled binary is not on any `PATH` - it lives inside the `.app` bundle - and a daemon launched from a GUI session inherits a minimal environment anyway. So the two halves of the system disagree: the binary exists and runs, and the daemon has no way to find it. - -## The fix - -Give the provider an explicit command in `~/.paseo/config.json` pointing at the bundled binary: - -```json -{ - "agents": { - "providers": { - "codex": { - "command": ["/Applications/ChatGPT.app/Contents/Resources/codex"] - } - } - } -} -``` - -Restart the daemon after saving. `command` is an array - first element the binary, the rest arguments - and it replaces the default launch entirely. The setting is documented under [Custom binary](https://paseo.sh/docs/custom-providers#custom-binary). - -This is the general answer for any CLI that ships inside a macOS `.app` rather than as a standalone install. If a daemon cannot see a binary that runs fine in a terminal, do not fight the discovery - name the path in its config and move on. diff --git a/data/devlog/betterfit-cold-simulator-ci.mdx b/data/devlog/betterfit-cold-simulator-ci.mdx new file mode 100644 index 0000000..40312a0 --- /dev/null +++ b/data/devlog/betterfit-cold-simulator-ci.mdx @@ -0,0 +1,41 @@ +--- +title: Getting BetterFit Through a Cold Simulator Start +summary: | + BetterFit's new iOS CI jobs were failing before the simulator fallback could run. I warmed CoreSimulator first, made selection failures recoverable and separated first-launch time from the test's total budget. +date: '2026-10-03 11:28:38 +1000' +tags: + - BetterFit + - iOS + - CI/CD +draft: false +icon: Bug +--- + +The new BetterFit iOS CI jobs exposed a difference between my warm local setup +and a fresh GitHub runner. The first `simctl` query could spend long enough +starting CoreSimulatorService to exceed the selection script's fifteen-second +timeout.[^fix] + +The fallback was there, but the failed pick exited before it could run. + +I changed the task to warm the daemon before querying, gave the pick sixty +seconds and let an empty or failed selection reach the any-iPhone fallback. +That addressed the setup path rather than just increasing the timeout around +the whole CI job. + +## First launch has its own cost + +There was a second budget problem after selection. The app-launch poll allowed +twenty seconds, and each test had a sixty-second wall that also included +`resetAndLaunch`. A fresh install on a loaded runner could consume most of that +before the assertions got going. + +The launch poll moved to sixty seconds and the per-test budget to 120. The local +e2e run passed four tests in 58 seconds after the fixes. The PR records the +cold-start and forced-empty-selection checks separately from the earlier CI run, +which had passed UI tests and coverage but hit the e2e time limit. + +The fix merged with the warm-machine path unchanged. It was a useful reminder +that a fallback is only real if the error path can actually reach it. + +[^fix]: [BetterFit's cold-runner and e2e timing fix](https://github.com/echohello-dev/betterfit/pull/26). diff --git a/data/devlog/betterfit-figma-to-ios.mdx b/data/devlog/betterfit-figma-to-ios.mdx new file mode 100644 index 0000000..85850ba --- /dev/null +++ b/data/devlog/betterfit-figma-to-ios.mdx @@ -0,0 +1,39 @@ +--- +title: Bringing BetterFit's Figma Work Into the App +summary: | + BetterFit's Figma direction and iOS code had drifted apart. I brought the electric-yellow identity, ledger components and rebuilt workout screens into the app, with the remaining lint debt called out separately. +date: '2026-09-28 17:30:29 +1000' +tags: + - BetterFit + - iOS + - Design Systems +draft: false +icon: Package +--- + +The BetterFit design work had moved on, but the iOS app was still carrying the +old ember-orange palette and earlier screen layouts. I used this pass to bring +the app back into agreement with the Figma direction.[^change] + +The identity is now electric yellow, black ink and neutral structure. Instead of +giving every status another colour, the yellow ladder uses brightness alongside +words. Glass is reserved for floating chrome, and yellow text on light surfaces +has its own darker token. + +I also added the `BFLedger` primitives: section rules, readouts, rows, quick actions +and the other pieces needed to make the screens feel like one app. WorkoutHome +and ActiveSession moved onto those components, alongside the profile, recovery, +sign-in and settings surfaces. + +## More than a palette swap + +The workout plan now has inline weight and reps editing, superset pairing and +equipment swapping. The active session carries the same direction through set +logging, ordering and the rest bar. + +This was the app-screens-to-code pass, not proof that every part of BetterFit was +finished. The PR also called out twelve pre-existing serious SwiftLint violations +in untouched files. Keeping that distinction visible was more useful than making +a design sync sound like a clean bill of health for the whole codebase. + +[^change]: [BetterFit's design-system and iOS screen sync](https://github.com/echohello-dev/betterfit/pull/23). diff --git a/data/devlog/bun-websockets-are-missing-pause-and-resume.mdx b/data/devlog/bun-websockets-are-missing-pause-and-resume.mdx deleted file mode 100644 index ff96d81..0000000 --- a/data/devlog/bun-websockets-are-missing-pause-and-resume.mdx +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Bun WebSockets Are Missing Pause And Resume -summary: | - Run a Node WebSocket server under Bun and every connection dies on arrival with `TypeError: ws.pause is not a function`. Bun's WebSocket is not a Node stream, so the stream control methods are simply not there. Guard the calls with optional chaining, or run the server on Node where the methods exist. -date: '2026-10-16 10:28:00 +1000' -tags: - - Bun - - WebSocket - - Runtime -draft: false -icon: Bug ---- - -Start a Node WebSocket server under Bun and every incoming connection is rejected the moment it arrives: - -``` -TypeError: ws.pause is not a function -``` - -The stack points at the socket attach path, wherever it calls `pause()` and `resume()` on the WebSocket to hold the stream while the auth handshake runs. Under Node those methods exist on the underlying stream and the same code has always been fine. Under Bun they are not implemented at all - the gap is tracked upstream at [oven-sh/bun#44453](https://github.com/oven-sh/bun/issues/44453). The report that surfaced it is [paseo#7055](https://github.com/getpaseo/paseo/issues/7055), and the guard is proposed in [paseo#5915](https://github.com/getpaseo/paseo/pull/5915). - -## Why it happens - -Bun's WebSocket is not a Node stream. It implements enough of the surface to send and receive, but `pause()`, `resume()` and `isPaused` are absent, so a bare call throws and takes the connection with it. Any server that reaches for stream control on a WebSocket is relying on a runtime detail that not every runtime honours. - -## The fix - -Guard the calls so the code tolerates either shape: - -```js -ws.pause?.() -ws.resume?.() -``` - -That keeps the buffering and resume behaviour under Node and degrades to a no-op under Bun rather than throwing. If you need real pause semantics on Bun you would have to buffer frames yourself - the optional chaining only buys compatibility, not stream control. - -If you would rather not carry the workaround, run the server on Node. The project's documented install path uses Node for exactly this reason, and Bun is not a supported runtime for the daemon. The general lesson holds whichever way you go - feature-detect stream control before you call it, because the methods are not part of the WebSocket contract everywhere. diff --git a/data/devlog/case-insensitive-disks-collapse-path-aliases.mdx b/data/devlog/case-insensitive-disks-collapse-path-aliases.mdx deleted file mode 100644 index e285cbc..0000000 --- a/data/devlog/case-insensitive-disks-collapse-path-aliases.mdx +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Case-insensitive Disks Collapse Path Aliases -summary: | - A repair step that compares two paths as strings can refuse to run with "requires distinct source and destination databases" even though the two paths look different, because on a case-insensitive filesystem `BA-EA` and `ba-ea` are the same physical file. Comparing filesystem identity instead of path strings is the fix. -date: '2026-10-17 09:10:00 +1000' -tags: - - OpenClaw - - SQLite - - macOS - - Gotcha -draft: false -icon: Bug ---- - -A migration refuses to run and reports that it needs two distinct databases, but the two paths it names are spelled differently and both exist. On macOS the culprit is often a single-letter case difference - an agent directory created as `BA-EA` while the configured identifier is `ba-ea`. This is [openclaw/openclaw#174089](https://github.com/openclaw/openclaw/issues/174089), fixed in [PR #174243](https://github.com/openclaw/openclaw/pull/174243). - -## Why it happens - -macOS and Windows default to case-insensitive filesystems. Two path strings that differ only in letter case resolve to the same inode, so `agents/BA-EA/state.sqlite` and `agents/ba-ea/state.sqlite` are one physical file. A guard that decides whether a source and a destination are distinct by comparing the raw path strings sees two different strings and concludes they are separate. The copy routine underneath, which correctly recognises the physical identity, then refuses because source and destination are the same database. - -The result is a repair that stops halfway - the higher-level step blocks every later migration, and nothing self-heals because both spellings keep resolving to the same file. - -## The fix - -Compare filesystem identity rather than path strings when deciding whether two stores are the same. Where you already have a physical-identity matcher, route selection, grouping and cleanup through it. If a copy genuinely targets the same physical database, treat it as a no-op instead of an error. - -The concrete failure is the string-based branch: - -```ts -// fragile - two spellings of one file look distinct -if (sourcePath === destinationPath) throw new Error( - 'copy requires distinct source and destination databases' -) -``` - -Recognise the alias before the check runs: - -```ts -const sameFile = fs.realpathSync(sourcePath) === fs.realpathSync(destinationPath) -if (sameFile) return // no-op - one physical database -``` - -The wider lesson is worth keeping for any tool that keys state on identifiers derived from names. If a name can differ in case but the state lives on a case-insensitive disk, canonicalise the key before it becomes a path. Two IDs that are distinct in the data model can be one file on disk, and a guard written against the data model will keep tripping over the disk. diff --git a/data/devlog/claude-app-skills-leak-into-other-providers.mdx b/data/devlog/claude-app-skills-leak-into-other-providers.mdx deleted file mode 100644 index 66773c8..0000000 --- a/data/devlog/claude-app-skills-leak-into-other-providers.mdx +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Claude App Skills Leak Into Other Providers -summary: | - Skills written for one agent surface turn up in threads on another, where none of the tool names they describe exist. The model spends its opening turns ruling out a product it is not running and then invents tool names that blend both vocabularies. Deny those skills per provider in `opencode.json` so each harness only sees skills whose tools it actually has. -date: '2026-10-16 10:05:00 +1000' -tags: - - Skills - - Agents - - OpenCode -draft: false -icon: Workflow ---- - -An agent thread on a non-Claude provider loads a skill written for the Claude desktop app. The skill names tools like `mcp__Claude_Browser__*` and a `ToolSearch` loading step that do not exist in that thread's catalog. Two things go wrong from there. The agent burns the start of the task searching for tools that were never going to be there, and when it does reach for the local equivalent it invents a hybrid - tool names stitched together from both products' vocabularies - and calls none of them. On a UI task it can also take the skill's wrong-nature guidance at face value and tell you the built-in browser is offline instead of retrying. - -Filed as [t3code#19978](https://github.com/pingdotgg/t3code/issues/19978). - -## Why it happens - -Skill discovery is global. OpenCode registers `~/.claude/skills` as a skill source and scans it for every provider, so a skill meant for one harness is offered to all of them. Nothing checks whether the tool surface a skill references exists in the thread that loaded it. The symptom looks like model confusion but it is a discovery scoping problem. - -## What to do - -Deny the offending skills per provider so they never reach threads where their tools are missing: - -```json -{ - "permission": { - "skill": { - "built-in-browser": "deny", - "chrome-browser": "deny", - "computer-use": "deny" - } - } -} -``` - -That is in `opencode.json`. Swap the names for whatever skills reference a tool surface your harness does not have. - -The broader fix is scoping skill injection to the provider - a skill whose referenced tools are absent should not be advertised at all. That is an upstream design question rather than something you can configure away today, so the deny list is the lever you have. If you write skills yourself, this is the argument for naming the tools a skill needs up front, so a host can filter on them instead of guessing. diff --git a/data/devlog/cleanup-timestamps-advance-before-the-work.mdx b/data/devlog/cleanup-timestamps-advance-before-the-work.mdx deleted file mode 100644 index 4e91360..0000000 --- a/data/devlog/cleanup-timestamps-advance-before-the-work.mdx +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Cleanup Timestamps Advance Before The Work -summary: | - A maintenance pass that stamps its completion time before it starts deleting lets one interrupted pass suppress every retry in the window. Short-lived commands exit mid-scan, the hour is marked done, and the cache grows to fifty times its documented cap. Stamp after the pass completes and let the cleanup outlive the process that started it. -date: '2026-10-16 13:10:00 +1000' -tags: - - Node.js - - Caching - - Gotcha -draft: false -icon: Bug ---- - -A CLI's compile cache directory grows to about 26 GB on a machine that runs short commands all day, roughly fifty times the 512 MiB cap its own maintenance code claims to enforce. The directory's modification time keeps advancing about once an hour, so the trim looks like it is running. It never finishes. This is [openclaw/openclaw#173568](https://github.com/openclaw/openclaw/issues/173568), fixed in [PR #173603](https://github.com/openclaw/openclaw/pull/173603). - -## Why it happens - -The pass is meant to run at most once an hour, and it records that fact by advancing a timestamp on the namespace directory. The problem is the order. The worker advanced the timestamp first, then started the directory walk and the sort that eventually reach a deletion. A short-lived command exits on its own schedule. When it exits after the stamp but before any file is removed, the pass is recorded as done and every invocation in the next hour skips cleanup entirely. - -A workload of quick commands - a scheduler firing a CLI every few minutes, release prewarming, a build hook - produces exactly this. The backlog compounds quietly because the timestamp keeps moving. - -## The fix - -Stamp the window only after a completed pass, and let the cleanup outlive the process that started it. The landed fix runs the work detached from the command that triggered it, streams and prunes incrementally, and serialises passes behind a filesystem lock so concurrent writers cannot interleave. It trims toward 512 MiB and 4,096 files with a seven-day expiry. - -Two things to carry beyond this bug. Do not read a maintenance timestamp as evidence that anything was trimmed - check the size. And treat a documented size cap enforced by in-process housekeeping as a target rather than a guarantee. A large backlog needs several bounded passes, and concurrent writers can push past the target in between. diff --git a/data/devlog/cmd-shims-warn-dep0190-with-args-and-shell.mdx b/data/devlog/cmd-shims-warn-dep0190-with-args-and-shell.mdx deleted file mode 100644 index 563dd61..0000000 --- a/data/devlog/cmd-shims-warn-dep0190-with-args-and-shell.mdx +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Windows Cmd Shims Warn DEP0190 When Args Meet Shell True -summary: | - Every spawn of a .cmd or .bat shim on Windows prints Node's DEP0190 deprecation warning if you pass an args array alongside shell: true. The warning is about a future behaviour change, not a bug in your code, but it pollutes every log line. Join the args into a single command string and pass an empty array instead. -date: '2026-10-15 09:30:00 +1000' -tags: - - Node.js - - Windows - - Gotcha -draft: false -icon: Bug ---- - -A Node CLI that shells out to npm-installed tools on Windows starts printing this on every single spawn once the runtime is recent enough: - -``` -[DEP0190] DeprecationWarning: Passing args to a child process with shell option true -can lead to security vulnerabilities, as the arguments are not escaped. -``` - -It shows up whether the child succeeds or fails, and it is not a problem with the child. It is Node warning that the calling convention you are using will change ([pingdotgg/t3code#19385](https://github.com/pingdotgg/t3code/pull/19385)). - -## Why it happens - -When `shell: true`, Node joins `[file, ...args]` with spaces and hands the result to `cmd.exe /d /s /c ""`. The args array is the problem: Node is telling you that this implicit joining is not escaped properly and will eventually stop working. The warning fires whenever both `shell: true` and a non-empty `args` array are present: - -```js -// warns on every call -spawn('npm.cmd', ['install'], { shell: true }) -``` - -## The fix - -Escape the path and each argument yourself, join them into one command string, and pass an empty args array. Node only warns when args are present, so the warning stops and the command line is byte-identical to what Node would have built: - -```js -const line = [file, ...args].map(escapeForCmd).join(' ') -spawn(line, [], { shell: true }) -``` - -Use the same escaping rules as [cross-spawn](https://github.com/moxystudio/node-cross-spawn) for spaces, quotes, `%VAR%`, carets, pipes, redirects, and trailing backslashes. On Node 24, `spawn(line, [], { shell: true })` prints no DEP0190 while `spawn(file, args, { shell: true })` does. - -One gotcha if you switch to spawning `cmd.exe /c` directly instead: you need `windowsVerbatimArguments` threaded through every caller, otherwise Node re-quotes the `/c` line and breaks paths with spaces. diff --git a/data/devlog/cold-coresimulator-hangs-on-fresh-runners.mdx b/data/devlog/cold-coresimulator-hangs-on-fresh-runners.mdx deleted file mode 100644 index 4ecc0ef..0000000 --- a/data/devlog/cold-coresimulator-hangs-on-fresh-runners.mdx +++ /dev/null @@ -1,69 +0,0 @@ ---- -title: Cold CoreSimulator Hangs on Fresh macOS Runners -summary: | - On a fresh GitHub Actions macOS runner, the first iOS simulator boot can hang indefinitely with no output. The job does not fail, it just stops making progress until timeout-minutes kills it. Shutting down and erasing the prewarmed simulators before the first boot is the fix, and bootstatus instead of sleep is what turns an invisible hang into a readable failure. -date: '2026-06-14 11:45:00 +1000' -tags: - - CI/CD - - iOS - - Testing -draft: false -icon: Bug ---- - -The symptom is unhelpful: your test job goes quiet after the build step, produces no further log lines, and eventually dies on `timeout-minutes`. No CoreSimulator error, no crash, no kernel log. It reads like the runner ran out of memory, which is why people burn hours on memory limits before looking anywhere useful. - -## What is actually happening - -GitHub's macOS images ship with CoreSimulator already warm. The image build boots devices, the runner starts, and the simulator service has been holding that state for however long the image has been sitting in the blob store. Sometimes the first boot on a fresh runner picks up a half-torn-down device and the `CoreSimulatorService` never finishes bringing it up. - -You cannot see this from inside the job, but you can see it from the device list: - -```bash -xcrun simctl list devices -``` - -If you see devices listed with `(unavailable)` next to a runtime that is not installed, that is your hung boot. The runner is trying to boot a device that no longer maps to a runtime it has. - -## The fix - -Shut down and erase everything before the first boot. It costs about ten seconds and it removes the whole class of problem: - -```yaml -- name: Reset simulators - run: | - xcrun simctl shutdown all || true - xcrun simctl erase all - -- name: Test - run: | - set -o pipefail - xcodebuild test \ - -scheme MyApp \ - -destination 'platform=iOS Simulator,name=iPhone 15,OS=latest' \ - -derivedDataPath "$RUNNER_TEMP/dd" \ - -parallel-testing-enabled NO \ - | xcpretty -``` - -Four things in there matter, and each one is a separate failure mode: - -- **Shutdown and erase before the first boot.** The actual fix. `|| true` because a cold runner may have nothing running and `simctl shutdown all` returns non-zero when there is no service to talk to. -- **`bootstatus`, never `sleep`.** `xcodebuild` does wait for the boot, but it waits quietly. If you need to boot explicitly, `xcrun simctl bootstatus -b` prints a line per state transition and fails loudly on timeout. A `sleep 30` in place of this is a coin flip that is green on fast runners and red on loaded ones. -- **`-parallel-testing-enabled NO`.** Parallel testing clones the destination device per worker. On a cold CoreSimulator that is several simultaneous boots against the same service, and it is the second most common cause of a silent hang. -- **Explicit `-derivedDataPath`.** The default is under the runner's home directory, which persists across jobs on a self-hosted runner and picks up stale build state. A path under `$RUNNER_TEMP` is fresh by definition. - -## The belt-and-braces bit - -`set -o pipefail` is not optional when you pipe `xcodebuild` into anything. Without it the pipeline's exit code is the last command's, so a formatter that succeeds will happily report a green build that never compiled. - -And always set `timeout-minutes` on the job. Not as a safety net for a hang you have not diagnosed, but because the failure mode of this bug is _time_, and an explicit ceiling converts an hour of wall clock into a clean red build you can read. - -## When it is not this - -Two other things produce a similar quiet stop: - -- **A test that waits on a real network or a real clock.** Common in UI suites with animations, and it looks identical in the log. Add `-only-testing:` to isolate a single test class and see whether the hang follows it. -- **Instruments and `xctrace`.** Much heavier than `xcodebuild test`, and it wants a booted destination beforehand. Boot it with `bootstatus -b` first, then hand the booted UDID to the test step. - -If shutdown and erase did not fix it and the hang survives `-only-testing:MyAppTests/testFoo`, it is your test, not the runner. diff --git a/data/devlog/container-restarts-take-full-repair-captures.mdx b/data/devlog/container-restarts-take-full-repair-captures.mdx deleted file mode 100644 index 95a7ab8..0000000 --- a/data/devlog/container-restarts-take-full-repair-captures.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Container Restarts Each Take a Full Repair Capture -summary: | - A plain stop and start of the OpenClaw Docker image spends minutes in its doctor pass and leaves another multi-gigabyte pre-repair copy of the whole state on disk, retained for 30 days, even when there is nothing to repair. The repair pass now keeps one capture per container and skips the repeat copy on later starts. -date: '2026-10-16 19:52:00 +1000' -tags: - - Docker - - State - - Gotcha -draft: false -icon: RefreshCw ---- - -Restart the container on the same image and the Gateway takes about ten minutes to become healthy, roughly seven and a half of them spent writing a copy of the state. That copy lands under the state directory's `update-captures/` and is kept for 30 days. On a large install the report behind [openclaw/openclaw#173564](https://github.com/openclaw/openclaw/issues/173564) measured about 11 GB per restart, so a host that restarts daily accumulates hundreds of gigabytes of captures it will never read. - -## Why it happens - -The image entrypoint runs `doctor --fix` before every Gateway start, which is deliberate - it is what repairs a torn state after an update. But standalone doctor takes a full pre-repair capture of every local SQLite store before it changes anything, every time it runs, even when the repair pass finds nothing to do. A restart is not an update, so the same stores get copied again on every boot and each copy ages out independently. - -## The fix - -[PR #173711](https://github.com/openclaw/openclaw/pull/173711) makes the first successful capture in a container the only full copy. Later starts still run the repair pass but skip the repeated full copy and keep the container's existing capture; a failed capture retries on the next start. Repair-specific backups such as schema migrations and rewritten databases are untouched, and manual `doctor --fix` and update-time doctor still capture as before. - -Until that reaches your image, prune `update-captures/` on a schedule the same way you would any other diagnostic dump, and treat slow container startup as capture time rather than repair time. The lesson generalises to any repair or migration tool wired into a boot path - if it snapshots state before it decides whether anything needs fixing, every restart becomes a backup whether or not there is anything to back up. Gate the capture on the repair being necessary, not on the tool being invoked. diff --git a/data/devlog/container-updates-drop-global-git-config.mdx b/data/devlog/container-updates-drop-global-git-config.mdx deleted file mode 100644 index 5372ac2..0000000 --- a/data/devlog/container-updates-drop-global-git-config.mdx +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Container Updates Drop Global Git Config -summary: | - An unattended push from a containerised job fails with "could not read Username" after an image update even though the token is still valid, because the credential helper lives in a global gitconfig on a path outside the persistent volume. Bind the helper on a persistent path and make the push step's failure visible. -date: '2026-10-16 01:35:00 +1000' -tags: - - Git - - Docker - - Gotcha -draft: false -icon: Bug ---- - -A scheduled job that commits and pushes over HTTPS runs fine for a while, then fails after the container image updates with - -```text -could not read Username for 'https://github.com': terminal prompts disabled -``` - -The token is still there and still valid. `gh auth status` reports it happily. Only the binding between git and the token has gone. This is [openclaw/openclaw#173059](https://github.com/openclaw/openclaw/issues/173059). - -## Why it happens - -`gh auth setup-git` writes the credential helper into the global `$HOME/.gitconfig`. In a container deployment only some paths are persistent volumes, and a global gitconfig usually is not one of them, so it lives in the ephemeral image layer and vanishes on every image update. Nothing warns you. The credential store itself may well be on a persistent path - the token survives, the helper that points git at it does not. - -The nastier half is the failure report. The local commit succeeds, so the job looks like it completed, while the remote falls behind a little more every run. A job whose whole purpose is off-host backup can report success for days. - -## The fix - -Put the binding on a persistent path instead of the global one. Either point the job at a repository-local helper inside the persistent volume, or persist the home directory itself where the deployment offers that option. For the OpenClaw container, the documented home volume does it: - -```text -OPENCLAW_HOME_VOLUME=openclaw_home -``` - -Whatever you choose, do not let the push step depend on ambient global git config that the image can drop. And make the failure visible - verify the remote ref after pushing rather than trusting a zero exit from `git commit`. A commit that never leaves the box is not a backup. diff --git a/data/devlog/diagnostic-paths-must-resolve-secrets.mdx b/data/devlog/diagnostic-paths-must-resolve-secrets.mdx deleted file mode 100644 index 8641532..0000000 --- a/data/devlog/diagnostic-paths-must-resolve-secrets.mdx +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Diagnostic Paths Must Resolve Secrets Like the Runtime -summary: | - A health check that reports "Invalid API key" while the running service authenticates fine is usually a code path that skips secret resolution. The runtime resolves SecretRefs before handing credentials to a provider; the diagnostic and status commands often do not, so they pass the raw reference and get a misleading failure. -date: '2026-10-15 09:35:00 +1000' -tags: - - Secrets - - Diagnostics - - Gotcha -draft: false -icon: Bug ---- - -A third-party memory plugin is configured with an exec SecretRef for its API token. The Gateway resolves the reference, the provider authenticates, recall and retain both work. Then you run the status command and see `Authentication failed: Invalid API key` - yet the command exits 0 and reports the provider as `ready` ([openclaw#172098](https://github.com/openclaw/openclaw/issues/172098)). - -The credential is fine. The diagnostic path is not resolving it. - -## Why it happens - -The runtime and the diagnostics take different routes to the same provider config. The runtime runs the full secret-resolution pass before handing the config to the provider. The status command and the doctor probe take a shortcut - a foreign-provider branch that returns early, or a raw-config handoff - and skip the resolution entirely. The provider receives the unresolved reference string as if it were the literal API key. - -Nothing fails loudly when this happens. The plugin's lazy init swallows the auth error - it logs the message and carries on - so the doctor probe reports no findings and the status command still exits 0. - -## The fix - -Every code path that loads a provider's config must run the same secret-resolution pass as the runtime. If the secret cannot be resolved, report the provider as unavailable rather than probing with the raw reference: - -```ts -// wrong: hands the raw SecretRef to the provider -const config = await loadProviderConfig(slot) - -// right: resolve declared secret targets first -const config = await resolveSecretTargets( - await loadProviderConfig(slot), - manifest.secretTargets -) -if (!config.apiToken) return { status: 'unavailable' } -``` - -The pattern generalises beyond secrets. If the runtime validates, resolves, or transforms a value before use, the diagnostic path that exercises the same component must do the same - otherwise you are testing a different code path and the failure you see belongs to the harness, not the thing being tested. diff --git a/data/devlog/discord-voice-state-overwrites-connection-state.mdx b/data/devlog/discord-voice-state-overwrites-connection-state.mdx deleted file mode 100644 index 119ad26..0000000 --- a/data/devlog/discord-voice-state-overwrites-connection-state.mdx +++ /dev/null @@ -1,34 +0,0 @@ ---- -title: Another Member's Voice State Overwrites Your Discord Connection -summary: | - A Discord voice bot that forwards every voice state update to its voice adapter lets any member's mute, deaf or channel change overwrite the bot's own connection config. Joins time out, the bot comes back self-muted, or it lands in the wrong channel. Forward only the packets whose user_id is the bot's own. -date: '2026-10-15 12:06:00 +1000' -tags: - - Discord - - Voice - - Gotcha -draft: false -icon: Bug ---- - -A Discord voice bot joins fine when the channel is quiet, then breaks as soon as the guild gets busy. Joins abort after the connect timeout, the bot comes back self-muted, or it quietly moves to someone else's channel. Nothing is wrong with the bot's own voice state. It is being overwritten by everyone else's. This is [openclaw/openclaw#172648](https://github.com/openclaw/openclaw/issues/172648), fixed in [PR #172660](https://github.com/openclaw/openclaw/pull/172660). - -## Why it happens - -The Discord gateway sends a `VOICE_STATE_UPDATE` for every member of the guild - joins, leaves, moves, mutes, deafens. If all of those reach the voice adapter, the adapter contract breaks. `@discordjs/voice` expects only the client's own voice state. Its connection stores the latest packet as the bot's own, so a packet about another member becomes the bot's session identity, its mute and deaf flags, and its channel. - -One member toggling mute is enough to make the next join fail. The failure looks like a networking or timeout problem, because the symptom is a join that never completes. - -## The fix - -Filter before forwarding. Record the bot's user id from the gateway `READY` event and pass on only that member's voice state. Everything that watches other people - rosters, occupancy, follow-user behaviour - keeps receiving the rest. - -```js -if (event.t === 'VOICE_STATE_UPDATE' && event.d.user_id === botUserId) { - adapter?.onVoiceStateUpdate?.(event.d) -} -``` - -Voice server updates carry no user id and keep routing by guild, so they are unaffected. Participant rosters and occupancy tracking still see every event. They just no longer reach the connection adapter. - -If a Discord voice bot misbehaves only when the channel is busy, look at whose voice state is reaching the connection layer before you blame the network. diff --git a/data/devlog/expo-fingerprint-drops-native-modules-without-sourcedir.mdx b/data/devlog/expo-fingerprint-drops-native-modules-without-sourcedir.mdx deleted file mode 100644 index 980031f..0000000 --- a/data/devlog/expo-fingerprint-drops-native-modules-without-sourcedir.mdx +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Expo Fingerprints Drop Native Modules Without sourceDir -summary: | - An Android fingerprint can hash zero Expo native modules because one Maven-resolved gradle plugin has no sourceDir and the collector's catch returns an empty list, so native changes stop moving the runtime version and fingerprint-policy OTA updates crash store builds that lack them. Skip sourceDir-less plugins when hashing, and refuse to publish an OTA from a fingerprint that covers no native modules. -date: '2026-10-17 09:50:00 +1000' -tags: - - Expo - - Fingerprint - - Gotcha -draft: false -icon: Bug ---- - -An OTA update crashes the store build on launch with a missing native view. The patch that added the view shipped normally, but the Android runtime version never moved, so the update was published as compatible with a binary that predates the native code. iOS behaves. Reproduced, fixed and gated in [pingdotgg/t3code#20384](https://github.com/pingdotgg/t3code/pull/20384). - -## Why it happens - -`@expo/fingerprint` collects Android autolinking sources in `getExpoAutolinkingAndroidSourcesAsync`. A module can declare a gradle plugin resolved from Maven by id and version, with no local `sourceDir` - `expo-widgets` ships one. The collector passes `plugin.sourceDir` to `path.relative()`, which throws on `undefined`, and the surrounding `catch` returns `[]`. One sourceless plugin therefore drops every Expo Android module from the hash, not just itself. In the reported app the fingerprint covered 0 `expoAutolinkingAndroid` sources where the fixed build covers 56. - -From there the failure is silent. Any native edit to any Expo module stops changing the Android fingerprint, which is exactly the wrong signal under a fingerprint-based OTA policy - the runtime looks unchanged, so the update is routed to binaries that predate the native code. The bug is in `@expo/fingerprint` 0.21.5 and current canaries. - -## The fix - -Skip gradle plugins without a `sourceDir` when collecting hash sources. They have no local files to hash, and their id and version are still fingerprinted: - -```js -if (typeof plugin.sourceDir !== 'string') { - continue -} -``` - -Carry it as a patch until it lands in a released `@expo/fingerprint` - the same skip was proposed upstream in [expo/expo#51710](https://github.com/expo/expo/pull/51710). - -The gate matters more than the patch. Refuse to publish an OTA from a fingerprint that hashes no Expo native modules. The sources carry a reason tag you can count: - -```bash -expo_modules="$(eas fingerprint:generate --platform android --environment production --json \ - | sed -n '/^{/,$p' \ - | jq '[.sources[] | select((.reasons // []) | index("expoAutolinkingAndroid"))] | length')" -[ "${expo_modules:-0}" -eq 0 ] && exit 1 -``` - -The general shape is worth keeping. A hashing pass that catches and continues can quietly cover nothing, and every "nothing changed" result downstream of it is then a lie. Verify what a fingerprint actually covered before trusting it to route updates. diff --git a/data/devlog/free-plan-sign-in-loops-on-plan-required.mdx b/data/devlog/free-plan-sign-in-loops-on-plan-required.mdx deleted file mode 100644 index fe9a8a5..0000000 --- a/data/devlog/free-plan-sign-in-loops-on-plan-required.mdx +++ /dev/null @@ -1,28 +0,0 @@ ---- -title: Free Plan Sign-in Loops on Plan Required -summary: | - Signing in to Cursor with a free account finishes in the browser and then reports "Not authenticated" forever, because a 403 plan_required rejection is treated as a generic catalog failure and the fresh credential gets cleared. Use a paid account or set CURSOR_API_KEY from one. -date: '2026-10-15 15:56:00 +1000' -tags: - - Cursor - - Authentication - - Gotcha -draft: false -icon: Bug ---- - -Signing in to Cursor with a free (Hobby) account completes in the browser, and the client then reports it is not authenticated. Retry sign-in runs the whole browser flow again and lands in the same place. Swap to a paid account and the identical flow works first go. This is [pingdotgg/t3code#19766](https://github.com/pingdotgg/t3code/issues/19766). - -## Why it happens - -Cursor's local SDK requires a paid plan - [confirmed on Cursor's own forum](https://forum.cursor.com/t/agent-create-local-apikey-model-fails-for-free-users-because-local-model-validation-calls-cloud-v1-models/160839). A free-plan key is answered with `403 plan_required`, but the client's catalog check only treats `401` as an authentication failure. The plan rejection surfaces as a generic catalog error, the verification step discards the credential that was just created, and the account view shows a fixed "sign-in failed or expired" message that never mentions the plan. The user retries a flow that cannot succeed. - -## What to do - -Sign in with a paid account, or set `CURSOR_API_KEY` from one: - -```bash -export CURSOR_API_KEY= -``` - -The wider lesson is worth keeping. When a sign-in reports failure immediately after a successful browser round trip, look at the status code before blaming the token. A `403` that is not `401` is usually the product saying no - a plan, a licence, an enterprise policy - and any client that folds every rejection into "your session expired" will send you chasing credentials that were never the problem. An enterprise account whose identity lookup returns `403` fails through the same path, so the same check applies there. The fix upstream is to surface the plan rejection instead of the generic message ([pingdotgg/t3code#15075](https://github.com/pingdotgg/t3code/issues/15075)). diff --git a/data/devlog/fresh-workers-rerun-full-sqlite-integrity-checks.mdx b/data/devlog/fresh-workers-rerun-full-sqlite-integrity-checks.mdx deleted file mode 100644 index bb0a229..0000000 --- a/data/devlog/fresh-workers-rerun-full-sqlite-integrity-checks.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: Fresh Workers Re-Run Full SQLite Integrity Checks -summary: | - Every write to a large SQLite database can cost a fixed couple of seconds because each newly spawned worker thread runs a full `PRAGMA integrity_check` when it opens the file, and workers that idle out are replaced constantly. The verification result lives in one thread's async context, so the next worker never learns the database was already proven good. Cache the proof against the file identity, or drop the per-worker gate to `quick_check`. -date: '2026-10-16 09:40:00 +1000' -tags: - - SQLite - - Performance - - Workers -draft: false -icon: Bug ---- - -A steady-state write to a large SQLite file costs a fixed couple of seconds, on a database that is not corrupt and is not being migrated. The slow-transaction log names a different thread every time, and a raw write to the same file with the sqlite3 CLI is instant. `PRAGMA quick_check` is fast enough to miss, but a full `PRAGMA integrity_check` accounts for the whole delay. - -This came up in [openclaw#173440](https://github.com/openclaw/openclaw/issues/173440), where a 250 MB state database paid roughly 2 s on almost every write. - -## Why it happens - -A short-lived worker thread opens the database and runs a full integrity check as part of its admission gate. The flag that says this file was already verified is scoped to one thread's async context, so a brand new worker thread never sees it and re-proves the file from scratch. When workers retire after a short idle timeout, a fresh worker is spawned for nearly every write, and the check is the write. - -The same shape bites any pool where a verification result is held in thread-local or request-local state instead of against the file itself. - -## The fix - -Key the proof on the file identity rather than on the thread or the async context - device and inode plus a schema or data generation - and reuse it across workers and reopens for as long as that identity holds. Where you cannot share the proof, drop the per-worker gate to `quick_check` and keep the full check for the one place that actually opens the file cold. - -```js -// Proof belongs to the file, not to the worker that opened it -const proven = new Map() // file identity -> integrity result - -function openWorker(path) { - const key = fileIdentity(path) // dev/ino + schema generation - if (!proven.has(key)) { - proven.set(key, assertIntegrity(path, 'quick_check')) - } - return open(path) -} -``` - -Upstream fixed it by transporting the schema and integrity facts into each new worker and publishing a reusable seal on clean close, so a later worker restores the proof instead of recomputing it. If you are on an older build, the `quick_check` swap alone takes the per-spawn cost from seconds to fractions of a second. diff --git a/data/devlog/git-pathspecs-resolve-from-cwd-not-repo-root.mdx b/data/devlog/git-pathspecs-resolve-from-cwd-not-repo-root.mdx deleted file mode 100644 index 63adf7e..0000000 --- a/data/devlog/git-pathspecs-resolve-from-cwd-not-repo-root.mdx +++ /dev/null @@ -1,29 +0,0 @@ ---- -title: Git Pathspecs Resolve From the Working Directory -summary: | - A diff call against a workspace opened at a repository subdirectory returns every changed file with its counts but empty hunks, because the per-file patch is requested with a repo-root-relative path while git runs from the subdirectory. Pathspecs resolve against the process working directory, not the repo root. -date: '2026-10-15 15:48:00 +1000' -tags: - - Git - - Tooling - - Gotcha -draft: false -icon: Workflow ---- - -A checkout diff against a workspace opened at a repository subdirectory lists every changed file with correct addition and deletion counts, but each file comes back with empty hunks. The same call from the repository root returns the patch, and so does the live changes subscription for the identical workspace. This is [getpaseo/paseo#6867](https://github.com/getpaseo/paseo/issues/6867), fixed in [PR #6882](https://github.com/getpaseo/paseo/pull/6882). - -## Why it happens - -The file list is produced with paths relative to the repository root. The per-file patch is then requested with one of those paths as a pathspec, while git runs from the workspace subdirectory. Pathspecs resolve against the process working directory, not the repository root, so the spec matches nothing and git prints nothing. No error - just an empty patch. - -```text -cd repo/sub && git diff HEAD -- ':(literal)sub/n.txt' # empty output -cd repo && git diff HEAD -- ':(literal)sub/n.txt' # the patch -``` - -## The fix - -Run git from the repository root and pass the paths as they come back, or rewrite each path relative to the current directory before calling. Anchoring the spec at the repository root works too. The fix here was to read diffs from the worktree root so callers keep passing their workspace directory and still get repository-relative paths out. - -The same mismatch bit the discard and commit-diff paths in the same project before it ([getpaseo/paseo#6769](https://github.com/getpaseo/paseo/issues/6769), [getpaseo/paseo#6799](https://github.com/getpaseo/paseo/issues/6799)). If your tooling shells out to git from a workspace subdirectory, audit every place a stored path meets a pathspec. The failure mode is silent, and it only shows up for people who do not open the repo root. diff --git a/data/devlog/git-refs-cannot-nest-under-existing-branches.mdx b/data/devlog/git-refs-cannot-nest-under-existing-branches.mdx deleted file mode 100644 index 835f1f7..0000000 --- a/data/devlog/git-refs-cannot-nest-under-existing-branches.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: Git Refs Cannot Nest Under Existing Branches -summary: | - Checking out a fork PR whose owner shares a name with a local branch dies on the fetch, because the generated branch name like main/patch-1 nests under an existing ref and Git refuses to create it. Branch names are ref paths - any a blocks a/b - so generated names need collision checks in both directions. -date: '2026-10-15 21:20:00 +1000' -tags: - - Git - - Branching - - Gotcha -draft: false -icon: Workflow ---- - -Checking out a fork pull request fails with `Unable to fetch change request refs for worktree branch main/patch-1` when the fork owner happens to share a name with a local branch - `main`, `master`, `dev`. The pull request itself is fine and the fetch would otherwise work. This is [getpaseo/paseo#6830](https://github.com/getpaseo/paseo/issues/6830), fixed in [PR #6829](https://github.com/getpaseo/paseo/pull/6829). - -## Why it happens - -A branch name is a ref path. `main` is the file `refs/heads/main`, and `main/patch-1` would have to live at `refs/heads/main/patch-1`, inside a directory that cannot exist while the file does. Git says so plainly: - -```text -fatal: cannot lock ref 'refs/heads/main/patch-1': 'refs/heads/main' exists; cannot create 'refs/heads/main/patch-1' -``` - -The collision runs both ways. An existing `bob/patch-1/child` blocks creating `bob/patch-1`, and an existing `bob/patch-1` blocks a later `bob`. The allocator that picked the name checked for exact matches only, so it handed git a name that could never be created, and the fetch failed before any checkout ran. - -## The fix - -[PR #6829](https://github.com/getpaseo/paseo/pull/6829) suffixes the conflicting path segment instead of failing, so `main-1/patch-1` sits alongside `main` and the checkout keeps the fork's identity. - -If you generate branch names as `owner/branch`, check for nesting collisions in both directions before handing the name to git, or use a separator that cannot nest. Exact-match lookups are not enough - the conflict is between a ref and any ref that would live under it. diff --git a/data/devlog/git-worktree-remove-refuses-initialised-submodules.mdx b/data/devlog/git-worktree-remove-refuses-initialised-submodules.mdx deleted file mode 100644 index 4a65086..0000000 --- a/data/devlog/git-worktree-remove-refuses-initialised-submodules.mdx +++ /dev/null @@ -1,31 +0,0 @@ ---- -title: Git Worktree Remove Refuses Initialised Submodules -summary: | - A cleanup that runs `git worktree remove` without `--force` can never delete a worktree whose submodules are initialised, even when the tree and every submodule are clean, and the same failure repeats on every run. Check cleanliness yourself and pass `--force`, because git applies the same hard refusal to a pristine worktree and the safety net never fires there. -date: '2026-10-17 09:55:00 +1000' -tags: - - Git - - Worktrees - - Gotcha -draft: false -icon: Bug ---- - -Every cleanup run reports `working trees containing submodules cannot be moved or removed` for the same worktree. The worktree is clean, its submodules are clean, and manual removal fails with the same message. This is [pingdotgg/t3code#20355](https://github.com/pingdotgg/t3code/issues/20355). - -## Why it happens - -`git worktree remove` without `--force` refuses any worktree with initialised submodules. Not just dirty ones - all of them. git-worktree(1) says it plainly: "Unclean worktrees or ones with submodules can be removed with --force". A cleanup that omits `--force` on purpose, so git still refuses when a tracked edit lands after the last status check, therefore turns every submodule worktree into a permanent failure that repeats on each run. - -## The fix - -Verify the tree yourself, then force it: - -```bash -git -C status --short -git -C worktree remove --force -``` - -If you script worktree cleanup, keep the explicit cleanliness check and pass `--force` after it, extending the check through `git submodule foreach --recursive` first when the tree has submodules - checking status and commits that are not on any remote, the same checks you already do on the superproject. The race guard survives, because the check runs right before the removal. Relying on remove to refuse is a real safety net for ordinary trees and no net at all for submodule trees, where the same refusal applies to a pristine checkout. - -One side effect worth knowing. A run can remove other worktrees fine and fail on only the submodule one, while the failure toast shows just the first reason - the sweep continues past a failure even when the message reads like it stopped. diff --git a/data/devlog/github-token-suppresses-pull-request-ci.mdx b/data/devlog/github-token-suppresses-pull-request-ci.mdx deleted file mode 100644 index 871e03b..0000000 --- a/data/devlog/github-token-suppresses-pull-request-ci.mdx +++ /dev/null @@ -1,76 +0,0 @@ ---- -title: GitHub Actions Silently Skips CI on PRs Opened by GITHUB_TOKEN -summary: | - GitHub does not run pull_request workflows for pull requests that GITHUB_TOKEN opened. Any release tool that opens its own PR (release-please, semantic-release, changesets, Mergify) will therefore never see its own release PR pass CI. Nothing errors, nothing is annotated, the checks just do not exist. Here is the rule, how to prove you have hit it, and the three ways out. -date: '2026-08-30 09:20:00 +1000' -tags: - - CI/CD - - GitHub Actions - - Release Automation -draft: false -icon: Workflow ---- - -GitHub will not trigger a `pull_request` workflow for a pull request that was created using the default `GITHUB_TOKEN`. This is deliberate, it is documented, and it is the single most common reason a release bot's PR sits there with no checks on it. - -## The rule - -Events triggered by `GITHUB_TOKEN` do not start new workflow runs. Not `pull_request`. Not `pull_request_target` either, because that fires on the base branch's workflow file and the _action_ still came from the token. The loop is cut at the source. - -So this: - -```yaml -on: - push: - branches: [main] - pull_request: - branches: [main] -``` - -will not run for the release PR that release-please opened. It runs perfectly well for every human PR, which is exactly what makes it hard to spot. - -## Proving it in ten seconds - -Open the release PR and look at the checks tab. No jobs at all, not even skipped ones. Then compare with a human PR on the same branch. That is the whole diagnosis. - -Two more signals if you want them in the log: - -- `gh pr view --json statusCheckRollup` returns an empty array, while a human PR on the same head returns a populated one. -- No run exists for the release commit. `gh run list --commit ` is empty, but `gh run list --branch ` shows the push-branch run from when the branch was created and nothing since. - -I lost an afternoon to a slightly different face of the same bug. A release-please PR had been sitting unmerged for a week. Not because CI was red. Because CI had never been asked. - -## The three ways out - -**1. Use a token that is not `GITHUB_TOKEN`.** A fine-grained PAT, or better, a GitHub App installation token. Pass it to the action: - -```yaml -- uses: googleapis/release-please-action@v4 - with: - token: ${{ secrets.RELEASE_PLEASE_TOKEN }} -``` - -The PR is now authored by a real identity and `pull_request` fires normally. This is the option to pick if you want full CI on release PRs. It costs you: a PAT with a rotation schedule, or App permissions you have to keep in scope. - -**2. Gate on `workflow_run` instead.** Let CI run on the branch push rather than the PR, and have release-please wait for it: - -```yaml -on: - pull_request: - branches: [main] - -jobs: - release-please: - runs-on: ubuntu-latest - needs: [ci] -``` - -On a bot PR the `ci` job is skipped, so `needs` resolves immediately and release-please proceeds. On a human PR release-please is blocked until CI passes. Correct in the common case, and it is a workaround rather than a fix, because you have quietly made "no CI result" mean "pass". - -**3. Add a `workflow_dispatch` escape hatch.** This is the belt-and-braces option and it is worth having regardless. Release-please actions generally support being pointed at a specific target, so a manual workflow can create the release PR, at which point the token _is_ a PAT and CI behaves. Note this in the repo README, because a release process that only exists in one workflow file is a release process nobody can use under pressure. - -## What I would do - -Fine-grained PAT, scoped to the one repository, with contents, pull requests and issues read/write. Nothing else. Keep the release pipeline on `GITHUB_TOKEN` and the PR-opening action on the PAT. That way the blast radius of the PAT is a single repo and a single job, and the release PR gets real CI on it. - -Whichever route you take, the thing to take away is narrower than "GitHub Actions is unreliable": **an event that is self-referential needs a different identity than the one that triggered it.** Any tool that reacts to your infrastructure by writing to that same infrastructure hits this. diff --git a/data/devlog/gui-launches-drop-shell-ca-trust-exports.mdx b/data/devlog/gui-launches-drop-shell-ca-trust-exports.mdx deleted file mode 100644 index d4f3654..0000000 --- a/data/devlog/gui-launches-drop-shell-ca-trust-exports.mdx +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: GUI Launches Drop Shell CA Trust Exports -summary: | - An app launched from the dock does not inherit the CA trust exports from your shell profile, so every Node child it spawns fails TLS with UNABLE_TO_GET_ISSUER_CERT_LOCALLY while the same command works in a terminal. Import SSL_CERT_FILE, SSL_CERT_DIR and NODE_EXTRA_CA_CERTS from the login shell, or launch the app from a terminal. -date: '2026-10-15 12:10:00 +1000' -tags: - - TLS - - Desktop - - Gotcha -draft: false -icon: Bug ---- - -An updater inside a desktop app fails with "Update command exited with code 1" while the same update works from a terminal. The command it runs exits with - -``` -Error: Could not determine latest pi version: fetch failed (UNABLE_TO_GET_ISSUER_CERT_LOCALLY) -``` - -TLS verification fails inside the app and nowhere else. This is [pingdotgg/t3code#19604](https://github.com/pingdotgg/t3code/pull/19604). - -## Why it happens - -An app launched from the dock or the Start menu does not inherit your shell profile's exports. It sees only the desktop session environment, plus whatever the app chooses to import. The desktop app imports a fixed list of variables from the login shell - tool prefixes, data directories - and CA trust variables were not in the list. - -The command works in a terminal because the shell profile exports `SSL_CERT_FILE`. It fails inside the app when the Node on the PATH has no bundled root store - a shared-OpenSSL build - so it needs those variables to find the issuer at all. The same class of failure hits anything else the app spawns that fetches over HTTPS. - -## The fix - -The app now imports `SSL_CERT_FILE`, `SSL_CERT_DIR` and `NODE_EXTRA_CA_CERTS` from the login shell, filling them in only when its own environment has none: - -```bash -export SSL_CERT_FILE=/path/to/your/ca-bundle.crt -export NODE_EXTRA_CA_CERTS=/path/to/your/ca-bundle.crt -``` - -On versions without the fix, launching the app from a terminal is the quickest workaround, because the profile's exports reach it that way. - -More generally, if something that runs Node children fails TLS only when launched from the desktop, compare `process.env` inside the app against your shell before you go hunting for certificate problems. diff --git a/data/devlog/helm-does-not-install-crds-on-upgrade.mdx b/data/devlog/helm-does-not-install-crds-on-upgrade.mdx deleted file mode 100644 index 2aff24e..0000000 --- a/data/devlog/helm-does-not-install-crds-on-upgrade.mdx +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Helm Does Not Install CRDs on Upgrade -summary: | - CRDs in a chart's crds/ directory are only applied by helm install. helm upgrade skips them entirely, with no warning. If your External Secrets Operator ClusterSecretStore or your cert-manager Issuer disappears, this is why, and the fix is not in your values file because the values file is never read. -date: '2026-07-22 14:10:00 +1000' -tags: - - Kubernetes - - Helm - - External Secrets Operator -draft: false -icon: Package ---- - -Anything in a chart's `crds/` directory is applied on `helm install` and skipped on `helm upgrade`. No warning, no diff, no error. Helm treats CRDs as cluster-scoped infrastructure that it does not own, on the reasonable grounds that it cannot safely diff or roll back a CRD, and that a failed CRD update can brick every CR in the cluster. - -The trap is that the CRDs only need to be missing once. After that your release PRs look clean, your values are correct, and your `ExternalSecret` resources are rejected with `no matches for kind "ExternalSecret" in version "external-secrets.io/v1"`. - -## The failure, in order - -This is what a broken bootstrap looks like: - -1. A fresh cluster. `helm upgrade --install external-secrets external-secrets` runs against a release that Helm believes already exists, or a previous uninstall cleaned up the release record but not the namespace. -2. Helm applies the Deployment and RBAC. It does not apply `externalsecrets.external-secrets.io`, `secretstores.external-secrets.io` or `clustersecretstores.external-secrets.io`. -3. The controller starts, watches for `SecretStore` and `ClusterSecretStore`, finds none, and logs nothing useful because nothing told it to look for anything. -4. Your `ClusterSecretStore` apply fails at the API server, not at the controller. `no matches for kind`. -5. Every workload that mounts a `secretRef` from that store CrashLoopBackOffs with a generic missing-secret error, which sends you looking at the store rather than at the CRDs. - -The failure surfaces three layers away from its cause, which is why this takes a while to spot the first time. - -## The fix, in the order you need it - -Assert the CRDs exist, and create them if they do not. This is idempotent and safe to run every time: - -```bash -for crd in externalsecrets.external-secrets.io \ - secretstores.external-secrets.io \ - clustersecretstores.external-secrets.io; do - kubectl get crd "$crd" >/dev/null 2>&1 || \ - kubectl apply -f "https://raw.githubusercontent.com/external-secrets/external-secrets/main/deploy/crds/bundle.yaml" -done -``` - -Then wait for them to be servable before applying any custom resource: - -```bash -kubectl wait --for=condition=Established --timeout=60s \ - crd/externalsecrets.external-secrets.io \ - crd/secretstores.external-secrets.io \ - crd/clustersecretstores.external-secrets.io -``` - -That `kubectl wait` matters even when the CRDs are already there, because `kubectl apply` returns as soon as the object is stored. Custom resources created in the same script will hit `no matches for kind` against an API server that has not finished its discovery refresh yet. - -## Why the ordering is not optional in Argo CD - -If you sync a bundle containing both the Helm release and the custom resources, Argo CD applies them as one wave. CRDs and the CRs that depend on them land in the same sync, and the CRs lose that race routinely. - -Two options that both work: - -- Put the CRD-manifest and the `helm install` in a **first sync wave**, and everything that references those CRDs in a second wave with a `sync-wave` delay. This is the least surprising option and I would start here. -- Drop the chart's CRDs and manage the CRDs as a pinned manifest in the same repo. More verbose, and it means you own the upgrade path for the CRD schema across chart versions, but it makes the dependency explicit instead of relying on Helm's install-time-only behaviour. - -## Verify it, do not assume it - -```bash -kubectl get externalsecrets,secretstores,clustersecretstores -A -kubectl describe clustersecretstore -n | tail -20 -``` - -If the store exists and the `ExternalSecret` exists but no `Secret` appears, the provider is the problem, not the plumbing. `kubectl describe` on the `ExternalSecret` carries the controller's reconciliation error in its events, and that message is more specific than anything the workload logs will tell you. - -One last trap. External Secrets only retries reconciliation on the `refreshInterval` you configured, so a store that was applied before its provider was ready can sit silently empty for the length of that interval. Set the interval low during bootstrap and raise it once the cluster is steady. A 24 hour default that was fine in dev will cost you an afternoon of staring at a healthy-looking pod. diff --git a/data/devlog/hetzner-provider-release-packaging.mdx b/data/devlog/hetzner-provider-release-packaging.mdx new file mode 100644 index 0000000..007c6c7 --- /dev/null +++ b/data/devlog/hetzner-provider-release-packaging.mdx @@ -0,0 +1,46 @@ +--- +title: Getting the Hetzner Provider Release Into Its Container +summary: | + The Karpenter Hetzner provider release reached Docker packaging and failed looking for go.sum. GoReleaser had staged a binary, but I was giving it a Dockerfile that expected the source tree. +date: '2026-10-09 23:18:38 +1100' +tags: + - Karpenter + - Hetzner + - Release Engineering +draft: false +icon: Workflow +--- + +The Karpenter Hetzner provider release finally got past signing, then stopped at +the first Docker `COPY` with `"/go.sum": not found`.[^fix] + +The file was not missing from the repository. I had two different build paths +colliding. The normal Dockerfile builds from source and expects `go.mod`, +`go.sum` and the code. GoReleaser was handing Docker a temporary context containing +the binary it had already built. + +I added a separate `Dockerfile.goreleaser` for that packaging step: + +```dockerfile +FROM gcr.io/distroless/static-debian12:nonroot +COPY karpenter-provider-hetzner /bin/karpenter-provider-hetzner +USER 65532:65532 +ENTRYPOINT ["/bin/karpenter-provider-hetzner"] +EXPOSE 8080 8081 9443 +``` + +Both GoReleaser Docker entries now use it. The source-building Dockerfile stayed +in place for the local `docker-build` task. + +## Package the thing already built + +The final image kept its distroless base, non-root user, entrypoint and ports. +What changed was the input contract: this path packages a staged executable, +not a checkout. + +That also removed a duplicate Go build inside the image, including the emulated +arm64 compile under QEMU. The PR records a configuration check and a real local +packaging build. I am keeping that separate from claiming every later release +stage was solved by this one change. + +[^fix]: [The provider's separate GoReleaser packaging Dockerfile](https://github.com/echohello-dev/karpenter-provider-hetzner/pull/66). diff --git a/data/devlog/hoist-light-theme.mdx b/data/devlog/hoist-light-theme.mdx new file mode 100644 index 0000000..8d528ba --- /dev/null +++ b/data/devlog/hoist-light-theme.mdx @@ -0,0 +1,38 @@ +--- +title: Giving Hoist a Proper Light Theme +summary: | + Hoist was still carrying dark-mode assumptions into macOS light mode. I added a warm surface palette, replaced hardcoded status colours and let the native glass follow the system appearance. +date: '2026-08-24 21:57:03 +1000' +tags: + - Hoist + - Design + - Electron +draft: false +icon: Bug +--- + +Hoist needed a proper light theme. Switching the system appearance was not enough +while the app still had dark-only status colours and a permanent tint on its native +glass. + +I brought the light side into the same design system as the dark side: warm paper +surfaces, off-white panels, readable text and borders, with shared tokens for +statuses and controls. The two appearances use the same depth ladder rather than +separate component styles.[^change] + +The native window needed attention too. Its liquid-glass treatment now follows +macOS instead of applying a dark tint regardless of the selected appearance. +That was a different layer from the renderer CSS, so fixing only the web colours +would have left the window looking wrong. + +## Checking both appearances + +The change passed the build, typecheck, lint and tests. I also checked Watchtower +in macOS light mode and the dark token path through browser emulation. + +The useful lesson from this pass was how many assumptions sit outside the main +background and text colours. Status badges, borders, controls and the native +window all need to agree. A theme can look nearly finished while one of those +layers still belongs to the other appearance. + +[^change]: [Hoist's light-theme change](https://github.com/echohello-dev/hoist/pull/44). diff --git a/data/devlog/host-revision-invalidates-every-cache-read.mdx b/data/devlog/host-revision-invalidates-every-cache-read.mdx deleted file mode 100644 index 963a4ca..0000000 --- a/data/devlog/host-revision-invalidates-every-cache-read.mdx +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Host-wide Revision Invalidates Every Cache Read -summary: | - Opening a second agent or workspace can sit on its loading screen for a whole turn while an unrelated agent streams, because every stream flush bumps a host-wide revision that invalidates every in-flight cache read and forces it to persist and retry. Scoping invalidation to the rows being read is the fix. -date: '2026-10-17 09:20:00 +1000' -tags: - - Paseo - - Performance - - Caching - - Gotcha -draft: false -icon: Bug ---- - -Open a second agent or workspace on a host where something else is already streaming, and the new tab stays cold until the stream pauses. On slow storage it can sit there for the entire turn. Nothing is wrong with the agent being opened - it is waiting on a cache read that keeps restarting. This is [getpaseo/paseo#4813](https://github.com/getpaseo/paseo/issues/4813), fixed in [PR #7074](https://github.com/getpaseo/paseo/pull/7074). - -## Why it happens - -The replica cache guards a read with a revision check - capture the host revision, read the rows, and return only if the revision has not moved and nothing is pending. The revision is tracked per host, not per row, and every stream flush advances it. A streaming agent flushes its timeline many times a second, so the revision moves constantly while that unrelated agent works. - -Each time the read loop retries it also forces a full persist first, which serialises every pending row and writes them out. The measurements in the report make the shape obvious - a single read of `agent-1` while `agent-2` streams cost 201 store reads and 200 forced writes, because each iteration saw the revision had moved again. With commits arriving faster than persist plus read, the loop effectively never settles. - -## The fix - -Invalidate an in-flight read only when something touches the rows that read asked for - an update or delete to those rows, a replacement of the directory baseline that covers them, or a host lifecycle change. Unrelated commits leave the read alone. - -```ts -// too broad - any commit restarts every read -if (hostRevision !== capturedRevision) retry() - -// scoped - only changes to what this read requested matter -if (requestedRowsChanged || baselineChanged || hostLifecycleChanged) retry() -``` - -Keep the persistence contract intact. Reads should still return persisted rows and fail closed when storage rejects pending writes - the earlier overlay approach fixed the churn but handed back unpersisted data after rejected writes. Row-scoped invalidation cuts the retry storm without weakening durability. - -The generalisable part is the guard. When a cache read is fenced by a revision, decide what that revision covers before wiring it to the read. A single host-level counter turns every unrelated write into a reason to restart every reader. diff --git a/data/devlog/idle-keep-alive-close-races-socket-reuse.mdx b/data/devlog/idle-keep-alive-close-races-socket-reuse.mdx deleted file mode 100644 index 6160ff7..0000000 --- a/data/devlog/idle-keep-alive-close-races-socket-reuse.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: Idle Keep-Alive Close Races Socket Reuse -summary: | - A loopback MCP server answers in time yet tool calls still fail with ECONNRESET and never run, because Node's idle keep-alive timeout closes a pooled socket at the moment the client reuses it. Send Connection: close on every loopback response and the race disappears - non-idempotent tools stop vanishing on the floor. -date: '2026-10-15 21:10:00 +1000' -tags: - - Node - - HTTP - - Gotcha -draft: false -icon: Bug ---- - -Tool calls to a local MCP HTTP server randomly fail with `ECONNRESET: The socket connection was closed unexpectedly`. The call never reaches the handler, so the tool does not run, and the caller cannot tell whether it did. Non-idempotent tools - messages, comments, writes - are simply lost. The rate tracks idle gaps rather than load: in one production count 798 of 100,982 loopback tool calls died this way over three and a half days. This is [openclaw/openclaw#172871](https://github.com/openclaw/openclaw/issues/172871), fixed in [PR #172980](https://github.com/openclaw/openclaw/pull/172980). - -## Why it happens - -`node:http` closes idle keep-alive sockets after `keepAliveTimeout` (5 s by default) plus `keepAliveTimeoutBuffer` (1 s). A client with a connection pool - Bun's fetch among them - reuses a socket from that pool without knowing the server is closing it. When the reuse lands in that window the write dies, and the request is gone before any handler runs. Nothing is wrong with the tool, the payload or the response. A standalone repro with the timeout scaled down rejected 29 of 9,600 pooled fetches with the defaults, and none at all once every response asked for a close. - -## The fix - -[PR #172980](https://github.com/openclaw/openclaw/pull/172980) sets `Connection: close` on every loopback response - successes, notifications and errors alike - so the response end owns the socket shutdown and no client can reuse a socket the server is closing. It removes the race instead of nudging its timeout window. The cost is one new loopback TCP connection per request, and long-running responses and SSE streams still stay open until they end. - -For any small local HTTP server where a request must not be silently lost, answer every response the same way before any branching: - -```js -res.setHeader('Connection', 'close') -``` - -The tradeoff is cheap on loopback and predictable. Leaving keep-alive on instead means tuning `keepAliveTimeout` above the client's pool behaviour, which only moves the race. diff --git a/data/devlog/instructed-silence-gets-a-fabricated-reply.mdx b/data/devlog/instructed-silence-gets-a-fabricated-reply.mdx deleted file mode 100644 index 1856fda..0000000 --- a/data/devlog/instructed-silence-gets-a-fabricated-reply.mdx +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Instructed Silence Gets a Fabricated Reply -summary: | - An automation turn told to end silently that emits no final text is booked as a failed answer, burns a tools-disabled retry, and has a made-up summary reply injected into the session. The harness cannot tell deliberate silence from a dropped answer, so have the automation emit its sentinel token out loud instead of just stopping. -date: '2026-10-17 09:06:00 +1000' -tags: - - OpenClaw - - Automation - - Gotcha -draft: false -icon: Bug ---- - -A bounded automation prompt ends with an instruction like "if nothing needs to be delivered, reply NO_REPLY." The turn runs its tools, everything lands, and the model ends without emitting any final text. The run is then booked as failed anyway - the harness logs that the turn lacked a final answer, runs one isolated retry with tools disabled, and injects a fabricated reply ("The tool run finished, but no final summary was produced.") into the session. Every such turn costs a retry and pollutes the transcript. This is [openclaw/openclaw#174620](https://github.com/openclaw/openclaw/issues/174620). - -## Why it happens - -A settled turn with successful tool calls and no final assistant text is indistinguishable from an answer that got dropped on the floor. The finaliser plays it safe, so deliberate silence gets the same recovery treatment as a real failure. Models make this worse by reading "reply NO_REPLY" as "reply nothing" rather than "type this string," especially after a long tool chain - so the instructed-silence path is the one that reliably produces an empty final answer. In one report, every fallback reply that day was this class of turn, with no provider or tool errors in any of them. - -## The fix - -Have the automation emit the sentinel out loud. The instruction is doing double duty, so phrase it so the literal token is the deliverable - for example "end your turn by typing exactly NO_REPLY and nothing else." When the model emits the token, the turn closes as a silent success and none of the recovery runs. - -Two smaller things help. Keep automation prompts short at the tail - the sentinel instruction after a long tool chain is the case that gets swallowed. And if you see the fabricated summary reply in a transcript, treat it as a signal that a silent turn was misclassified rather than as content, so it does not get carried into later context as if the model wrote it. - -For context, the empty-answer handling is being repaired on the product side - current main accepts an empty completion on optional turns without the retry or the fabricated reply - but a released install still does this, and the prompt fix works everywhere a sentinel is expected. diff --git a/data/devlog/invisible-tcc-dialogs-block-headless-macs.mdx b/data/devlog/invisible-tcc-dialogs-block-headless-macs.mdx deleted file mode 100644 index c1ebc89..0000000 --- a/data/devlog/invisible-tcc-dialogs-block-headless-macs.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Invisible Privacy Dialogs Block Headless Macs -summary: | - A command that touches a protected folder on a Mac nobody is sitting at stays "running" with no output and no error, and Stop does nothing, because macOS shows the privacy prompt to the console user only. Grant Full Disk Access to whatever launches the agent CLI before you run headless. -date: '2026-10-15 15:44:00 +1000' -tags: - - macOS - - Headless - - Gotcha -draft: false -icon: Bug ---- - -An agent run on a Mac driven remotely through a web UI grinds to a halt mid-command. The command item stays `running` with no output. The thread shows no pending request and no error. Messages queue behind it, and an interrupt request has no visible effect for the best part of a minute. Dismiss the dialogs on the Mac itself and everything resumes as if nothing happened. This is [pingdotgg/t3code#19776](https://github.com/pingdotgg/t3code/issues/19776). - -## Why it happens - -The first time a process touches a privacy-protected folder - Downloads, Documents, Desktop - macOS raises a consent dialog. The dialog goes to the console user, on the physical screen. A remote session never sees it. The blocked process just waits in a syscall, so there is no output, no error and no signal a remote client could detect. The automation stack looks wedged when it is actually parked behind a prompt nobody can see. - -## What to do - -Grant Full Disk Access under System Settings, Privacy and Security to the app or terminal that launches the agent CLI, or pre-approve the Downloads, Documents and Desktop folders once while you are at the Mac. Both stop the prompt appearing during a headless session. Full Disk Access is the more future-proof of the two, since it covers protected folders the agent has not reached yet. - -If a remote Mac run ever sits at "running" with zero output, suspect a TCC prompt first. The stuck point is usually a folder the agent touched for the first time, and no amount of retrying from the remote side will clear it. The sluggish interrupt is the same trap - the run cannot act on a stop request while the process is parked in the syscall. diff --git a/data/devlog/ipv6-only-services-fail-behind-ipv4-proxies.mdx b/data/devlog/ipv6-only-services-fail-behind-ipv4-proxies.mdx deleted file mode 100644 index 19db99d..0000000 --- a/data/devlog/ipv6-only-services-fail-behind-ipv4-proxies.mdx +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: IPv6 Only Services Return 502 Through Proxies That Hardcode IPv4 -summary: | - A dev service that binds only to ::1 works on localhost but returns 502 through a reverse proxy or workspace URL, because the proxy forces 127.0.0.1 as the upstream address. Try both loopback literals instead of resolving localhost or hardcoding one family. -date: '2026-10-15 09:40:00 +1000' -tags: - - Networking - - Proxy - - Gotcha -draft: false -icon: Bug ---- - -A workspace service binds to `::1` and answers perfectly on `http://[::1]:3000`. Open it through the generated workspace URL and you get `502 Bad Gateway` from the proxy, with no error from the service itself ([getpaseo/paseo#678](https://github.com/getpaseo/paseo/issues/678)). - -The service is running. The proxy just cannot reach it. - -## Why it happens - -The proxy's forwarding and health-check code hardcodes `127.0.0.1` as the upstream address. When the service listens only on `::1`, that IPv4 loopback literal has nothing to connect to. Resolving `localhost` instead does not reliably help either - DNS may return only `127.0.0.1` even when a working IPv6 loopback listener exists. - -The confusing part is where the error surfaces. The 502 is manufactured by the proxy, so it lands in the proxy log and in the browser, while the service's own log stays completely quiet. That asymmetry is what sends people debugging a perfectly healthy service when the fault is one address literal away. - -## The fix - -Try both loopback literals and let the connection succeed on whichever family the service actually bound to. Bracket only the IPv6 literal - `http://[127.0.0.1]:3000` is not a valid URL and throws before any connection is attempted: - -```js -const targets = ['127.0.0.1', '::1'] -for (const host of targets) { - const authority = host.includes(':') ? `[${host}]` : host - try { - const res = await fetch(`http://${authority}:${port}${path}`) - return res - } catch { - continue - } -} -``` - -Use the same selection logic for health probes so a service that answers on `::1` is not reported as unhealthy. Keep IPv4 as the first candidate to preserve connection pooling behaviour on stacks where both families are available. - -The same trap applies to any local reverse proxy, dev tunnel, or port-forwarding tool. If the upstream address is a literal, the service needs to be listening on that exact literal - dual-stack listeners avoid the problem entirely. diff --git a/data/devlog/locale-aware-key-sorting-breaks-thumbprints.mdx b/data/devlog/locale-aware-key-sorting-breaks-thumbprints.mdx deleted file mode 100644 index 4f3becd..0000000 --- a/data/devlog/locale-aware-key-sorting-breaks-thumbprints.mdx +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: Locale Aware Key Sorting Breaks Thumbprints -summary: | - Sort object keys with localeCompare() and the order follows the host machine's locale, so the same data serialises differently on different machines. When that serialisation feeds an RFC 7638 JWK thumbprint the host computes a different hash than its clients and rejects valid credentials. Sort keys by code unit instead. -date: '2026-10-15 14:10:00 +1000' -tags: - - JavaScript - - Canonical JSON - - Gotcha -draft: false -icon: Bug ---- - -Every remote device is refused with - -``` -Connection failed: The environment credential is invalid. -``` - -while the host itself keeps working. The trigger is the host's OS locale. The report lists Latvian, Lithuanian, Basque and Azerbaijani hosts breaking while English ones work. This is [pingdotgg/t3code#19610](https://github.com/pingdotgg/t3code/issues/19610). - -## Why it happens - -The shared stringify that builds canonical JSON sorted object keys with `localeCompare()` and no locale argument, so the sort order follows the host's default locale. RFC 7638 wants keys in code-unit order, and a handful of locales disagree about where certain letters sit. The keys of a DPoP thumbprint input are enough to see it: - -```js -const keys = ['crv', 'kty', 'x', 'y'] -keys.sort((a, b) => a.localeCompare(b, 'lv')) -// -> ['crv', 'y', 'kty', 'x'], not ['crv', 'kty', 'x', 'y'] -``` - -The host hashes one string, the client and the cloud hash another, and the credential bound to the real thumbprint fails the comparison. Nothing in the error mentions a locale, so the hunt starts at the network layer. - -## The fix - -Sort keys by code unit, which is what RFC 7638 asks for and what never moves with the machine. [PR #19616](https://github.com/pingdotgg/t3code/pull/19616) switches the stringify to `<` and `>` comparisons, and the output is unchanged on English hosts. - -```js -Object.keys(obj).sort() // code-unit order, locale independent -``` - -The rule is wider than thumbprints. Any canonical JSON that feeds a hash, a signature or a cache key must sort by code unit. A locale-aware comparison is a presentation sort, and using one in a hash input turns the host's regional settings into part of your protocol. diff --git a/data/devlog/looking-at-durable-agents.mdx b/data/devlog/looking-at-durable-agents.mdx new file mode 100644 index 0000000..e60eb96 --- /dev/null +++ b/data/devlog/looking-at-durable-agents.mdx @@ -0,0 +1,50 @@ +--- +title: Looking at Durable Agents With OpenCode and Pi +summary: | + I researched what it would take for an agent to survive a restart and remain usable across interfaces. The useful distinction was between saving a conversation, recovering execution and safely repeating an external action. +date: '2026-10-15' +tags: + - AI Agents + - OpenCode + - Pi +draft: false +icon: Workflow +--- + +I looked into durable agents with OpenCode v2 and Pi v1, using the Syntax discussion +about a shared backend across terminal, web and desktop interfaces alongside the +framework documentation.[^discussion] +This was a research pass, not a new service I had deployed. + +The part I wanted to get clear was what actually survives a restart. Being able +to reload the conversation is useful, but it does not tell me whether unfinished +work resumes, whether the checkout still exists or whether a tool call is about +to happen twice. + +## Four things to keep separate + +- The conversation and its configuration. +- The execution state of unfinished work. +- The files and environment the tools operate on. +- External actions that may already have succeeded. + +The Cloudflare integrations helped make the distinction concrete. The OpenCode +example replaces unsupported local file and shell tools rather than pretending +a Worker has a normal coding checkout.[^opencode] Pi's recovery documentation +distinguishes tools safe to replay from other interrupted tools, which return +control to the model instead of automatically running again.[^pi] + +Neither observation gives arbitrary external actions an exactly-once guarantee. +A saved transcript cannot, by itself, tell me whether an API request succeeded +just before the process died. + +The prototype I would want next is deliberately small: one persistent +conversation, one replay-safe read and one idempotent write, with crashes injected +around the write and the result recording. That remains a proposed test, not a +result from this research pass. + +[^discussion]: [The OpenCode and personal-agent section of Syntax Weekly](https://www.youtube.com/watch?v=Ov_KPnwuu0U&t=773s). + +[^opencode]: [Cloudflare's OpenCode harness integration](https://developers.cloudflare.com/agents/harnesses/opencode/). + +[^pi]: [Cloudflare's Pi harness recovery behaviour](https://developers.cloudflare.com/agents/harnesses/pi/#recovery). diff --git a/data/devlog/major-upgrades-import-state-one-way.mdx b/data/devlog/major-upgrades-import-state-one-way.mdx deleted file mode 100644 index 7f50cb9..0000000 --- a/data/devlog/major-upgrades-import-state-one-way.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: Major Upgrades Import State One Way -summary: | - A major upgrade that copies the old version's database into the new version's own store leaves two copies that diverge the moment either side writes. An edit made in the old app after the import never appears in the new one. Treat the migration as one way and finish the work in the version you are keeping. -date: '2026-10-16 13:20:00 +1000' -tags: - - T3 Code - - Upgrades - - Gotcha -draft: false -icon: RefreshCw ---- - -Upgrade to a major version that reworks its state store and the migration looks clean. Everything is there. Then you fix a thread in the old app, switch back to the new one, and the fix is missing. Both copies are intact and they no longer agree. [T3 Code v0.1.0](https://github.com/pingdotgg/t3code/releases/tag/v0.1.0) ships this in its upgrade notes, tracked in [issue #14871](https://github.com/pingdotgg/t3code/issues/14871). - -## Why it happens - -The first run of the new version copies the existing database and imports from it. The previous version keeps its own copy and keeps writing to it. Nothing syncs the two afterwards, so any change made on the old side after the import exists only there. There is no error and no warning, which is what makes it expensive to discover late. - -The same release flags the same trap one layer down. OpenCode 2 is a separate package that converts the shared database the first time it runs, so 1.x and 2.x cannot be installed side by side on one machine at all. - -There is a wire-format half too. A server on the new major and a client on the old one refuse to talk to each other, and whichever side is older is the one that says so. A single stale desktop app or background service is enough to make a whole upgrade look broken. - -## What to do - -Treat the migration as one way. Do the remaining work in the version you are keeping before you switch, and do not plan on bouncing between the two. - -Update every installation that talks to the server at the same time - desktop apps, mobile apps, background services and remote machines. The side that refuses the connection will name the machine that is behind, which is the fastest way to find the one you missed. - -Where two majors of a tool share one on-disk database and the newer one migrates it in place, install one or the other. Running both is not a fallback. It is how the store gets converted underneath the older binary. diff --git a/data/devlog/malformed-question-answers-consume-the-request.mdx b/data/devlog/malformed-question-answers-consume-the-request.mdx deleted file mode 100644 index 9437e1e..0000000 --- a/data/devlog/malformed-question-answers-consume-the-request.mdx +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: Malformed Question Answers Consume the Request -summary: | - Answering a Claude question permission in any shape but updatedInput.answers reports success, delivers nothing, and burns the request so a retry fails with "No pending permission request". The request is deleted before the answer is validated. Send the answers keyed by the question, and the fix keeps bad answers retryable. -date: '2026-10-16 01:40:00 +1000' -tags: - - Paseo - - MCP - - Gotcha -draft: false -icon: Bug ---- - -Answer an `AskUserQuestion` permission with the answer in the wrong place and the call reports success. Nothing reaches the waiting agent - it is told "The user did not answer the questions." - and the retry then fails with `No pending permission request`. The responder thinks it worked, the agent thinks the human declined, and neither can see that anything broke. This is [getpaseo/paseo#6916](https://github.com/getpaseo/paseo/issues/6916). - -## Why it happens - -`respondToPermission` deletes the request out of `pendingPermissions` on entry, before the answer payload is validated. Any shape that is not `updatedInput.answers` keyed by one of the questions resolves as `allow`, delivers no answer, and leaves the request gone. Both of these do exactly that: - -```json -{"behavior": "allow", "selectedActionId": "Viridian"} -{"behavior": "allow", "answers": {"Which colour?": "Viridian"}} -``` - -A correct-looking answer one level too shallow is enough to burn it. Both shapes are accepted without complaint at send time, so nothing warns the caller that the question is about to be eaten. - -## The fix - -[PR #6936](https://github.com/getpaseo/paseo/pull/6936) makes malformed answers return an error while keeping the question pending, so a corrected retry can still land. It has merged and is waiting on a release past v0.11.2. - -Until then, send the answers under `updatedInput`, keyed by the question text or header: - -```json -{ - "behavior": "allow", - "updatedInput": { - "answers": { "Which colour?": "Viridian" } - } -} -``` - -The pattern is worth carrying elsewhere. Anything that deletes a request before it validates the answer turns a malformed payload into an unrecoverable failure, and reporting success on top leaves both sides certain the other one is at fault. Validate first, then consume. diff --git a/data/devlog/markdown-image-paths-need-decoding.mdx b/data/devlog/markdown-image-paths-need-decoding.mdx deleted file mode 100644 index be08986..0000000 --- a/data/devlog/markdown-image-paths-need-decoding.mdx +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: Markdown Image Paths Need Decoding First -summary: | - An inline markdown image at a path with spaces or non-ASCII characters fails with ENOENT even though the file exists, because the renderer reads the percent-encoded path from disk. Decode the destination before the file read and accept the angle-bracket form so spaces survive CommonMark parsing. -date: '2026-10-15 15:52:00 +1000' -tags: - - Markdown - - Rendering - - Gotcha -draft: false -icon: Bug ---- - -An inline markdown image fails to render when the absolute local path contains spaces or non-ASCII characters. The widget reports the file as missing: - -```text -ENOENT: no such file or directory, open '/Volumes/Disk/%D0%9C%D0%BE%D0%B8%20.../icon/sheet-app.png' -``` - -The file is there. The same image at an ASCII-only path renders fine. This is [getpaseo/paseo#6871](https://github.com/getpaseo/paseo/issues/6871), fixed in [PR #6877](https://github.com/getpaseo/paseo/pull/6877). - -## Why it happens - -All three spellings of the destination fail for different reasons. A bare path with a space does not render at all, because CommonMark ends the URL at the space. The wrapped and percent-encoded forms parse, but the renderer then opens the encoded string as a filesystem path. Nothing decodes it first, so the file lookup runs against the literal percent escapes and misses. The reproduction is minimal - drop a PNG into a directory whose name contains a space and a non-ASCII word, reference it with either parsing form, and watch the ENOENT name the escapes rather than the file. - -```md -![test]() -![test](/tmp/My%20Apps/Мои/test.png) -``` - -## The fix - -Decode the destination - `decodeURIComponent` - before the file read, and accept the `<...>` destination form so spaces survive parsing. Bare destinations containing spaces still need the angle brackets or percent encoding, because that part is the CommonMark spec, not a bug. - -This is the same class of failure as local file links. The link renderer in the same app had already been fixed to decode, while the image renderer kept reading the encoded path. If one consumer of URL-ish paths in a codebase decodes and another does not, the one that does not is a bug waiting for a file with a space in its name. diff --git a/data/devlog/mcp-tools-stay-missing-one-turn-after-sign-in.mdx b/data/devlog/mcp-tools-stay-missing-one-turn-after-sign-in.mdx deleted file mode 100644 index f38cb83..0000000 --- a/data/devlog/mcp-tools-stay-missing-one-turn-after-sign-in.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: MCP Tools Stay Missing One Turn After Sign-In -summary: | - A bundled MCP server that failed to start caches its failure and serves that cache while the retry runs in the background, so the turn after you sign in still has none of the server's tools. Signing in does not change the server's config fingerprint, so nothing invalidates the cached failure. The first turn after a due retry now waits for the server to list instead of trusting the stale cache. -date: '2026-10-15 12:08:00 +1000' -tags: - - MCP - - Caching - - Gotcha -draft: false -icon: Bug ---- - -Sign in to an MCP server mid-session and the next turn still comes back with none of that server's tools. - -``` -openclaw mcp login -# sign-in succeeds, and the next turn runs without the server's tools -``` - -The turn after that works. It looks like the login did not take, so people sign in twice. This is [openclaw/openclaw#172614](https://github.com/openclaw/openclaw/issues/172614), fixed in [PR #172654](https://github.com/openclaw/openclaw/pull/172654). - -## Why it happens - -When a bundled server fails to start - it needs OAuth and has no credentials yet, or its transport refuses once - the catalog owner stores a failure catalog and schedules a retry. While the retry is pending it serves the stored catalog and recovers in the background. For a server that worked before, that is the right trade. For a server that has never listed in this runtime, the stored catalog is the failure, with none of its tools, so the first turn after the retry comes due runs against a catalog that is already wrong. - -Signing in does not help on its own. The login does not change the server's config fingerprint and does not touch the retry schedule, so nothing invalidates the cached failure. The tools are sitting right there. The session just has not looked again. - -## The fix - -The first turn after a retry is due now waits for the catalog loader until that server has listed successfully, instead of returning the cached failure and recovering in the background. If startup still fails, the catalog stays empty for that server and reports its unavailable reason, the next retry time, and reachability guidance. Healthy servers keep their existing background recovery. - -On affected versions, expect one turn of missing tools after `openclaw mcp login `. If you cannot wait, restarting the session picks the tools up immediately. Upgrade for the fix. diff --git a/data/devlog/metacharacter-scans-run-before-quote-parsing.mdx b/data/devlog/metacharacter-scans-run-before-quote-parsing.mdx deleted file mode 100644 index e969827..0000000 --- a/data/devlog/metacharacter-scans-run-before-quote-parsing.mdx +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Metacharacter Scans Run Before Quote Parsing -summary: | - A shell-safety check that scans raw command text for `;`, `&`, `|`, `<` and `>` before any quoting is parsed rejects a correctly quoted literal ampersand as if it were syntax. A callback URL with two query parameters never reaches approval. Parse the argv first and test only what survives as syntax. -date: '2026-10-16 13:15:00 +1000' -tags: - - Shell - - Security - - Gotcha -draft: false -icon: Bug ---- - -An approval-gated `exec` refuses a command that prints one URL. - -```sh -/usr/bin/printf '%s\n' 'http://127.0.0.1:9876/oauth2/callback?state=synthetic&code=synthetic' -``` - -The URL is single-quoted, so the ampersand is a literal. The refusal comes back anyway, and no approval request is ever created. - -```text -SYSTEM_RUN_DENIED: approval cannot safely bind this interpreter/runtime command -``` - -Double quotes reproduce it. Escaping the ampersand reproduces it. The same call with a single query parameter passes. This is [openclaw/openclaw#173240](https://github.com/openclaw/openclaw/issues/173240), fixed in [PR #173273](https://github.com/openclaw/openclaw/pull/173273). - -## Why it happens - -The binder that decides whether a shell payload can be attached to a stable script operand starts by testing the raw command text against a metacharacter pattern. That test runs before the string is split into argv, so it cannot tell a quoted literal from active shell syntax. An OAuth callback URL legitimately carries `&` between query parameters, and the raw-text match treats it as a command separator. The request is refused during preparation, which is earlier than approval - so the operator never even gets a prompt to decline. - -## The fix - -Parse the shell payload into argv first, then run the metacharacter test over what is left as syntax. The landed fix points the binder at the canonical shell-argv parser for the `sh` payloads node exec generates, so quoted and escaped separators survive as literal characters while unquoted globs and nested shells still go to a human. The existing execution policy, working-directory identity and mutable-script binding checks all stay in place. - -The general shape is worth keeping. Any safety check that inspects text before the text has been tokenised is answering a question about the wrong string. Decide what counts as syntax after quoting is resolved, not before. diff --git a/data/devlog/mid-turn-images-are-sent-at-full-size.mdx b/data/devlog/mid-turn-images-are-sent-at-full-size.mdx deleted file mode 100644 index f155990..0000000 --- a/data/devlog/mid-turn-images-are-sent-at-full-size.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: Images Attached Mid-turn Are Sent at Full Size -summary: | - A photo attached while a turn is already running can kill the conversation with repeated image errors, while the same file that starts a turn works fine - images queued into a running turn were sent at full size and anything already within the dimension cap skipped compression entirely. Claude Code v2.1.299 fixes it upstream. -date: '2026-10-16 19:56:00 +1000' -tags: - - Claude Code - - T3 Code - - Gotcha -draft: false -icon: Bug ---- - -Attach a PNG or JPEG while Claude is busy with a turn - a 1800 by 1200 screenshot at 6.5 MB, say - and the steer fails with image errors. The conversation can then lose every later attachment on that turn. Attach the identical file to start a turn and it goes through fine. This is [pingdotgg/t3code#20106](https://github.com/pingdotgg/t3code/issues/20106). - -## Why it happens - -Two halves meet in the middle. Images that start a turn get downsized before they are sent, but images queued into a running turn were sent at full size. On the client side, compression only looks at dimensions - anything already within the 2000px long-edge limit skips resizing entirely, so a file between about 3.75 MB and the attachment cap sails through untouched and arrives over the API's image size limit. - -The dimension checks pass, so nothing warns you. The size only matters at the moment the bytes are sent, which is why timing decides whether the same file is fine or fatal. - -## The fix - -[Anthropic fixed the sending half in Claude Code v2.1.299](https://github.com/anthropics/claude-code/releases/tag/v2.1.299) - the release notes call out images attached while Claude is working being sent at full size, which led to repeated image errors in long conversations. Clients that shell out to the user's installed Claude Code pick the fix up by updating it. - -```bash -claude update -``` - -Until you are on 2.1.299 or later, attach images while Claude is idle rather than steering mid-turn, or keep them under about 3.7 MB. And if an image error does land mid-turn, start a fresh turn instead of retrying the attach - the old behaviour poisoned the rest of the conversation. diff --git a/data/devlog/nextjs-16-4-1-and-15-5-28-security-patches.mdx b/data/devlog/nextjs-16-4-1-and-15-5-28-security-patches.mdx deleted file mode 100644 index 064d88c..0000000 --- a/data/devlog/nextjs-16-4-1-and-15-5-28-security-patches.mdx +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: Next.js 16.4.1 And 15.5.28 Patch Two Unauthenticated RCEs -summary: | - Three security advisories landed with Next.js 16.4.1 and 15.5.28 - two critical unauthenticated remote code execution flaws and a Server Actions denial of service. The useful detail is in the version matrix. There is no patch on the older minor lines, so anything from 10.x up to 15.5.27 has to take 15.5.28, and the image optimisation fix rides in on a sharp bump rather than on Next.js code itself. -date: '2026-10-15 13:30:00 +1000' -tags: - - Next.js - - Security - - Release Notes -draft: false -icon: Package ---- - -Next.js shipped [v16.4.1](https://github.com/vercel/next.js/releases/tag/v16.4.1) and [v15.5.28](https://github.com/vercel/next.js/releases/tag/v15.5.28) with fixes for three advisories, and the two critical ones need no account at all - the requests just have to reach the server. - -## What the advisories are - -[GHSA-673q-mjm6-r5xr](https://github.com/vercel/next.js/security/advisories/GHSA-673q-mjm6-r5xr) - critical. Unauthenticated remote code execution through `use cache` functions that return multiple typed arrays. It needs App Router with Cache Components and a matching function, but the advisory says to upgrade immediately even if nothing in your app looks like that. - -[GHSA-2464-fhr4-w9xw](https://github.com/vercel/next.js/security/advisories/GHSA-2464-fhr4-w9xw) - critical. Unauthenticated remote code execution in the Image Optimization API when SVG files are used. The flaw is a memory issue in the upstream image library, and Next.js on glibc-based Linux is the exposure. The workaround is the interesting part - the advisory names `sharp` 0.35.5 as the fixed version. - -[GHSA-p3m5-59mp-jv5r](https://github.com/vercel/next.js/security/advisories/GHSA-p3m5-59mp-jv5r) - high. Crafted requests against an App Router app with at least one Server Action burn CPU until the process stops answering anything else. Apps on Pages Router, or with no Server Actions at all, are not vulnerable. - -## The detail that matters in the version matrix - -The patched versions are exactly 15.5.28 and 16.4.1. Nothing else got a release. The advisory ranges run from 10.x (the image optimisation one) and 13.x (the denial of service) all the way up to 15.5.27, so a site parked on 14.x or 15.2 or 15.3 is in scope and has no patch on its own line. Taking 15.5.28 is the upgrade. - -## What to do - -```bash -npm install next@16.4.1 # on the 16 line -npm install next@15.5.28 # on the 15 line or anything older -``` - -Then check what your lockfile actually resolved for the image library. Fresh installs pick up a fixed `sharp` on their own, but an existing lockfile will happily keep the old one: - -```bash -npm ls sharp # or: pnpm why sharp -``` - -If you cannot move Next.js yet, the image optimisation advisory can be held off by pinning the fixed library version through your package manager: - -```json -{ - "overrides": { - "sharp": "^0.35.5" - } -} -``` - -That covers one of the three. The `use cache` remote code execution has no workaround, and neither does the Server Actions denial of service beyond upgrading, so the override is breathing room and not a substitute. diff --git a/data/devlog/nextjs-16-4-2-fixes-lazy-dynamic-chunk-collisions.mdx b/data/devlog/nextjs-16-4-2-fixes-lazy-dynamic-chunk-collisions.mdx deleted file mode 100644 index 3a2c51b..0000000 --- a/data/devlog/nextjs-16-4-2-fixes-lazy-dynamic-chunk-collisions.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Next.js 16.4.2 Fixes Turbopack Lazy Import Chunk Collisions -summary: | - Dev on Turbopack falls over with a Build Error about different assets landing on the same output path, and the page 500s, when lazy dynamic imports reach a shared module through different parent chunk groups. Next.js 16.4.2 patches the chunk-list identity so the collision cannot happen. If `next/dynamic` and nested `React.lazy` have been flaky in dev, this is the patch to take. -date: '2026-10-16 09:12:00 +1000' -tags: - - Next.js - - Turbopack - - Release Notes -draft: false -icon: Package ---- - -Dev on Turbopack throws a Build Error saying assets with different content were emitted to the same output path, and the page comes back as a 500. It shows up when lazy dynamic imports reach a shared target through different parent chunk groups - two routes that both pull in the same module with `next/dynamic`, with a nested `React.lazy` inside it. - -[Next.js v16.4.2](https://github.com/vercel/next.js/releases/tag/v16.4.2) ships the fix. - -## Why it happens - -Dynamic HMR chunk-list contents depend on which modules the parent chunk group already has available. The chunk-list identity left that availability context out, so two different contents hashed to the same output path and Turbopack reported the collision as a Build Error. The fix folds the availability identity into the key while keeping HMR paths and lazy-compilation activation keys stable, in [PR #99940](https://github.com/vercel/next.js/pull/99940). - -The reliable reproduction is two parallel page branches that both `next/dynamic`-import a shared target, with that target itself pulling in a nested `React.lazy` import. Both leaves fail to hydrate and the error overlay shows the collision. The same fault showed up in a streaming markdown component rendering three copies of one lazy diagram - the page 500ed and the overlay complained about the shared output path. - -## What to do - -```bash -npm install next@16.4.2 -``` - -The rest of the patch is Turbopack dev stability work - Pages Router dev module graph chaining, two garbage collection paths that used to leave half-torn-down dependency edges, module resolution tolerating realpath errors the way Node does, and code frames snapping to character boundaries. None of it changes production output. - -This drop is on the 16 line. The current 15.5 patch is still 15.5.28 and the release list carries no 15.5 build with these fixes, so if you are on 15 and hitting the collision there is nothing to take yet. diff --git a/data/devlog/non-interactive-ssh-misses-profile-path.mdx b/data/devlog/non-interactive-ssh-misses-profile-path.mdx deleted file mode 100644 index 44e293b..0000000 --- a/data/devlog/non-interactive-ssh-misses-profile-path.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Non-Interactive SSH Misses the Profile Path -summary: | - A remote host probe reports "Could not connect to this host over SSH" while plain ssh to the same machine works, because the remote command never sees Node. An SSH command runs a non-interactive shell that reads only ~/.zshenv, so a PATH export living in ~/.zprofile or ~/.zshrc is invisible to it. Put the export in ~/.zshenv and the host connects. -date: '2026-10-17 09:00:00 +1000' -tags: - - T3 Code - - SSH - - Gotcha -draft: false -icon: Bug ---- - -A device host added over SSH refuses to connect and the summary says "Could not connect to this host over SSH." The transport is fine - plain `ssh host echo ok` succeeds from the same machine, with the same key and the same hostname. Node is installed on the remote host and an interactive shell finds it without trouble. This is [pingdotgg/t3code#20373](https://github.com/pingdotgg/t3code/issues/20373). - -## Why it happens - -The probe runs a small remote command before it runs `node`, and that command prints `Node is missing from the non-interactive SSH PATH` and exits when Node is not on the PATH. The error mapping then turns every probe failure into the same connection message, so the real reason only appears in the trace while the summary claims the host is unreachable. A working SSH connection gets reported as a failed one. - -The PATH miss is a shell-profile effect. `ssh host command` runs a non-interactive, non-login shell. zsh reads only `~/.zshenv` in that mode - `~/.zprofile` and `~/.zshrc` are never sourced - so an `export PATH=...` placed in either of those, which is where Node managers usually put it, is invisible to the probe. The same command typed at an interactive login shell finds Node, which is what makes the failure look impossible. - -## The fix - -Export the PATH from `~/.zshenv` on the remote machine so non-interactive sessions pick it up too: - -```bash -echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshenv -``` - -Adjust the path to wherever `which node` points from an interactive shell. `~/.zshenv` is sourced for every zsh invocation, so this does not change interactive behaviour. - -Two habits follow from this one. When a remote probe claims a host is unreachable but the transport works, read the trace rather than the summary - the first failure message is usually the true one and the mapping above flattens it. And when tooling has to find a binary over `ssh host command`, treat the non-interactive PATH as a separate environment from the one you get at a prompt. The same trap shows up in CI runners, `authorized_keys` forced commands, and anything launched from `launchd` or `systemd` - each runs a shell class that skips the profile files people rely on. diff --git a/data/devlog/npx-cache-can-drop-the-platform-binary.mdx b/data/devlog/npx-cache-can-drop-the-platform-binary.mdx deleted file mode 100644 index 3dc469b..0000000 --- a/data/devlog/npx-cache-can-drop-the-platform-binary.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: npx Cache Can Drop the Platform Binary -summary: | - `npx t3@latest serve` can refuse to start with "no T3 Code CLI build is available for this platform" while naming the platform in its own supported list, because the cached install is missing the optional per-platform package. Clearing the npx cache is the fix, and checking what npm is told to omit stops it coming back. -date: '2026-10-16 19:50:00 +1000' -tags: - - T3 Code - - Node.js - - Gotcha -draft: false -icon: Bug ---- - -The launcher refuses to start and says no CLI build exists for this platform (linux-x64), then prints a supported-platform list that includes linux-x64. The package is published and its tarball downloads fine, so the error reads like a packaging gap when it is not. This is [pingdotgg/t3code#20186](https://github.com/pingdotgg/t3code/issues/20186). - -## Why it happens - -The launcher resolves its binary through an optional dependency - one package per platform triple, `@t3code/t3-linux-x64` and friends. npx builds its cached install once per package version. If that cache entry was created before the platform package existed, or npm was configured to skip optional dependencies, the cache holds an install with no binary for your triple and every later run reuses it. The platform is fine. The cache is lying to you. - -## The fix - -Throw the cache away and let npm refetch the optional dependency: - -```bash -npx clear-npx-cache # or: rm -rf ~/.npm/_npx -npx t3@latest serve -``` - -If it fails again after a clean fetch, check that npm is allowed to install optional dependencies at all - `npm config get omit` and `npm config get optional` should not be filtering them out. That combination is the whole failure mode. - -One heads-up for later versions. [PR #20148](https://github.com/pingdotgg/t3code/pull/20148), merged after 0.1.0 was published, drops the optional-dependency install path and has the launcher download the binary from GitHub Releases instead, so the cached install stops mattering. It is not in a published release yet, so on 0.1.0 the cache is still in the loop. - -The general shape is worth remembering for any npx-run tool that ships per-platform binaries behind optional dependencies. A stale `_npx` cache produces errors that point at the platform rather than at npm, and the supported list printed in the error message can contradict the error itself. diff --git a/data/devlog/openchamber-2-3-2-project-knowledge-for-agents.mdx b/data/devlog/openchamber-2-3-2-project-knowledge-for-agents.mdx deleted file mode 100644 index 320d8e6..0000000 --- a/data/devlog/openchamber-2-3-2-project-knowledge-for-agents.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: OpenChamber 2.3.2 Adds Project Knowledge for Agents -summary: | - A patch release that hands agents a tool for the project's notes, todos and plans, adds .deb and .rpm Linux packages, and quietly fixes config writes that could leave a file truncated mid-save. If project notes have been living outside the chat because agents could not see them, this is the release that moves them into the session. -date: '2026-10-15 09:45:00 +1000' -tags: - - OpenChamber - - Release Notes - - Agents -draft: false -icon: Package ---- - -[OpenChamber v2.3.2](https://github.com/openchamber/openchamber/releases/tag/v2.3.2) is a patch release that gives agents their own tool for the project's notes, todos and plans, plus Linux packages and a batch of fixes that used to bite at the worst times. - -## What changed for you - -Project knowledge is the headline. Agents read and change the project's notes, todos and plans through their own tool, switched on in Settings next to memory. A plan an agent writes opens straight into the Project knowledge panel, so the artefact lands where you will look for it instead of scrolling away in chat history. The panel itself picked up tabs and search in its header, one shared add field across notes, todos and plans, Markdown formatting preserved while editing, and a five-second undo on deletes. - -Linux gets proper packages. The .deb and .rpm builds for x64 and arm64 sit next to the AppImage, register in the app menu with an icon, and update from inside the app once the system prompts for your password. No more replacing an AppImage by hand. - -Links to issues, pull requests and merge requests from GitHub, GitLab and Linear now open on the issues board next to the session. On desktop, a new "Open links in the built-in browser" setting routes everything else into the browser panel. - -## The quieter fixes worth knowing - -Config writes are now atomic. The OpenCode config file is replaced in one step, so a crash can no longer leave a half-written file behind for the next start to trip over. Agent and command files save the same way, a symlinked config keeps pointing into your dotfiles, and a read-only config stays untouched. If you have ever chased a mysteriously truncated config after a bad shutdown, that is the fix. - -Agent settings got honest failure modes. A yes/no tool setting sent as text, such as `returnResult` or `goal`, now fails with a message naming the bad value so the agent can retry. It used to quietly do the opposite of what you asked. - -## What to do - -Update in-app or grab the packages from the [v2.3.2 release](https://github.com/openchamber/openchamber/releases/tag/v2.3.2). Turn on project knowledge in Settings next to memory if you want agents working against the same notes and plans you are. If you are upgrading from further back, 2.3.0 and 2.3.1 landed earlier the same week with movable panels, mirrored Arabic and Hebrew layouts, and iOS Shortcuts session starts. diff --git a/data/devlog/openchamber-2-3-3-agents-draw-charts-and-mockups.mdx b/data/devlog/openchamber-2-3-3-agents-draw-charts-and-mockups.mdx deleted file mode 100644 index b3fabb1..0000000 --- a/data/devlog/openchamber-2-3-3-agents-draw-charts-and-mockups.mdx +++ /dev/null @@ -1,34 +0,0 @@ ---- -title: OpenChamber 2.3.3 Lets Agents Draw Charts and Mockups -summary: | - A patch release that lets an agent render a chart, table, diagram or interface mockup above its reply, adds end-to-end encrypted mobile push that names the outcome, and fixes checkouts on repositories where Git LFS is installed at both the global and local level. If agents have been describing visuals in text because there was nowhere to put them, this is the release that gives them a canvas. -date: '2026-10-15 12:04:00 +1000' -tags: - - OpenChamber - - Release Notes - - Agents -draft: false -icon: Package ---- - -[OpenChamber v2.3.3](https://github.com/openchamber/openchamber/releases/tag/v2.3.3) is a patch release that gives agents somewhere to put visuals, makes mobile push say what actually happened, and clears a batch of papercuts that used to bite mid-session. - -## What changed for you - -Charts and mockups are the headline. An agent can show a chart, table, diagram or interface mockup directly above its answer. The page follows your theme, opens full screen, and downloads as HTML on desktop. If it is not your thing, turn it off under OpenChamber Tools in Settings. - -Mobile push now says what happened. A notification carries the session name and the agent's reply, its question, the command it asks to run, or the error. The text is end-to-end encrypted, so the relay and the push services see only a generic title. - -The issues board got review threads. Replies sit under the comment that started them, resolved threads fold to one line, and you can answer, resolve and reopen threads on GitHub and GitLab. Videos now play in chat and in issue and pull request descriptions and comments, including clips an agent recorded on your machine. The Plans tab shows how many steps of each plan are done. - -## The quieter fixes worth knowing - -Git LFS at both levels no longer breaks worktrees. If you ran `git lfs install` globally and again inside a repository, the canonical filter keys repeat across the config layers and worktree setup used to reject the repo outright ([openchamber#4869](https://github.com/openchamber/openchamber/issues/4869)). Checkouts work again on those repositories. - -A worktree from a pull request can be created after the PR was force-pushed. A leftover branch that diverged is kept as a backup rather than deleted, and a message names it. - -The providers page lists your providers when no project is added, instead of saying "No providers yet" while chat happily shows the models and you sign in again and again. GitLab merge requests show correct line counts and keep their labels when you edit them. A session started in a plain folder next to a project no longer gets filed inside that project. - -## What to do - -Update in-app or grab the build from the [v2.3.3 release](https://github.com/openchamber/openchamber/releases/tag/v2.3.3). The bundled OpenCode is now 2.0.29. If you are upgrading from further back, 2.3.2 landed just before it with project knowledge for agents and proper Linux packages. diff --git a/data/devlog/openclaw-2026-10-3-clears-update-blockers.mdx b/data/devlog/openclaw-2026-10-3-clears-update-blockers.mdx deleted file mode 100644 index 019e90c..0000000 --- a/data/devlog/openclaw-2026-10-3-clears-update-blockers.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: OpenClaw 2026.10.3 Clears Update Blockers and Compaction Failures -summary: | - Mostly an update and repair release, and the trap it removes is nasty - an interrupted update could leave a record behind that blocked every later upgrade, with the stuck updater unable to clear it. Recovery comes first now, Doctor repairs keep saved sign-ins and memory files, and conversations no longer die at a compaction boundary. -date: '2026-10-15 09:50:00 +1000' -tags: - - OpenClaw - - Release Notes - - Updates -draft: false -icon: Package ---- - -[OpenClaw 2026.10.3](https://github.com/openclaw/openclaw/releases/tag/v2026.10.3) is a maintenance release - 79 pull requests - and its [changelog](https://github.com/openclaw/openclaw/blob/main/CHANGELOG/2026.10.3.md) reads like a list of ways an upgrade used to strand itself. - -## The update trap - -An interrupted update could leave a completed update record behind, and that record blocked the next upgrade. The stuck updater could not work around it either, so the fix was never another self-update. This release teaches updates to archive those records and clear eligible leftover recovery work. If you are already blocked, the way out is the manual installation or the recovery first step - the independent recovery helper can preserve the completed record and unblock the original updater. - -A related one. An abandoned updater copy can stop the next update entirely. Wait for updater workers to finish, run `openclaw doctor --fix` from the affected installation, then retry, and keep the recovery journal and its permissions intact. - -## What else changed for you - -Repairs now preserve the things that are painful to lose. Doctor keeps saved sign-ins and memory files during affected repairs, and new recovery copies include older Canvas documents and Memory Core files before migration. If malformed UTF-8 has ever blocked merging `MEMORY.md` and the older `memory.md`, the repair preserves the original bytes. - -Conversations survive their own internals. After compaction the next message waits for saved history to become readable instead of failing while it rebuilds, and a ChatGPT reply cut off by a connection close can continue from the saved conversation. The Mac app also connects again in eligible "On this Mac" setups that used to fail with "Gateway rejected auth". - -## What to do - -If an update is blocked today, recover first and update second - the stuck updater cannot save itself. Follow the recovery steps in the [update guide](https://docs.openclaw.ai/cli/update) rather than retrying the same updater, and keep the recovery journal until the upgrade is through. diff --git a/data/devlog/openclaw-2026-10-4-survives-sleep-and-clock-jumps.mdx b/data/devlog/openclaw-2026-10-4-survives-sleep-and-clock-jumps.mdx deleted file mode 100644 index 545710a..0000000 --- a/data/devlog/openclaw-2026-10-4-survives-sleep-and-clock-jumps.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: OpenClaw 2026.10.4 Survives Sleep And Clock Jumps -summary: | - A stability patch for 2026.10.3, and the fix worth having is the one that stops a Gateway restarting itself after a laptop sleeps or the system clock moves. Path aliases and loopback OAuth redirects get the same treatment, and nothing here needs a config change. -date: '2026-10-16 13:05:00 +1000' -tags: - - OpenClaw - - Release Notes - - Gateway -draft: false -icon: Package ---- - -[OpenClaw 2026.10.4](https://github.com/openclaw/openclaw/releases/tag/v2026.10.4) is a stability patch for 2026.10.3. The changelog is explicit - no new features, no configuration changes - and 83 commits of repair behind it. - -## What changed for you - -**The Gateway stops restarting itself across a sleep.** Close the laptop, come back, and the Gateway used to decide its ownership heartbeat had gone stale and restart itself. A sleep is enough to trigger it. So is a system clock jump. Ownership state is now recovered after both instead of being declared lost, and `openclaw gateway restart` works from a verified external macOS terminal. - -**Shutdown no longer hangs.** A stalled transcript worker or in-flight session lifecycle event could hold a restart open indefinitely. Both waits are now bounded. - -**Manual `/compact` settles the run first.** It used to abort the active run and drop the pending request. Now it waits for the run to settle before compacting. - -**Two path and redirect traps close.** `/tmp` and `/private/tmp` are the same directory on macOS, and treating the two spellings as different strings restarted channels whenever both met. Loopback OAuth redirect URIs registered without a port now complete login instead of stalling. - -## The quieter fixes worth knowing - -Long replies that quote tool-call XML no longer re-parse Markdown on every token, so a reply that reproduces a tool transcript stays cheap to stream. Group DM authors are never treated as the owner, which closes an owner-impersonation path. Headless browser pages keep accepting clicks while a site waits for a passkey. Reasoning-model tool turns on the OpenAI Completions route no longer 400. Enabled standalone plugin files start with the Gateway, and worker heap limits hold even when the managed service sets a process-wide heap flag. - -Nothing here needs a migration. It is the release to take if 2026.10.3 has been restarting your Gateway on a machine that sleeps. diff --git a/data/devlog/openclaw-2026-9-33-brings-critical-fixes-to-extended-stable.mdx b/data/devlog/openclaw-2026-9-33-brings-critical-fixes-to-extended-stable.mdx deleted file mode 100644 index 408b20b..0000000 --- a/data/devlog/openclaw-2026-9-33-brings-critical-fixes-to-extended-stable.mdx +++ /dev/null @@ -1,28 +0,0 @@ ---- -title: OpenClaw 2026.9.33 Brings Critical Fixes to the Extended Stable Line -summary: | - The LTS-equivalent line gets a big refresh - September OpenClaw plus critical security updates, updates that finally finish on pnpm 12, npm and Windows, and work that survives a Gateway restart. If you pin 2026.9 to skip the 2026.10 feature churn, this is the release that makes that choice safe. -date: '2026-10-16 01:25:00 +1000' -tags: - - OpenClaw - - Release Notes - - Updates -draft: false -icon: Package ---- - -[OpenClaw 2026.9.33](https://github.com/openclaw/openclaw/releases/tag/v2026.9.33) is a gateway-only extended-stable release - the current equivalent to an LTS - and it is OpenClaw from the end of September plus critical security updates, reliability fixes and new model support. The newest release overall is still [2026.10.3](https://github.com/openclaw/openclaw/releases). This one is for anyone staying on the 2026.9 line on purpose. - -## What changed for you - -Updates finish now. pnpm 12 Gateway services and launchers survive native updates, retained updater runtimes no longer block npm recovery, and Windows updates no longer roll back after Doctor rechecks driver ancestry. Updates on this line keep state, plugin data, service ownership and Windows state paths intact too, and refuse an automatic restore over unattributed Doctor writes. - -Work survives a Gateway restart. Interrupted automation turns, steered sessions, cloud worker claims and partial worker replies resume or persist instead of disappearing with the process. - -One responsiveness fix is worth knowing on its own. A long streamed reply that starts with a reply tag such as `[[reply_to_current]]` used to re-parse the whole accumulated reply on every delta, which could keep the Gateway busy for tens of seconds. That re-parse is gone from the delta path. - -New model support rides along - Meta Muse Wonder, MiniMax M3.1 thinking effort levels, and MiniMax portal OAuth that stays signed in past token expiry. - -## What to do - -If you are on the 2026.9 line, take it. It is the usual in-place update path, and the update and repair fixes are the reason to stop putting upgrades off. If you are on this line to avoid feature churn, nothing here forces your hand - that is what the extended-stable channel is for. The full note is on the [release page](https://github.com/openclaw/openclaw/releases/tag/v2026.9.33). diff --git a/data/devlog/pidex-pi-cube-and-newsreader.mdx b/data/devlog/pidex-pi-cube-and-newsreader.mdx new file mode 100644 index 0000000..c52f83a --- /dev/null +++ b/data/devlog/pidex-pi-cube-and-newsreader.mdx @@ -0,0 +1,35 @@ +--- +title: Putting the Pi Mark Back on Pidex +summary: | + I brought the Pi mark back onto Pidex's isometric cube, added the variants needed for a dark canvas and paired the interface with bundled Newsreader. +date: '2026-09-28 20:29:36 +1000' +tags: + - Pidex + - Branding + - Typography +draft: false +icon: Package +--- + +I put the Pi mark back on Pidex's isometric cube and worked through the variants +the app actually needs: mono, inverse, stacked and icon treatments that hold up +on the dark canvas.[^change] + +It was a small identity pass, but it crossed more than one surface. The change +included the app's mark component, the cube artwork and a brand demo board, rather +than leaving the logo as a standalone image that the interface did not use. + +Newsreader came in as bundled body type, with Georgia as the fallback. That gave +the text side of the app a deliberate pairing with the geometric mark instead of +trying to make the cube carry the whole identity. + +## Check the mark where it lives + +The useful part of making a demo board was seeing the family together. A mark +that reads well by itself still needs to survive as a small icon, an empty-state +element and an inverse treatment. + +The result is a set of usable pieces rather than one finished-looking logo. +The cube, small mark and body type can now be checked together in the interface. + +[^change]: [Pidex's Pi cube, mark variants and Newsreader change](https://github.com/echohello-dev/pidex/pull/22). diff --git a/data/devlog/pruned-transcripts-strand-system-markers.mdx b/data/devlog/pruned-transcripts-strand-system-markers.mdx deleted file mode 100644 index 9c67820..0000000 --- a/data/devlog/pruned-transcripts-strand-system-markers.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: Pruned Transcripts Strand System Markers And Fail Every Turn -summary: | - Rewrite a provider transcript to prune old turns and you can strand an injected system marker after an assistant message. The provider rejects the ordering with 400 "role 'system' must follow a 'user' message", and because the same rewrite runs on every turn, the whole conversation is poisoned until you reset it. Keep marker messages attached to a following user turn or drop them. -date: '2026-10-15 13:36:00 +1000' -tags: - - LLM APIs - - Transcript Pruning - - Gotcha -draft: false -icon: Bug ---- - -A chat session works fine, then one turn fails with - -``` -400 role 'system' must follow a 'user' message -``` - -and from that point on every turn fails identically. Short typed messages fail. Nothing recovers it except resetting the conversation. The trigger in the report was a run of long pastes that aged past the window the history pruner keeps ([openclaw/openclaw#172241](https://github.com/openclaw/openclaw/issues/172241)). - -## Why it happens - -Providers with strict message-ordering rules do not just validate the message you send. They validate the history you rebuilt. Anything that trims a transcript - compaction, media pruning, a keep-recent-turns safeguard - can drop a turn that an injected system marker was anchored to and leave the marker sitting after an assistant message instead of in a position the provider accepts. - -The failure is sticky for a specific reason. The rewrite is deterministic. The same stored transcript produces the same broken history on the next turn, and the turn after that. It looks like the account, the model or the request is broken, when the payload was poisoned one turn earlier and is being rebuilt faithfully every time. - -## What to do - -When a rewrite step removes turns, treat injected marker messages as belonging to a user turn. Merge a marker into the next surviving user turn, or drop it when the turn it was attached to is gone. Never leave one stranded between two assistant turns or at the end of the history. - -```js -// after pruning, before sending -for (const msg of history) { - if (msg.role === 'system' && !followsUser(msg, history)) { - mergeIntoNextUser(msg, history) || drop(msg, history) - } -} -``` - -If you are staring at this 400 right now, do not retry. The request will fail the same way until the conversation is reset, because the poisoned history is still there. Reset it, then add a check on the rewrite boundary so the ordering rule is asserted before the payload goes out rather than being discovered as a 400 mid-session. diff --git a/data/devlog/quota-exhaustion-is-not-a-rate-limit.mdx b/data/devlog/quota-exhaustion-is-not-a-rate-limit.mdx deleted file mode 100644 index 3a7b2a7..0000000 --- a/data/devlog/quota-exhaustion-is-not-a-rate-limit.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: Quota Exhaustion Is Not a Rate Limit -summary: | - A model provider returning 429 for an exhausted weekly or monthly quota gets the transient same-model retry treatment - nine retries at thirty seconds each - before failover finally moves on, even though the quota cannot reset during that window. Recognising long-window quota wording as a dead end sends the turn straight to fallback. -date: '2026-10-16 19:54:00 +1000' -tags: - - Failover - - Agents - - Gotcha -draft: false -icon: Bug ---- - -The turn stalls for minutes and then falls back to another model, and the log shows the retry budget being burned one attempt at a time against something that cannot succeed: - -```text -transient same-model retry 9/9 for zai/glm-5.3 reason=rate_limit: delayMs=30000 -rawError=429 您已达到每周/每月使用上限，您的限额将在 ... 重置 -``` - -The raw error is a weekly or monthly usage cap with a reset time in it. This is [openclaw/openclaw#173436](https://github.com/openclaw/openclaw/issues/173436). - -## Why it happens - -Failover classification maps HTTP 429 to `rate_limit`, and a rate limit is treated as short-lived throttling worth waiting out. Quota exhaustion is long-lived - the cap does not move until its window resets - so every retry in the nine-attempt budget is doomed, and the user waits about four and a half minutes before the fallback that should have happened immediately. - -The two look identical at the status-code level. The only reliable tell is the error text, and a per-minute throttle wording has to stay retryable while the weekly or monthly wording does not. - -## The fix - -[PR #173485](https://github.com/openclaw/openclaw/pull/173485) extends the shared periodic-usage matcher so long-window quota exhaustion skips same-model retries and goes straight to the existing fallback path, while temporary throttles keep their retry budget. It has merged but is not in 2026.10.4 yet. - -If you are building failover yourself, the transferable rule is this - classify by whether the failure can clear within the retry window, not by status code. A 429 whose message names a reset time minutes or hours away is a billing-class failure wearing a rate-limit coat. Sending it to the fast fallback path is what the retry budget is supposed to be protecting. diff --git a/data/devlog/s2000-bench-studio-and-oled-enclosure.mdx b/data/devlog/s2000-bench-studio-and-oled-enclosure.mdx new file mode 100644 index 0000000..65c59ce --- /dev/null +++ b/data/devlog/s2000-bench-studio-and-oled-enclosure.mdx @@ -0,0 +1,40 @@ +--- +title: A Bench Studio for the S2000 Dash +summary: | + I revised the S2000 cluster preview and gave it a bench workspace for telemetry, scenarios and playback. There is now a separate 3D inspector for the OLED and Raspberry Pi enclosure prototype. +date: '2026-09-20 21:14:52 +1000' +tags: + - S2000 + - Design + - Prototyping +draft: false +icon: Package +--- + +I moved the S2000 dash preview towards something I could use as a bench tool, +rather than just a picture of the cluster. + +This pass corrected the readout proportions and shared warning artwork, then +grouped the face, scenario and playback controls around the display. Telemetry is +readable without opening a JSON dump, and the raw JSON is still available when I +need it. Light, dark and system appearance settings persist between visits.[^change] + +The separate display route hides the web controls and uses mock telemetry, so the +cluster can be previewed fullscreen on the Pi. It is a preview, not a claim that +the live car integration is finished. + +## Looking at the physical setup + +I also added a 3D workspace for inspecting and downloading the OLED/Pi enclosure +prototype. That puts the face design and its housing next to each other in the +workflow, instead of treating the enclosure as something to work out afterwards. + +There is still an important physical unknown: the exact panel model, native +resolution and active area need recording. The target is a seven-inch OLED and +Raspberry Pi for the AP1, but the downloadable geometry is not evidence of a +measured fit in the car. + +That boundary matters. The studio makes the design easier to inspect and iterate; +the hardware measurements still have to decide what gets built. + +[^change]: [The cluster layout, bench studio and CAD inspector](https://github.com/johnnyhuy/s2000-digital-dash/pull/69). diff --git a/data/devlog/server-url-skips-the-major-version-gate.mdx b/data/devlog/server-url-skips-the-major-version-gate.mdx deleted file mode 100644 index 22966f9..0000000 --- a/data/devlog/server-url-skips-the-major-version-gate.mdx +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Server URL Skips the Major Version Gate -summary: | - Pointing a T3 Code provider at an OpenCode 2.x server is accepted and then fails mid-thread with 404s and a dead session, because the version check is skipped whenever a serverUrl is set and the only gate that runs is a minimum version. Pin OpenCode 1.x for that instance, or run a nightly with the v2 driver. -date: '2026-10-16 01:30:00 +1000' -tags: - - T3 Code - - OpenCode - - Gotcha -draft: false -icon: Bug ---- - -T3 Code stable 0.0.45 accepts an `opencode` provider pointed at an OpenCode 2.x server and then fails in the middle of a thread. Prompts 404, the session exits with `Cannot read properties of undefined (reading 'info')`, and resuming silently starts a different session. Point the same instance at OpenCode 1.x and everything works first go. This is [pingdotgg/t3code#19881](https://github.com/pingdotgg/t3code/issues/19881). - -## Why it happens - -When `serverUrl` is set, the version check is skipped entirely. The only gate that runs is a minimum-version check (`>= 1.14.19`), and 2.0.29 sails past it because 2 is greater than 1 - there is no upper bound and no capability probe. The health check does not catch it either. `GET /global/health` against 2.x answers `200 text/html` from the SPA fallback, so connect time looks fine. - -The thread then runs against v1 paths (`/session/{id}/prompt_async`, `/question`, `/permission`) that 2.x does not serve. Prompts 404, sometimes with an uninterpolated `{sessionID}` still in the path. Session resume fails the same way, because 2.x keeps sessions in separate `session_v2` tables, so the lookup misses and the driver quietly starts a fresh session instead. - -## The fix - -[PR #14239](https://github.com/pingdotgg/t3code/pull/14239) detects OpenCode 1.x versus 2.x for external `serverUrl` instances too, and routes 2.x to the dedicated v2 adapter. It is merged and shipping in nightlies now, with the next stable (0.0.46) to follow. - -Until that lands in stable, pin a 1.x server for that provider instance - 1.14.19 is the recommended build - or move the instance to a T3 Code nightly against OpenCode 2.0.18 or newer. The field under discussion is `serverUrl`, as in `serverUrl: http://127.0.0.1:4096`, and the lesson is the gate behind it. A minimum-version check with no upper bound treats every future major as compatible, and a health endpoint that answers 200 with HTML hides the mismatch at exactly the moment a version hint would have helped. diff --git a/data/devlog/shadcn-4-22-0-moves-schemas-to-zod-4.mdx b/data/devlog/shadcn-4-22-0-moves-schemas-to-zod-4.mdx deleted file mode 100644 index 4d5e11e..0000000 --- a/data/devlog/shadcn-4-22-0-moves-schemas-to-zod-4.mdx +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: shadcn 4.22.0 Moves Registry Schemas to Zod 4 -summary: | - shadcn@4.22.0 and @shadcn/registry@0.2.0 upgrade to Zod 4, so the exported schemas no longer typecheck against Zod 3's `z` helpers and error objects carry `issues` instead of `errors`. Registry authors on Zod 3 switch to the exported types and the schema's own methods to stay compiling. -date: '2026-10-17 09:30:00 +1000' -tags: - - shadcn - - Zod - - Release Notes -draft: false -icon: Package ---- - -Updating the shadcn CLI past 4.21.5 can break a registry build that never touched the CLI - the schema types stop compiling against Zod 3. [shadcn@4.22.0](https://github.com/shadcn-ui/ui/releases/tag/shadcn%404.22.0) and `@shadcn/registry@0.2.0` both upgrade to Zod 4 via [PR #12298](https://github.com/shadcn-ui/ui/pull/12298). - -## What changed - -`shadcn/schema` and `@shadcn/registry/schema` now export Zod 4 schemas. Passing them to Zod 3's `z` helpers no longer typechecks, so a registry that leans on `z.infer` or `z.array` over these schemas gets compile errors after an otherwise routine dependency bump. Three details bite: - -- Type inference goes through the exported types - `RegistryItem`, `RegistryItemFile`, `Config` - rather than `z.infer`. -- Composition uses the schema's own methods, for example `registryItemSchema.array()`, rather than wrapping it in `z.array`. -- Zod 4 error objects expose `issues`, not `errors`, and validation messages use Zod 4 wording, so anything that parsed or asserted on the old shape needs updating. - -## What to do - -If you are staying on Zod 3, stop inferring through `z` and use what the package exports directly: - -```ts -import type { RegistryItem, RegistryItemFile, Config } from '@shadcn/registry/schema' - -// instead of z.infer -const item: RegistryItem = { ... } - -// instead of z.array(registryItemSchema) -const items = registryItemSchema.array().parse(raw) -``` - -If you control your own Zod version, move to Zod 4 alongside the schemas and the `z.infer` and `z.array` forms keep working - the break is only the Zod 3 pairing. - -This is the kind of minor that hides its impact. The changelog line reads like a routine dependency bump, but the schemas are a public export, so the version floor moves for every consumer that typed against them. Worth checking before the next registry publish. diff --git a/data/devlog/sliced-stream-suffixes-retain-whole-message.mdx b/data/devlog/sliced-stream-suffixes-retain-whole-message.mdx deleted file mode 100644 index c526f42..0000000 --- a/data/devlog/sliced-stream-suffixes-retain-whole-message.mdx +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: Sliced Stream Suffixes Retain The Whole Message -summary: | - A streamed reply that turns every chunk into its own timeline row by slicing the new suffix out of the accumulated message keeps every intermediate snapshot of that message alive. V8 shares a slice's storage with its source, so retained heap grows with the square of the stream length and a single 128 KB part can hold over 100 MB after garbage collection. Copy the suffix out instead of slicing it. -date: '2026-10-16 04:20:00 +1000' -tags: - - Node.js - - Memory - - Gotcha -draft: false -icon: Bug ---- - -A long reasoning stream takes the whole daemon down with `SIGABRT` and every agent on the host is interrupted. The reply itself is modest - one 128,000-byte part - yet a standalone reproduction of the timeline store retains about 123 MiB after an explicit garbage collection, and the projection it feeds contains one row. Double the message length and the retained memory roughly quadruples. This is [getpaseo/paseo#5882](https://github.com/getpaseo/paseo/issues/5882), fixed in [PR #6262](https://github.com/getpaseo/paseo/pull/6262). - -## Why it happens - -Streaming code often builds rows like this: - -```js -const suffix = message.slice(offset) // shares `message`'s backing store -timeline.push({ text: suffix }) // and now every row pins the whole message -``` - -V8 can share a string's storage with the string it was sliced from. A 64-byte suffix is not 64 bytes of heap - it is a window onto the whole accumulated message as it stood when the chunk arrived. Keep every suffix in a growing store and the retained memory sums over all of those full-length snapshots, which is where the square comes from. A 128 KB stream of 64-byte chunks is 2,000 rows, each pinning up to 128 KB. - -Anything that slices per delta is exposed. The fix here went into the one path that slices on every streamed chunk. Suffixes cut once per completed part hold at most one copy and are fine. - -## The fix - -Emit an independent copy of each suffix rather than a view onto the accumulated string, and copy in UTF-16 code units so a surrogate pair split across two chunks still arrives intact. After the change, memory grows with the length of the text, not its square. - -If you are writing your own timeline or token stream display, the check is simple. Anywhere a row holds `message.slice(n)` where `message` keeps growing, that row is a snapshot of the whole message. Copy, or hold offsets and slice at render time from one live string. - -Reported with the standalone reproduction by a contributor on the linked issue. diff --git a/data/devlog/stale-oauth-bearers-outlive-re-login.mdx b/data/devlog/stale-oauth-bearers-outlive-re-login.mdx deleted file mode 100644 index 655495e..0000000 --- a/data/devlog/stale-oauth-bearers-outlive-re-login.mdx +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Stale OAuth Bearers Outlive Re-Login In Long-Lived Processes -summary: | - A long-running service that resolved its OAuth bearer once at startup keeps sending the expired token after you re-login, and every call fails with 401 token_expired until the process restarts. A fresh process with the same profile works, and local expiry checks stay green, because the provider invalidated the token server-side while the stored expiry still looks days away. -date: '2026-10-15 13:33:00 +1000' -tags: - - OAuth - - Credentials - - Gotcha -draft: false -icon: Bug ---- - -Embeddings and semantic search in a long-running service start failing with a 401 that nothing seems able to fix: - -``` -openai embeddings failed (401): Provided authentication token is expired. [code=token_expired] -``` - -You re-login. The login reports success. Nothing changes. You run the same query from a freshly started command-line process with the same profile and it works. The service still fails. Only a restart clears it. - -This is [openclaw/openclaw#172156](https://github.com/openclaw/openclaw/issues/172156), and the trap generalises to any client that resolves credentials once and then reuses them forever. - -## Why it happens - -Two things collide. The provider invalidated the token server-side, so the stored expiry is still days in the future and every local status check keeps reporting that the credential is fine. And the client captured its bearer at construction time, so the re-login writes a fresh profile to disk that the running process never reads. The in-memory headers are the old token until the process dies. - -That combination is what makes it hard to diagnose. The health check reports all clear, the login succeeds, and only one process is affected - which sends you looking at the account instead of at the client. - -## The fix - -Do not capture a bearer once. Resolve credentials at dispatch time through whatever owns the profile, and when a request comes back `token_expired`, release that response, refresh once, and retry the request once. That is what [PR #172237](https://github.com/openclaw/openclaw/pull/172237) does for the embedding client - it resolves OAuth through the same auth-profile owner as model requests before each HTTP dispatch and retries once on an expired-token response. Retry nothing else. Any other auth failure is a real failure. - -```js -// the shape, not the library -const res = await send(withBearer(await auth.current())) -if (res.status === 401 && isTokenExpired(res)) { - return send(withBearer(await auth.refresh())) -} -``` - -If you are hitting this today in a client you cannot patch, restarting the service after the re-login is the only way the new token lands. - -One more thing worth stealing. A credential check that only compares the local expiry date cannot see a server-side invalidation. The check was green the whole time here. If your monitoring asserts freshness that way, it is measuring the wrong thing. diff --git a/data/devlog/stale-pid-records-block-container-restarts.mdx b/data/devlog/stale-pid-records-block-container-restarts.mdx deleted file mode 100644 index 0ce9e3e..0000000 --- a/data/devlog/stale-pid-records-block-container-restarts.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Stale Pid Records Block Container Restarts -summary: | - A server refuses to start after an unclean container restart with an "already running" error, because its preflight only checks that a recorded pid is alive - and in a container the recycled pid is the new server itself. Move the leftover runtime record aside to start, and treat a recorded pid equal to your own as stale. -date: '2026-10-17 09:02:00 +1000' -tags: - - T3 Code - - Containers - - Gotcha -draft: false -icon: Bug ---- - -After a container is killed and restarted, the server inside it refuses to boot and reports that an instance is already running. Nothing else is in the container, nothing is listening on the port, and restarting again does not help. Every start attempt fails the same way. This is [pingdotgg/t3code#20320](https://github.com/pingdotgg/t3code/issues/20320). - -## Why it happens - -The startup preflight reads a persisted runtime record and refuses to launch when the recorded pid is alive. Liveness is a bare `kill(pid, 0)`, so it reports only that some process holds that pid, not that it is the recorded server. The record stores a pid and a wall-clock start time but nothing that ties the pid to a particular process. - -In a container the pid namespace always hands out the same small numbers, so a record left behind by a killed instance meets the new server's own pid on the next boot. The preflight then matches the process that is trying to start - it never skips `process.pid` - and startup is refused on every restart. A recycled pid that happens to belong to an unrelated process blocks you the same way. On a host with a wide pid space this is rare, which is why the bug hides until it runs in a container. - -## The fix - -Move the leftover record aside before launch: - -```bash -mv userdata/server-runtime.json userdata/server-runtime.json.bak -``` - -Then start the server. The path is wherever the install keeps its runtime state. - -If you are writing the check yourself, the pattern is worth stealing in reverse. A pid-only liveness probe is not a process identity check. Record something that identifies the process - a process start time, a boot id, a random token written by the instance at startup - and compare that alongside the pid. At minimum, treat a recorded pid equal to `process.pid` as stale, since the only way that can match is the record outliving the process that wrote it. A stale record from a crash should never be able to lock out the one process that could replace it. diff --git a/data/devlog/stream-end-without-a-terminal-leaves-the-run-running.mdx b/data/devlog/stream-end-without-a-terminal-leaves-the-run-running.mdx deleted file mode 100644 index da05f68..0000000 --- a/data/devlog/stream-end-without-a-terminal-leaves-the-run-running.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: Stream End Without a Terminal Leaves the Run Running -summary: | - A provider event subscription that ends normally without its terminal event leaves the run marked running forever, because the consumer sees a clean stream end and returns without settling anything. A real run sat on a compaction step for nearly seven hours and later steering failed with "provider turn is not active". Treat a stream that ends without the terminal marker as an ingestion error. -date: '2026-10-16 04:30:00 +1000' -tags: - - Async - - State - - Gotcha -draft: false -icon: Bug ---- - -A delegated run stays marked "running" on a compaction step for six hours and forty-one minutes. The compaction it was stuck on finished in under three minutes and the native turn completed after it, but nothing ever recorded that. Steering the run fails with - -``` -provider turn is not active -``` - -and restarting the app is what finally clears it. The run never persisted a terminal state because none was ever seen. This is [pingdotgg/t3code#19127](https://github.com/pingdotgg/t3code/issues/19127), fixed in [PR #19131](https://github.com/pingdotgg/t3code/pull/19131). - -## Why it happens - -In this protocol the turn ends when a terminal event arrives, not when the stream closes. The consumer loop iterates its subscription, and when the subscription ended without that event it closed cleanly and returned. A normal end of stream looked like a normal end of work. Nothing failed the run, the attempt or the run-owned subagents, and the provider thread was never returned to idle, so every later call saw a run that was still current. - -The hole is deterministic - any subscription that ends early, drops or routes wrong walks into it - and the symptom is a run that is only dead in the app's mind. - -## The fix - -Stream end without a terminal marker now raises an ingestion error instead of returning, and the existing failure finalisation does the rest. The shape of that change looks like this: - -```js -for await (const event of subscription) { - if (event.type === 'turn.terminal') return finalize(event) - ingest(event) -} -// without this line the run stays "running" forever -throw new IngestionError('provider stream ended without a terminal') -``` - -The pattern generalises to any job whose completion is an event rather than the end of its stream. A loop that exits normally on stream end has to ask whether the completion marker ever arrived, and fail the job when it did not. Otherwise the cleanest possible failure mode is the worst one - a state machine convinced work is still in progress that nothing will ever finish. diff --git a/data/devlog/sweep-status.md b/data/devlog/sweep-status.md index 2259152..a4b6a09 100644 --- a/data/devlog/sweep-status.md +++ b/data/devlog/sweep-status.md @@ -1,7 +1,7 @@ -# Devlog Sweep Status +# Devlog Publication History -Cutoff table for the 4-hourly devlog sweep. Each run reads this first, folds in only -what is genuinely new, then appends one row. Runs that find nothing leave no row. +Publication history for devlog entries. Append a row when entries are added, +corrected or withdrawn. This file is `.md`, not `.mdx`, so contentlayer does not pick it up as an entry. @@ -33,3 +33,7 @@ This file is `.md`, not `.mdx`, so contentlayer does not pick it up as an entry. | 2026-10-17 | johnnyhuy/* and echohello-dev/* PRs and releases, getpaseo/paseo issues PRs and releases, openclaw/openclaw issues PRs and releases, openchamber/openchamber issues PRs and releases, pingdotgg/t3code issues PRs and releases, deepseek-ai/deepseek-harness releases, Next.js / Tailwind / ContentLayer2 / shadcn-ui releases | One minor release and two resolutions from after the previous row. shadcn@4.22.0 and @shadcn/registry@0.2.0 upgrade to Zod 4, so shadcn/schema and @shadcn/registry/schema no longer typecheck against Zod 3 z helpers and error objects expose issues rather than errors - registry authors switch to the exported types and the schema's own methods (shadcn@4.22.0, PR #12298). On a case-insensitive filesystem an agent directory and a configured ID that differ only in case resolve to one physical SQLite file, so a repair guard comparing path strings sees two distinct stores, refuses with "requires distinct source and destination databases" and blocks every later migration - fixed by matching filesystem identity instead of path strings (openclaw #174089 / PR #174243). A cache read fenced by a host-wide revision gets restarted by every unrelated stream flush and each retry forces a full persist, so opening a second agent stalls for the whole turn while another streams - fixed by invalidating only on changes to the requested rows, a directory baseline replacement or a host lifecycle change (paseo #4813 / PR #7074). Dropped - a Retry-After 0 cooldown bypass and an @ file mention 20k walk cap both closed as duplicates with the canonical issue still open (paseo #7155 / #5881, #7134 / #3694); MCP server instructions dropped through the loopback bridge has only proposed fixes in open PRs (openclaw #174272); an abandoned Codex agent message finalized as a partial closed with no landed fix (t3code #20221). Nothing new from johnnyhuy/* or echohello-dev/* beyond dependency bumps and devlog PRs. No new stable releases elsewhere - Next.js 16.4.2, Tailwind 4.3.3, ContentLayer2 0.5.8, paseo v0.11.4, OpenChamber v2.3.3, openclaw 2026.10.4 and 2026.9.33, T3 Code v0.1.0 and pidex 2026.10.14-2 unchanged since the row above; openclaw 2026.10.10-beta.1, t3code nightlies and Android beta 2.0.0, Next.js canaries, deepseek-harness v0.2.1-alpha.5 and paseo pre-releases skipped as pre-releases. | case-insensitive-disks-collapse-path-aliases, host-revision-invalidates-every-cache-read, shadcn-4-22-0-moves-schemas-to-zod-4 | | 2026-10-17 | johnnyhuy/* and echohello-dev/* PRs and releases, getpaseo/paseo issues PRs and releases, openclaw/openclaw issues PRs and releases, openchamber/openchamber issues PRs and releases, pingdotgg/t3code issues PRs and releases, deepseek-ai/deepseek-harness releases, Next.js / Tailwind / ContentLayer2 / shadcn-ui releases | Four resolutions from after the previous row. A device host probe reports "Could not connect to this host over SSH" while plain ssh to the same host works, because Node is missing from the non-interactive SSH PATH - zsh reads only ~/.zshenv for an SSH command, so an export in ~/.zprofile or ~/.zshrc never runs, and the probe's error mapping flattens the real message into a connection failure (t3code #20373). A pid-only "already running" preflight left by an unclean container restart matches the new server's own pid in a recycled container pid namespace and refuses every later start - move userdata/server-runtime.json aside, and treat a recorded pid equal to process.pid as stale (t3code #20320). A Windows in-app update fails silently when electron-updater's 20 s PowerShell signature check times out, since the summariser drops killed and signal fields and a timeout reads as a bad signature - install from the releases page or verify the leftover pending\temp-*.exe against the release SHA-512 (t3code #20345). An automation turn instructed to end silently that emits no final text is booked as failed, burns a tools-disabled retry and has a fabricated fallback reply injected - have the model emit the sentinel token out loud; repaired on main for optional turns and not yet in a released build (openclaw #174620). Dropped - closed duplicates of still-open canonicals with no landed fix, on a drained work scope and on suspended completion delivery (openclaw #174697 / #167410, #174564 / #155163), on updater inspection and restart-intent cleanup (openclaw #174515 / #169865, #174506 / #153483), and on stale Claude session resume with the fix still in an open PR (t3code #20334 / #15854); a template-lookup report closed with the mechanism disproven against the published artifact (openclaw #174557); two reports already fixed in earlier shipped releases (openclaw #174707, #174479); a SIGSEGV crash loop closed not planned with the evidence pointing at host memory or storage rather than the release (t3code #20387); a missing t3 symlink that is PATH setup on the user's machine with the installer symlink already correct (t3code #20335); nested background helper routing that is upstream Claude Code behaviour with the workaround documented there (t3code #20332); a child-repo PR refresh request, custom-theme artwork fallback and a duplicate compaction-threshold request, all intended behaviour or ideas (t3code #20378, #20356, #20344 / #19823, #20369 / #17767); an outdated Codex CLI generic probe message with only "update the CLI" as the workaround (t3code #20351); a full-width image render already fixed on main with no user-side trick, a question-response UI recovery fix in an unreleased PR, and three feature requests routed to Discussions (paseo #7166, #7161 / PR #7098, #7132, #7173, #7138). Nothing new from johnnyhuy/* or echohello-dev/* beyond dependency bumps and devlog PRs. No new stable releases anywhere - Next.js 16.4.2, Tailwind 4.3.3, ContentLayer2 0.5.8, shadcn@4.22.0, paseo v0.11.4, OpenChamber v2.3.3, openclaw 2026.10.4 and 2026.9.33, T3 Code v0.1.0 and pidex 2026.10.14-2 unchanged since the row above; openclaw 2026.10.10-beta.1, paseo v0.12.0-beta.1, t3code nightlies and previews, Next.js canaries, deepseek-harness v0.2.1-alpha.5 and paseo pre-releases skipped as pre-releases. | non-interactive-ssh-misses-profile-path, stale-pid-records-block-container-restarts, windows-updates-fail-silently-on-signature-timeout, instructed-silence-gets-a-fabricated-reply | | 2026-10-17 | johnnyhuy/* and echohello-dev/* PRs and releases, getpaseo/paseo issues PRs and releases, openclaw/openclaw issues PRs and releases, openchamber/openchamber issues PRs and releases, pingdotgg/t3code issues PRs and releases, deepseek-ai/deepseek-harness releases, Next.js / Tailwind / ContentLayer2 / shadcn-ui releases | Two resolutions from after the previous row. `@expo/fingerprint` can hash zero Expo native modules because a Maven-resolved gradle plugin with no `sourceDir` makes `path.relative()` throw and the collector's catch returns an empty list, so one sourceless plugin drops every Expo Android module from the hash, native changes stop moving the Android fingerprint, and fingerprint-policy OTA updates reach store binaries that lack them and crash on launch - fixed by skipping plugins without a `sourceDir` and by refusing to publish an OTA from a fingerprint that covers no native modules (t3code PR #20384; the same skip was proposed upstream as expo/expo#51710 and is not in a released `@expo/fingerprint` yet, so 0.21.5 and canaries still carry the bug). `git worktree remove` without `--force` refuses any worktree with initialised submodules even when the tree and every submodule are clean, so a cleanup that omits `--force` to guard against late tracked edits fails permanently on those trees with "working trees containing submodules cannot be moved or removed" - check cleanliness yourself and pass `--force`, extending the check through `git submodule foreach --recursive` (t3code #20355). Dropped - a Fastmail calendar-invite `end` parameter report closed completed with no traceable fix, so the current behaviour could not be verified (openclaw #174577); OpenAI image generation offered on Free ChatGPT plans but failing HTTP 400 is open with no fix (openclaw #174733); system-prompt series markers persisted without prefix or hash, forcing full prefix rewrites on every cold start, is open with no landed fix (openclaw #174690); a /mcp reconnect discoverability request and an idle-thread auto-compact report both closed not planned (t3code #20298, #20287); Codex startup failure reporting and shared-startup teardown fixes plus a cold-start memory index fix are internal merges with no user-side trick (openclaw PR #174706, #174718); open plugin MCP tool loss, preview key capture, coordinator interrupt walk and skill chip reports carry no fix yet (t3code #20374, #20364, #20342, #20367, #20366, #20395). Nothing new from johnnyhuy/* or echohello-dev/* beyond dependency bumps and devlog PRs. No new stable releases anywhere - Next.js 16.4.2, Tailwind 4.3.3, ContentLayer2 0.5.8, shadcn@4.22.0, paseo v0.11.4, OpenChamber v2.3.3, openclaw 2026.10.4 and 2026.9.33, T3 Code v0.1.0 and pidex 2026.10.14-2 unchanged since the row above; openclaw 2026.10.10-beta.1, paseo v0.12.0-beta.1, t3code nightlies and previews, Next.js canaries and deepseek-harness alpha builds skipped as pre-releases. | expo-fingerprint-drops-native-modules-without-sourcedir, git-worktree-remove-refuses-initialised-submodules | +| 2026-10-17 | Editorial review of existing entries | Marked 55 entries dated 15-17 October as drafts pending verification. Source text and publication history preserved. | No new entries; 55 entries marked draft | +| 2026-10-17 | Editorial verification and technical source review | Removed the 55 unverified entries and four original seeds. Replaced the inaccurate GitHub-token explanation with the approval-required, zero-job diagnosis; added verified lessons on TOML table scope, card-fan geometry, MDX footnotes and interpreting a kernel panic. | release-pr-ci-can-wait-for-approval-with-zero-jobs, toml-parent-options-must-precede-child-tables, card-fans-need-a-width-derived-from-their-stride, footnote-references-do-not-wrap-the-cited-text, a-panicked-task-is-not-proof-of-the-leaking-process | +| 2026-10-17 | Implementation evidence review | Removed the diagnostic-only kernel-panic entry. The remaining four entries are supported by concrete code or configuration changes. | Four entries retained | +| 2026-10-17 | Daily work and learning review | Replaced the four generic tips with seven dated entries about Hoist, the S2000 dash, BetterFit, Pidex, the Hetzner provider release and durable-agent research. Entries distinguish merged changes, prototypes and research. | hoist-light-theme, s2000-bench-studio-and-oled-enclosure, betterfit-figma-to-ios, pidex-pi-cube-and-newsreader, betterfit-cold-simulator-ci, hetzner-provider-release-packaging, looking-at-durable-agents | diff --git a/data/devlog/temp-cleanup-fails-while-children-hold-handles.mdx b/data/devlog/temp-cleanup-fails-while-children-hold-handles.mdx deleted file mode 100644 index 2abd299..0000000 --- a/data/devlog/temp-cleanup-fails-while-children-hold-handles.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Temp Cleanup Fails While Children Hold Handles -summary: | - An installer that validates a runtime and then deletes its temporary profile can report a failure after a perfectly successful run, because the removal hits EBUSY on Windows while spawned children still hold the executable and unpacked DLLs. Kill the process tree before the pipes close and retry the removal, and a green validation stops looking like a disk fault. -date: '2026-10-16 04:25:00 +1000' -tags: - - Windows - - Processes - - Gotcha -draft: false -icon: Bug ---- - -The download and extraction succeed. Validation succeeds. Then the install fails at "Checking the downloaded runtime" with a message telling you to check disk space and directory access, and there is nothing wrong with either. This is [pingdotgg/t3code#18551](https://github.com/pingdotgg/t3code/issues/18551), fixed in [PR #18472](https://github.com/pingdotgg/t3code/pull/18472). - -## Why it happens - -The validation step runs the runtime from a disposable temporary profile. That runtime is itself a launcher, and its children - a PyInstaller-packaged process here - keep handles on the executable and the unpacked DLLs. Closing the validation wrapper does not wait for those children to exit, so when the installer removes the temporary directory the removal fails with `EBUSY` or `EPERM`. The run is marked failed and the error blames the disk. - -Windows is where this bites hardest. A file with open handles cannot be deleted, and the delete fails immediately rather than waiting. A retry loop alone is nearly enough - the children exit on their own shortly after - but the timing window is wide enough to fail a whole install. - -## The fix - -Terminate the process tree before the wrapper's pipes close, while the launcher still exists for the tree kill to walk, then retry the cleanup: - -```bash -taskkill /T /F /PID -``` - -The upstream fix does exactly that, and retries disposable-profile removal up to ten times at 100 ms intervals for the `Busy`, `PermissionDenied` and `Unknown` errors that linger while handles drain. - -The general shape matters more than the product. Any installer, test harness or sandbox that spawns children out of a temp directory owns their exit before it owns the delete. Join child exit first, remove second, and never let a cleanup failure overstate itself as a corrupt download. diff --git a/data/devlog/terraform-plan-is-not-drift-detection.mdx b/data/devlog/terraform-plan-is-not-drift-detection.mdx deleted file mode 100644 index 10dcf46..0000000 --- a/data/devlog/terraform-plan-is-not-drift-detection.mdx +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: Terraform Plan Is Not Drift Detection -summary: | - A plan on a schedule detects pending configuration changes. It does not detect drift, because drift is unrecorded change to live infrastructure and a plan has no way to see it unless you ask for a refresh-only run. It also exits 0 with real differences present. Two facts, and most homegrown drift detection is wrong because it missed one of them. -date: '2026-09-12 08:05:00 +1000' -tags: - - Terraform - - Infrastructure as Code - - Drift Detection -draft: false -icon: RefreshCw ---- - -There is a difference between two things people both call drift, and conflating them is the reason most hand-rolled drift detection is either noisy or blind. - -- **Configuration drift:** your code changed, nobody ran apply. `terraform plan` reports this and is very good at it. -- **Infrastructure drift:** something outside Terraform changed the live world. A console edit, a controller that reconciled a default back onto a resource, another team's apply with different state. Your code did not change and your plan will be empty. - -Only the second one is what people are actually worried about when they ask for drift detection. - -## Why a plain plan cannot see it - -Terraform's plan is a diff between your configuration and the state it last recorded. It is not a diff between your configuration and reality. State is the only record of reality Terraform has, and the whole point of state is that it is stale until you refresh it. - -So a plan on a schedule catches "the merge queue has a pending infra change nobody applied". Useful, and not the same thing. To actually read reality you need a refresh-only plan: - -```bash -terraform plan -refresh-only -detailed-exitcode -no-color > drift.json -``` - -- **`-refresh-only`** updates state from the live providers _without_ proposing any configuration change. This is the only plan mode that observes drift. -- **`-detailed-exitcode`** is the part people miss. Without it, `terraform plan` exits 0 whether or not it found changes. You cannot tell a clean run from a dirty one without parsing stdout, and parsing stdout is how you end up with a regex that silently stops matching when Terraform reformats a plan. - -Exit codes: `0` no changes, `1` error, `2` changes present. - -## The trap in refresh-only - -A refresh-only plan is a diagnostic, not a remediation. It tells you what changed. Running `terraform apply -refresh-only` writes those discovered changes into state and makes the drift disappear from every future plan, which is often exactly what you want and occasionally the opposite of what you want. - -Use it when the drift is expected. Someone fixed a broken resource by hand and you want Terraform to stop shouting about it. Do not use it when the drift is a security finding, because you will have laundered the evidence and the next plan will be clean. - -For a security finding you want the diff, keep the state untouched, and open a pull request that reverts the live change in code. - -## Making it a real signal - -The part that turns this into a detector rather than a curiosity: - -- **Run it on a schedule against production.** Nightly is enough. Hourly is expensive and the interesting drift is always someone editing by hand, which is a human timescale. -- **Alert on exit code 2, not on plan text.** Anything else is a rule you will have to maintain. -- **Snapshot state before you refresh it.** A refreshed state is your only record of what reality used to be. Commit it as an artefact, or you will lose the before-picture. -- **Differentiate the two drift types before you page anyone.** A plan with configuration changes means the pipeline is broken. A refresh-only plan with changes means someone touched production. Those go to different people and the wrong page is how you earn a reputation for noisy tooling. - -## On managed and serverless services - -Some resources cannot be drifted in the way you would expect, because the provider itself is the controller. A cloud database with automated backups will have fields that change underneath you. A service with a platform-managed ingress will report a live config that no Terraform configuration could reproduce. - -For these, diff only the fields you actually manage, and treat everything else as ignored rather than trying to converge it. A detector that reports thirty differences nobody can action will be muted within a week, and a muted detector is worse than no detector because it still looks like coverage. diff --git a/data/devlog/tilde-aliases-skip-prompt-cache-markers.mdx b/data/devlog/tilde-aliases-skip-prompt-cache-markers.mdx deleted file mode 100644 index 1ebd9c4..0000000 --- a/data/devlog/tilde-aliases-skip-prompt-cache-markers.mdx +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Tilde Aliases Skip Prompt Cache Markers -summary: | - An OpenRouter ~anthropic/*-latest model ref never gets Anthropic cache markers, so cacheRead stays 0 and every turn re-bills the full prompt at roughly eight times the cost of the exact ref. Pin the exact model ID until the alias normalisation fix reaches your release. -date: '2026-10-15 15:40:00 +1000' -tags: - - OpenRouter - - Prompt Caching - - Gotcha -draft: false -icon: Bug ---- - -Two model refs that route to the same model on OpenRouter behave very differently on the bill. Point an agent at `openrouter/~anthropic/claude-haiku-latest` and `usage.cacheRead` stays 0 on every turn. Point it at `openrouter/anthropic/claude-haiku-5.5` and the first turn writes the cache while every later turn reads it. This is [openclaw/openclaw#172642](https://github.com/openclaw/openclaw/issues/172642). - -Nothing errors. The alias run just re-bills the whole prompt each call. Measured on a stable 24k-token prefix, the alias cost $0.0025 every turn while the exact ref settled at about $0.0003 per turn - roughly eight times the input cost for the identical conversation. - -## Why it happens - -Anthropic prompt caching is opt-in per request. The client marks the stable prefix with `cache_control` markers before sending, and that marking is conditional on the model looking like an Anthropic model. The check reads the raw model ID and matches the `anthropic/` prefix, so a tilde alias like `~anthropic/claude-haiku-latest` classifies as not-Anthropic and no markers are injected. The shared Chat Completions path had the same gap. - -## The fix - -[PR #172652](https://github.com/openclaw/openclaw/pull/172652) normalises the alias family before classification, so `~anthropic/*-latest` refs get the same cache treatment as exact IDs. It needs no configuration change once it reaches your release. - -On versions without it, pin the exact model ID instead of the floating alias: - -```yaml -agents: - defaults: - modelPolicy: - allow: - - openrouter/anthropic/claude-haiku-5.5 -``` - -More generally, if `cacheRead` never moves across consecutive turns in one session, check your model ref spelling before you go hunting for cache TTLs. An alias form the provider resolves server-side can still be a form your client does not recognise. diff --git a/data/devlog/tool-prose-latches-quadratic-trace-scrub.mdx b/data/devlog/tool-prose-latches-quadratic-trace-scrub.mdx deleted file mode 100644 index e21e8dc..0000000 --- a/data/devlog/tool-prose-latches-quadratic-trace-scrub.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Tool Prose Latches a Quadratic Trace Scrub -summary: | - A streamed reply that merely mentions a tool call in ordinary prose stalls the Gateway event loop for seconds, because a substring latch turns the trace scrub on and then re-scans the whole accumulated reply on every delta. The scrub now activates only on bounded trace-line heads - if you write per-delta filters, gate the expensive pass on candidate heads, not substrings. -date: '2026-10-15 21:05:00 +1000' -tags: - - Streaming - - Performance - - Gotcha -draft: false -icon: Bug ---- - -A long streamed reply locks up the process serving it. Measured against 2026.10.3, streaming a reply in the 4-character deltas providers send took 4.3 s of main-thread time at 83 KB, 18.9 s at 166 KB and 85.1 s at 333 KB - roughly four times the cost for every doubling. The reply does not have to contain any internal trace output at all. Ordinary prose that says "tool call" or "function call" is enough to trigger it. This is [openclaw/openclaw#172629](https://github.com/openclaw/openclaw/issues/172629), fixed in [PR #172631](https://github.com/openclaw/openclaw/pull/172631). - -## Why it happens - -The visible-text projection runs a chain of per-delta filters, and one of them scrubs internal trace lines out of the accumulated reply. Its trigger is a substring match. The moment the reply text contains `tool call` or `function call`, the filter latches on and stays on. From then on every incoming delta re-runs the scrub over the entire accumulated buffer, so the work per delta grows with the reply length and the total cost is quadratic. A prose reply that never mentions those phrases streams at close to the linear rate - the table in the report shows a 67 KB reply finishing in 610 ms while the 83 KB reply that trips the latch takes 4.3 s. - -## The fix - -[PR #172631](https://github.com/openclaw/openclaw/pull/172631) replaces the substring latch with a bounded check for canonical trace-line heads, so mentioning a tool call in prose no longer switches the rest of the reply into whole-buffer scrubbing. Genuine trace lines still get stripped. - -The shape is worth stealing for any streaming filter. If an expensive pass re-processes accumulated text, its trigger must be cheap and region-scoped, not a substring that can appear in ordinary content. Otherwise one unlucky sentence turns a linear stream into an O(n^2) one, and the cost only shows up on long replies. The follow-up cases - replies that really do carry trace lines or fenced code blocks - still re-scrubbed the whole buffer and are fixed separately in [PR #172832](https://github.com/openclaw/openclaw/pull/172832). diff --git a/data/devlog/undici-body-timeout-cuts-quiet-http-streams.mdx b/data/devlog/undici-body-timeout-cuts-quiet-http-streams.mdx deleted file mode 100644 index f5aa8da..0000000 --- a/data/devlog/undici-body-timeout-cuts-quiet-http-streams.mdx +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Undici Body Timeout Cuts Quiet HTTP Streams -summary: | - A long-running HTTP stream that goes quiet for more than about five minutes is closed by undici's default body timeout, even when your own request deadline is much longer. The failure lands as a client-side request timeout although the server answered in time. Set headersTimeout and bodyTimeout on your dispatcher, or raise the transport stream budget rather than only the request deadline. -date: '2026-10-15 14:05:00 +1000' -tags: - - HTTP - - Node.js - - Gotcha -draft: false -icon: Bug ---- - -A Streamable HTTP MCP tool call whose server stays quiet for longer than about five minutes never completes. The call eventually fails with - -``` -MCP error -32001: Request timed out -``` - -even though `requestTimeoutMs` is set to ten minutes and the server did answer in time. It looks intermittent, because whether it bites depends on what else has configured streaming timeouts in the process first. This is [openclaw/openclaw#172488](https://github.com/openclaw/openclaw/issues/172488), fixed in [PR #172510](https://github.com/openclaw/openclaw/pull/172510). - -## Why it happens - -Two different clocks are involved and only one of them is the deadline you set. Your request timeout covers the RPC. Undici separately enforces inactivity limits on the stream carrying the response, and its `headersTimeout` and `bodyTimeout` both default to 300 s. A response that stays silent past that has its stream closed, so the RPC dies against a stream that is already gone and the error names your request timeout instead of the real cause. - -Anything built on undici inherits this. A fetch with no explicit timeout is not unlimited, it is capped at five minutes of silence. - -## The fix - -Give the dispatcher its own stream budget instead of inheriting the defaults: - -```js -import { Agent, setGlobalDispatcher } from 'undici' - -setGlobalDispatcher(new Agent({ - headersTimeout: 600_000, - bodyTimeout: 600_000, -})) -``` - -Raise both. Raising only the request deadline keeps the symptom exactly as it is. For the bundled MCP client the fix was to pass an independent stream inactivity budget through the guarded fetch dispatcher, so quiet calls now wait at least as long as the configured request timeout and no configuration change is needed. - -If a five minute silence is legitimate for your server, audit every layer that fetches before you trust the timeout you set. The number in your config is one of at least two that decide when the stream dies. diff --git a/data/devlog/window-bounds-rewrites-churn-symlinked-settings.mdx b/data/devlog/window-bounds-rewrites-churn-symlinked-settings.mdx deleted file mode 100644 index 2f96237..0000000 --- a/data/devlog/window-bounds-rewrites-churn-symlinked-settings.mdx +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Window Bounds Rewrites Churn A Symlinked Settings File -summary: | - Keep a settings file in a dotfiles repo and symlink it into the app, and every window move dirties the repo. The file stores preferences and machine-local window state together, and saves write through the symlink. Keep the file unlinked until window state is split out from preferences. -date: '2026-10-15 14:20:00 +1000' -tags: - - Dotfiles - - Desktop Apps - - Gotcha -draft: false -icon: Bug ---- - -Symlink the desktop app's settings file into a dotfiles repo so preferences stay versioned, then move or resize the main window. Check `git diff`. The dotfiles copy changed, and it will change again on every move, resize, maximise and unmaximise, debounced at 500 ms. This is [pingdotgg/t3code#19595](https://github.com/pingdotgg/t3code/issues/19595). - -## Why it happens - -The settings file holds both things you care to track and things only this machine should know. Preferences like the update channel sit next to `mainWindowBounds` and the maximised flag, and the window handlers write those through the same save path as everything else. Since saves started writing through the symlink, every window event rewrites the tracked target with state that is meaningless everywhere else. - -It is not a bug in the symlink handling. The write-through is working. The problem is what gets written. - -## What to do - -Keep the settings file unlinked. That is the stated workaround until window state is separated from preferences, so a settings file in version control stops picking up machine-local churn. - -If you want the preferences tracked anyway, the general rule is to symlink only files whose entire contents are worth versioning. A config that mixes durable preferences with per-machine state, like window geometry, last opened path, device identifiers or cached sizes, will fight you every time the app moves. Split the state out, or leave the file out of the repo and track only the parts you would actually restore on a new machine. The test is simple. If you would not paste a value from the file into your dotfiles on purpose, it does not belong in a tracked file. diff --git a/data/devlog/windows-update-repair-refuses-on-protected-processes.mdx b/data/devlog/windows-update-repair-refuses-on-protected-processes.mdx deleted file mode 100644 index 274d2df..0000000 --- a/data/devlog/windows-update-repair-refuses-on-protected-processes.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Windows Update Repair Refuses on Protected Processes -summary: | - Update repair on Windows can refuse with "Handoff descendants remain alive or unverified" while naming only a protected antivirus helper as the blocker, and a reboot just hands it a new PID. The fix lets same-account repair exclude system processes it cannot read instead of treating the whole host as suspect. -date: '2026-10-16 19:58:00 +1000' -tags: - - Windows - - Updates - - Gotcha -draft: false -icon: Bug ---- - -`update repair` refuses to run and names one PID as a surviving handoff holder. The process is Microsoft Defender's protected `DefenderSessionHelper.exe`, parent `MsMpEng.exe`. Running the repair from an elevated shell changes nothing, and after a reboot the error names Defender's fresh PID instead. Nothing about the update is actually blocked - the check is looking in the wrong place. This is [openclaw/openclaw#173430](https://github.com/openclaw/openclaw/issues/173430). - -## Why it happens - -The process census covers every PID on the host rather than the update's own job or process tree. Any row whose command line or working directory cannot be read lands in an unverified bucket and is treated as a possible holder of the update's files. Protected system processes are exactly the ones an ordinary account can never read, so on any machine with antivirus running the repair holds itself hostage. - -## The fix - -[PR #173783](https://github.com/openclaw/openclaw/pull/173783) lets same-account repair exclude fully access-denied system processes and positively unrelated readable ones, while genuine retained-artifact holders and unresolved surviving update work still block. The refusal it keeps is the conservative one for unreadable processes on old handoffs with no recorded launching account, and the error now says to run repair as the same Windows account that started the update. - -The transferable rule is the census scope. When a cleanup waits for children, scope the check to the process tree it started. A host-wide scan cannot distinguish your descendants from the operating system's, and the rows you most want to ignore are the ones you are least allowed to inspect. diff --git a/data/devlog/windows-updates-fail-silently-on-signature-timeout.mdx b/data/devlog/windows-updates-fail-silently-on-signature-timeout.mdx deleted file mode 100644 index 2eee553..0000000 --- a/data/devlog/windows-updates-fail-silently-on-signature-timeout.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: Windows Updates Fail Silently On Signature Timeout -summary: | - An in-app update on Windows can fail with no useful error when the installer signature check takes longer than electron-updater's 20 second PowerShell limit, because the timeout is reported like a signature failure and the summary drops the fields that would show it. Download the installer from the releases page, or verify the leftover temp file against the published checksum and run it. -date: '2026-10-17 09:04:00 +1000' -tags: - - Windows - - Electron - - Gotcha -draft: false -icon: RefreshCw ---- - -An in-app update on Windows downloads and then goes quiet. No error dialog, no rollback message, and the app is still on the old version after a restart. The log carries a signature-check failure, which reads like the installer was rejected, but the installer from the releases page runs fine. This is [pingdotgg/t3code#20345](https://github.com/pingdotgg/t3code/issues/20345). - -## Why it happens - -electron-updater wraps the download in a PowerShell `Get-AuthenticodeSignature` check and kills that check after 20 seconds. On a busy or cold machine the check can exceed the limit, and the killed process is recorded as a plain failure. The error summariser then keeps only the tag, the code and the first line of the message, so a timeout and a genuine bad signature come out looking the same. The update fails, the cause is flattened away, and the download it already verified well enough to run sits on disk unused. - -## The fix - -Take the update by hand. Close the app and install from the releases page. - -If you want the copy the updater already fetched, it is waiting in the updater's temp directory as `pending\temp-*.exe`. Check it against the SHA-512 in the release's `.yml` before you run it: - -```powershell -Get-FileHash .\pending\temp-1234.exe -Algorithm SHA512 -# compare with the sha512 entry in the release's latest.yml -``` - -If the hash matches the release you were updating to, that file is the installer. - -The wider lesson is for anyone shipping an Electron app. A verification step with a hard timeout will fail on slow machines as surely as on a corrupt download, so the two cases have to be distinguishable in the error - keep the `killed` and `signal` fields when you summarise a child-process failure. And a timed-out check is not the same as a failed one, so retrying the check, or verifying in-process without a shell, is the fix rather than failing the update. diff --git a/data/devlog/worktree-moves-drop-the-composer-draft.mdx b/data/devlog/worktree-moves-drop-the-composer-draft.mdx deleted file mode 100644 index 1365462..0000000 --- a/data/devlog/worktree-moves-drop-the-composer-draft.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Worktree Moves Drop the Composer Draft -summary: | - Text typed into a chat composer vanishes when the session moves to a worktree, because the draft is keyed on the session's directory and the move re-keys that directory without migrating the draft. The typed text is saved under the old key and the new key loads empty. Migrate the draft whenever the identity it is keyed on changes. -date: '2026-10-16 04:35:00 +1000' -tags: - - State - - Worktrees - - Gotcha -draft: false -icon: Bug ---- - -Type a message, move the session to a worktree before sending, and the input field is empty. Nothing was sent and nothing errored - the draft is simply gone from view. This is [openchamber/openchamber#4833](https://github.com/openchamber/openchamber/issues/4833), fixed in [PR #4974](https://github.com/openchamber/openchamber/pull/4974). - -## Why it happens - -The composer persists drafts under a key built from the full identity of the conversation: - -```js -const key = JSON.stringify([runtimeKey, directory, sessionId]) -``` - -Move the session to a worktree and its directory changes, so the identity changes with it. On that change the composer saves the outgoing draft under the old directory key and loads the incoming one - which is empty for the new worktree. The typed text is not lost, it is filed under the previous identity where nothing looks for it. Every move path behaves this way. - -It is a small cousin of the classic stale-key bug. Any piece of state keyed on a tuple is only findable while every member of the tuple holds still, and a change to any member silently orphans it. The text is never actually lost - it sits in the draft store under the previous identity, which is how the report was traced. - -## The fix - -Migrate the draft when the identity changes. The move now carries the typed text across with the session rather than leaving it under the old key. - -If you key client-side state on an identity that can change mid-session, give the change handler one job - move the state to its new key before anything reads the new one. Renaming the key without moving the value reads exactly like data loss. diff --git a/data/devlog/wrapper-spawns-that-drop-stderr-hide-cli-exits.mdx b/data/devlog/wrapper-spawns-that-drop-stderr-hide-cli-exits.mdx deleted file mode 100644 index aa36f9b..0000000 --- a/data/devlog/wrapper-spawns-that-drop-stderr-hide-cli-exits.mdx +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: Wrapper Spawns That Drop Stderr Hide Early CLI Exits -summary: | - A supervised CLI that exits early shows only a generic turn failure with no reason, because the wrapper spawns it with stderr sent to /dev/null to keep credentials out of server logs. The exit error carries just a code or signal. Run the binary directly in a terminal to see what it actually printed. -date: '2026-10-15 14:15:00 +1000' -tags: - - CLI Tools - - Debugging - - Gotcha -draft: false -icon: Bug ---- - -A CLI exits before the session gets going and the interface reports one line: - -``` -Provider turn failed. -``` - -Nothing in the thread, nothing in the server logs, no stack. The only trace is a warning that the query stream failed. This is [pingdotgg/t3code#19571](https://github.com/pingdotgg/t3code/issues/19571). - -## Why it happens - -On macOS and Linux the supervisor spawns the CLI as a process-group leader, so plugin MCP servers the CLI starts get cleaned up when the session closes ([PR #19322](https://github.com/pingdotgg/t3code/pull/19322)). The spawn uses `stdio: ['pipe', 'pipe', 'ignore']`. - -The SDK's own spawn captures and redacts stderr before it lands anywhere. A wrapper that pipes stderr through itself loses that redaction, and unredacted CLI output can carry credentials, so the safe move is to send it to `/dev/null`. The cost is that the exit error keeps only the code or signal. The thing that would have explained the failure is discarded on purpose. - -``` -spawn(command, args, { - stdio: ['pipe', 'pipe', 'ignore'], - detached: true, -}) -``` - -## What to do - -Run the configured binary or wrapper directly in a terminal. Its stderr is the diagnosis the supervisor threw away, and in the report the wrapper's refusal reason was sitting right there. - -If you are the one writing the supervisor, do not just turn stderr back on. The redaction exists for a reason. Capture a redacted tail, or at least surface the exit code and signal as a first-class error so an early exit reads as "exited with code 2" instead of a bare failure. A redacted stderr tail is not shipped behaviour today, so the direct run is the workaround. diff --git a/scripts/generate-llms.mjs b/scripts/generate-llms.mjs index 0449f47..2b1c0f3 100644 --- a/scripts/generate-llms.mjs +++ b/scripts/generate-llms.mjs @@ -1,4 +1,4 @@ -import { writeFileSync, mkdirSync } from 'fs'; +import { writeFileSync, mkdirSync, readdirSync, rmSync } from 'fs'; import path from 'path'; import { fileURLToPath } from 'url'; @@ -45,6 +45,13 @@ async function generateLlms() { // Markdown versions of each entry at /devlog/.md const devlogOutDir = path.join(projectRoot, outputFolder, 'devlog'); mkdirSync(devlogOutDir, { recursive: true }); + // Remove generated markdown for entries that were withdrawn or deleted. + const publishedMarkdown = new Set(entries.map((entry) => `${entry.slug}.md`)); + for (const filename of readdirSync(devlogOutDir)) { + if (filename.endsWith('.md') && !publishedMarkdown.has(filename)) { + rmSync(path.join(devlogOutDir, filename), { force: true }); + } + } for (const entry of entries) { const body = entry.body.raw .split('\n') diff --git a/scripts/generate-redirects.mjs b/scripts/generate-redirects.mjs index 2d85670..0b9df2b 100644 --- a/scripts/generate-redirects.mjs +++ b/scripts/generate-redirects.mjs @@ -1,4 +1,4 @@ -import { mkdirSync, writeFileSync, copyFileSync, existsSync } from 'fs' +import { mkdirSync, writeFileSync, copyFileSync, existsSync, readdirSync, rmSync } from 'fs' import path from 'path' // Keep old /notes/ URLs alive after the surface was renamed to /devlog. @@ -38,6 +38,14 @@ async function main() { const notesRoot = path.join(projectRoot, outputFolder, 'notes') mkdirSync(notesRoot, { recursive: true }) + // Do not retain generated redirects for entries that no longer publish. + const publishedSlugs = new Set(published.map((entry) => entry.slug)) + for (const entry of readdirSync(notesRoot, { withFileTypes: true })) { + if (entry.isDirectory() && !publishedSlugs.has(entry.name)) { + rmSync(path.join(notesRoot, entry.name, 'index.html'), { force: true }) + } + } + // /notes/ -> /devlog/ const indexTarget = `${basePath}/devlog/` writeFileSync(path.join(notesRoot, 'index.html'), html(indexTarget))