Skip to content

Commit a5038dc

Browse files
suleimanshclaude
andauthored
Three small fixes: the shared checkout's index lock, the website build, a dead module (#1791)
The daemon's data sync and the scheduler's tick each write .branches/agent-data from their own process on the same 60-second clock; when their phases align, git refuses the second with "index.lock: File exists" every minute until they drift, and the scheduler loses the tick. A write cycle now waits out that one refusal: reset, half a second, run again, three times at most. Vike reads every +name.* file beside a page as page configuration, .md included, so the +config.LOGIC.md and +config.BUG-ANALYSIS.md sidecars broke the website build, and every deploy since 08-23 failed on it. The site's vite config tells Vike's crawl to skip them. cli.ts imported five names from maintenance.ts and used none; nothing else imported the module. It goes, with its test and their sidecars. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
1 parent dcb2a3b commit a5038dc

14 files changed

Lines changed: 117 additions & 500 deletions

‎packages/agent-data/src/file-branch.LOGIC.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ Implements a branch of the project's repository used as a file store: files that
2828
- **The write cycle** - sync with origin, apply the change to the checkout, commit whatever changed under the caller's message, push whenever the branch is ahead of origin's copy.
2929
- **Sync: rebase onto origin, and origin wins a conflict** - unpushed local commits are rebased onto origin's copy; when the rebase fails, the checkout is reset to origin's copy and those commits are dropped, unreported.
3030
- **A push that loses a race re-applies the change once** - the attempt's commit is wound back, the cycle re-syncs and re-applies; a second failed push keeps the commit local and reports it, for the next cycle to carry out; never a force push.
31+
- **Another process holding the index is waited out** - git's refusal to take the checkout's index lock, which two processes on one clone hand each other, resets the checkout, waits half a second and runs the cycle again, three times at most.
3132
- **A failed change leaves the checkout clean** - any other failure, a timeout included, resets the checkout to its last commit, removes stray files, and is reported rather than thrown.
3233
- **The pull** - a write cycle with no change, run on the daemon's clock so this machine converges on what others pushed and pushes what an earlier cycle left stranded; a repository with no remote is an error it names.
3334
- **Reads from anywhere, and never a failure** - a file or a directory listing is read off the checkout, the local branch, or origin's copy, from any directory of the repository, an agent's checkout included; whatever is missing reads as absent.
@@ -116,6 +117,16 @@ Without a remote, a sync does nothing. With one, the branch is fetched from orig
116117

117118
The change is an intent and the commit only its serialization, so the caller's change must be safe to run again. When the push fails on the first attempt, the attempt's commit is wound back to the tip the cycle started from (when a commit was made), the cycle syncs again, bringing in what the other writer pushed, and runs the change again against the fresher files, so the change lands exactly once. When the push fails on the second attempt too (the network, most likely), the commit stays local in the checkout [2] and the cycle reports the failure as "the <branch> branch could not be pushed: <git's reason>", marked as committed: the next write cycle [3] or pull rebases that commit onto whatever origin has by then, and its push carries it out together with the new change. A push killed on its time budget counts as a failed push and may have landed anyway; the next sync's rebase absorbs a commit origin already has. The branch is never force-pushed.
118119

120+
### Another process holding the index is waited out
121+
122+
#### Context
123+
124+
**Problem**: the one-at-a-time rule is a process's own; two processes on one clone (the daemon's pull and a scheduler's, each on its own clock) can run a cycle on the same checkout at once, and git refuses the second with "Unable to create '…/index.lock': File exists" rather than waiting. On the scheduler's side a refused pull loses a whole tick.
125+
126+
#### Business logic
127+
128+
When a cycle fails with git's index-lock refusal, the checkout is reset as for any failure (next section), the cycle waits half a second and runs again, the change included, up to three more times. A lock that never lifts is then reported as the failure it is.
129+
119130
### A failed change leaves the checkout clean
120131

121132
#### Context

‎packages/agent-data/src/file-branch.test.LOGIC.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ What the tests cover, against real git repositories with a bare `origin` and a s
55
- **A write** - commits on the branch under the caller's message, pushes it to origin, and leaves `main` untouched; a change that writes nothing commits nothing and reports no change; in a repository with no remote the write lands locally and reports that nothing was pushed.
66
- **Stranded commits and conflicts** - a write syncs in what another machine pushed and carries out an earlier local-only commit together with its own; a stranded commit that conflicts with origin's version resolves toward origin, and the change is re-applied on top of origin's version.
77
- **A lost race** - a push rejected because another writer landed in between re-runs the change once against the fresher files: the change is on the branch exactly once, next to the other writer's file.
8+
- **The index held by another process** - git's index-lock refusal is recognized and a plain push failure is not; a change refused twice runs a third time and lands on origin exactly once, as one commit; a lock that never lifts is reported with git's reason and leaves the checkout at its last commit.
89
- **The pull** - converges the checkout on what another machine pushed; in a repository with no remote it reports an error naming the missing remote.
910
- **One write at a time** - three concurrent writes run one after another, never interleaving, and all three land in order.
1011
- **Reads** - a file reads off the checkout, from an agent's own checkout of the same repository too, which resolves to the project as its repository; a missing file reads as absent; after another machine moved origin on, a plain read still shows the checkout while a fresh read shows origin; a clone that holds no local branch reads origin's copy and lists a directory off it, and a missing directory lists as empty.

‎packages/agent-data/src/file-branch.test.ts‎

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,12 @@ import { mkdir, mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promi
33
import { tmpdir } from 'node:os'
44
import { dirname, join } from 'node:path'
55
import { test } from 'node:test'
6-
import { nodeGitRunner } from './git.js'
6+
import { nodeGitRunner, type GitRunner } from './git.js'
77
import {
88
ensureFileBranch,
99
fileBranchPath,
1010
fileBranchRepo,
11+
isIndexLocked,
1112
listBranchDir,
1213
pullFileBranch,
1314
readBranchFile,
@@ -210,6 +211,38 @@ test('a push lost to another writer re-applies the intent once, not twice', asyn
210211
}
211212
})
212213

214+
test('another process holding the index is waited out: the cycle runs again and the write lands once', async () => {
215+
const { repo, bare, cleanup } = await initSyncedRepos()
216+
try {
217+
// git's own refusal, as the daemon and the scheduler hand it to each other on one clone.
218+
const refusal = new Error("Command failed: git add -A\nfatal: Unable to create '/x/.git/worktrees/store/index.lock': File exists.\n\nAnother git process seems to be running in this repository")
219+
assert.equal(isIndexLocked(refusal), true)
220+
assert.equal(isIndexLocked(new Error('Command failed: git push')), false)
221+
let refusals = 2
222+
const locked: GitRunner = (args, cwd) => {
223+
if (args[0] === 'add' && refusals-- > 0) return Promise.reject(refusal)
224+
return git(args, cwd)
225+
}
226+
let runs = 0
227+
const result = await withFileBranch(repo, BRANCH, 'append', async dir => {
228+
runs++
229+
await writeFile(join(dir, 'queue.md'), '- once\n')
230+
}, { git: locked })
231+
assert.deepEqual(result, { ok: true, changed: true, pushed: true })
232+
assert.equal(runs, 3, 'the change re-ran after each refusal')
233+
assert.equal(await git(['show', `${BRANCH}:queue.md`], bare), '- once\n')
234+
assert.equal((await git(['log', '--oneline', BRANCH], repo)).trim().split('\n').length, 2, 'born, then one commit')
235+
// A lock that never lifts is reported like any other failure, and the checkout is clean.
236+
refusals = Number.POSITIVE_INFINITY
237+
const stuck = await withFileBranch(repo, BRANCH, 'again', async dir => writeFile(join(dir, 'queue.md'), '- twice\n'), { git: locked })
238+
assert.equal(stuck.ok, false)
239+
assert.match((stuck as { error: string }).error, /index\.lock/)
240+
assert.equal(await readFile(join(fileBranchPath(repo, BRANCH), 'queue.md'), 'utf8'), '- once\n')
241+
} finally {
242+
await cleanup()
243+
}
244+
})
245+
213246
test('the eager pull converges a machine on what others pushed, and names a repo with no remote', async () => {
214247
const { repo, other, cleanup } = await initSyncedRepos()
215248
try {

‎packages/agent-data/src/file-branch.ts‎

Lines changed: 64 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -170,11 +170,26 @@ function resolveMessage(message: CommitMessage): string {
170170
return typeof message === 'function' ? message() : message
171171
}
172172

173+
/**
174+
* git's refusal when another process holds the checkout's index: the one failure two processes
175+
* on one clone (a daemon and a scheduler, each with its own in-process chain) hand each other.
176+
* git does not wait for the index lock, so the cycle waits instead and runs again.
177+
*/
178+
const INDEX_LOCK_RETRIES = 3
179+
const INDEX_LOCK_WAIT_MS = 500
180+
export function isIndexLocked(err: unknown): boolean {
181+
return /index\.lock['"]?: File exists/.test(errorMessage(err))
182+
}
183+
173184
/**
174185
* Apply one change to the branch through its persistent checkout: sync with origin, run `op`
175186
* against the checkout, commit whatever it changed, push. The single funnel a long-lived
176187
* process's writes go through.
177188
*
189+
* Another process holding the checkout's index (its `index.lock`) is waited out: the checkout
190+
* is reset, the cycle waits half a second and runs again, three times at most, before the
191+
* failure is reported like any other.
192+
*
178193
* `op` must be re-runnable: when the push loses a race with another writer, the cycle re-syncs
179194
* and runs it again against the fresher state rather than force-fitting a stale commit — the op
180195
* *is* the intent, the commit is just its serialization. Two attempts; a push that still fails
@@ -193,43 +208,59 @@ export async function withFileBranch(
193208
const r = resolveDeps(deps)
194209
const path = fileBranchPath(repo, branch)
195210
return serialize(repo, branch, async () => {
196-
try {
197-
await ensureCore(repo, branch, r)
198-
const remote = await hasRemote(repo, r.git)
199-
for (let attempt = 0; ; attempt++) {
200-
await syncCore(repo, branch, r)
201-
// The tip before this op's commit: what a lost push winds back to before re-applying, so
202-
// the op's first serialization is dropped rather than rebased under its second run.
203-
const before = (await r.git(['rev-parse', 'HEAD'], path)).trim()
204-
await op(path)
205-
await r.git(['add', '-A'], path)
206-
const staged = (await r.git(['status', '--porcelain'], path)).trim()
207-
if (staged) await r.git(['commit', '-m', resolveMessage(message)], path)
208-
if (!remote) return { ok: true, changed: Boolean(staged), pushed: false }
209-
// Unpushed commits — this cycle's, or an earlier cycle's that the sync just rebased. The
210-
// push is owed whenever any exist, even when this op itself wrote nothing new.
211-
const ahead = (await refExists(repo, `refs/remotes/origin/${branch}`, r.git))
212-
? (await r.git(['rev-list', '--count', `origin/${branch}..${branch}`], repo)).trim() !== '0'
213-
: true
214-
if (!ahead) return { ok: true, changed: false, pushed: false }
215-
try {
216-
await r.git(['push', 'origin', `${branch}:${branch}`], path)
217-
return { ok: true, changed: Boolean(staged), pushed: true }
218-
} catch (err) {
219-
if (attempt >= 1) return { ok: false, committed: true, error: `the ${branch} branch could not be pushed: ${errorMessage(err)}` }
220-
if (staged) await r.git(['reset', '--hard', before], path)
221-
}
222-
}
223-
} catch (err) {
224-
// The op's half-written files must not ride a later, unrelated commit: put the checkout
225-
// back to its committed state before reporting.
226-
await r.git(['reset', '--hard'], path).catch(() => {})
227-
await r.git(['clean', '-fd'], path).catch(() => {})
228-
return { ok: false, committed: false, error: errorMessage(err) }
211+
for (let locked = 0; ; locked++) {
212+
const outcome = await cycle(repo, branch, path, message, op, r)
213+
if (outcome.ok || !isIndexLocked(outcome.error) || locked >= INDEX_LOCK_RETRIES) return outcome
214+
await new Promise(resolve => setTimeout(resolve, INDEX_LOCK_WAIT_MS))
229215
}
230216
})
231217
}
232218

219+
/** One write cycle, unserialized: sync, apply, commit, push (twice on a lost race). Never throws. */
220+
async function cycle(
221+
repo: string,
222+
branch: string,
223+
path: string,
224+
message: CommitMessage,
225+
op: (dir: string) => Promise<void>,
226+
r: Resolved,
227+
): Promise<FileBranchWrite> {
228+
try {
229+
await ensureCore(repo, branch, r)
230+
const remote = await hasRemote(repo, r.git)
231+
for (let attempt = 0; ; attempt++) {
232+
await syncCore(repo, branch, r)
233+
// The tip before this op's commit: what a lost push winds back to before re-applying, so
234+
// the op's first serialization is dropped rather than rebased under its second run.
235+
const before = (await r.git(['rev-parse', 'HEAD'], path)).trim()
236+
await op(path)
237+
await r.git(['add', '-A'], path)
238+
const staged = (await r.git(['status', '--porcelain'], path)).trim()
239+
if (staged) await r.git(['commit', '-m', resolveMessage(message)], path)
240+
if (!remote) return { ok: true, changed: Boolean(staged), pushed: false }
241+
// Unpushed commits — this cycle's, or an earlier cycle's that the sync just rebased. The
242+
// push is owed whenever any exist, even when this op itself wrote nothing new.
243+
const ahead = (await refExists(repo, `refs/remotes/origin/${branch}`, r.git))
244+
? (await r.git(['rev-list', '--count', `origin/${branch}..${branch}`], repo)).trim() !== '0'
245+
: true
246+
if (!ahead) return { ok: true, changed: false, pushed: false }
247+
try {
248+
await r.git(['push', 'origin', `${branch}:${branch}`], path)
249+
return { ok: true, changed: Boolean(staged), pushed: true }
250+
} catch (err) {
251+
if (attempt >= 1) return { ok: false, committed: true, error: `the ${branch} branch could not be pushed: ${errorMessage(err)}` }
252+
if (staged) await r.git(['reset', '--hard', before], path)
253+
}
254+
}
255+
} catch (err) {
256+
// The op's half-written files must not ride a later, unrelated commit: put the checkout
257+
// back to its committed state before reporting.
258+
await r.git(['reset', '--hard'], path).catch(() => {})
259+
await r.git(['clean', '-fd'], path).catch(() => {})
260+
return { ok: false, committed: false, error: errorMessage(err) }
261+
}
262+
}
263+
233264
/** How a pull went: converged with origin, or why it could not. */
234265
export type FileBranchSync = { ok: true } | { ok: false; error: string }
235266

‎packages/framework/src/LOGIC.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Everything of The Framework that runs in Node: the `the-framework` command, the
3333
- **The daemon** (`daemon.ts`, `daemon-runtime.ts`, `daemon-services.ts`, `daemon-tick.ts`, `loopback-host.ts`, `project-hooks.ts`) - one foreground process that serves the dashboard, gives each agent a checkout [5] and a process, runs every background job on one clock, and runs the shell lines each project's own `.the-framework/hooks.yml` names when the dashboard opens and closes.
3434
- **Projects on this machine** (`registry.ts`, `project.ts`, `install.ts`, `layout.ts`, `framework-gitignore.ts`, `project-errors.ts`, `project-pass.ts`, `pick-directory.ts`, `config.ts`, `config-layers.ts`, `preference-defaults.ts`, `agent-options.ts`) - the one user file listing the projects and the preferences, what activating a repository does to it, the per-repository defaults that travel with the code, and the order in which the layers of a decision are resolved.
3535
- **Checkouts and what may be removed** (`worktrees.ts`, `merged-worktrees.ts`, `agent-locks.ts`) - one checkout [5] per agent under `.branches/`, and one rule for reclaiming it: only what is already on the remote may go.
36-
- **Spending** (`quota-boundary.ts`, `quota-poller.ts`, `maintenance.ts`, `handoff-level.ts`, `on-before-mergeable-prompt.ts`) - the share of the quota [6] week that may be spent by now, whether unattended work may start under it, the per-repository record of the last maintenance review, and the ladder a finished agent publishes itself by.
36+
- **Spending** (`quota-boundary.ts`, `quota-poller.ts`, `handoff-level.ts`, `on-before-mergeable-prompt.ts`) - the share of the quota [6] week that may be spent by now, whether unattended work may start under it, and the ladder a finished agent publishes itself by.
3737
- **Getting the work out** (`ci-watch.ts`, `update-check.ts`) - the pull requests The Framework opened are merged once their checks pass and fixed by an agent when they go red.
3838
- **Work that runs somewhere else** (`cloud-work.ts`, `cloud-run-state.ts`, `cloud-scratch-refs.ts`, `bridge-browser.ts`) - a Claude Code cloud session is recognized by the commit its branch descends from, its state is worded the same way on every surface, and the daemon can run its own browser so the bridge works with no Chrome of the user's open.
3939
- **The agent's browser** (`browser.ts`, `browser-stream.ts`) - a real Chrome an agent can drive, whose page the dashboard shows live and hands to the user at a login wall.

‎packages/framework/src/cli.ts‎

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -50,13 +50,6 @@ import { AgentMessageQueue } from './agent-messages.js'
5050
import { createGateKeepalive } from './gate-keepalive.js'
5151
import { ensureDaemonToken, readDaemonToken, readPreferences } from './registry.js'
5252
import { DEFAULT_SPEND_OFFSET } from './preference-defaults.js'
53-
import {
54-
planMaintenanceSweep,
55-
maintainSweep,
56-
mergeMaintenanceState,
57-
short,
58-
type RepoReview,
59-
} from './maintenance.js'
6053
import {
6154
listProjectWorktrees,
6255
removeProjectWorktree,

‎packages/framework/src/maintenance.BUG-ANALYSIS.md‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)