Skip to content

Commit 63ed6a2

Browse files
committed
chore(gates): drop the test-file size pin map, keep the merge-base ratchet
The exact-length pin map duplicated what the merge-base already records and made every shrink a two-file edit. The gate now has one rule: a test file over the 1,000-line tripwire may be no longer than at the merge-base with origin/main, and no new test file may cross the tripwire.
1 parent e624ef9 commit 63ed6a2

3 files changed

Lines changed: 42 additions & 173 deletions

File tree

docs/agents/testing.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -178,8 +178,9 @@ or remove flakes.
178178
along source topology is a performance win, not just a readability win.
179179
- The slow-test reporter enforces unit and integration budgets. Existing pins only shrink; a new
180180
pin needs measured justification.
181-
- Test files over 1,000 lines are pinned to their merge-base size and may only shrink. Split the
182-
family before adding tests; never raise the pin.
181+
- Test files over 1,000 lines may be no longer than at the merge-base with `origin/main`, and no
182+
new test file may cross that line. Split the family before adding tests; shrinking needs no
183+
gate edit.
183184
- Keep isolation enabled and the pool on forks — both alternatives were measured and did not help.
184185
The useful optimization is importing the module under test, not a platform barrel.
185186
- Local Vitest runs use a four-worker cap. Override it when a run needs a different host share:

scripts/__tests__/eager-closure-budgets.ts

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,8 @@ export type EagerClosureBudget = {
3737
* A `<=` ceiling looks stricter than it is: the moment an entry legitimately shrinks, the
3838
* unchanged row silently becomes headroom, and the next regression up to the old number passes
3939
* unnoticed. Equality is what "only ever ratchets down" actually requires -- the same shape
40-
* `test-file-size-ratchet.test.ts` uses for file length and R9/R10 use for cycle size and
41-
* writer counts: growing fails, and shrinking ALSO fails until the row is lowered in the same
42-
* PR, so the gain is kept rather than banked as slack.
40+
* R9/R10 use for cycle size and writer counts: growing fails, and shrinking ALSO fails until
41+
* the row is lowered in the same PR, so the gain is kept rather than banked as slack.
4342
*
4443
* Seeded from measurement, never rounded up. The regression this catches is a single static
4544
* import dragging a subtree in, measured by #1969 at 5-12% of the whole suite's import work

scripts/__tests__/test-file-size-ratchet.test.ts

Lines changed: 37 additions & 168 deletions
Original file line numberDiff line numberDiff line change
@@ -5,60 +5,27 @@ import { walkFiles } from '../lib/walk-files.ts';
55
import { runCmdSync } from '@agent-device/host-kit/command';
66

77
/**
8-
* Test-file size ratchet (AGENTS.md "Scope & shape": past 1,000 lines is architecture debt,
9-
* and tests are not exempt; the topology rule says a test file mirrors its source module and
10-
* splits when the source does).
8+
* Test-file size ratchet (AGENTS.md "Module and test topology": past 1,000 lines is architecture
9+
* debt, and tests are not exempt; a test file mirrors its source module and splits when the
10+
* source does).
1111
*
1212
* The slow-test ratchet keeps the unit suite's wall clock honest; this one keeps its files
13-
* readable in one bounded read. Every test file over the tripwire is pinned at its exact
14-
* length, R9-style (#1781 A6): growing a pinned file fails ("split it, don't add to it"), and
15-
* shrinking one fails until the pin is lowered, so the list only ever ratchets down. A file
16-
* that drops under the tripwire leaves the list; a new file may not cross it.
13+
* readable in one bounded read. The rule is history-backed: every test file over the tripwire
14+
* may be no longer than it was at the merge-base with origin/main, and a file that did not
15+
* exist there may not cross the tripwire. Shrinking needs no gate edit; the merge-base is the
16+
* only record of the previous length.
1717
*
18-
* The pin map alone could be edited alongside the file (raise a pin and grow into it; add a
19-
* pin with a new giant file), so the gate is history-backed as well: every test file over the
20-
* tripwire may be no longer than it was at the merge-base with origin/main (or no longer than
21-
* the tripwire if it did not exist there), and no pin may exceed its file's base length. Both
22-
* pin-edit bypasses go red against git, not against the map.
23-
*
24-
* Catches: a >1,000-line test file growing (with or without a matching pin edit), or a new one
25-
* appearing (with or without a pin).
18+
* Catches: a >1,000-line test file growing, or a new one appearing.
2619
* Evidence: 26 test files were over the line when this landed (2026-08-18); the largest,
2720
* `snapshot-handler.test.ts`, gained 55 lines in the PR before, under a rule with no gate.
28-
* Cost: one directory walk and a line count per test file — well under a second.
29-
* Kill criterion: the pin list is empty. Delete this file with the last pin.
21+
* Cost: one directory walk, a line count per test file, and one `git cat-file --batch` spawn
22+
* for the files over the tripwire — well under a second.
23+
* Kill criterion: no test file under src/, packages/, test/, or scripts/ exceeds the tripwire at
24+
* the merge-base; delete this file when that holds.
3025
*/
3126

3227
const TRIPWIRE_LINES = 1_000;
3328

34-
// Exact current lengths. Lower a pin when its file shrinks; never raise one — extract instead.
35-
const PINNED_TEST_FILE_LINES: Readonly<Record<string, number>> = Object.freeze({
36-
'src/__tests__/remote-connection.test.ts': 2973,
37-
'src/daemon/handlers/__tests__/snapshot-handler.test.ts': 2120,
38-
'src/commands/interaction/runtime/settle.test.ts': 2359,
39-
'src/daemon/replay/internal/__tests__/session-replay-runtime-maestro.test.ts': 1963,
40-
'packages/platform-apple/src/runner/__tests__/runner-session.test.ts': 1957,
41-
'src/daemon/client/__tests__/daemon-client.test.ts': 1873,
42-
'packages/platform-android/src/__tests__/snapshot.test.ts': 1435,
43-
'packages/platform-apple/src/runner/__tests__/runner-client.test.ts': 1441,
44-
'src/__tests__/client.test.ts': 1554,
45-
'test/integration/provider-scenarios/android-lifecycle.test.ts': 1556,
46-
'src/daemon/client/__tests__/daemon-client-lifecycle.test.ts': 1409,
47-
'packages/platform-apple/src/runner/__tests__/runner-command-retry.test.ts': 1280,
48-
'src/__tests__/cli-client-commands.test.ts': 1304,
49-
'src/__tests__/cli-config.test.ts': 1282,
50-
'src/daemon/interaction/internal/__tests__/find.test.ts': 1199,
51-
'packages/platform-apple/src/core/__tests__/perf.test.ts': 1222,
52-
'src/mcp/__tests__/command-tools.test.ts': 1216,
53-
'src/daemon/replay/internal/__tests__/session-replay-divergence.test.ts': 1100,
54-
'packages/platform-apple/src/core/__tests__/apps.test.ts': 1146,
55-
'src/daemon/replay/internal/__tests__/session-replay-repair-transaction.test.ts': 1202,
56-
'src/daemon/replay/internal/__tests__/session-replay-target-verification-runtime.test.ts': 1182,
57-
'src/__tests__/client-metro.test.ts': 1105,
58-
'src/__tests__/cli-network.test.ts': 1092,
59-
'packages/platform-android/src/__tests__/snapshot-helper.test.ts': 1002,
60-
});
61-
6229
const REPO_ROOT = path.resolve(import.meta.dirname, '..', '..');
6330
const TEST_ROOTS = ['src', 'packages', 'test', 'scripts'];
6431
const SKIPPED_DIRECTORIES = new Set(['node_modules', 'dist', 'dist-types', '.tmp']);
@@ -79,56 +46,12 @@ function countLines(file: string): number {
7946
return lines;
8047
}
8148

82-
/** The ratchet decision, separated from the filesystem so the tests below can plant each red. */
83-
function ratchetFindings(
84-
measured: ReadonlyMap<string, number>,
85-
pinned: Readonly<Record<string, number>>,
86-
tripwire: number,
87-
): string[] {
88-
const findings: string[] = [];
89-
for (const [file, lines] of [...measured].sort()) {
90-
const pin = pinned[file];
91-
if (pin === undefined) {
92-
if (lines > tripwire) {
93-
findings.push(
94-
`${file} is ${lines} lines, over the ${tripwire}-line tripwire and not pinned: split it ` +
95-
`along the source module it mirrors (docs/agents/testing.md) rather than pinning it.`,
96-
);
97-
}
98-
continue;
99-
}
100-
if (lines <= tripwire) {
101-
// Pins exist only for files over the tripwire: one on a smaller file grows the map for
102-
// nothing (900 pinned at 900 would satisfy equality and history alike) and defeats the
103-
// only-shrink kill criterion.
104-
findings.push(
105-
`${file} is ${lines} lines, at or under the ${tripwire}-line tripwire, but has a pin (${pin}): remove it — pins are only for files over the tripwire.`,
106-
);
107-
continue;
108-
}
109-
if (lines > pin) {
110-
findings.push(
111-
`${file} grew to ${lines} lines (pinned ${pin}): extract instead of adding to a file over the tripwire.`,
112-
);
113-
} else if (lines < pin) {
114-
findings.push(
115-
`${file} shrank to ${lines} lines (pinned ${pin}): lower its pin in this PR so the ratchet keeps the gain.`,
116-
);
117-
}
118-
}
119-
for (const file of Object.keys(pinned)) {
120-
if (!measured.has(file)) {
121-
findings.push(`${file} is pinned but does not exist: remove its pin.`);
122-
}
123-
}
124-
return findings;
125-
}
126-
12749
/**
12850
* Line counts of the given repo paths at the merge-base with origin/main, following renames, in
12951
* one `git cat-file --batch` spawn. `undefined` = the file did not exist there.
13052
*/
13153
function baseLineCounts(paths: readonly string[]): ReadonlyMap<string, number | undefined> {
54+
if (paths.length === 0) return new Map();
13255
const mergeBase = runCmdSync('git', ['merge-base', 'origin/main', 'HEAD'], {
13356
cwd: REPO_ROOT,
13457
allowFailure: true,
@@ -191,13 +114,9 @@ function parseCatFileBatch(
191114
return counts;
192115
}
193116

194-
/**
195-
* The history-backed half: measured against the merge-base, not against the pin map, so
196-
* editing the map alongside the file cannot admit growth.
197-
*/
198-
function historyFindings(
117+
/** The ratchet decision, separated from the filesystem so the test below can plant each red. */
118+
function ratchetFindings(
199119
measured: ReadonlyMap<string, number>,
200-
pinned: Readonly<Record<string, number>>,
201120
baseLines: ReadonlyMap<string, number | undefined>,
202121
tripwire: number,
203122
): string[] {
@@ -207,112 +126,62 @@ function historyFindings(
207126
const base = baseLines.get(file);
208127
if (base === undefined) {
209128
findings.push(
210-
`${file} is ${lines} lines and did not exist at the merge-base: a new test file may not cross the ${tripwire}-line tripwire, pinned or not.`,
129+
`${file} is ${lines} lines and did not exist at the merge-base: a new test file may not cross the ${tripwire}-line tripwire.`,
211130
);
212131
} else if (lines > Math.max(base, tripwire)) {
213132
findings.push(
214-
`${file} is ${lines} lines, ${base} at the merge-base: a test file over the tripwire may not grow, whatever its pin says.`,
215-
);
216-
}
217-
}
218-
for (const [file, pin] of Object.entries(pinned).sort()) {
219-
const base = baseLines.get(file);
220-
if (base !== undefined && pin > base) {
221-
findings.push(
222-
`${file} is pinned at ${pin} but was ${base} lines at the merge-base: a pin may not be raised above its file's base length.`,
133+
`${file} is ${lines} lines, ${base} at the merge-base: a test file over the tripwire may not grow; split it along the source module it mirrors (docs/agents/testing.md).`,
223134
);
224135
}
225136
}
226137
return findings;
227138
}
228139

229-
test('no test file over the tripwire grows, and every pin matches its file exactly', () => {
140+
test('no test file over the tripwire is longer than at the merge-base, and no new file crosses it', () => {
230141
const measured = new Map<string, number>();
231142
for (const root of TEST_ROOTS) {
232143
for (const file of walkFiles(path.join(REPO_ROOT, root), isTestFile)) {
233144
measured.set(path.relative(REPO_ROOT, file).split(path.sep).join('/'), countLines(file));
234145
}
235146
}
236147
expect(measured.size).toBeGreaterThan(500);
237-
expect(ratchetFindings(measured, PINNED_TEST_FILE_LINES, TRIPWIRE_LINES)).toEqual([]);
238-
239-
const ofInterest = [
240-
...new Set([
241-
...[...measured].filter(([, lines]) => lines > TRIPWIRE_LINES).map(([file]) => file),
242-
...Object.keys(PINNED_TEST_FILE_LINES),
243-
]),
244-
];
245-
const baseLines = baseLineCounts(ofInterest);
246-
expect(historyFindings(measured, PINNED_TEST_FILE_LINES, baseLines, TRIPWIRE_LINES)).toEqual([]);
247-
});
248148

249-
test('planted reds: growth, shrink, unpinned crossing, and a stale pin each name their fix', () => {
250-
const pinned = { 'a.test.ts': 1200, 'b.test.ts': 1500, 'e.test.ts': 900, 'gone.test.ts': 1100 };
251-
const measured = new Map([
252-
['a.test.ts', 1201], // grew
253-
['b.test.ts', 900], // shrank under the tripwire: the pin must go
254-
['e.test.ts', 900], // unchanged sub-tripwire file that someone pinned at its own length
255-
['c.test.ts', 1001], // new offender
256-
['d.test.ts', 1000], // at the line, fine
257-
]);
258-
expect(ratchetFindings(measured, pinned, 1000)).toEqual([
259-
'a.test.ts grew to 1201 lines (pinned 1200): extract instead of adding to a file over the tripwire.',
260-
'b.test.ts is 900 lines, at or under the 1000-line tripwire, but has a pin (1500): remove it — pins are only for files over the tripwire.',
261-
'c.test.ts is 1001 lines, over the 1000-line tripwire and not pinned: split it along the source module it mirrors (docs/agents/testing.md) rather than pinning it.',
262-
// The arbitrary-new-pin bypass: equality (900 == 900) and history (900 <= base) both pass,
263-
// so this rule is the one that rejects it.
264-
'e.test.ts is 900 lines, at or under the 1000-line tripwire, but has a pin (900): remove it — pins are only for files over the tripwire.',
265-
'gone.test.ts is pinned but does not exist: remove its pin.',
266-
]);
267-
expect(ratchetFindings(new Map([['a.test.ts', 1200]]), { 'a.test.ts': 1200 }, 1000)).toEqual([]);
149+
const overTripwire = [...measured]
150+
.filter(([, lines]) => lines > TRIPWIRE_LINES)
151+
.map(([file]) => file);
152+
const baseLines = baseLineCounts(overTripwire);
153+
expect(ratchetFindings(measured, baseLines, TRIPWIRE_LINES)).toEqual([]);
268154
});
269155

270-
test('planted reds against history: raising a pin, growing into it, and pinning a new giant file are all red', () => {
156+
test('planted reds: growth, a new giant file, and a file crossing the tripwire each name their fix', () => {
271157
const baseLines = new Map<string, number | undefined>([
272-
['a.test.ts', 1200], // existed, 1200 at base
158+
['a.test.ts', 1200],
273159
['b.test.ts', 1500],
274-
['fresh.test.ts', undefined], // did not exist at base
275-
['small.test.ts', 900], // existed, under the tripwire at base
160+
['fresh.test.ts', undefined],
161+
['small.test.ts', 900],
276162
]);
277-
// Bypass 1: grow a pinned file and raise its pin so the equality pin stays green.
278-
expect(
279-
historyFindings(new Map([['a.test.ts', 1230]]), { 'a.test.ts': 1230 }, baseLines, 1000),
280-
).toEqual([
281-
'a.test.ts is 1230 lines, 1200 at the merge-base: a test file over the tripwire may not grow, whatever its pin says.',
282-
"a.test.ts is pinned at 1230 but was 1200 lines at the merge-base: a pin may not be raised above its file's base length.",
163+
expect(ratchetFindings(new Map([['a.test.ts', 1230]]), baseLines, 1000)).toEqual([
164+
'a.test.ts is 1230 lines, 1200 at the merge-base: a test file over the tripwire may not grow; split it along the source module it mirrors (docs/agents/testing.md).',
283165
]);
284-
// Raising the pin alone (before growing into it) is already red.
285-
expect(
286-
historyFindings(new Map([['a.test.ts', 1200]]), { 'a.test.ts': 1230 }, baseLines, 1000),
287-
).toEqual([
288-
"a.test.ts is pinned at 1230 but was 1200 lines at the merge-base: a pin may not be raised above its file's base length.",
166+
expect(ratchetFindings(new Map([['fresh.test.ts', 1400]]), baseLines, 1000)).toEqual([
167+
'fresh.test.ts is 1400 lines and did not exist at the merge-base: a new test file may not cross the 1000-line tripwire.',
289168
]);
290-
// Bypass 2: add a new >1,000-line file together with a pin for it.
291-
expect(
292-
historyFindings(new Map([['fresh.test.ts', 1400]]), { 'fresh.test.ts': 1400 }, baseLines, 1000),
293-
).toEqual([
294-
'fresh.test.ts is 1400 lines and did not exist at the merge-base: a new test file may not cross the 1000-line tripwire, pinned or not.',
295-
]);
296-
// Same for a file that existed but was under the tripwire at base.
297-
expect(
298-
historyFindings(new Map([['small.test.ts', 1001]]), { 'small.test.ts': 1001 }, baseLines, 1000),
299-
).toEqual([
300-
'small.test.ts is 1001 lines, 900 at the merge-base: a test file over the tripwire may not grow, whatever its pin says.',
301-
"small.test.ts is pinned at 1001 but was 900 lines at the merge-base: a pin may not be raised above its file's base length.",
169+
expect(ratchetFindings(new Map([['small.test.ts', 1001]]), baseLines, 1000)).toEqual([
170+
'small.test.ts is 1001 lines, 900 at the merge-base: a test file over the tripwire may not grow; split it along the source module it mirrors (docs/agents/testing.md).',
302171
]);
303-
// Allowed: shrink with a lowered pin, a pin that disappears, an unchanged file.
172+
// Allowed: unchanged, shrunk (with no gate edit), under the tripwire, or new and small.
304173
expect(
305-
historyFindings(
174+
ratchetFindings(
306175
new Map([
307176
['a.test.ts', 1100],
308177
['b.test.ts', 1500],
178+
['small.test.ts', 1000],
179+
['fresh.test.ts', 1000],
309180
]),
310-
{ 'a.test.ts': 1100, 'b.test.ts': 1500 },
311181
baseLines,
312182
1000,
313183
),
314184
).toEqual([]);
315-
expect(historyFindings(new Map([['a.test.ts', 950]]), {}, baseLines, 1000)).toEqual([]);
316185
});
317186

318187
test('cat-file --batch output is parsed per request, in order, with misses as undefined', () => {

0 commit comments

Comments
 (0)