Bug: a hunk header that over-declares its line counts is discarded, and misreported as a truncated reply

[thedancingdeveloper]

Problem

When the implementer's reply is cut off mid-hunk, the harness correctly
detects it — and then fails the whole item. The model call itself returned
200, so the retry ladder never engages; validate_diff reports a fatal
patch problem and _execute returns.

One truncated response therefore costs a full item: the planner call, the
implementer call, and an attempt off the item's budget. Recovering it takes a
manual requeue, which re-plans and re-implements from scratch.

Why that is the wrong call

Truncation is the most obviously retryable failure there is. It says nothing
about the prompt and nothing about the model's suitability — the reply simply
stopped. It is not the same class of thing as "the model returned prose
instead of a diff", which is what the fatal path was built for.

Compare what happens either side of it: an HTTP 503 is retried up to six times
with backoff, and now also falls through to the next model in the chain
(#141). A reply that arrives complete but unusable fails once and stops. The
harness is most patient with the failure it can do least about, and least
patient with the one a single re-ask usually fixes.

Observed

Three times in one evening's runs, on the same item, across two different
models:

T3 patch_malformed — line 21: the patch ends mid-hunk — the reply was truncated
   FAIL T3: plan -> implement -> parse
   the implementer did not produce a usable diff — line 21: the patch ends mid-hunk

T2 patch_malformed — line 17: the patch ends mid-hunk — the reply was truncated
   FAIL T2: plan -> implement -> parse

Each was recovered by requeue + a fresh run, i.e. by paying for the whole
item again. The second and third occurrences were on gpt-5.4 reached through
a fallback chain, so this is not one bad model.

What would fix it

Treat "the reply was truncated" as a retryable call failure rather than a
fatal item failure:

• re-ask the implementer, at the call level, so the planner's output is not
  thrown away;
• with a chain configured, prefer the next route on the retry — a model that
  truncated once on a given prompt is a reasonable thing to route around;
• keep the existing fatal path for replies that are malformed in ways a
  re-ask will not fix (no hunk header at all, prose instead of a diff).

PatchProblem already carries fatal, and the truncation case is already
detected and named precisely — the classification simply sits on the wrong
side of the line. Worth also capping how many re-asks one item may spend, so a
model that truncates every time still terminates.

[thedancingdeveloper]

Correcting this issue: the reply is not truncated, and the title is wrong. I filed it from the harness's own message without reading the preserved patch. Having read it, the diagnosis is different and more actionable.

The preserved patch is complete and correct prose:

@@ -1,10 +1,21 @@
-from calc import add, subtract
+import pytest
+
+from calc import add, divide, subtract
 ...
+def test_divide_by_zero():
+    with pytest.raises(ZeroDivisionError, match="cannot divide by zero"):
+        divide(1, 0)

Nothing stopped mid-sentence. What is wrong is the hunk header:

	old	new
header declares	10	21
body supplies	9	20

test_calc.py has 9 lines. The model over-declared by exactly one on both sides — it counted the trailing newline as a tenth line. The patch is 38 lines and git reports corrupt patch at line 39: it parsed by the declared counts and ran out of input.

Two real defects, neither of which is the model truncating.

1. The diagnostic says "truncated" for a reply that is complete

line 17: the patch ends mid-hunk — the reply was truncated sent me looking for a token limit. There isn't one: the call used 266 output tokens. The message should say what is actually true — that the hunk header declares more lines than the body supplies — because that names a different problem with a different fix.

2. The ladder does not rescue the case it says it rescues

APPLY_LADDER documents wrong hunk line numbers → --unidiff-zero, and PatchProblem.fatal distinguishes damage "routinely repaired, notably hunk headers whose line counts are wrong" from damage no rung can fix. That holds for under-declared counts, which is what test_a_hand_written_hunk_header_is_rescued covers. It does not hold for over-declared counts: git hits EOF before any tolerance applies, and --unidiff-zero fails identically to strict git apply.

So a patch that is byte-for-byte correct in its body is discarded over an arithmetic error in its header.

What would fix it

Recompute the hunk counts from the body before applying. The body is the truth — the counts are derivable from it, deterministically, with no guessing about placement (unlike #133, where the header carried the only placement information and there was nothing to recompute). That rescues this patch and every one like it.

Re-asking the implementer, as originally suggested, is a strictly worse fix here: it pays for another call to work around arithmetic the harness can do itself.

Reproduced on four consecutive runs of the same item, across two models — so this is a common model error, not one bad response.