Agentic-loop data point: thinking off produced zero edits where thinking on produced edits (n=1 pair)

[Defilan]

A single controlled pair on the question your §5d/§2 material leaves open:
does turning thinking off help or hurt on a long agentic coding task?

Everything we published before was single-turn probes. This is the same task run
twice through a real 100+ turn coder loop, with only enable_thinking differing.

Setup

Same issue (a controller ownership bug in our repo), same harness build, same
max_tokens (16384), same context budget (90k), same model, same prompt. The
only change between runs is the kwarg.

Both runs happened after we fixed two harness defects that were previously
confounding this task, so neither is fighting a broken loop.

	thinking OFF (enable_thinking: false)	thinking ON (kwarg absent)
reasoning chars in transcript	0	1,340
turns	41	54
edits made	0	2
first edit	never	turn 35
tool mix	grep 22, read_file 17, bash 2	grep 36, bash 16, read_file 7, str_replace 2
stuck-loop detector that fired	EditFreeStreak	RepeatedToolCall
wall clock	112 min	21.6 min

The one thing worth taking from it

With thinking off it never edited anything. 41 turns of grep and read_file,
then the harness killed it for going too long without a write. With thinking on,
same task, it committed to an edit at turn 35 and made two.

That points the opposite way to "thinking-ON is net-negative on long coding
work". At least on this task, the reasoning block is what let it stop surveying
and act.

What this is not

Both runs failed. Thinking-on did not solve the issue either; it edited, then
got stuck re-running tests (the last 16 calls are all bash). So this is not
"thinking on wins", it is "thinking off removed the ability to start".

More importantly: this is n=1 per arm. I flagged your attenuation number as
resting on a single run pair and said I would not lean on it. The same standard
applies here and I am not going to pretend otherwise. Two runs is an
observation, not a result.

The wall-clock difference is confounded too and I would ignore it: the
thinking-off run grew a much larger context, so its turns were individually far
slower. It is not evidence that thinking-off is slow.

Why I am sending it anyway

It is the only agentic-loop data any of the three of us has on this, and it
points against the current guidance rather than for it. If it were the other way
round I would still want to see it, so here it is. Worth a line in §5d as an
unreplicated counter-observation at most, and I would not put it in a table
alongside your scenario battery.

Happy to run more pairs if the direction is worth pinning down. It is roughly
two hours of hardware per pair, so a proper n would take a while.

[Blackwellboy]

Possibly relevant to your ON arm: with the kwarg absent and no reasoning-resend, every prior turn renders an empty <think></think> on this template family, which suppresses firing hard in accumulated sessions (#16 — stripped 0/10 vs preserved 10/10 on identical transcripts). 1,340 reasoning chars over 54 turns is consistent with that stripped-arm suppression; a resend-enabled re-run might separate "thinking helped" from "thinking barely survived."