Skip to content

Latest commit

 

History

History
112 lines (93 loc) · 5.79 KB

File metadata and controls

112 lines (93 loc) · 5.79 KB

09 — The Mixed-Unit A4 Test: The Whole A4 Arc Was Surface Behaviour

Headline. Stating the budget in packs decouples value-parity (λ = 1) from numeral-equality — and the decoupling dissolves everything the previous two docs reported about A4:

  • 7B, identical displays, 5× different real stakes-per-credit (C2: λ=1 vs C4: λ=5): spend 0.582 vs 0.582 — indistinguishable.
  • 7B, equal-numeral displays, 5× different λ (C1: λ=1 vs C3: λ=0.2): 0.704 vs 0.683 — a 0.021 gap against numeral effects of +0.12/+0.24.
  • 3B, fixed pack count, λ ∈ {0.2, 1, 5}: spend 0.241 / 0.240 / 0.235 — completely stakes-blind.

When display and value are separated, λ has no measurable effect on either model. Doc 08's parity peak was numeral-parity; and therefore doc 07/08's "falsification of monotone arousal" was itself an artifact — a numeral effect read as a value effect. The honest final status of A4 at this capability level is inconclusive: the subjects never engaged the value-state the test is about. The kill has been killed.

1. The design and the data

Budget stated as P credit-packs of s credits each (E = P·s), stakes as K points; spend fraction = packs spent / packs held; 16 seeded scenarios per cell (python3 sim/real/a4_mixed_unit.py, 159/160 parsed).

cell K pts packs × size λ = K/E numerals 3B spend 7B spend
C1 300 300 × 1 1.00 equal 0.495 ± .022 0.704 ± .034
C2 300 60 × 5 1.00 unequal 0.240 ± .004 0.582 ± .035
C3 60 60 × 5 0.20 equal 0.241 ± .004 0.683 ± .046
C4 300 60 × 1 5.00 unequal 0.235 ± .009 0.582 ± .035
C5 60 300 × 1 0.20 unequal 0.380 ± .027 0.444 ± .017

The three contrasts the design was built for (7B):

  • Value-parity effect (C2 vs mean(C4, C5), λ = 1 vs λ ≠ 1 at fixed unequal display): +0.069 — and even this is carried entirely by C5; against the exact C2 = C4 tie it cannot be read as a λ effect.
  • Numeral effect at λ = 1 (C1 − C2): +0.122.
  • Numeral effect at λ = 0.2 (C3 − C5): +0.240.

The 3B's structure is purer still: its spend fraction is a function of the pack-count numeral (60-pack cells all ≈ 0.24; 300-pack cells 0.38–0.50), with stakes contributing nothing at all.

2. What this retracts, and in what order

The A4 arc across three runs, stated plainly:

  1. Doc 07 reported the arousal identity falsified and read the data as "K and E as separate registers."
  2. Doc 08 corrected the reading (ratio-dependence held across a 9× rescale) but kept the kill: "monotone-λ arousal stays dead," with a parity-peaked spend curve as the surviving shape.
  3. This doc retracts the kill itself. In a same-unit design, every λ-relation is also a numeral-relation, so doc 08's rescaling — which preserved both — could not tell them apart. Once separated: the parity peak follows the numerals (C1, C3 high; C2, C4 low, regardless of λ), and no λ-response remains anywhere. The "falsification of monotone arousal" was a numeral artifact, not a measurement of any value-state.

The scoreboard row is accordingly rewritten: not falsified, not supported — inconclusive, because the instrument bottomed out at the agents' shallowness. These models' allocation behaviour reads the prompt's surface statistics; they do not maintain the resource-stakes state that A4 is a test of. A test of arousal needs a subject that has arousal's prerequisites.

Two cautions against comfortable readings:

  • This is not evidence for the theory. "The subjects lacked the state" is the epistemically weakest escape available, and it is earned here only because the C2 = C4 tie demonstrates the insensitivity directly, not because the theory needs rescuing.
  • It does convert the capability-grading observed in docs 07–08 into a methodological law for this repo: at 3B–7B scale, stated stakes and budgets are cheap talk. Every A4-family result on such models measures numeral statistics unless proven otherwise.

3. The fix: real budgets, not stated ones

The root failure is that K and E were asserted in text rather than borne by the agent. The redesign that survives this critique makes both load-bearing:

  • Real E: an enforced token/compute budget (max_tokens the model must ration across sub-tasks it actually executes), not a number in the prompt.
  • Real K: stakes that gate something the model's run actually optimizes (score on a benchmark it is being evaluated on, retries it does or does not receive).
  • Frontier-scale subjects, where stated-stakes sensitivity is known to be measurable at all.

This is now the ROADMAP's A4 entry. Until it runs, the dictionary's arousal row carries the honest label: untested — all three attempted operationalizations measured the prompt, not the agent.

4. What survives from the three A4 runs

  • The methodological result, which is real and transferable: same-unit designs cannot test ratio-dependence claims on LLMs; numeral relations must be decoupled by construction, and 3B–7B models fail even the decoupled version's engagement precondition.
  • The capability ladder observation, now in its strongest form: the 3B is stakes-blind and pack-count-driven; the 7B adds a numeral-equality response and mild display sensitivity; neither reaches value-state behaviour. Lawful structure arrives with capability — and so does the eligibility to be a subject of this theory's tests.
  • The L1/L2 results of doc 07 are untouched by this retraction: Z there was the model's actual correctness and A its actual report — nothing was asserted-in-text — and the alexithymia bound and appetitive-arm dissociation stand as measured.