Epic: Stabilize per-turn extraction token demand at quality parity (bounded growth over 300+ turns)

[daviburg]

Epic: Stabilize per-turn extraction token demand at quality parity

Ultimate goal: Per-turn extraction token demand must STABILIZE as a session grows to 300+ turns (NOT grow unbounded) WHILE preserving near-ideal semantic extraction quality.

This is the permanent tracker for the per-turn extraction token-stabilization work over the coming weeks. All findings below were established from a 344-turn run (eval-qwen36-344t-full), re-derived from code + logs by the token-economist and quality-analyst specialists.

Predecessor / lineage: #385 (Context budget for relationship mapper and entity detail). Related: #447 (strategic-redirection epic), #460 and #468 (mistargeted compression -- see Decision below).

---

Root cause: the runaway

• Per-turn total input tokens grow LINEARLY UNBOUNDED: approximately 15182 + 88.6 * turn_index, with NO plateau (~16K at turn 30 -> ~42K at turn 300 -> ~46K at turn 344).
• There is NO rolling transcript/history window -- each LLM phase uses only the current turn's text. History is NOT the driver.
• DRIVER = the entity_detail phase (64% of load, ~9.8K -> ~27.8K/turn):
  1. Detail-call count rises 3.25 -> 5.93/turn toward the cap _MAX_DETAIL_ENTITIES_PER_TURN=6 (tools/semantic_extraction.py ~L1161).
  2. The PC (player character) prior-state is UNCAPPED -- _format_prior_entity_context (~L1500-1610) caps non-PC relationships at _SCENE_MAX_RELATIONSHIPS=8, but the player keeps ALL relationships (109 at turn 344, 100% active, 0 ever resolved), and the PC appears in nearly every turn.
  3. ~15-17K tokens/turn of PURE REPETITION -- the system template + full turn text is re-sent across ~8.9 LLM calls/turn at scale.
• Secondary: relationship_mapper grows 2.3K -> 7K/turn (near its 0.2 * ctx budget).

This epic CONTINUES #385, which capped NON-PC entity detail + the relationship mapper but LEFT THE PC PATH UNCAPPED. That gap is the dominant remaining driver.

---

Decision on #460 / PR #468

The "adaptive pressure-gated context compression" (#460 / PR #468) is MISTARGETED:

• It gates only the known-entities catalog block (~13% of load, dormant in the A/B, <4% savings even if engaged).
• It is latently quality-risky: the catalog block is the coreference anchor; dropping an entry causes the entity to be re-extracted as a NEW duplicate.

Resolution:

• The GATE is being DROPPED.
• The INSTRUMENTATION (per-turn / per-phase token metrics) is being SALVAGED into a separate instrumentation-only PR (orchestrator task already queued).
• Smart context compression (#393) regresses entity retention — rework before merge #460 / feat: adaptive pressure-gated context compression #468 to be closed/repurposed.

---

Levers (checklist, ranked by quality-safety)

A deep data-driven design report is being authored before implementation; it will refine L1/L2 scoping. Each lever is evaluated independently.

• L1 -- Type-tiered PC relationship cap (SHIP-FIRST, low risk). Keep permanent bonds uncapped (kinship / adversarial / mentorship / political / captor-debt); recency + mention-filter the volatile spatial/social tail (~cap 10); force-keep anything mentioned this turn. PC web 109 -> ~30-40. MUST tier by relationship TYPE (0 rels ever resolve, so recency/status is an unreliable freshness signal). Needs a thin callback-retention A/B.
• L4 -- relationship_mapper budget reuses L1 type-tiering (ships with L1, low risk / low yield).
• L2 -- Batch entity_detail calls to kill the ~15K/turn template + turn-text REPETITION (BIGGEST token win). Risk: cross-entity attribute bleed -> keep PC + new + high-confidence SOLO, batch only the low-salience existing tail 2-3/call with hard-delimited per-entity blocks. NEEDS an A/B before trust.
• L3 -- Do NOT lower the detail cap (rejected as a token lever). It only bites on busy high-info turns where dropping a detail pass is most costly; 525 updates were already dropped at cap=6. Instead FIX entity SELECTION: prioritize by state-change-this-turn + mention-salience + staleness, not discovery-time confidence.

---

Deliverables

• Salvage feat: adaptive pressure-gated context compression #468 instrumentation into an instrumentation-only PR (orchestrator task queued).
• Deep data-driven design report (per-phase token breakdown + per-lever target mapping + honest expected-return estimates) -- being authored.
• Add a token-budget / lever-analysis visualization panel to the Extraction KPI Dashboard (private repo saas/orchestrator/dashboard.py) as a permanent reference of the initial ESTIMATES to compare against later MEASURED impact.
• Fix the KPI dashboard baseline: the headline baseline is the 100-turn slice (eval-qwen36-100t-temp0-clean), which UNDERSTATES the runaway (only visible at scale). Promote the 344-turn full run (eval-qwen36-344t-full) as the meaningful baseline + add a token-growth-trend KPI.
• Implement levers L1 + L4, then L2 (each with the standard A/B per docs/ab-test-standard.md), measuring actual token reduction vs the dashboard estimates, at quality parity (entity-set retention, no new duplicates / phantoms).

---

Success criteria

Per-turn token demand flattens (asymptotically bounded, not linearly growing) across a 300+ turn session, with extraction quality at parity (no entity drops, no new duplicates, no attribute corruption) vs the 344-turn baseline (eval-qwen36-344t-full).

---

Notes

• Sub-issues are intentionally NOT created yet; the design report will refine L1 / L2 scoping first. This epic holds the checklist.
• References: perf: per-turn extraction degrades linearly with catalog growth (7.5x at turn 300) #385 (predecessor), Smart context compression (#393) regresses entity retention — rework before merge #460, feat: adaptive pressure-gated context compression #468, docs: establish source-quality-first principle (extend Rule 9 to thresholds and sweeps) #447 (strategic-redirection epic).

---

Lessons learned + current state (consolidated) -- updated 2026-06-11

Authored by @pm after the A/B campaign on eval-qwen36-344t-full. This section is the durable record. Future analyses MUST build on it, not re-derive from the older design docs. Grounded on origin/main f85b665 + measured logs. Replaces the lever-by-lever checklist above as the current-truth status.

The 7 consolidated facts (verified)

1. L1 relationship_type_tiering (feat: type-tiered PC relationship cap (L1+L4, config-gated default OFF) #480/feat: enable relationship_type_tiering by default #481) is MERGED + default-ON. PC permanent-bond types (kinship/adversarial/mentorship/political/partnership) are UNCAPPED; the volatile tail is capped at 10; non-PC capped 8. This OBSOLETES the "uncapped PC web" premise of design PRs docs: per-turn token-stabilization design report #478/docs: bounded per-turn context architecture (A1+A2) design report #479.
2. A0 checkpoint-compaction (feat: A0 periodic checkpoint-compaction digest body (config-gated, default OFF) [epic #477] #482/feat: A0 periodic checkpoint-compaction digest body (#482) #483) = NULL / ICED. It compressed the volatile block, which a downstream max_snapshots=3 cap ALREADY bounded -> ~0 token lever. Its A/B was measurement-blind (it scored uncompacted raw the feature never touched, fix: entity_detail prompt-token instrumentation is blind to compaction (raw_tokens captured post-compression) #484). PR feat: A0 periodic checkpoint-compaction digest body (#482) #483 closed unmerged; branch feat/issue-482-a0-checkpoint-compaction PRESERVED (substrate for lever c-ii / feat: checkpoint context reset/rebuild with identity-anchor floor (epic #477, lever c-ii) #496).
3. L2 batch-entity_detail (feat: batch entity_detail calls to eliminate per-call template+turn-text repetition (L2) #491/feat: batch entity_detail calls to collapse template+turn duplication (L2, #491) #493) = MERGED default-OFF, WITHIN-NOISE as shipped. conf>=0.8 tiering left only 0.01 entities/turn batchable (91.5% of entities sit at conf0.9). No quality harm, no token win at the shipped config. N=3 paired verdict NOT SEPARABLE from noise. It is a throughput/wall-clock enabler, not a per-turn bounding lever.
4. A1a anchor-floor (feat: identity-anchor floor + checkpoint digest body (A1a) -- bound catalog-coupled context growth #492) = DEAD PREMISE. The identity/relationship body it targeted is now bounded by L1; what remains (identity + stable ~190 tok) is STATIC -> A1a is a pure constant downshift (slope unchanged) plus a quality risk (merge needs identity/stable visible). Its still-valid anchor-floor idea is FOLDED into lever c-ii (feat: checkpoint context reset/rebuild with identity-anchor floor (epic #477, lever c-ii) #496).
5. DECISIVE STRUCTURAL FINDING. Per-turn token growth = calls/turn (3.2 -> 5.7, cap-6 SATURATED, ~290-525 updates/run silently dropped) x (template 2175 fixed + turn_text 212 -> 972 + per-entity prior ~flat) + PC permanent web (uncapped). Late-window slope ~151 tok/turn, NO plateau; segmentation (auto-warned at 100 turns) does NOT reset injected context. CONCLUSION: NO per-turn COMPACTION lever can asymptotically BOUND the curve -- compaction multiplies each term by a constant < 1; it cannot change the COUNT of terms nor the linear growth of the carried web. Only de-dup (flatten) or constant-trim. Bounding requires attacking call MULTIPLICITY (relevance-scoped detail = lever b / feat: relevance-scoped entity_detail selection (A2a) -- bound calls/turn (epic #477, lever b) #494) and ACCUMULATION (cap PC web = c-i / feat: cap+age PC permanent-bond web -- extend L1 type-tiering (epic #477, lever c-i) #495; checkpoint reset = c-ii / feat: checkpoint context reset/rebuild with identity-anchor floor (epic #477, lever c-ii) #496).
6. COST FACT. GPU wall-clock is 86.8% DECODE / 13.2% prefill; llama.cpp single-slot prefix-cache already reuses duplicated input. So input-token reduction (option a / L2) gives ~0-2% real speedup -- NOT an A/B-cost lever. What cuts A/B cost = fewer GENERATING calls = lever (b). (b) is the DUAL lever: it bounds the curve AND cheapens every future A/B.
7. METHODOLOGY. The A/B is non-deterministic at 344-turn depth (noise floor ~5-13 tok/call). Use feat: paired multi-run A/B scoring on the entity_detail token metric (#487) #488 paired multi-run (>=3/variant) + per-turn scoring. Every future lever must pass TWO pre-checks BEFORE GPU spend: (i) projected effect EXCEEDS the noise floor, and (ii) it actually FLATTENS the curve (late-window slope -> 0), not merely beats the baseline by X%.

Lever scoreboard

Lever	Issue(s)	Status	Verdict
L1 type-tiered PC rel cap	#480/#481	MERGED, default-ON	WIN (bounds volatile tail)
L4 rel_mapper reuses L1	#480	MERGED	WIN (rides L1)
A0 checkpoint-compaction	#482/#483	ICED / NULL, branch preserved	~0 lever; substrate feeds c-ii
L2 batch entity_detail (option a)	#491/#493	MERGED, default-OFF	WITHIN-NOISE as shipped; throughput enabler only
A1a anchor-floor	#492	DEAD premise; folded into c-ii	superseded by L1 + #496
(b) relevance-scoped detail (A2a)	#494	NEW -- FIRST	bounds call multiplicity; the bounding + A/B-cost lever
(c-i) PC permanent-web cap+age	#495	NEW	bounds carried PC web (parallel-safe with #494)
(c-ii) checkpoint reset/rebuild	#496	NEW -- LAST	bounds carried context to a floor

The bounding path = (b) + (c). Compaction (A0/L2/A1a) is closed as a bounding strategy. Dependency order: #494 (S0 shadow -> S1 gate) FIRST; #495 parallel-safe with #494's S1; #496 LAST (highest coreference risk). Design: docs/design-bounded-context-relevance-reset.md.

STANDING RULE (applies to every agent, every future analysis)

1. Ground in current origin/main + measured logs, NOT design docs. git fetch first. Treat docs: per-turn token-stabilization design report #478/docs: bounded per-turn context architecture (A1+A2) design report #479 (and the A1a feat: identity-anchor floor + checkpoint digest body (A1a) -- bound catalog-coupled context growth #492 framing) as SUPERSEDED by docs/design-bounded-context-relevance-reset.md wherever they conflict. The specific stale premise is the pre-L1 "uncapped PC web".
2. TWO pre-checks before ANY GPU spend: (i) effect > feat: paired multi-run A/B scoring on the entity_detail token metric (#487) #488 noise floor, (ii) late-window slope -> 0 (the asymptotic-bound test, distinct from beating the baseline). A positive-slope result fails the bounding claim regardless of intercept improvement.
3. Non-blind by construction. Every lever must move a per-turn field the feat: paired multi-run A/B scoring on the entity_detail token metric (#487) #488 paired scorer reads (the A0 fix: entity_detail prompt-token instrumentation is blind to compaction (raw_tokens captured post-compression) #484 lesson). No A/B against a field the feature does not touch.

[daviburg]

Token-Stabilization GPU A/B Backlog + L1/L4 Task Specs (planning)

Author: @pm. Planning artifact only -- NO tasks submitted, NO pipeline code written.
@developer submits the specs in section 4 in a follow-up. Read-only repo + orchestrator
inspection. Every GPU-topology assumption I could not verify from code is flagged for
@b70-optimizer in section 6.

Scope: how to use the task orchestrator to serialize + prioritize the GPU-bound A/B eval
runs for epic #477 across the two B70 lanes, plus the ordered backlog and the two
ready-to-submit L1+L4 task specs (impl + A/B).

---

0. TL;DR recommendations

• COMBINE L1 + L4 into ONE variant. They share the type-tiering mechanism, both are
  low-risk, and a combined A/B costs one GPU lane-run instead of two. Attribution (L1 vs
  L4 split) is nice-to-have, not decision-critical -- the design report already estimates
  each, and the dashboard records those estimates. If the combined A/B underperforms the
  summed estimate we can split afterward. Net: 1 long run now, not 2.
• RUN LENGTH: run the FULL 344t (session-import / concord) for the P1 gate. The runaway
  and the PC-relationship explosion only manifest at scale; the L1/L4 savings RAMP from ~0
  early to their t300 value, so a 150t run under-shows the win (same lesson as the dormant
  feat: adaptive pressure-gated context compression #468 compression A/B). 150t is the floor (acceptable only if a lane is contended); 344t
  is the standard for this epic.
• RESOURCE SLOTS ALREADY EXIST -- do NOT add new ones. config-arclight.json already
  defines port-8000:1, port-8001:1, gpu-0:1, gpu-1:1, copilot-cli:6. The two eval lanes
  are already modeled. Use them: an A/B task declares resources:["port-8000"] (lane A) or
  resources:["port-8001"] (lane B); capacity 1 serializes eval runs per lane automatically.
• Single logical GPU-eval queue = priority-ordered pending tasks each requiring one port
  slot. The coordinator already claims "highest-priority pending task whose resources are
  free" (coordinator.py _claim_next_eligible_task), so priority + the cap-1 port slot give
  us serialized, priority-ordered eval execution for free.

---

1. GPU A/B backlog framework

1.1 The scarce resource

An A/B EXTRACTION run is a CLIENT of an inference endpoint; it does NOT itself compute on
the GPU inside the orchestrator (recall the Variant B A/B task ran with resources=[]). The
scarce resource is INFERENCE THROUGHPUT on a single llama-server endpoint. temp-0
byte-determinism (ab-test-standard sec 1.2) requires:

• a SINGLE pinned endpoint (no base_urls round-robin), and
• control + variant on the SAME endpoint / SAME GPU (the two B70s are not bit-identical,
  so a cross-GPU control-vs-variant comparison is invalid).

Therefore, WITHIN one experiment, control (flag OFF) and variant (flag ON) SERIALIZE on
one endpoint. The ab_test harness already runs variant-a runs then variant-b runs
sequentially against the injected base_url, so this serialization is intrinsic.

1.2 Two concurrent EXPERIMENT LANES

• Lane A = endpoint :8000 -> Vulkan0 / gpu-0 -> orchestrator slot "port-8000" (cap 1)
• Lane B = endpoint :8001 -> Vulkan1 / gpu-1 -> orchestrator slot "port-8001" (cap 1)

Two DIFFERENT experiments can run concurrently, one per lane. An A/B task pins itself to a
lane by:

1. metadata.base_url_a AND metadata.base_url_b BOTH set to the SAME endpoint (e.g. both
   http://localhost:8000/v1) -- this guarantees control+variant share one GPU, and
2. resources:["port-8000"] -- this makes the orchestrator hold lane A for the whole task
   so no second eval task can grab :8000 mid-run.

Because each port slot has capacity 1, the coordinator will never dispatch two eval tasks
onto the same endpoint; it WILL dispatch a P-lower task onto the other free lane. That is
exactly the 2-lane concurrency we want: priority-ordered serialization PER lane,
parallelism ACROSS lanes.

NOTE on gpu-0/gpu-1 slots: declaring only the port slot is sufficient (port:GPU is 1:1).
Do NOT also declare gpu-0 in the same task unless we later add non-endpoint GPU jobs that
must mutually exclude with an eval on the same card; double-declaring just risks an
accounting mismatch. Keep eval tasks port-slot-only.

1.3 Eval coexisting with "prod"

In this research setup there is no live user traffic, so an eval IS the endpoint load
(confirmed by the human). The orchestrator port slot only serializes ORCHESTRATOR tasks;
it does not know about the externally-launched prod llama-server. Since prod is launched
outside the orchestrator (systemd/nssm, not a task), it does not itself hold the
port-8000/port-8001 slot -- so an eval task is the sole orchestrator claimant of its lane.
That is fine here precisely because the eval is the only meaningful load. (Flagged in
sec 6 for @b70-optimizer to confirm prod-launch is external and the box has headroom.)

1.4 How the priority queue behaves

• All eval tasks land in one logical queue (pending tasks requiring a port slot).
• priority (int, higher first) orders them; ties broken by the existing claim ordering.
• When a lane frees, the coordinator claims the highest-priority pending eval whose
  required port slot is free. A P2 task will start on lane B while a P1 task still owns
  lane A -- they do not block each other; they just cannot share a lane.
• A long P1 run holds its lane for its whole duration but does NOT block other experiments
  -- they use the other lane. It only blocks LOWER-priority runs that want the SAME lane.

---

2. Standard A/B run spec for this epic

Every token-stabilization A/B (P1..Pn) MUST use this template (extends
docs/ab-test-standard.md):

• Length: 344 turns (session-import / concord) standard; 150t floor only under lane
  contention. Rationale: savings ramp ~0 -> t300; short runs under-show.
• Determinism: temperature 0 + explicit top_k 1 / top_p 1.0 / min_p 0.0 / seed 42; SINGLE
  pinned endpoint; capture the client [sampler] + server /props provenance log.
• Control (variant A): the MERGED commit with the new flag DEFAULT OFF (== main HEAD after
  the impl PR merges). Variant B: same commit, flag ON.
• base_url_a == base_url_b == one endpoint (same GPU). resources:[the matching port slot].
• Runs: 3 per variant minimum (temp-0 single-GPU noise floor expected ~0; any nonzero
  floor is a CONFIG bug to investigate, not inherent churn). Use run+confirm (run2/3
  majority) for the quality verdict.
• Quality gate (ALL must hold vs the 344t baseline eval-qwen36-344t-full):
  1. Entity-set RETENTION: no net entity drops beyond the A-vs-A noise floor (per-type),
     via tools/entity_retention_diff.py.
  2. ZERO new duplicates / phantoms (the PC web cut must not re-introduce duplicate
     re-extraction -- this is L1's primary risk: long-arc callback anchors).
  3. No attribute corruption.
  4. Callback-retention check: long-range (<=t100) relationship anchors still resolve
     (L1-specific; the 23 oldest PC rels are the callback anchors).
• TOKEN gate (the point of the epic): measured per-turn input-token SLOPE reduced vs the
  88.62 tok/turn baseline (15182 + 88.62*t). Report the re-fit slope + intercept for
  control and variant, and the t300 per-turn delta, bucketed by turn band (1-20, 21-50,
  51-100, 101-200, 201-300, 301-344). Session averages are insufficient (PR feat: smart entity context compression for extraction performance #393/feat: rework smart-context-compression to preserve entities (#460) #463
  lesson). Compare measured delta against the design-report ESTIMATE (L1 2-4K, L4 1.5-2.9K
  @t300) and record on the dashboard estimate-vs-actual panel.

---

3. The ordered GPU backlog

Priority is the int submitted on each A/B task (higher = sooner). Lane is a recommendation;
either lane works -- the point is one experiment per lane at a time. GPU-hours are ESTIMATES
(flagged sec 6); 344t ~= 7h/run -> 3 control + 3 variant ~= 6 runs ~= ~42h wall per full
A/B if serial, but control+variant share one lane so a full 3+3 A/B on one lane is ~42 lane-
hours. (If determinism holds, run+confirm can cut variant runs to 2 -> ~35h.)

P	Experiment	Lever(s)	Lane	Runs	Est lane-hours (344t)	Depends on
P1	L1+L4 combined type-tiered relationship cap	L1 + L4	A (:8000)	3+3	~35-42h	impl PR merged (sec 4a)
P2	L2 batch entity_detail	L2	B (:8001)	3+3	~35-42h	L2 impl PR merged; sequence its delta vs the L1/L4-tightened baseline
P3	A1+A2 architecture spike validation	A1/A2	A or B	3+3	~35-42h	A1/A2 design (not yet written) + spike impl
P0*	Re-baseline / dashboard baseline promote (344t full, no code change)	--	either	1-3	~7-21h	only if a fresh 344t baseline is needed beyond eval-qwen36-344t-full

Sequencing notes:

• P1 can start NOW (after 4a merges) -- L1+L4 are low-risk and help regardless of the
  A1+A2 architecture decision. Put P1 on lane A.
• P2 (L2) is the biggest token win but highest risk (attribute bleed); it should be
  measured against the L1/L4-tightened baseline, so its A/B is sequenced AFTER P1 lands.
  But its A/B run can occupy lane B CONCURRENTLY with P1 on lane A if its impl is ready --
  they do not share a lane. Just measure L2's delta against the post-L1/L4 control.
• P3 depends on the A1+A2 design (separate, pending) -- placeholder here so the lane
  budget accounts for it.
• P0* re-baseline: only schedule if the dashboard baseline-promote work needs a fresh full
  run; eval-qwen36-344t-full already exists as the reference, so this is likely a
  dashboard-data task, not a new GPU run. Keep it lowest priority / opportunistic.

Lane budget reading: with 2 lanes, P1 (lane A) and an in-flight L2 impl-ready run (lane B)
can overlap; P3 waits for a lane to free or for its design. Do not queue a 3rd concurrent
experiment -- there is no 3rd lane.

---

4. L1+L4 orchestrator task specs (ready for @developer to submit)

Two tasks: (a) implementation (copilot-cli, NO GPU) then (b) A/B validation (GPU lane, P1,
LONG). (b) depends on (a)'s PR being MERGED, not just the impl task succeeding.

4a. IMPLEMENTATION task (copilot-cli worker, no GPU)

TaskDefinition (fields per saas/orchestrator/models.py):

id:            impl-l1l4-pc-rel-type-tier
name:          Implement L1+L4 type-tiered PC relationship cap (config-gated, default OFF)
adapter:       local            # routes to copilot-cli per task_routing.default
target_machine: null            # any worker; no GPU needed
resources:     ["copilot-cli"]  # cap 6; does NOT touch a port/gpu lane
priority:      50
timeout:       5400
branch:        feat/l1l4-pc-relationship-type-tier
base:          main
prompt_file:   prompts/feat-l1l4-pc-relationship-type-tier.prompt.md  # @developer authors
success_criteria: [ {type: exit_code, expected: 0} ]
metadata:      { epic: 477, levers: ["L1","L4"] }

Worker brief (the .prompt.md body @developer writes -- summary of intent):

• GOAL: cap the char-player relationship web in the entity_detail prompt with a TYPE-TIERED
  policy, and reuse the same tiering for the relationship_mapper budget. Config-gated,
  DEFAULT OFF.
• Code locus 1 (L1): tools/semantic_extraction.py _format_prior_entity_context (~L1500-1610).
  Today the PC branch (is_pc) gets arc-compaction (_compact_relationships_with_arcs) or a
  history-trim, but NOT the mention/recency filter + cap that non-PC entities get
  (_filter_relationships_for_scene -> _SCENE_MAX_RELATIONSHIPS=8 at L1158). Add a
  type-tiered cap for the PC path:
  • KEEP UNCAPPED: permanent-bond types -- kinship, adversarial, mentorship, political,
    captor-debt (the long-arc anchors). Tier by relationship TYPE, NOT recency/status:
    0 PC rels ever resolve, so status/recency is an unreliable freshness signal (design
    report sec 2.3 + L1 sec (d)).
  • RECENCY + MENTION FILTER the volatile tail (spatial/social), cap ~10.
  • FORCE-KEEP anything in mentioned_ids (mentioned this turn).
  • Target: 109 -> ~30-40 retained.
• Code locus 2 (L4): the relmap budget at _REL_TOKEN_BUDGET_FRACTION=0.2 (L1154), applied
  ~L1827. Reuse the L1 type-tiering in the relmap scoring so permanent-bond types survive
  budget pruning and the volatile tail is trimmed first; optionally tighten the fraction
  toward ~0.10-0.11. Keep the highest-priority tier intact (quality guardrail).
• CONFIG GATE: add a context_optimizations block to config/llm.json with a boolean
  (e.g. "pc_relationship_type_tiering": false), read via the llm_config.get(...) pattern
  already used for checkpoint_interval / dedup_audit_interval (L4793/L4817) and the relmap
  budget. DEFAULT OFF so main behaviour is unchanged until the A/B blesses it (mirrors the
  perf: per-turn extraction degrades linearly with catalog growth (7.5x at turn 300) #385 budgeted-feature pattern). Both L1 and L4 paths gate on the SAME flag (they ship
  together).
• TESTS: unit tests for the tiering function -- permanent types retained when old,
  volatile tail capped, mentioned force-kept, flag OFF == current behaviour (golden), flag
  ON == reduced web. Add a fixture asserting a <=t100 kinship/adversarial anchor SURVIVES.
• DOCS (Rule 8): update docs/architecture.md (PC relationship tiering in entity_detail +
  relmap), docs/usage.md (the new config flag), and check off L1/L4 progress notes. The
  .prompt.md MUST include a Documentation Updates section.
• PR: gh pr create --body-file, Closes/refs Epic: Stabilize per-turn extraction token demand at quality parity (bounded growth over 300+ turns) #477, label enhancement. Do NOT flip the flag
  default to ON in this PR.
• WHAT NOT TO DO: do not lower _MAX_DETAIL_ENTITIES_PER_TURN (that is L3, rejected as a
  token lever); do not touch L2 batching; do not add hardcoded domain word lists (Rule 9);
  do not add the relationship-type list as a Python constant if it can live in/near the
  existing scene-trim config -- but a small fixed set of structural relationship-TYPE names
  (kinship/adversarial/mentorship/political/captor-debt) is acceptable as these are schema
  relationship types, not domain content words.

4b. A/B VALIDATION task (GPU lane A, priority P1, LONG run)

Submit ONLY after 4a's PR is MERGED (variant A = merged main HEAD). The token-stabilization
A/B compares SAME COMMIT, flag OFF vs flag ON. Because the ab_test harness keys variants on
BRANCHES, express the toggle as two thin branches off the merged commit:

• variant_a_branch = main (or a tag at the merge commit) -- flag default OFF.
• variant_b_branch = a 1-line branch off that same commit that flips
  context_optimizations.pc_relationship_type_tiering to true (config-only, ZERO code diff).

TaskDefinition:

id:            ab-l1l4-pc-rel-type-tier-p1
name:          A/B L1+L4 type-tiered PC relationship cap (344t, single-GPU, P1)
adapter:       ab_test          # routes via task_routing "ab-test"
target_machine: arclight
resources:     ["port-8000"]    # lane A; cap 1 -> serializes eval on :8000
priority:      100              # P1 -- ahead of P2/P3 eval tasks
timeout:       180000           # ~50h headroom for 3+3 x 344t (tune to measured tok/s)
dependencies:  ["impl-l1l4-pc-rel-type-tier"]   # plus a human MERGE gate (see note)
metadata:
  mode:             ab
  pr_number:        <the merged impl PR number>
  variant_a_branch: main                                   # flag OFF (control)
  variant_b_branch: ab/l1l4-flag-on                        # same commit, flag ON only
  base_url_a:       http://localhost:8000/v1               # SAME endpoint as b
  base_url_b:       http://localhost:8000/v1               # -> same GPU (gpu-0)
  runs_per_variant: 3
  turns:            344
  session:          session-import
  output_dir:       ab-l1l4-pc-rel-type-tier-p1
  long_running:     true
success_criteria: [ {type: exit_code, expected: 0} ]

Operational notes for @developer:

• DEPENDENCY is on the PR being MERGED, not merely impl-task success. The
  dependencies:[impl-l1l4-...] edge gates on the impl TASK; add a human merge gate: submit
  4b only after merge, OR set not_before, OR keep 4b in a held/pending state and release it
  post-merge. Do not let 4b dispatch against an unmerged variant_a_branch.
• LANE: resources:["port-8000"] holds lane A for the whole run. It will NOT block other
  experiments -- a P2/P3 eval can run on lane B (:8001) concurrently. It only blocks
  lower-priority eval tasks that also want :8000.
• Confirm BOTH base_url_a and base_url_b point to the SAME endpoint (single-GPU
  determinism). The harness emits a warning if multiple endpoints are detected -- treat any
  such warning as a BLOCK.
• Gate the result on section 2 (entity retention >= noise floor, ZERO new duplicates, no
  attribute corruption, callback anchors survive, AND measured token-slope reduction vs
  88.62 with per-band token table). Post the report as a PR/issue comment per
  ab-test-standard.
• Do NOT flip the config default to ON until this A/B passes; the default-ON flip is a
  follow-up PR justified by this report.

---

5. Recommendation summary

1. COMBINED vs SEPARATE: COMBINE L1+L4 into one variant/one A/B. They share the
   type-tiering mechanism and both are low-risk; one combined long run saves a full lane-
   run of GPU time vs attributing L1 vs L4 separately. Attribution is recoverable later
   (split A/B) if the combined delta misses the summed estimate (L1 2-4K + L4 1.5-2.9K =
   ~3.5-6.9K @t300). Only split up front if a stakeholder needs the per-lever number now.
2. RUN LENGTH: 344t standard, 150t floor. Signal strength dominates here -- the effect is a
   late-turn phenomenon and a 150t run systematically under-reports the win and the
   duplicate-regression risk. Pay the GPU hours once and get a trustworthy late-band slope.
3. RESOURCE SLOTS: do NOT add new slots -- port-8000/port-8001/gpu-0/gpu-1 already exist in
   config-arclight.json. Just USE the port slots on eval tasks (sec 1.2). The only possible
   improvement is naming/aliasing (e.g. an "eval-lane-a" alias) for clarity, but that is
   cosmetic and not required; recommend NOT changing slot config now.

---

6. Assumptions to confirm (@b70-optimizer)

I could not verify these from code alone; please confirm before P1 launches:

1. Endpoint->GPU mapping is current: :8000 -> Vulkan0/gpu-0 and :8001 -> Vulkan1/gpu-1
   (taken from the brief + slot names, not a live probe).
2. Prod llama-servers are launched OUTSIDE the orchestrator (systemd/nssm), so they do NOT
   hold the port-8000/port-8001 orchestrator slots -- i.e. an eval task is the sole
   orchestrator claimant of its lane. (The slot mechanism only mutexes orchestrator tasks.)
3. The box has VRAM/RAM headroom to run a long 344t eval on one card while the other card
   serves the second lane -- two concurrent long runs, one per B70.
4. The two B70s are NOT bit-identical (cross-GPU control-vs-variant compare is invalid),
   confirming control+variant MUST share one endpoint.
5. GPU-hours: 344t ~= 7h/run is an estimate; please re-measure effective tok/s for
   Qwen3.6-35B-A3B Q4_K_M at long (40k+) context so the timeouts and lane-hour budget in
   sec 3 are accurate (the old 52.7 tok/s figure was a different model/config).

(No secrets, env dumps, or config values were read or printed in producing this plan.)