TR-only optimization: target the 26 TR failure modes from iter27 baseline

[wsuli615]

Goal

Push LongMemEval temporal-reasoning (TR) subset accuracy from the
current iter27 baseline of 80.5% (107/133) toward ≥87% without
regressing other types. Track this work on branch
tr-only-optimization.

Baseline

Stack	N	strict	TR subset
iter17 TR-only experiments	133	—	81.2% peak
iter21/iter25 TR-only	133	—	83.5% peak
iter19 full N=500	500	86.80%	78.9%
iter27 full N=500	500	86.80%	80.5%
Mastra (paper)	500	94.87%	est. ~95%

TR failure clusters (from iter27 wrong_cases.json, 26 cases)

Cluster	Count	Example	Hypothesis
Group A: duration_since_start	10	"how long had I been taking guitar lessons when I bought the amp?"	writer doesn't reliably extract a START_EVENT concept; _try_duration_activity fall-through silently picks latest_value or returns nothing
Group B: order_among	4	"order of 6 museums earliest→latest"	_try_order_among returns partial list; retrieval cap loses some museums
Group C: named_day disambig	3	"what airline on Valentine's day" (multi flight day)	_try_named_day_recall picks lowest-BM25 candidate; need multi-candidate disambig
Group D: date_diff off-by-one	3	"how many days between X and Y" → 37 vs GT 38/39	reader picks exclusive boundary
Group E: refusal with data	5	"how long had I been bird watching" → "I don't have a memory"	covered by iter29 TR-NEW-7 NO-REFUSAL but compliance varies
Group F: DERIVED-TIME	1	"wake time T/Th" → 7:00 instead of derived 6:45	covered by iter22 #9 but compliance varies
Group G: other	0-2	which_first / count_among	already disabled in iter30 (were 0%-acc poisoners)

Validation pattern

TR-only runs are N=133 (vs N=500 full) and finish in ~1/4 the time
and cost. Use them to A/B test fixes before promoting to N=500.

# extract TR-only qids
grep -l '"question_type": "temporal-reasoning"' \
  < benchmarks/longmemeval/data/longmemeval_s_cleaned.json \
  | jq -r '.[] | select(.question_type=="temporal-reasoning") | .question_id' \
  > benchmarks/longmemeval/qid_sets/tr_only.txt
bash scripts/run_iter31.sh benchmarks/longmemeval/qid_sets/tr_only.txt \
     iter31_tr_only_baseline 5

Workflow

1. Open PR from tr-only-optimization against iter30_cleanup (this issue link).
2. Each TR fix lands as its own commit on the branch.
3. Before merging back: TR-only N=133 must show net positive vs iter31 baseline.

Acceptance

• TR ≥ 85% on N=133 (iter31 TR baseline + 4-5pp)
• No other type regresses > 1pp on N=500 spot-check

[duanyiqun]

Out of date, we achieve 93%