Observed on the first scored LoCoMo run (conv-26, 19 sessions): all 135 claims have claim_valid_kind = 'unknown', no claim_valid_from, empty asserted_at — although every rendered turn carries an explicit timestamp ([D1:3 | 1:56 pm on 8 May, 2023] …).
Consequences:
entity_timeline for the main entity collapses to a single bucket keyed by ingestion year, which looks like data but is noise.- Every LoCoMo category-2 (temporal) question is structurally unanswerable, regardless of retrieval quality.
Suggested shape: E2 emits valid-time from in-document timestamps; an eval canary asserts "a dated source line yields a dated claim"; until populated, entity_timeline should return a typed negative with a reason rather than an ingestion-time histogram.
Found during RS-LoCoMo-Full-v3 trace forensics, 2026-07-26.
🤖 Generated with Claude Code
https://claude.ai/code/session_01GKENhTLJg1HqhbdwCmmkbc
Observed on the first scored LoCoMo run (conv-26, 19 sessions): all 135 claims have
claim_valid_kind = 'unknown', noclaim_valid_from, emptyasserted_at— although every rendered turn carries an explicit timestamp ([D1:3 | 1:56 pm on 8 May, 2023] …).Consequences:
entity_timelinefor the main entity collapses to a single bucket keyed by ingestion year, which looks like data but is noise.Suggested shape: E2 emits valid-time from in-document timestamps; an eval canary asserts "a dated source line yields a dated claim"; until populated,
entity_timelineshould return a typed negative with a reason rather than an ingestion-time histogram.Found during RS-LoCoMo-Full-v3 trace forensics, 2026-07-26.
🤖 Generated with Claude Code
https://claude.ai/code/session_01GKENhTLJg1HqhbdwCmmkbc