Skip to content

Record a per-item disposition from selection through render #1147

Description

@kevinhermawan

Summary

Six of the nine issues in the 2026-08-11 batch shipped a lower-scored item over a higher-scored one in the same section, or left a section empty while 1.00-scored candidates sat in the accepted pool. Nobody can say which stage discarded them, because nothing records a per-item disposition.

The worst case: DSSA began transferring 9.63 billion treasury shares on August 10. article-analysis accepted that story six times — three rows at dealsAndMovements 1.00, one at 0.80, four at quickHits 0.60 — and the shipped newsletter has neither section. A DSSA shareholder read the issue and learned nothing about their own company's disposal.

The gap

Ticker What was available What shipped
DSSA 6 rows, 0.60–1.00, across two sections neither section rendered
CCSI 4 rows at 1.00 across three sections none of them, no Industry Pulse
BMRI industryPulse 1.00 the 0.80, which carries a wrong ratio
ANTM two quickHits at 0.80 a 0.60 internship notice
AMAN regulatoryPolicyWatch 0.60 (Indonesian) the 0.40 (Vietnamese)
TLKM issuerPerformance 0.60 two 0.40 rows

selectArticles takes the top MAX_ARTICLES_PER_SECTION per section by score, so these candidates were selected. The loss is downstream: on-demand body fetch (triage-fetch-requests.ts, fetch-source-bodies.ts), generation itself, or a post-generation filter. selectSectionCoverageSeeds only guarantees a body for the top candidate of each section, which fits the pattern that the top-scored item usually survives and the rest do not.

That is a hypothesis. It should not be fixed on a hypothesis.

Scope

  • In scope: emit a per-item disposition from selection through render — selected, body requested, body fetched, summarised, shipped — with the reason on every drop, keyed by dataSourceId.
  • Out of scope: changing any selection or generation behaviour. This ticket only makes the existing behaviour observable.

Acceptance criteria

  • Every candidate that reaches selectArticles has a terminal disposition recorded
  • A drop records which stage dropped it and why
  • The record is keyed by dataSourceId so it joins to data_source_ticker_section
  • Replaying the 2026-08-11 DSSA run identifies the stage that discarded all six treasury-share rows

Notes

Found during the 2026-08-11 newsletter review. Evidence in reviews/2026-08-11/DSSA.md and reviews/2026-08-11/IMPROVEMENTS.md item 4.

This blocks any real fix for the selection gap, and it is the reason that gap is not being fixed in the same cycle as the other defects from this batch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions