Skip to content

Umbrella: source ingest reliability — difficult pages + failure recovery #2196

Description

@zachdunn

Tracking umbrella for making source ingest resilient on the two axes the 2026-07 incidents exposed: pages our extraction path can't see properly, and failures that don't self-heal.

Grew out of the 2026-07-23→28 provider outage and the langchain/cloudflare-blog investigations (PRs #2191, #2192). Sub-issues cover:

  • extraction blind spots on difficult pages (line-window misses, thin-content strategies, challenge-blocked sources, large-body tool-loop rollout and its checkpointing)
  • recovery gaps where a failed ingest stays failed until a human notices (change-driven sources after an outage, provider attribution in the operator digest)
  • correctness of ingest-time gates on the bulk paths

One line per child in the sub-issue list; detail lives in each issue.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions