feat: coverage guard for page-builder one-pagers - #47
Merged
Conversation
Wires lib/coverage-guard.js (Task 1) into convertWithReadability: measure body text length before Readability runs (it mutates the document), then after pickBest chooses a candidate, re-attempt extraction from the dominant container when coverage is suspiciously low. Adds integration tests against a page-builder one-pager fixture built in-test.
- Renamed "stays silent on a small page" to "leaves an ordinary small article untouched" (body text ~600 chars, already below size and coverage floors) - Added "stays silent below the body-size floor even when every other condition holds" (body 17,635 chars / 9.3% coverage / dominant container / 10.8x gain) to isolate the MIN_BODY_TEXT threshold The new test fails when MIN_BODY_TEXT is lowered to 0, confirming it properly discriminates on the body-size floor (verified via temporary patch).
- extractor=playwright no longer silently disables the guard: gate on the two static extractors it actually defers to (readability/trafilatura), not on any truthy extractor value - a guard hit on the Playwright-rendered pass no longer loses its coverage-guard label: extractorReason now keeps both the render reason and the guard reason instead of the guard's markers being overwritten - SITE-RECIPES.md and README/SKILL X-Source lists updated to include coverage-guard (and recipe-content, which was also missing) - dropped a dangling reference to a gitignored design doc path - findDominantContainer no longer recomputes a parent's textContent it already measured in the previous iteration (behavior unchanged) Adds two regression tests (1033 -> 1035 passing, 0 failing).
…ecovered Three findings from the review of this branch. The dominance threshold is checked per descent level, not against the body, so the shares multiply: in a six-level 91% chain the recovered container holds 56% of the body, not the >= 90% the trigger description implies. The sheddings are chrome siblings and dropping them is the point, but that is a judgement for whoever reads the reason afterwards - so state the measured share instead of leaving the threshold to be read as a guarantee. containerLabel echoed a page-controlled id or class verbatim into a string that is persisted in the extraction log and emitted as frontmatter. A 5000 character id produced a 5088 character reason. Capped at 60 characters. (YAML stays valid either way - the reason always contains ": ", so the frontmatter builder quotes and escapes it; verified with a crafted id.) bodyTextLength walked the whole body on every default-path request. A serialization is never shorter than the text inside it, so a document below the size floor cannot reach it - skip the walk on those instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hh7VhsxbwhJJGbb3RpYzo2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #46
What
Adds
lib/coverage-guard.jsand wires it into the default extraction path. It notices when an extraction kept only a sliver of the page and recovers the container that holds the body - effectively an auto-writtenselect.content, using the same conversion path a recipe takes.Why the failure is invisible today
Readability distributes a paragraph's score across its ancestors, divided down with depth. On a page-builder one-pager the preamble has its paragraphs directly in its container and wins the top candidate; the chapters split their prose across many accordion items three or four levels deeper, so their common ancestor only collects fractions. The chapters are not siblings of the top candidate either, so they are never appended. Result: well-formed markdown containing 3% of the page.
Trigger
All five conditions must hold:
select.content, no forced static extractor, not the comments pathCondition 4 is the discriminator. It separates "one article split across siblings" from "many separate teasers", and it is what keeps listing pages out.
Calibration
85 live pages: the Hacker News front page, the p2-1 eval corpus, 12 server-rendered forum threads and comment-heavy blog posts, wikis, long single-document pages (GPL-3.0, RFC 9110, MDN, Python docs), shops, listing homepages, and four page-builder landing pages.
Forum threads: 0 of 12 fire, their coverage runs 17.7-258%. The nearest non-target miss is a listing homepage at 1.9% coverage, blocked by condition 4.
Threshold sensitivity: the first false positive appears at coverage < 20% (a forum thread at 19.8%) or at body >= 10k with coverage < 15% (a landing page at 11.7%). The shipped thresholds sit with roughly a factor of 2 in hand.
The sweep was re-run against the shipped code before opening this PR, with the gate model and the real pipeline cross-checked against each other on every page. They agree everywhere.
Safety
The guard can only grow a result, never shrink it - condition 5 requires a 3x gain, and anything that does not hold leaves today's output untouched. The whole block is wrapped in try/catch and degrades to current behaviour. Opt out with
PULLMD_COVERAGE_GUARD=off. Every intervention carriessource: coverage-guardand records the figures it acted on inmetadata.extractorReason, so an operator can judge afterwards whether it was warranted.Side effect worth noting: on the motivating page the recovered extract raises the quality score from 0.45 to 0.8, which puts it above the
renderDecisionthreshold. The page needed no JavaScript, so this also removes a pointless Playwright round-trip.Deliberately not changed
pickBest. Relaxing its heading gate would let Trafilatura's heading-less output win on these pages, push coverage to ~95% and silence the guard, trading a structured result for a flat one.Tests
1035 passing, 0 failing (baseline before the branch: 995). The new tests cover the descent algorithm, the trigger thresholds, and the cases where the guard must stay silent: a structurally flat twin of the firing fixture with identical content, a page below the size floor that meets every other condition, a recipe
select.content, forced static extractors, and the opt-out. Test fixtures are generated in-test from neutral prose.Python sidecar tests pass, and the query-extract goldens are byte-identical - no re-capture needed.
Known limitation
emit('extracting', ...)still reportsplaywrightwhen the guard fires on a rendered pass. That is a transient progress event only; the persistedsourceandextractorReasonare correct.🤖 Generated with Claude Code
https://claude.ai/code/session_01J1rQhLmVXHnJvurKvfGgfg