fix(#39): resolve X-Article inline + cover images on the real X payload shape#71
Merged
Merged
Conversation
Ground-truth fixtures trimmed verbatim from the 3 real captured Articles (article_results.result subtree). First failing test: art-OpenWiki must resolve its atomic MEDIA block to the inline CDN image via the LIST entityMap + media_entities indirection (currently yields zero images). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ad shape The #39 parser was pinned to a constructed fixture and resolved ZERO images on live X Articles (validated against 3 real bookmarked Articles, #66). Three real-shape divergences fixed in extract/article.py: - entityMap is a LIST of {key, value} keyed by entry.key (not a dict) — index by key, with the old dict shape kept as a defensive path. - a MEDIA entity's URL is indirect: data.mediaItems[].mediaId resolves against the sibling media_entities[] (media_id -> media_info.original_img_url). - cover_media is a separate lead image, prepended as the first ArticleImageBlock (deduped against inline). Also render header-two -> "## " and unordered-list-item -> "- " by baking the markdown prefix into the text run (flattened-text invariant preserved). Degrades to text-only on any shape miss; no model change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real-payload acceptance tests (tests/test_article_real.py): exact image inventory per Article (OpenWiki cover+inline, Wiki Memory cover-only, Headcount none), cover-is-lead-block, LINK-never-image, headings/bullets markdown, flattened-text invariant via the model validator, and degradation (missing media siblings drop images but keep all text; foreign payloads -> (None, [])). Baseline the 3 fixtures' base64 node-ids: each decodes to "ArticleEntity:<public rest_id>" — benign GraphQL global IDs in real-payload ground-truth fixtures, not credentials. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… selection Review panel (silent-failure + test-quality + spec-conformance) hardening of the real-payload image fix: - Multi-image gallery: a MEDIA block's `mediaItems` now yields ONE ArticleImageBlock per resolvable item (was first-only), so a gallery is no longer silently truncated to its first image; a partial resolve WARNs for the lost remainder. - Image tripwire: WARN when the content_state carries media indicators (atomic blocks / media_entities / cover_media) but resolves 0 images — the exact regression the original #39 defect produced silently. Text-only articles are not flagged. - Richest-payload selection (fetch_x._structured_article): parse ALL captured payloads and pick the one with the most blocks (was first-with-blocks), so a preview/skeleton article response can't ship as the complete body; WARN if >1 parsed. - Atomic-ness decided from the entity, not blank text: a caption-bearing atomic image that fails to resolve is logged as a drop, not demoted to a text run. - Failed gallery no longer falls back to a stray click-through `url` on the entity data (only the no-mediaItems defensive shape uses the URL-in-entity fallback). - Per-item alt text; DEBUG log on an unparseable content_state string; blockquote/ header-one/three/four prefixes; TypeGuard on _is_image_entity. - Rewrote the stale module docstring (was "pinned against a constructed fixture, validate before production") to reflect the now-completed real-payload validation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…h, titles Close the review-flagged test gaps: - Multi-image gallery (one block per item), partial-gallery WARN, int/str media_id coercion, list-shape MEDIA drop WARN, caption-atomic-drop, stray-URL guard, ordered-list/header-one/three/blockquote prefixes (constructed, test_article.py). - Real payloads (test_article_real.py): title extraction per fixture; cover==inline dedup; media-tripwire WARN vs no-warn on a text-only article; replaced the TAUTOLOGICAL invariant test with an independent text-run separator-structure check plus a model-validator check via fetch_x._flatten_blocks. - Production wrapper (test_fetch_x.py): _structured_article on the REAL OpenWiki payload (cover-first + inline, title, off the sibling media_entities/cover_media), and richest-of-multiple-payloads selection with the ambiguity WARN. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
#39 shipped X-Article structured capture pinned to a constructed fixture. Validated against 3 real bookmarked Articles (#66), the parser resolved body text correctly but zero images — the live X shape differs in three ways the constructed fixture didn't model:
entityMapis a LIST of{key, value}keyed byentry.key(not a dict) — the old helper returned{}, blinding all image resolution.MEDIAentity's URL is indirect:data.mediaItems[].mediaId→ siblingmedia_entities[].media_info.original_img_url(never on the entity itself).cover_mediaimage lives in a separate sibling.The fix (
extract/article.py,fetch_x.py)entityMapindexing byentry.key(dict shape kept as a defensive path).mediaId→media_entitiesresolution; multi-image galleries emit one block per resolvable item.cover_mediaprepended as the lead image (deduped vs inline).header-two→##, lists→-/1.,blockquote→>baked into text runs (flattened-text invariant preserved).Validation
tests/fixtures/art-*.json); acceptance locked: OpenWiki cover+inline, Wiki Memory cover-only, Headcount none.Addresses #66 (real-payload validation of #39). Follow-up #67 (
fetch --forcebackfill of the older text-onlyx_article) still applies.🤖 Generated with Claude Code