Skip to content

Manifest reports complete: true unconditionally; truncation is undetectable downstream #139

Description

@gregor-em-cg

Labels: bug, render

When a capture truncates, nothing in the output says so. The manifest's complete flag is a literal, the one signal that would reveal the problem is never recorded, and a consumer reading tiles.json afterwards has no way to tell a complete capture from a truncated one.

This is a claim about the manifest contract, not about any particular truncation bug, so it is stated below from the code on current main (c1dae49) rather than from a reproduction. It is independent of why a capture truncated: it held for the height-formula path that #131 fixed, it holds for the document.fonts.ready path in #133, and it will hold for whatever causes the next one. #131 does not address it — that PR changed how the height is measured, not what the manifest reports about it.

1. complete: true is a literal

On current main (c1dae49), the manifest is assembled with the flag hardcoded:

https://github.com/StarTrail-org/PixelRAG/blob/c1dae49/render/src/pixelrag_render/backends/cdp.py#L500-L510

manifest = {
    "url": url,
    "page_height": page_height,
    "tiles": tiles,
    "complete": True,
}

There is no code path that sets it to False. A caller checking complete learns nothing.

Worth noting that the repro in #133 shows this directly — {"page_height": 1568, "tiles": ["tile_0000.jpg"], "complete": true} for a page several thousand pixels tall.

2. page_height == tile_height is never surfaced, and tile_height is never recorded

That equality is the cheap local signal that a capture is untrustworthy: it means the measured height tracked the emulated viewport rather than the content. Two things make it unusable downstream:

  • It produces no warning at render time, on either the measured path or the fallback at L440-L443 — which assigns page_height = tile_h in precisely the case where the probe failed, the case most deserving of one.
  • tile_height is not written to the manifest. URL manifests carry url, page_height, tiles, complete and nothing else, so a later consumer cannot reconstruct the comparison at all without being told out-of-band which --tile-height the capture used. Any downstream checker is reduced to guessing from a candidate list.

3. What this costs downstream: the pixelbrowse plugin

The plugin is where the silence turns into a confidently wrong answer, which is why this matters beyond tidiness.

pixelbrowse@0.1.0's SKILL.md mandates --tile-height 1568, reasoning correctly that Claude's vision downscales images with a long edge over 1568 px. But --tile-height is the emulated viewport height, so the recommended setting maximises truncation. The skill then instructs:

"Do NOT run ls — just read tile_0000.jpg. If it doesn't exist, the page had no content."
"If the page is long, also read tile_0001.jpg, tile_0002.jpg…"

With one tile written and complete: true in the manifest, there is no tile_0001.jpg, the reader is told not to list the directory, and "no second tile" is indistinguishable from "the page ended." On en.wikipedia.org/wiki/Python_(programming_language) at the mandated 1568, that is 1568 px of a ~29,200 px article — 5.4 % captured, 94 % missing — read and summarised as though it were the whole page.1

Repro — against v0.4.0, and it will not reproduce on main

Please read this note before running anything below, or the result will be misleading.

The repro uses v0.4.0, the current release, where the formula path still truncates and so gives
something visible to point at. On main it will not show the collapse#131 fixed the
measurement, so the heights come back real and the loop looks healthy. That is expected and does
not bear on this issue.

Sections 1 and 2 hold regardless, and are verifiable by inspection at c1dae49 rather than by
running anything: complete is still a literal, and tile_height is still absent from the
manifest. What #131 removed was one cause of truncation. The reporting of truncation is
unchanged, so any future cause — the Brave path in #133, a new layout pattern, a timeout — will be
equally silent.

pixelshot "https://en.wikipedia.org/wiki/Python_(programming_language)" \
  -o ./probe --tile-height 1568 --wait-network-idle

cat probe/*/tiles.json
# {"url": "...", "page_height": 1568, "tiles": ["tile_0000.jpg"], "complete": true}

ls probe/*/            # tile_0000.jpg only — the step SKILL.md says not to run

The reported height follows the flag rather than the page, and complete never moves:

for TH in 1568 4096 8192; do
  pixelshot "https://en.wikipedia.org/wiki/Python_(programming_language)" \
    -o "./probe_$TH" --tile-height "$TH" --wait-network-idle >/dev/null
  python3 -c "
import glob, json
m = json.load(open(glob.glob('probe_$TH/*/tiles.json')[0]))
print(f\"tile_height=$TH  page_height={m['page_height']}  tiles={len(m['tiles'])}  complete={m['complete']}\")"
done

# tile_height=1568  page_height=1568  tiles=1  complete=True
# tile_height=4096  page_height=4096  tiles=1  complete=True
# tile_height=8192  page_height=8192  tiles=1  complete=True

Three captures of the same page, at three different heights, all reported complete, none of them the page's height (~29–30k px, and itself viewport-dependent). Run the same loop on main and the heights are correct — while the manifest still carries a hardcoded complete and still omits tile_height.

Environment: stock Google Chrome 151.0.7922.71, macOS 26 (Darwin 25.5.0), arm64, pixelrag==0.4.0 (PyPI), pixelbrowse@0.1.0. The percentages are from v0.4.0; sections 1 and 2 describe main at c1dae49.

Suggested direction

Not a patch, just the shape of one:

  1. Set complete from something real — False when the probe fell back, or when page_height equals the emulated viewport height.
  2. Record tile_height in the manifest so consumers can check the equality themselves.
  3. Warn on that equality at render time, on both the measured and fallback paths.
  4. In SKILL.md, replace "do not run ls" with a manifest check — one file read, and the failure mode disappears.

Happy to open a PR for any of these if the direction is welcome.

Footnotes

  1. Adjacent, and probably its own issue rather than part of this one: a render that fails before writing any tile leaves an orphan .tiles directory behind, and the same "if tile_0000.jpg doesn't exist, the page had no content" instruction then reports an empty page rather than an error.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions