feat(epub): add the edge-case corpus for the EPUB extraction fixes - #10
Merged
Conversation
Add 36 EPUB files under epub/edge-cases, one or more for each defect that xberg-io/xberg pull request #1498 fixes: named entities and byte order marks, the navigation heuristic, deep nesting, unsafe spine hrefs, Dublin Core metadata and covers, rendering, encodings and DRM, and spine media types. Each file was run through the base and head builds of that pull request, and EPUB_EDGE_CASES.md records the measured before and after per file, the source URL, and the licence with its evidence. 22 files are published documents from Project Gutenberg, Standard Ebooks, the IDPF samples, the W3C EPUB tests, the epubcheck test suite, epub.js, epublib and the Internet Archive. 14 are synthesized by scripts/build_epub_edge_cases.py for the cases no public file exhibits, and the document says which and why. scripts/epub-edge-cases.json pins every file by sha256 and names its source: a URL, the members of an epubcheck directory at a pinned commit, or the generator. scripts/fetch_epub_edge_cases.py materialises all of them, and scripts/test_epub_edge_cases.py checks the generator against the pins without the network. _typos.toml allows the word "opf". corpus.lock.json is untouched. The bytes are staged locally and need a publish by a maintainer with write access to the bucket.
Goldziher
approved these changes
Aug 25, 2026
Goldziher
reviewed
Aug 25, 2026
tobocop2
force-pushed
the
feat/epub-edge-cases
branch
from
August 25, 2026 05:59
b863669 to
74dc6aa
Compare
tobocop2
force-pushed
the
feat/epub-edge-cases
branch
from
August 25, 2026 05:59
74dc6aa to
cc1b894
Compare
Each synthesized package now carries an NCX (EPUB 2) or a nav document (EPUB 3), so epubcheck reads the fixture as a conforming EPUB and the defect under test is the only thing wrong with it. The depth fixture wraps its nested run in a div. The build is deterministic and the 14 affected pins in scripts/epub-edge-cases.json are regenerated to match.
Member
|
@tobocop2 - can you get this merged? |
Member
Author
|
it will be merged shortly. I'm mid publish to GCP. |
The 36 edge-case files are uploaded to the corpus bucket. Their pins are added to corpus.lock.json; no existing pin is dropped or changed.
tobocop2
marked this pull request as ready for review
August 25, 2026 17:39
Member
Author
|
Everything is now published, and this is being merged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
36 EPUB files under
epub/edge-cases/, one or more per defect that xberg-io/xberg#1498 fixes.EPUB_EDGE_CASES.mdrecords each file's source, licence evidence, and the observation measured on the base and head builds of that pull request.Fix to file
text/htmlbook, 1 synthesizedLicences
All 36 are vendor lane: US public domain, CC0, W3C, BSD-3, Apache-2.0, MIT. IDPF samples that show these cases under CC-BY-SA or GFDL are recorded, not hosted.
Not found
No public file exists for entities beside a stylesheet link, a BOM, legacy or UTF-16 encodings, a cover meta pointing at XHTML, or the odd media-type labels; those 14 are synthesized. The head of #1498 carries no media-type commit, so the
text/htmlrows show no change yet.Publish
The bytes are staged at
~/projects/xberg-worktrees/test-documents-epub/epub/edge-cases/and need a publish by @tobocop2.corpus.lock.jsonis untouched.