Skip to content

perf(bench): add enwik8 large-text corpus + zopfli ceiling for L9 ratio evaluation #2700

Description

@kim-em

Add a large pure-text corpus (enwik8, the first 100 MB of English Wikipedia, from the Large Text Compression Benchmark) so we can evaluate L9 against the zopfli ceiling on zopfli's home turf.

Why

Our L9 now sits right at zopfli's operating point (slightly faster, slightly worse ratio). Zopfli ships no corpus of its own; its published evaluation used standard corpora including enwik8, and its design target is large text / web content. Big pure-text is exactly the regime where near-optimal parsing pays off most, and where our current ratio gap to zopfli is widest. Canterbury is tiny and Silesia, while large and modern, is heterogeneous; a single large prose file is the missing data point for judging L9-vs-zopfli.

What to do

  • Add bench/corpora/enwik8/ wired through bench/fetch_corpora.sh (SHA-256-pinned, gitignored cache, fetched on demand like Silesia). The harness discovers corpora by directory, so it slots into the per-level Pareto/summary/heatmap charts with no plot.py change.
  • Generate the zopfli ceiling for it (bench-report --zopfli-ceiling) so the L9-vs-zopfli overlay appears on the enwik8 ratio graph. Note: zopfli on 100 MB at default iterations is very slow (~100x zlib); if the one-time run is impractical, use a fixed slice (e.g. enwik8 first 10-20 MB, recorded in the corpus name) — deterministic and SHA-pinned either way.
  • Run the reduced (single-pass timing) matrix like Silesia, since it is large.

Acceptance

enwik8 (or the chosen slice) appears in bench/graphs/enwik8_* with the zopfli ceiling overlaid on the ratio graph, and bench/results/latest.json carries its rows. This becomes the reference for the "is L9 strictly better ratio than libdeflate, and how close to zopfli" question (ties to #2638's ratio floor).

Context: requested to evaluate L9 against zopfli on a large-text corpus; sibling of the bench-regeneration chore #2699.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requesttrack-dBenchmarking and verified optimization (PLAN.md Track D)

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions