Check that the radix sort and the in-memory sort agree

[e-n-f]

Important

Do not merge before #402 and #403. This target fails on main on purpose: the two inputs it generates are what detect the two bugs those PRs fix. It goes green once both are in.

Approach

The result of a sort shouldn't depend on how the sort was performed. So rather than checking sorted output against a committed copy of it, this checks that --prefer-radix-sort — which lowers the memory limit to 8K so the radix subdivision has to recurse — produces the same tiles as sorting in memory. Same shape as the existing parallel-test, which compares linear-file.json against parallel-file.json rather than against a golden file.

Two things follow from that:

• Nothing new to keep up to date. No expected-output files, so no rebaselining when unrelated tiling behavior changes.
• It doesn't matter how deep the recursion goes. How many buckets each level splits into comes from how many files the machine will let us open, so the path through radix1() varies from machine to machine — but both runs happen on the same machine and the sorted result is the same either way, so the assertion is stable even though the path isn't. (This is what I'd wrongly worried about when I said in Distinguish duplicate feature locations during the index sort #402 that an -aR test would risk being flaky. A golden-file test would have been fine too; it's the crash in Keep the radix sort from recursing forever when it runs out of files #403 that was the real hazard, and that's now fixed.)

Inputs

What sends the sort down the branches that are otherwise almost never taken is the shape of the input, not the size, so the two purpose-built inputs are small and adversarial rather than large:

• bigfeatures.json — eight well-separated 2000-point linestrings. Each is bigger than the lowered memory limit on its own, so each is sorted as a bucket of one feature.
• onelocation.json — 500 points at identical coordinates. They share the whole index, so they have to be subdivided until there are no bits left.

Then tests/feature-filter, tests/ne_110m_ocean, tests/border, tests/loop, tests/tl_2022_11_tract, and tests/epsg-3857 for breadth.

Instrumenting the branch taken in radix1() confirms all three are covered:

bigfeatures.json      8 direct-write single-feature   prefix=0
onelocation.json      1 direct-write prefix-exhausted prefix=60
feature-filter/in     1 direct-write prefix-exhausted prefix=59
                      3 memory-sort
                      1 memory-sort (no files left)      <-- the fallback #403 adds

Worth noting that an earlier version of bigfeatures.json had a single feature and passed on main even though it did reach the buggy path: with only one feature in the whole tileset, the stray byte lands right before the EOF marker and happens to be zero, so it reads as EOF and the corruption is invisible. It needs a following record to desync against, hence eight features rather than one.

Runtime

About ten seconds. Wired into both test and fewer-tests — the latter because the out-of-bounds geomfiles[which] write described in #403 is exactly what that build's address sanitizer is for.

Not included

I left out the TIPPECANOE_SORT_MEMORY / TIPPECANOE_MAX_FILES overrides I'd suggested, to keep this to just the test. They aren't needed for it to be correct — -aR is enough of a lever, and the comparison is machine-independent regardless. They'd still be worth having so a test could sweep several budgets and pin the recursion depth by construction instead of inheriting it from ulimit -n; happy to do that separately if you want it.

---

Generated by Claude Code