Skip to content

Property-based tests over randomly generated SDF trees#63

Merged
kmatzen merged 2 commits into
mainfrom
sdf-property-tests
Jul 25, 2026
Merged

Property-based tests over randomly generated SDF trees#63
kmatzen merged 2 commits into
mainfrom
sdf-property-tests

Conversation

@kmatzen

@kmatzen kmatzen commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Part of #52 — the CPU-side half. The differential CPU-vs-GLSL testing that issue also proposes needs a headless WebGL context and is not included here.

evaluate.test.ts checks hand-picked points on hand-built shapes. These check invariants that must hold for any tree, which is where the cases nobody thought to write live.

Properties

property why it matters
1-Lipschitz (globally, and near the surface) Sphere tracing steps by the returned distance. A field that over-reports makes the march overshoot — which is how thin walls turn into holes in a render.
finite everywhere NaN/Inf propagates into codegen and the mesher
union commutative, idempotent, exactly min at k=0 boolean algebra the codegen relies on
translation commutes with evaluation catches inverse-transform sign errors, which move a shape the wrong way
smooth union never exceeds sharp union a blend must add a fillet, not carve material

Design notes

Deterministic, no new dependency. A seeded mulberry32 PRNG rather than a fuzzing library — failures reproduce exactly and CI cannot flake.

The generator is restricted to shapes that claim exactness. ellipsoid and cone are approximations in their usual formulations, patterns break the bound at cell boundaries, and text is piecewise. Asserting exactness of those would produce noise rather than findings. Exclusions are documented with reasons in the file.

Two guards against passing for the wrong reason. The near-surface test bisects to the zero level set, because random points in an 8-unit cube mostly land far from any surface where the field is smooth and the check is weak. And it asserts probed > 50, so a bisection that never finds surfaces fails rather than silently skipping every sample.

Verified by mutation

Over-reporting sphere distance by 1.5x in evaluate.ts makes both Lipschitz tests fail, at ratios of 1.500x, 1.497x, 1.496x, with the failure output naming the offending trees.

That check is what makes the result meaningful — without it, "8 tests pass" is indistinguishable from "8 tests that assert nothing."

Worth recording that my first mutation attempt was invalid: I assumed non-uniform scale breaks the Lipschitz bound, without reading evaluate.ts:113, where the Math.min(sx, sy, sz) factor already handles it conservatively — and the generator pins scale to 1, so the mutation never executed. Had I stopped there I would have concluded the tests were toothless when the real problem was my choice of defect.

Result

No bugs found. The evaluator is sound on every property tested. A negative result, but a verified one — and the generator is reusable for the GPU differential tests when those become possible.

One honest limit: I predicted only the near-surface variant would catch the mutation, and both did. A uniform 1.5x over-report shows up everywhere, so random sampling finds it fine. The near-surface variant should matter for localized violations at boolean seams, but I have not demonstrated a case where it catches something the plain one misses.

🤖 Generated with Claude Code

kmatzen and others added 2 commits July 18, 2026 21:11
Fixes #61.

It is TypeScript's incremental-build cache, not source. `npm run build`
runs `tsc -b`, which rewrites it, so anyone who builds locally gets a dirty
working tree and the file lands in unrelated diffs as churn. Its contents are
machine-specific, so concurrent developers would conflict on it continuously.

Nothing consumes the committed copy -- tsc -b regenerates it when absent, at
the cost of one non-incremental build.

dist/ and test-results/ were already ignored, so this was the only tracked
build artifact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Part of #52 -- the CPU-side half. The differential CPU-vs-GLSL testing that
issue also proposes needs a headless WebGL context and is not included.

evaluate.test.ts checks hand-picked points on hand-built shapes. These check
invariants that must hold for any tree: 1-Lipschitz (globally and near the
surface), finiteness, union commutative / idempotent / exactly min at k=0,
translation commuting with evaluation, and smooth union never exceeding sharp
union.

The Lipschitz bound is the load-bearing one. Sphere tracing steps by the
returned distance, so a field that over-reports makes the march overshoot the
surface -- which is how thin walls turn into holes in a render.

Deterministic by construction: a seeded mulberry32 PRNG rather than a fuzzing
library, so failures reproduce exactly, CI cannot flake, and no dependency is
added.

The generator is restricted to shapes that claim exactness. ellipsoid and
cone are approximations in their usual formulations, patterns break the bound
at cell boundaries, and text is piecewise -- asserting exactness of those
would produce noise rather than findings. The exclusions are documented with
reasons in the file.

Two guards against the tests passing for the wrong reason:

- The near-surface test bisects to the zero level set, because random points
  in an 8-unit cube mostly land far from any surface where the field is
  smooth and the check is weak.
- It asserts probed > 50, so a bisection that never finds surfaces fails
  rather than silently skipping every sample.

Verified by mutation: over-reporting sphere distance by 1.5x in evaluate.ts
makes both Lipschitz tests fail at ratios of 1.500x, 1.497x, 1.496x, and the
failure output names the offending trees. Without that check, "8 tests pass"
would not be distinguishable from "8 tests that assert nothing".

No bugs found. The evaluator is sound on every property tested.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kmatzen
kmatzen merged commit 2670cda into main Jul 25, 2026
4 checks passed
@kmatzen
kmatzen deleted the sdf-property-tests branch July 25, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant