bench: compute-bound benchmark harness with hand-written Python baselines - #4
Merged
Conversation
…ines Three benchmarks (expr_eval: ADT trees + nested matching; collatz: call/integer throughput; map_build: fold-pass dict lowering + lookups), each paired with the Python a Pythonista would write by hand as the ceiling reference. bench/run.py times both sides wall-clock (median of N after a warmup, spread reported, byte-identical-output check); --python selects the interpreter so the same harness measures GraalPy/PyPy/mypyc variants later. Also records the ROADMAP "Performance beyond CPython" scoping this serves, with CPython 3.14.6 status quo: expr_eval 2.37x, collatz 1.18x, map_build 1.64x vs hand-written. Claude-Session: https://claude.ai/code/session_011PBhfjrzsUtwupYtJtzAWu
This was referenced Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measurement infrastructure for the ROADMAP "Performance beyond CPython" scoping (also included here): three compute-bound benchmarks, each as a
.pyfunprogram paired with a hand-written Python baseline as the ceiling reference, and a wall-clock runner.expr_eval— builds/simplifies/evaluates ADT expression trees (allocation + nested pattern matching): 2.37x vs hand-writtencollatz— recursive arithmetic (function-call + integer throughput): 1.18xmap_build— 500k string-keyed Map inserts through the fold pass, then 500k lookups: 1.64xbench/run.py: median of N after a warmup, spread reported, refuses to report timings unless both sides print byte-identical output;--pythonselects the interpreter (GraalPy/PyPy later),--skip-compilereusesbench/out/*.pyfor hand-edited experiments (mypyc). Wall-clock only, no call-graph profilers, per the fold-pass postmortem in ROADMAP.All inputs LCG-derived: deterministic, no data files. Numbers above from CPython 3.14.6 on the dev machine, release compiler at HEAD.
https://claude.ai/code/session_011PBhfjrzsUtwupYtJtzAWu