Skip to content

Commit 2375f68

Browse files
TheMeinerLPclaude
andauthored
ci: bring the header comment level with the class-based planner (#35)
The planner changed in #33 and the comment above it did not. It still said the suite is 494 configurations — JMH counts 488 — and it still explained the split as a balancing decision. Balance is the secondary reason. The unit is the class because splitting a comparison across runners measures the hardware, which is a correctness argument and the one a reader needs first. Claude-Session: https://claude.ai/code/session_01QzEdy5fN5JKGxJo8gwtNeu Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent ec35f61 commit 2375f68

1 file changed

Lines changed: 7 additions & 5 deletions

File tree

.github/workflows/benchmark.yml

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ name: Benchmark
55
# custom — one filter, one job. For reproducing a single table or checking one change.
66
# full — the whole suite, split across parallel jobs.
77
#
8-
# Sequentially the suite is 494 configurations and several hours; a timeout at hour five would
8+
# Sequentially the suite is 488 configurations and several hours; a timeout at hour five would
99
# discard everything, including the classes that finished in minute three. Split into shards the
1010
# wall clock is roughly the longest single shard, and a failure costs one shard.
1111
#
@@ -18,10 +18,12 @@ name: Benchmark
1818
# would spend two to four minutes building and one measuring, so the build would be most of the
1919
# bill.
2020
#
21-
# Shards are balanced by configuration count, not by method count. JMH's -lp reports the parameter
22-
# cross product per method and it is wildly uneven — ScalingBenchmark declares 75 combinations per
23-
# method against 2 for the region file comparison — so splitting by method alone would leave one
24-
# shard running twenty times longer than another.
21+
# A shard is a whole class, and the balancing is by configuration count. The unit is the class for
22+
# correctness, not for balance: a comparison class holds both sides of its comparison, shards land
23+
# on different runners with different CPUs, and a split pair measures the hardware — observed once
24+
# at 2.13x where one machine gives 1.13x. The weighting matters because JMH's -lp reports a wildly
25+
# uneven cross product, ScalingBenchmark declaring 300 configurations against 8 for the region file
26+
# comparison, so ScalingBenchmark alone sets the wall clock.
2527
#
2628
# The plan is derived from the jar at run time. A hard-coded list is a count that goes stale, and
2729
# this suite went from sixteen classes to seventeen inside one day.

0 commit comments

Comments
 (0)