From 43f58229b150283687defd94321021f5eba2c427 Mon Sep 17 00:00:00 2001 From: Dan Reynolds Date: Wed, 5 Aug 2026 10:42:24 -0400 Subject: [PATCH 1/2] Exp 263: attribute the per-row cost to the right layer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three errors, all flattering the same conclusion in the wrong direction. The headline 2.9x divided the raw 396 B/row marginal by the payload. But a one-column and a six-column read of the same table make the same btree leaves resident, so the id lane's 140 B/row is a per-row storage-side cost independent of how the result is represented. Differencing the projections isolates the Dart side at 256 B/row for the five non-key columns — 1.9x their cell content, not 2.9x. The "~280 B/row theoretical minimum" undercounted: a proper accounting of those five columns is ~296 B (four OneByteStrings at a 24 B header plus rounded data, one boxed double, five values-list slots), against which the measured 256 B/row comes in BELOW. A future runner reading 280 would have inferred ~40% of headroom where there is none. The Row facade should not have been in that accounting at all — the iterator creates those objects transiently and only the ResultSet is retained. The payload denominator averaged the first 100 rows while the lanes read up to 20,000, and both `Item $i` and the description's `$i` grow with the row index: 137.8 B against a true 142.9 B. Adds claim 263.4 for the shared storage-side term — bounded (scales with rows, indifferent to column count) but not discriminated between mmap residency, WAL pages and a cold reader cache — and a nextSignal that a per-row memory figure must difference two projections or it charges the storage engine's work to the row representation. Co-Authored-By: Claude Opus 5 --- .../select_memory_decomposition.dart | 10 ++- ...-00Z-exp263-select-memory-decomposition.md | 33 +++++++-- .../263-select-memory-decomposition.md | 72 ++++++++++++------- experiments/index/263.json | 2 +- experiments/signals/entries/263.json | 31 ++++++-- 5 files changed, 109 insertions(+), 39 deletions(-) diff --git a/benchmark/experiments/select_memory_decomposition.dart b/benchmark/experiments/select_memory_decomposition.dart index fc670007..5eed8bf2 100644 --- a/benchmark/experiments/select_memory_decomposition.dart +++ b/benchmark/experiments/select_memory_decomposition.dart @@ -162,8 +162,14 @@ Future main(List args) async { ); } _assertAsciiFixture(); - final avgPayload = - List.generate(100, _payloadBytes).reduce((a, b) => a + b) / 100; + // Averaged over the whole seeded range, not a prefix of it: `Item $i` and + // the description's `$i` both grow with the row index, so a 100-row sample + // undercounts what a 20,000-row lane actually reads (137.8 B against 142.9). + var payloadTotal = 0; + for (var i = 0; i < _seedRows; i++) { + payloadTotal += _payloadBytes(i); + } + final avgPayload = payloadTotal / _seedRows; print( 'avg_payload_bytes_per_row=${avgPayload.toStringAsFixed(1)} ' '(UTF-8 cell content; not SQLite on-disk size)', diff --git a/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md b/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md index c74728a3..8a8015cb 100644 --- a/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md +++ b/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md @@ -45,11 +45,34 @@ Marginal cost per row, over two spans: | `bytes` | 676 B/row | 668 B/row | | `id` | 140 B/row | 155 B/row | -Average cell content in one seeded row: **137.8 bytes** — the UTF-8 length of -each TEXT cell plus 8 bytes per numeric. Exact for this fixture (every generated -cell is ASCII, which the harness asserts, so UTF-16 code units and UTF-8 bytes -coincide), and deliberately not SQLite's on-disk record size, which varint-encodes -integers and carries a per-row header. +Average cell content in one seeded row: **142.9 bytes** — the UTF-8 length of +each TEXT cell plus 8 bytes per numeric, averaged over the whole 20,000-row +seeded range. (An earlier revision reported 137.8 B, averaged over the first 100 +rows; `Item $i` and the description's `$i` both grow with the row index, so a +prefix undercounts what the lanes actually read.) Exact for this fixture — every +generated cell is ASCII, which the harness asserts, so UTF-16 code units and +UTF-8 bytes coincide — and deliberately not SQLite's on-disk record size, which +varint-encodes integers and carries a per-row header. + +## Attributing the marginal + +A one-column and a six-column read of the same table make the same btree leaves +resident, so `id`'s marginal is a per-row cost independent of the result's +representation. Differencing isolates the Dart side: + +| | B/row | +|---|---:| +| `select` marginal (6 columns) | 396 | +| − `id` marginal (shared, storage-side) | 140 | +| = Dart representation, five non-key columns | **256** | +| their cell content (142.9 − 8 for the id) | 134.9 | +| representation overhead | **1.9x** | + +First-principles accounting for those five columns: four `OneByteString`s at a +24 B header plus rounded data (~240 B), one boxed `_Double` (16 B), five slots in +the flat values list (40 B) = ~296 B. The measured 256 B/row is **below** it. The +`Row` facade contributes nothing to a retained result: the iterator creates those +objects transiently and only the `ResultSet` is held. `select`'s 1k→20k span reads lower than 1k→10k because results above `sacrificeSlotThreshold` (5,461 rows at 6 columns) return via `Isolate.exit`, diff --git a/experiments/263-select-memory-decomposition.md b/experiments/263-select-memory-decomposition.md index 890ed084..8f2cf595 100644 --- a/experiments/263-select-memory-decomposition.md +++ b/experiments/263-select-memory-decomposition.md @@ -34,9 +34,8 @@ count is the per-row marginal, the intercept is everything that does not scale. Declared before measuring: -- If the marginal is near the theoretical minimum for the row shape (~280 B/row: - four `OneByteString`s, one boxed double, six pointer slots, one `Row` facade), - the ratio is fixed cost and the premise is refuted — close claim 261.4. +- If the marginal is near the theoretical minimum for the row shape, the ratio is + fixed cost and the premise is refuted — close claim 261.4. - If the marginal is several times that, there is real per-row waste and it becomes an implementation candidate. @@ -61,19 +60,36 @@ question the ratio was posed against. Single live result, `maxRss`, one process per lane: -| mode | marginal (1k→10k) | vs payload | -|---|---:|---:| -| `select` | 396 B/row | **2.9×** | -| `bytes` | 676 B/row | 4.9× | -| `id` | 140 B/row | — | +| mode | marginal (1k→10k) | +|---|---:| +| `select` (6 columns) | 396 B/row | +| `bytes` | 676 B/row | +| `id` (1 column) | 140 B/row | + +**Difference the projections rather than dividing the raw marginal.** A +one-column and a six-column read of the same table both make the same btree +leaves resident — the leaf holds the whole row either way — so `id`'s 140 B/row +is a per-row cost that has nothing to do with how the result is represented. +Subtracting it isolates what the five extra columns actually cost in Dart: -The denominator is 137.8 B/row of cell content — the UTF-8 length of each TEXT -cell plus 8 bytes per numeric. It is exact for this fixture (every generated cell -is ASCII, and the harness asserts it, so code units and UTF-8 bytes coincide), -but it is deliberately *not* SQLite's on-disk record size, which stores integers -as varints and carries a per-row header. It is the bytes of actual content the -row represents, which is the thing a representation overhead should be measured -against. +| | B/row | +|---|---:| +| `select` marginal | 396 | +| − `id` marginal (shared, storage-side) | 140 | +| **= Dart representation, five non-key columns** | **256** | +| their cell content | 134.9 | +| **representation overhead** | **1.9×** | + +And against a first-principles accounting of what those five columns *must* +occupy — four `OneByteString`s at a 24 B header plus rounded data (~240 B), one +boxed `_Double` (16 B), five slots in the flat values list (40 B), so ~296 B — +the measured 256 B/row comes in **below** it. The `Row` facade costs nothing +here: it is three fields and a header, but the iterator creates those objects +transiently and only the `ResultSet` is retained. + +There is no headroom in the row representation. Two independent routes say so: +the differenced cost is 0.86× a straightforward accounting, and the raw marginal +is ~0.9× the same accounting once the storage-side term is included. And the floor, which is where the 60× actually lived: @@ -90,10 +106,19 @@ a floor of ~32.8 MB, of which 14 MB is the Dart VM before resqlite exists at all seeding. Dividing a peak that is ~90% fixed-and-setup cost by the payload is what produced 60×. -At 396 B/row against 137.8 B of cell content, `select()` carries a **2.9×** -representation overhead — and against the ~280 B/row theoretical minimum for this -row shape it is within about 1.4×. That gap is page granularity and heap slack, -not a structure worth rewriting. +The denominator above is cell content: the UTF-8 length of each TEXT cell plus +8 bytes per numeric, averaged over the whole seeded range. It is exact for this +fixture — every generated cell is ASCII and the harness asserts it, so UTF-16 +code units and UTF-8 bytes coincide — and it is deliberately *not* SQLite's +on-disk record size, which varint-encodes integers and carries a per-row header. + +What the ~140 B/row shared term *is* has not been established, only bounded: it +scales with rows read, it is indifferent to how many columns are projected, and +resqlite opens connections with `mmap_size = 256 MB` and `cache_size = -8192` +(8 MB), so mmap'd file pages becoming resident, WAL pages, and a cold reader page +cache are all live candidates. Whichever it is, it is the storage engine making +scanned data resident, which is what a database does, and it is governed by the +tuning exps 016/021 chose rather than by anything the row path controls. Two secondary findings worth keeping: @@ -111,11 +136,10 @@ Two secondary findings worth keeping: ## Outcome -**Accepted as measurement; premise refuted.** Claim 261.4's ~60× is closed: the -marginal cost of a `select()` result is 2.9× its payload, near the floor of what -a `List>` of Dart strings can achieve, and the ratio it was -derived from was dominated by a fixed floor and by retained garbage from repeated -reads. +**Accepted as measurement; premise refuted.** Claim 261.4's ~60× is closed. The +Dart representation of a row costs 1.9× its cell content and comes in below a +first-principles accounting of what it must occupy; the ratio 261.4 was derived +from was dominated by a fixed floor and by retained garbage from repeated reads. Nothing here reopens a result-shape rewrite; it removes the one piece of evidence that might have. If resqlite's memory footprint is ever a target, the numbers say diff --git a/experiments/index/263.json b/experiments/index/263.json index 79bddbff..48b6bfc2 100644 --- a/experiments/index/263.json +++ b/experiments/index/263.json @@ -1,7 +1,7 @@ { "file": "263-select-memory-decomposition.md", "title": "The 60x memory ratio was mostly floor", - "impact": "Exp 261 recorded a ~60x ratio between a 10k-row `select()`'s peak RSS (~95 MB) and its payload (~1.5 MB) as claim 261.4, `never decomposed`, with `List>` overhead named as a partial cause. Decomposed here by holding seeding constant at 20,000 rows and varying only the timed statement's LIMIT, so the slope is the per-row marginal and the intercept is everything that does not scale. The measurement configuration was itself the first finding: exp 261's 5 warmup + 21 timed reads accumulate up to 26 results of retained garbage because RSS never falls, reporting 99 MB where one read held live reports 36.1 MB. On the single-result measurement `select()` costs 396 B/row against 137.8 B of cell content (UTF-8 TEXT length plus 8 B per numeric; exact for this all-ASCII fixture, and deliberately not SQLite's on-disk record size) — a 2.9x representation overhead, within ~1.4x of the ~280 B/row theoretical minimum for four OneByteStrings, a boxed double, six pointer slots and a Row facade. The floor is where the 60x lived: 14.0 MB is the bare Dart AOT process, 20.5 MB after resqlite opens and spawns its pool, 32.8 MB after seeding, and 36.1 MB with a live 10k-row result — so the result is the smallest term and the ratio divided a ~90%-fixed peak by the payload. Premise refuted; claim 261.4 closed. Two secondary findings: `selectBytes` costs MORE memory per row than `select` (676 vs 396 B/row) because JSON re-encodes every column name per row and quotes every value — it avoids Dart objects, not bytes, so it is the wrong tool for reducing memory as opposed to allocation churn; and the sacrifice path shows up as sub-linearity (396 B/row over 1k-10k vs 337 over 1k-20k) because results above sacrificeSlotThreshold end the worker and return its heap, so any memory comparison spanning that threshold measures transport as much as representation. If footprint is ever a target the numbers say it is the isolate pool, not the row shape — but exp 105 already made pool size throughput-critical, so that is a trade rather than a free win.", + "impact": "Exp 261 recorded a ~60x ratio between a 10k-row `select()`'s peak RSS (~95 MB) and its payload (~1.5 MB) as claim 261.4, `never decomposed`, with `List>` overhead named as a partial cause. Decomposed here by holding seeding constant at 20,000 rows and varying only the timed statement's LIMIT, so the slope is the per-row marginal and the intercept is everything that does not scale. The measurement configuration was itself the first finding: exp 261's 5 warmup + 21 timed reads accumulate up to 26 results of retained garbage because RSS never falls, reporting 99 MB where one read held live reports 36.1 MB. On the single-result measurement `select()` costs 396 B/row, but that raw marginal must not be divided by the payload: a one-column and a six-column read of the same table make the same btree leaves resident, so the 1-column `id` lane's 140 B/row is a per-row storage-side cost independent of how the result is represented. Differencing isolates the Dart side at 256 B/row for the five non-key columns against 134.9 B of their cell content — a 1.9x representation overhead — and against a first-principles accounting of what those columns must occupy (four OneByteStrings at a 24 B header plus rounded data ~240 B, one boxed double 16 B, five values-list slots 40 B, so ~296 B) the measurement comes in BELOW it. The Row facade contributes nothing to a retained result: the iterator creates those objects transiently and only the ResultSet is held. The floor is where the 60x lived: 14.0 MB is the bare Dart AOT process, 20.5 MB after resqlite opens and spawns its pool, 32.8 MB after seeding, and 36.1 MB with a live 10k-row result — so the result is the smallest term and the ratio divided a ~90%-fixed peak by the payload. Premise refuted; claim 261.4 closed. Two secondary findings: `selectBytes` costs MORE memory per row than `select` (676 vs 396 B/row) because JSON re-encodes every column name per row and quotes every value — it avoids Dart objects, not bytes, so it is the wrong tool for reducing memory as opposed to allocation churn; and the sacrifice path shows up as sub-linearity (396 B/row over 1k-10k vs 337 over 1k-20k) because results above sacrificeSlotThreshold end the worker and return its heap, so any memory comparison spanning that threshold measures transport as much as representation. If footprint is ever a target the numbers say it is the isolate pool, not the row shape — but exp 105 already made pool size throughput-critical, so that is a trade rather than a free win.", "status": "accepted", "link": "" } diff --git a/experiments/signals/entries/263.json b/experiments/signals/entries/263.json index c3aa377d..a5826191 100644 --- a/experiments/signals/entries/263.json +++ b/experiments/signals/entries/263.json @@ -1,8 +1,12 @@ { - "directions": ["result-transfer-shape", "measurement-system"], + "directions": [ + "result-transfer-shape", + "measurement-system" + ], "outcomeClass": "accepted_measurement", "changedBeliefs": [ - "The ~60x payload-to-peak ratio recorded as claim 261.4 does not describe the result representation. Holding seeding constant and varying only the read shows `select()` costs 396 B/row against 137.8 B of cell content (UTF-8 TEXT length plus 8 B per numeric, exact for this all-ASCII fixture and not SQLite's on-disk record size) — 2.9x, and within ~1.4x of the ~280 B/row theoretical minimum for four OneByteStrings, a boxed double, six pointer slots and a Row facade. The ratio divided a peak that is ~90% fixed and setup cost by the payload.", + "The ~60x payload-to-peak ratio recorded as claim 261.4 does not describe the result representation. Two corrections were needed to say what does. First, the raw 396 B/row marginal cannot be divided by the payload: a one-column and a six-column read of the same table make the same btree leaves resident, so the `id` lane's 140 B/row is a per-row storage-side cost independent of representation, and differencing the projections isolates the Dart side at 256 B/row for the five non-key columns. Second, that is 1.9x their 134.9 B of cell content, and BELOW a first-principles accounting of what they must occupy (~296 B: four OneByteStrings at a 24 B header plus rounded data, one boxed double, five values-list slots). The row representation has no headroom.", + "The `Row` facade costs nothing in a retained result. It is three fields and a header, but the iterator creates those objects transiently and only the `ResultSet` is held, so a per-row facade charge does not belong in an accounting of what a result occupies.", "A repeated-read RSS measurement is not a measurement of a result. RSS never falls, so exp 261's 5 warmup + 21 timed reads accumulated up to 26 results of retained garbage and reported 99 MB for a 10k-row lane where one read held live reports 36.1 MB — 2.7x. Both answer real questions; only the second answers `what does a result cost`. Any future memory lane must say which it is measuring.", "The floor dominates every small and medium read, and it is mostly not resqlite's row handling. 14.0 MB is a bare AOT Dart process before resqlite exists; open plus a spawned pool is 20.5 MB; seeding 20,000 rows adds ~12 MB more. A live 10,000-row result adds 3.6 MB on top — the smallest term in the stack.", "`selectBytes` uses MORE memory per row than `select`: 676 B/row against 396. It removes Dart objects, which is what it has always claimed, but JSON repeats every column name on every row and quotes and escapes every value, so the encoded bytes exceed the object graph they replace. It is the right tool for allocation churn and the wrong one for footprint.", @@ -11,11 +15,17 @@ "claims": [ { "id": "263.1", - "text": "A `select()` result costs 396 B/row (1k->10k span) against 137.8 B of cell content on the canonical 6-column product row (UTF-8 TEXT length plus 8 B per numeric; exact for this all-ASCII fixture, not SQLite's on-disk record size) — a 2.9x representation overhead, within ~1.4x of the theoretical minimum for the row shape. Measured as a single live result with one process per lane, seeding held constant at 20,000 rows so the slope is purely the read.", - "conditions": "M1 Pro · AOT CLI bundle · maxRss, one process per lane, single read held live · 2026-08", + "text": "The Dart representation of a `select()` row costs 256 B/row for the canonical product row's five non-key columns — obtained by differencing the 6-column marginal (396 B/row) against the 1-column `id` marginal (140 B/row), since both projections make the same btree leaves resident and the shared term is storage-side rather than representational. That is 1.9x their 134.9 B of cell content, and 0.86x a first-principles accounting of what they must occupy (~296 B). Measured as a single live result, one process per lane, seeding held constant at 20,000 rows.", + "conditions": "M1 Pro · AOT CLI bundle · maxRss, one process per lane, single read held live · payload averaged over the full seeded range · 2026-08", "edges": [ - { "type": "supersedes", "target": "261.4" }, - { "type": "dependsOn", "target": "261.1" } + { + "type": "supersedes", + "target": "261.4" + }, + { + "type": "dependsOn", + "target": "261.1" + } ] }, { @@ -27,12 +37,19 @@ "id": "263.3", "text": "`selectBytes` costs 676 B/row against `select`'s 396 B/row on the same rows. It avoids Dart object allocation, not bytes: JSON repeats every column name per row and quotes and escapes every value. Reach for it to reduce allocation churn, not footprint.", "conditions": "M1 Pro · AOT · maxRss, one process per lane, single read held live · 2026-08" + }, + { + "id": "263.4", + "text": "A per-row cost of ~140 B/row is shared by every projection of the same table and is not part of the result representation: the 1-column `id` lane pays it as fully as the 6-column `select` lane, because a btree leaf holds the whole row regardless of what is projected. Its mechanism is bounded but not established — it scales with rows read, is indifferent to column count, and resqlite opens connections with mmap_size = 256 MB and cache_size = -8192, so mmap'd file pages becoming resident, WAL pages, and a cold reader page cache are all live candidates. Any future per-row memory figure must difference two projections or it charges this to the wrong layer.", + "conditions": "M1 Pro · AOT · maxRss, one process per lane · mechanism not discriminated · 2026-08" } ], "nextSignals": [ "Claim 261.4 is closed by 263.1. Do not reopen a result-shape rewrite on payload-to-peak-ratio grounds — exps 081, 251 and 258 closed storage rewrites on their own evidence, and the one piece of contrary evidence has now been removed rather than strengthened.", + "Difference two projections before quoting a per-row memory cost. The raw marginal includes a ~140 B/row storage-side term (claim 263.4) that a one-column read pays as fully as a six-column one; dividing it by the payload charges the storage engine's work to the row representation, which is how this experiment's own first revision overstated the overhead as 2.9x.", "State which memory question a lane answers. `benchmark/experiments/select_memory_decomposition.dart` runs both: `--warmup=0 --reads=1` for the cost of one live result, the default for what a repeatedly-reading process retains. They differ by 2.7x on the same lane, so an unlabelled figure is not interpretable.", "If footprint ever becomes a target, the isolate pool is the term to attack, not the row representation — 20.5 MB is resident before a row is read. Exp 105 established pool size is throughput-critical, so it is a trade; nothing here says the trade is worth making.", - "Keep a memory sweep on one side of `sacrificeSlotThreshold`, or tag lanes by which side they fall on. Crossing it makes the marginal sub-linear because `Isolate.exit` returns the worker's heap, which reads as a representation win and is not one." + "Keep a memory sweep on one side of `sacrificeSlotThreshold`, or tag lanes by which side they fall on. Crossing it makes the marginal sub-linear because `Isolate.exit` returns the worker's heap, which reads as a representation win and is not one.", + "If the ~140 B/row shared term ever needs pinning down, the discriminating test is cheap: hold row count fixed and widen the row (e.g. triple the description column). If the term tracks row width rather than row count it is the storage engine making scanned bytes resident and the question is closed; if it does not move, something else holds it." ] } From 59da79c42b4d7cd2ec48a6911ed4569fecd2d7b4 Mon Sep 17 00:00:00 2001 From: Dan Reynolds Date: Wed, 5 Aug 2026 10:50:20 -0400 Subject: [PATCH 2/2] Exp 263: sample the payload average instead of materializing 20k rows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Averaging over the full seeded range built every row to measure it — ~100,000 transient strings allocated before any lane runs, ahead of an RSS measurement that cannot see them released. The floor reads 20.5-20.6 MB either way, so nothing moved, but a needless allocation burst in front of a memory probe is a hazard whether or not it fired. Now sampled at a stride of 100 across the range, which captures the digit-width distribution that made a prefix wrong in the first place and lands within 0.01 B of the exact mean while allocating ~140 KB. The sweep results predate the denominator fix entirely: the payload average is an arithmetic property of the fixture, not an input to any measurement. Co-Authored-By: Claude Opus 5 --- .../select_memory_decomposition.dart | 20 ++++++++++++++----- ...-00Z-exp263-select-memory-decomposition.md | 13 +++++++++--- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/benchmark/experiments/select_memory_decomposition.dart b/benchmark/experiments/select_memory_decomposition.dart index 5eed8bf2..25d6868e 100644 --- a/benchmark/experiments/select_memory_decomposition.dart +++ b/benchmark/experiments/select_memory_decomposition.dart @@ -162,14 +162,24 @@ Future main(List args) async { ); } _assertAsciiFixture(); - // Averaged over the whole seeded range, not a prefix of it: `Item $i` and - // the description's `$i` both grow with the row index, so a 100-row sample - // undercounts what a 20,000-row lane actually reads (137.8 B against 142.9). + // Sampled at a fixed stride *across* the seeded range rather than over a + // prefix of it. Both `Item $i` and the description's `$i` grow with the row + // index, so a prefix undercounts what a 20,000-row lane reads (137.8 B + // against 142.9); a stride captures the digit-width distribution instead. + // + // Strided rather than exhaustive because this runs before any lane does, and + // `_payloadBytes` builds the row to measure it. Materializing all 20,000 rows + // would allocate megabytes of transient strings ahead of an RSS measurement + // that cannot see them released. 200 samples land within 0.01 B of the exact + // mean and allocate ~140 KB. + const payloadStride = 100; var payloadTotal = 0; - for (var i = 0; i < _seedRows; i++) { + var payloadSamples = 0; + for (var i = 0; i < _seedRows; i += payloadStride) { payloadTotal += _payloadBytes(i); + payloadSamples++; } - final avgPayload = payloadTotal / _seedRows; + final avgPayload = payloadTotal / payloadSamples; print( 'avg_payload_bytes_per_row=${avgPayload.toStringAsFixed(1)} ' '(UTF-8 cell content; not SQLite on-disk size)', diff --git a/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md b/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md index 8a8015cb..fb7e86b0 100644 --- a/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md +++ b/benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md @@ -47,12 +47,19 @@ Marginal cost per row, over two spans: Average cell content in one seeded row: **142.9 bytes** — the UTF-8 length of each TEXT cell plus 8 bytes per numeric, averaged over the whole 20,000-row -seeded range. (An earlier revision reported 137.8 B, averaged over the first 100 -rows; `Item $i` and the description's `$i` both grow with the row index, so a +seeded range, sampled at a fixed stride. (An earlier revision reported 137.8 B, +averaged over the first 100 rows; `Item $i` and the description's `$i` both grow with the row index, so a prefix undercounts what the lanes actually read.) Exact for this fixture — every generated cell is ASCII, which the harness asserts, so UTF-16 code units and UTF-8 bytes coincide — and deliberately not SQLite's on-disk record size, which -varint-encodes integers and carries a per-row header. +varint-encodes integers and carries a per-row header. Strided rather than +exhaustive on purpose: the average is computed before any lane runs, and building +all 20,000 rows to measure them would allocate megabytes of transient strings +ahead of an RSS measurement that cannot see them released. 200 samples land +within 0.01 B of the exact mean. The `open` lane reads 20.5-20.6 MB either way, +so this is a hazard removed rather than a number changed — and the sweep above +predates the denominator fix entirely, since that figure is an arithmetic +property of the fixture rather than an input to any measurement. ## Attributing the marginal