Exp 263: the 60x memory ratio was mostly floor

benchmark/experiments/select_memory_decomposition.dart

@@ -0,0 +1,289 @@
+// ignore_for_file: avoid_print
+//
+// Focused memory decomposition for [EXP-263]: where does a `select()`'s
+// resident memory actually go?
+//
+// [EXP-261](../../experiments/261-focused-memory-guard.md) measured the repo's
+// canonical 6-column product row at 10k rows peaking at ~95 MB while the table
+// holds roughly 1.5 MB of data, and flagged the ~60x ratio as never decomposed.
+// Its own instrument cannot decompose it — process RSS cannot resolve anything
+// below a doubling, and an AOT binary has no VM service to ask for heap
+// composition.
+//
+// What process RSS *can* do is separate fixed cost from marginal cost, if the
+// only thing that varies is the amount read. Every lane here seeds the same
+// 20,000-row table and differs only in how many rows the timed statement
+// returns, so a fit of peak RSS against row count gives the per-row marginal
+// directly, and the intercept is everything that does not scale with the read
+// (process, VM, connection, page cache, seeding).
+//
+// Three modes over the same rows isolate the parts:
+//
+//   select  — `select()`, the full Dart object graph (flat values list plus
+//             the lazy `Row` facade over it).
+//   bytes   — `selectBytes()`, the same rows serialized in C with no Dart
+//             object graph at all. The difference between this and `select`
+//             is what the Dart representation costs.
+//   id      — `select()` of the INTEGER primary key alone. Smis live inline in
+//             the values list, so this is structure without payload.
+//
+// The `select` sweep crosses `sacrificeSlotThreshold` (32768 structural slots,
+// so 5,461 rows at 6 columns) between its 5,000 and 7,500 row lanes. Results
+// above it return via `Isolate.exit` and end the worker; results below take a
+// `SendPort`. Lanes are tagged with which path they took, because a
+// discontinuity there is a transport artifact rather than a representation one
+// ([EXP-258](../../experiments/258-columnar-result-store.md)).
+//
+// Per [EXP-261](../../experiments/261-focused-memory-guard.md): the reported
+// figure is `maxRss`, and it is only per-lane clean when the lane had the
+// process to itself. Run one lane per process with `--lane=`.
+//
+// Usage:
+//   dart run benchmark/experiments/select_memory_decomposition.dart \
+//     [--reads=21] [--lane=select-5000]
+import 'dart:io';
+
+import 'package:resqlite/resqlite.dart' as resqlite;
+
+import '../shared/memory_probe.dart';
+
+/// Rows seeded into every lane's table, held constant so the only variable is
+/// how many of them the timed statement returns.
+const _seedRows = 20000;
+
+const _defaultReads = 21;
+const _defaultWarmup = 5;
+
+/// `sacrificeSlotThreshold` in `lib/src/reader/read_worker.dart`.
+const _sacrificeSlots = 32 * 1024;
+
+const _rowCounts = [1000, 2500, 5000, 7500, 10000, 20000];
+
+enum _Mode {
+  select('select', 6),
+  bytes('bytes', 6),
+  id('id', 1),
+
+  /// Open the database and read nothing. Isolates the fixed floor — VM,
+  /// native library, SQLite connections and the reader/writer isolate pool —
+  /// from anything the result contributes. Ignores the row count.
+  open('open', 0);
+
+  const _Mode(this.label, this.columns);
+  final String label;
+
+  /// Structural slots a row of this mode occupies, for the sacrifice estimate.
+  final int columns;
+}
+
+const _standardCreate = '''
+  CREATE TABLE items(
+    id INTEGER PRIMARY KEY,
+    name TEXT NOT NULL,
+    description TEXT NOT NULL,
+    value REAL NOT NULL,
+    category TEXT NOT NULL,
+    created_at TEXT NOT NULL
+  )
+''';
+const _standardInsert =
+    'INSERT INTO items(name, description, value, category, created_at) '
+    'VALUES (?, ?, ?, ?, ?)';
+List<Object?> _standardRow(int i) => [
+  'Item $i',
+  'This is a description for item number $i with some padding text to '
+      'simulate real data',
+  i * 1.5,
+  'category_${i % 10}',
+  '2026-04-0${(i % 9) + 1}T12:00:00Z',
+];
+
+/// Logical in-memory payload of one seeded row, in bytes: the UTF-8 length of
+/// each TEXT cell plus 8 bytes for each numeric one.
+///
+/// This is the denominator the per-row marginal is reported against, so be
+/// precise about what it is and is not. It is **not** SQLite's on-disk record
+/// size — integers are stored as varints and the row carries a header — and it
+/// is not what a Dart `String` occupies, which is the point of the comparison.
+/// It is the number of bytes of actual cell content the row represents.
+///
+/// `String.length` counts UTF-16 code units, which equals the UTF-8 byte count
+/// only for ASCII. Every cell this fixture generates is ASCII, and
+/// [_assertAsciiFixture] enforces that, so the two coincide here and the figure
+/// is exact for this fixture rather than an approximation of it.
+int _payloadBytes(int i) {
+  final row = _standardRow(i);
+  var bytes = 8; // id, INTEGER
+  for (final cell in row) {
+    bytes += cell is String ? cell.length : 8;
+  }
+  return bytes;
+}
+
+/// Fails loudly if the fixture ever stops being ASCII, which would silently
+/// turn [_payloadBytes] from an exact count into an undercount.
+void _assertAsciiFixture() {
+  for (var i = 0; i < 100; i++) {
+    for (final cell in _standardRow(i)) {
+      if (cell is! String) continue;
+      for (final unit in cell.codeUnits) {
+        if (unit > 0x7F) {
+          throw StateError(
+            'fixture is no longer ASCII, so _payloadBytes undercounts: $cell',
+          );
+        }
+      }
+    }
+  }
+}
+
+Future<void> main(List<String> args) async {
+  var reads = _defaultReads;
+  var warmup = _defaultWarmup;
+  String? only;
+  for (final arg in args) {
+    if (arg.startsWith('--reads=')) {
+      reads = int.parse(arg.substring('--reads='.length));
+    } else if (arg.startsWith('--warmup=')) {
+      warmup = int.parse(arg.substring('--warmup='.length));
+    } else if (arg.startsWith('--lane=')) {
+      only = arg.substring('--lane='.length);
+    } else {
+      throw ArgumentError('unknown argument: $arg');
+    }
+  }
+
+  print('=== select() memory decomposition ===');
+  print('seed_rows=$_seedRows warmup=$warmup reads_per_lane=$reads');
+  if (warmup == 0 && reads == 1) {
+    print(
+      'mode=single-live-result — one read, held alive across the sample, so '
+      'the marginal is one result rather than accumulated retention',
+    );
+  }
+  _assertAsciiFixture();
+  final avgPayload =
+      List.generate(100, _payloadBytes).reduce((a, b) => a + b) / 100;
+  print(
+    'avg_payload_bytes_per_row=${avgPayload.toStringAsFixed(1)} '
+    '(UTF-8 cell content; not SQLite on-disk size)',
+  );
+
+  for (final mode in _Mode.values) {
+    for (final rows in mode == _Mode.open ? const [0] : _rowCounts) {
+      final label = mode == _Mode.open ? 'open' : '${mode.label}-$rows';
+      if (only != null && label != only) continue;
+      await _runLane(
+        mode,
+        rows,
+        reads: reads,
+        warmup: warmup,
+        laneIsolated: only != null,
+      );
+    }
+  }
+}
+
+Future<void> _runLane(
+  _Mode mode,
+  int rows, {
+  required int reads,
+  required int warmup,
+  required bool laneIsolated,
+}) async {
+  final temp = await Directory.systemTemp.createTemp('bench_memdecomp_');
+  try {
+    final db = await resqlite.Database.open('${temp.path}/test.db');
+    await db.execute(_standardCreate);
+
+    if (mode == _Mode.open) {
+      // One trivial statement so the reader pool has actually spawned; the
+      // pool is lazy and a floor measured before it exists is not the floor a
+      // reading workload pays.
+      await db.select('SELECT id FROM items LIMIT 1');
+      final probe = MemoryProbe.start();
+      probe.sample();
+      final reading = probe.finish(laneIsolated: laneIsolated);
+      await db.close();
+      print(
+        'shape=open mode=open rows=0 slots=0 sacrifices=false '
+        '${reading.format()}',
+      );
+      return;
+    }
+
+    const chunk = 500;
+    for (var start = 0; start < _seedRows; start += chunk) {
+      final end = start + chunk < _seedRows ? start + chunk : _seedRows;
+      await db.executeBatch(_standardInsert, [
+        for (var r = start; r < end; r++) _standardRow(r),
+      ]);
+    }
+
+    final sql = switch (mode) {
+      _Mode.select => 'SELECT * FROM items ORDER BY id LIMIT ?',
+      _Mode.bytes => 'SELECT * FROM items ORDER BY id LIMIT ?',
+      _Mode.id => 'SELECT id FROM items ORDER BY id LIMIT ?',
+      // Unreachable: the open lane returns above, before any statement.
+      _Mode.open => throw StateError('open lane has no statement'),
+    };
+    final params = [rows];
+
+    // The result is held in `live` across the sample. Without that the VM may
+    // reclaim it before RSS is read, and the lane would measure a result that
+    // no longer exists.
+    Object? live;
+    Future<int> read() async {
+      if (mode == _Mode.bytes) {
+        final r = await db.selectBytes(sql, params);
+        live = r;
+        return r.rowCount;
+      }
+      final r = await db.select(sql, params);
+      live = r;
+      // Read a cell from every row so the lazy `Row` facade actually
+      // materializes. The cell *values* are built by `decodeQuery` either way
+      // — what this adds is the per-row `Row` object a consumer holds.
+      if (mode == _Mode.select) {
+        for (final row in r) {
+          if (row['name'] == null) throw StateError('null name');
+        }
+      }
+      return r.length;
+    }
+
+    for (var i = 0; i < warmup; i++) {
+      if (await read() != rows) {
+        throw StateError('lane ${mode.label}-$rows returned the wrong count');
+      }
+    }
+    live = null;
+
+    final probe = MemoryProbe.start();
+    for (var i = 0; i < reads; i++) {
+      if (await read() != rows) {
+        throw StateError('lane ${mode.label}-$rows returned the wrong count');
+      }
+      probe.sample();
+    }
+    if (live == null) throw StateError('result was not retained');
+    final reading = probe.finish(laneIsolated: laneIsolated);
+    await db.close();
+
+    // `selectBytes` never sacrifices — the result is native bytes, so
+    // `Isolate.exit` would need a copy first and saves nothing.
+    final slots = rows * mode.columns;
+    final sacrifices = mode != _Mode.bytes && slots > _sacrificeSlots;
+
+    print(
+      'shape=${mode.label}-$rows '
+      'mode=${mode.label} '
+      'rows=$rows '
+      'slots=$slots '
+      'sacrifices=$sacrifices '
+      '${reading.format()}',
+    );
+  } finally {
+    await temp.delete(recursive: true);
+  }
+}

benchmark/results/2026-08-05T15-30-00Z-exp263-select-memory-decomposition.md

@@ -0,0 +1,92 @@
+# Experiment 263: where a select()'s memory actually goes
+
+Collected 2026-08-05 on arm64 macOS 26.2 (Apple M1 Pro) with Dart 3.12.2, from
+`main` at `7225903`. Harness:
+[`benchmark/experiments/select_memory_decomposition.dart`](../experiments/select_memory_decomposition.dart).
+
+Every figure is `maxRss` with **one process per lane** (`--lane=`), per exp 261.
+
+## Method
+
+Exp 261 flagged the canonical 6-column product row at 10k rows peaking at ~95 MB
+against a table holding ~1.5 MB, a ~60x ratio, and noted its own instrument could
+not decompose it: process RSS cannot resolve below a doubling, and an AOT binary
+has no VM service to ask for heap composition.
+
+What RSS *can* do is separate fixed from marginal, if the only thing that varies
+is the amount read. Every lane seeds the same 20,000-row table and differs only
+in the timed statement's `LIMIT`, so the slope against row count is the per-row
+marginal and the intercept is everything that does not scale with the read.
+
+Two configurations, and the difference between them is the experiment's first
+finding:
+
+- `--warmup=5 --reads=21` — the shape exp 261 used. RSS never falls, so 26 reads
+  accumulate up to 26 results' worth of retained garbage.
+- `--warmup=0 --reads=1`, result held live across the sample — one result,
+  which is what "what does a result cost" actually asks.
+
+## Single live result (maxRss, MB)
+
+| rows | `select` | `bytes` | `id` |
+|---:|---:|---:|---:|
+| 1,000 | 32.7 | 33.0 | 32.8 |
+| 2,500 | 33.0 | 33.7 | 33.0 |
+| 5,000 | 33.9 | 35.3 | 33.3 |
+| 7,500 | 35.1 | 37.0 | 33.6 |
+| 10,000 | 36.1 | 38.8 | 34.0 |
+| 20,000 | 38.8 | 45.1 | 35.6 |
+
+Marginal cost per row, over two spans:
+
+| mode | 1k→10k | 1k→20k |
+|---|---:|---:|
+| `select` | 396 B/row | 337 B/row |
+| `bytes` | 676 B/row | 668 B/row |
+| `id` | 140 B/row | 155 B/row |
+
+Average cell content in one seeded row: **137.8 bytes** — the UTF-8 length of
+each TEXT cell plus 8 bytes per numeric. Exact for this fixture (every generated
+cell is ASCII, which the harness asserts, so UTF-16 code units and UTF-8 bytes
+coincide), and deliberately not SQLite's on-disk record size, which varint-encodes
+integers and carries a per-row header.
+
+`select`'s 1k→20k span reads lower than 1k→10k because results above
+`sacrificeSlotThreshold` (5,461 rows at 6 columns) return via `Isolate.exit`,
+which ends the worker and returns its heap. The sub-linearity is a transport
+artifact, not a representation one.
+
+## Fixed floor
+
+| stage | maxRss |
+|---|---:|
+| bare AOT Dart process (measured separately) | 14.0 MB |
+| + resqlite open, pool spawned, one trivial read (`open` lane) | 20.5 MB |
+| + seeding 20,000 rows via `executeBatch` (`id-1000` lane) | 32.8 MB |
+| + one live 10,000-row `select()` result | 36.1 MB |
+
+## Repeatability
+
+Three runs per lane, isolated processes, maxRss MB:
+
+| lane | runs |
+|---|---|
+| `open` | 20.5, 20.5, 20.5 |
+| `select-1000` | 32.8, 32.9, 32.8 |
+| `select-10000` | 36.8, 36.4, 36.8 |
+| `select-20000` | 38.9, 42.1, 38.9 |
+
+## Accumulated-retention configuration, for contrast
+
+The same lanes under `--warmup=5 --reads=21` (26 reads, nothing released):
+
+| rows | `select` | `bytes` | `id` |
+|---:|---:|---:|---:|
+| 1,000 | 33.9 | 39.4 | 33.3 |
+| 5,000 | 99.4 | 51.3 | 35.6 |
+| 10,000 | 99.0 | 74.5 | 38.4 |
+| 20,000 | 105.0 | 97.8 | 45.2 |
+
+This is the configuration exp 261 measured, and it is ~2.7x the single-result
+figure at 10,000 rows. It answers a real question — what a process doing
+repeated reads holds — but not the one the 60x ratio was posed against.