fix(storage): cache open HDF5 read handles for multi-bag bag files by pmrv · Pull Request #786 · pmrv/fleche

pmrv · 2026-07-23T12:14:37Z

Fixes the BagOfHoldingH5File multi-bag regression flagged in #625: since PR #746, every contains() (and each bag in list()) paid a full HDF5 file open/close, ~8.6× slower than the per-key layout (240 µs vs 28 µs).

What changed

bagofholding_file.py gains a process-wide cache of open read-only h5py.File handles (_BagHandleCache), keyed by absolute bag-file path:

A WeakValueDictionary indexes the open handles; a bounded strong-ref MRU (_MAX_OPEN_BAGS = 32) keeps the most recently used ones alive between operations. A handle evicted from the MRU is never closed eagerly — the strong reference is dropped and the interpreter closes the file once the last in-flight user releases it.
Every acquisition re-validates the handle against an (inode, mtime_ns, size) stat signature, so bags rewritten by other processes or other storage instances are detected and reopened. One stat (~1 µs) replaces one HDF5 open (~120 µs).
Cached handles are opened with locking=False: a long-lived reader holding HDF5's shared OS lock would otherwise make every write from another process fail for as long as it lives. Cross-process write mutual exclusion was always provided by the filelock sidecar locks, not by HDF5's locking, so nothing is lost.
Within a process, HDF5 refuses any open whose mode or locking flags differ from a live handle's. So every path that opens the bag file itself — put/_to_file, get/_from_file, _evict, rebag (H5Bag uses default flags) — runs inside _bag_writer(key): it closes the cached handle and holds a per-bag in-process lock across the operation. Readers (_contains, list) hold the same lock while using the handle, so a writer can never close a handle mid-read.
The cache is process-wide rather than literally per-instance for the same reason: a handle cached by one storage instance must be closable by any other instance (equal or not) addressing the same file, or that instance's writes would fail. The per-instance WeakKeyDictionary pattern used for lock tables would leave two non-equal instances on the same root able to brick each other's writes.
Fork-safety: the cache detects a changed pid and drops all inherited state (read-only handles close harmlessly in the child); locking=False falls back to a default-flag open on h5py < 3.5 / HDF5 < 1.12.1.

Numbers

op	before	after	pre-regression (#625 baseline)
`contains` (hit)	240 µs	29 µs	28 µs
`contains` (miss)	~240 µs	17 µs	—
`evict`	490 µs	unchanged	79 µs (per-key layout)

evict is deliberately untouched: it genuinely mutates the bag, so it must pay the filelock acquisition (~100 µs) plus an HDF5 write open/del/close (~130 µs+). That cost is structural for a durable per-key delete inside a shared file; caching write handles across operations would hold HDF5's exclusive state hostage against every other process. If evict throughput ever matters, the realistic lever is batching (one write open per bag for N evictions, e.g. a evict_many on the storage or transaction-style deferral), not handle caching.

`list()` — remaining cost and options

list() now reuses cached handles, so a warm repeated list() costs ~0 opens for up to _MAX_OPEN_BAGS bags. But a cold list() over a root with many bags still opens every file once, and roots with more than 32 bags will churn the MRU. Options, roughly in order of effort:

Leave it (this PR). Warm-path wins only; cold list() on a default prefix_length=2 root is up to 256 opens (~30 ms worst case).
Cache each bag's key list next to its handle, invalidated by the same stat signature. Cheap to add on top of this PR's infrastructure; makes repeated list() on an unchanged root free even beyond the MRU bound (key lists survive handle eviction). Doesn't help the truly cold first call.
A persistent per-root key index (sidecar manifest updated under the bag's filelock on put/evict). Makes cold list() one file read, but adds cost and a consistency obligation to every write, plus a repair story for crashes between bag write and index write. This is the only option that fixes the cold path, and the only one with real correctness surface.
prefix_length=0 already gives O(readdir) list() today for workloads that are list-heavy and don't hit the many-small-files problem multi-bagging exists to solve — worth remembering before building 3.

Happy to follow up with 2 (small) or 3 (needs design agreement) in a separate PR.

Behavioral notes

Code that opens a bag file directly with h5py.File(path) (default flags) in the same process while a cached handle is live will now get the flag-mismatch OSError; go through the storage API, or open with locking=False.
The stat signature uses st_mtime_ns: on filesystems with coarse mtime granularity, an external rewrite that changes neither size nor inode within the same timestamp tick could go briefly unnoticed by contains. Same-process writes are exact (explicit invalidation); cross-process reads were already best-effort.
Cached handles keep files open between operations; on Windows that makes deleting a bag file out from under a warm storage fail (_evict itself closes the handle under the bag lock before unlinking, so the backend's own deletes are safe; CI is Linux-only).

Testing

8 new tests pin: handle reuse across contains/list (open counting), invalidation on sibling-key put, get after a warm contains (the flag-mismatch trap), evict with a warm cache including file removal, writes from a second non-equal instance on the same root, external-process writes being neither blocked nor masked (subprocess test — also pins locking=False), and the MRU bound.
Full suite: 1640 passed, 11 skipped. ty check src/ clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014DWRGyuEoH3qXNSQbSXddp

Generated by Claude Code

The multi-bag layout introduced in PR #746 made every contains() and list() pay a full HDF5 file open/close (~120 us), regressing contains 8.6x against the per-key layout (240 us vs 28 us, #625). Add a process-wide cache of open read-only h5py handles, keyed by absolute path: a WeakValueDictionary index plus a bounded strong-ref MRU that keeps the most recently used handles alive between operations. Cached handles are validated against an (inode, mtime_ns, size) stat signature on every acquisition, so bags rewritten by other processes or other storage instances are detected and reopened. Handles are opened with locking=False so a long-lived cached reader cannot make writes from other processes fail. Within a process, HDF5 rejects any open whose mode or locking flags differ from a live handle's, so every operation that opens the bag file itself - put, get, evict, and rebag - closes the cached handle first and holds a per-bag in-process lock across the operation; readers hold the same lock while using the handle, so a writer can never close it mid-read. The cache is process-wide rather than per-instance because a handle cached by one instance must be closable by any other instance addressing the same file. contains(hit) drops from 240 us back to the pre-regression ~29 us. evict is unchanged: it genuinely writes, and the filelock acquisition plus HDF5 write open/close are structural for a durable per-key delete inside a shared file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014DWRGyuEoH3qXNSQbSXddp

github-actions · 2026-07-23T12:19:37Z

Benchmark Results

Baseline: 071d395 (cached)

Delta

Significant changes (|Δ| > 10%): 161 — 79 other rows hidden

Value Storage

configuration	workload	function	time @ `071d395`	time @ head	Δ vs base
BagOfHoldingH5File	small_strings	contains_hit	280 µs	45 µs	🟢 -83.9%
BagOfHoldingH5File	nested_structures	contains_hit	270 µs	46 µs	🟢 -83.0%
BagOfHoldingH5File	numpy_arrays	contains_hit	270 µs	46 µs	🟢 -83.0%
CloudpickleFile_Signed	small_strings	contains_miss	33 µs	24 µs	🟢 -27.3%
DillFile_Signed	small_strings	contains_miss	33 µs	24 µs	🟢 -27.3%
PickleFile	numpy_arrays	contains_miss	33 µs	24 µs	🟢 -27.3%
CloudpickleFile_Signed	nested_structures	contains_miss	34 µs	25 µs	🟢 -26.5%
CloudpickleFile_Signed	nested_structures	evict	58 µs	43 µs	🟢 -25.9%
PickleFile_Signed	nested_structures	evict	58 µs	43 µs	🟢 -25.9%
PickleFile	nested_structures	evict	58 µs	43 µs	🟢 -25.9%
DillFile_Signed	numpy_arrays	evict	110 µs	82 µs	🟢 -25.5%
DillFile	numpy_arrays	evict	110 µs	82 µs	🟢 -25.5%
PickleFile_Signed	numpy_arrays	evict	110 µs	82 µs	🟢 -25.5%
DillFile_Signed	numpy_arrays	contains_miss	32 µs	24 µs	🟢 -25.0%
CloudpickleFile	small_strings	load	280 µs	210 µs	🟢 -25.0%
DillFile_Signed	nested_structures	evict	57 µs	43 µs	🟢 -24.6%
DillFile	nested_structures	evict	57 µs	43 µs	🟢 -24.6%
CloudpickleFile_Signed	numpy_arrays	evict	110 µs	83 µs	🟢 -24.5%
PickleFile	numpy_arrays	evict	110 µs	83 µs	🟢 -24.5%
CloudpickleFile	numpy_arrays	contains_miss	33 µs	25 µs	🟢 -24.2%
CloudpickleFile_Signed	nested_structures	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile	numpy_arrays	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile	nested_structures	contains_hit	33 µs	25 µs	🟢 -24.2%
CloudpickleFile	numpy_arrays	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile_Signed	nested_structures	contains_miss	33 µs	25 µs	🟢 -24.2%
DillFile_Signed	nested_structures	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile_Signed	small_strings	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile_Signed	numpy_arrays	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile	small_strings	contains_miss	33 µs	25 µs	🟢 -24.2%
CloudpickleFile_Signed	small_strings	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile	nested_structures	contains_miss	33 µs	25 µs	🟢 -24.2%
CloudpickleFile	small_strings	contains_miss	33 µs	25 µs	🟢 -24.2%
CloudpickleFile	nested_structures	contains_hit	33 µs	25 µs	🟢 -24.2%
CloudpickleFile	nested_structures	contains_miss	33 µs	25 µs	🟢 -24.2%
PickleFile	small_strings	contains_miss	33 µs	25 µs	🟢 -24.2%
PickleFile	nested_structures	contains_hit	33 µs	25 µs	🟢 -24.2%
PickleFile_Signed	small_strings	contains_hit	33 µs	25 µs	🟢 -24.2%
PickleFile_Signed	nested_structures	contains_hit	33 µs	25 µs	🟢 -24.2%
PickleFile_Signed	nested_structures	contains_miss	33 µs	25 µs	🟢 -24.2%
CloudpickleFile_Signed	numpy_arrays	contains_miss	33 µs	25 µs	🟢 -24.2%
PickleFile	nested_structures	contains_miss	33 µs	25 µs	🟢 -24.2%
PickleFile_Signed	small_strings	contains_miss	33 µs	25 µs	🟢 -24.2%
PickleFile_Signed	numpy_arrays	contains_miss	33 µs	25 µs	🟢 -24.2%
PickleFile	numpy_arrays	contains_hit	33 µs	25 µs	🟢 -24.2%
DillFile	nested_structures	load	290 µs	220 µs	🟢 -24.1%
DillFile_Signed	small_strings	evict	100 µs	76 µs	🟢 -24.0%
CloudpickleFile	numpy_arrays	evict	110 µs	84 µs	🟢 -23.6%
CloudpickleFile	nested_structures	evict	56 µs	43 µs	🟢 -23.2%
CloudpickleFile	nested_structures	load	270 µs	210 µs	🟢 -22.2%
PickleFile	small_strings	evict	100 µs	78 µs	🟢 -22.0%
CloudpickleFile_Signed	small_strings	evict	100 µs	78 µs	🟢 -22.0%
DillFile	small_strings	evict	100 µs	78 µs	🟢 -22.0%
CloudpickleFile	small_strings	evict	100 µs	78 µs	🟢 -22.0%
PickleFile_Signed	small_strings	evict	100 µs	78 µs	🟢 -22.0%
CloudpickleFile_Signed	numpy_arrays	contains_hit	32 µs	25 µs	🟢 -21.9%
CloudpickleFile	small_strings	contains_hit	32 µs	25 µs	🟢 -21.9%
DillFile	small_strings	contains_hit	32 µs	25 µs	🟢 -21.9%
DillFile	numpy_arrays	contains_miss	32 µs	25 µs	🟢 -21.9%
PickleFile	small_strings	contains_hit	32 µs	25 µs	🟢 -21.9%
PickleFile_Signed	numpy_arrays	contains_hit	32 µs	25 µs	🟢 -21.9%
DillFile	small_strings	load	280 µs	220 µs	🟢 -21.4%
PickleFile	nested_structures	load	280 µs	220 µs	🟢 -21.4%
PickleFile_Signed	nested_structures	load	290 µs	230 µs	🟢 -20.7%
CloudpickleFile_Signed	nested_structures	load	290 µs	230 µs	🟢 -20.7%
CloudpickleFile_Signed	small_strings	load	290 µs	230 µs	🟢 -20.7%
PickleFile_Signed	small_strings	load	290 µs	230 µs	🟢 -20.7%
Memory	numpy_arrays	load	15 µs	12 µs	🟢 -20.0%
CloudpickleFile	numpy_arrays	load	300 µs	240 µs	🟢 -20.0%
PickleFile	numpy_arrays	load	300 µs	240 µs	🟢 -20.0%
DillFile_Signed	nested_structures	load	300 µs	240 µs	🟢 -20.0%
DillFile_Signed	small_strings	load	300 µs	240 µs	🟢 -20.0%
DillFile	numpy_arrays	load	320 µs	260 µs	🟢 -18.8%
PickleFile	small_strings	load	270 µs	220 µs	🟢 -18.5%
BagOfHoldingH5File	nested_structures	evict	270 µs	220 µs	🟢 -18.5%
PickleFile_Signed	numpy_arrays	load	370 µs	320 µs	🟢 -13.5%
CloudpickleFile_Signed	numpy_arrays	load	370 µs	320 µs	🟢 -13.5%
Memory	numpy_arrays	save	15 µs	13 µs	🟢 -13.3%
BagOfHoldingH5File	small_strings	contains_miss	34 µs	30 µs	🟢 -11.8%
BagOfHoldingH5File	small_strings	save	1.7 ms	1.5 ms	🟢 -11.8%
BagOfHoldingH5File	small_strings	load	1.7 ms	1.5 ms	🟢 -11.8%
PickleFile	nested_structures	save	360 µs	320 µs	🟢 -11.1%
PickleFile_Signed	nested_structures	save	370 µs	330 µs	🟢 -10.8%
DillFile_Signed	numpy_arrays	load	380 µs	340 µs	🟢 -10.5%

Integration

configuration	workload	function	time @ `071d395`	time @ head	Δ vs base
Memory	lightweight	contains_miss	82 µs	60 µs	🟢 -26.8%
Memory(Raw)	compute_heavy	hit	150 µs	110 µs	🟢 -26.7%
Memory(Raw)	lightweight	hit	150 µs	110 µs	🟢 -26.7%
SizeLimitedCache(Memory,max=10)	data_heavy	hit	190 µs	140 µs	🟢 -26.3%
Memory(Raw)	lightweight	contains_miss	76 µs	56 µs	🟢 -26.3%
Memory	lightweight	miss	350 µs	260 µs	🟢 -25.7%
SizeLimitedCache(Memory,max=100)	lightweight	contains_hit	82 µs	61 µs	🟢 -25.6%
SizeLimitedCache(Memory,max=100)	data_heavy	contains_hit	82 µs	61 µs	🟢 -25.6%
SizeLimitedCache(Memory,max=10)	compute_heavy	contains_hit	82 µs	61 µs	🟢 -25.6%
Memory	data_heavy	contains_miss	82 µs	61 µs	🟢 -25.6%
Memory	data_heavy	contains_hit	82 µs	61 µs	🟢 -25.6%
SizeLimitedCache(Memory,max=10)	compute_heavy	contains_miss	82 µs	61 µs	🟢 -25.6%
Memory(Raw)	data_heavy	contains_hit	75 µs	56 µs	🟢 -25.3%
Memory(Raw)	compute_heavy	contains_miss	75 µs	56 µs	🟢 -25.3%
Memory(Raw)	compute_heavy	contains_hit	75 µs	56 µs	🟢 -25.3%
Memory(Raw)	data_heavy	contains_miss	75 µs	56 µs	🟢 -25.3%
SizeLimitedCache(Memory,max=100)	data_heavy	contains_miss	83 µs	62 µs	🟢 -25.3%
SizeLimitedCache(Memory,max=10)	data_heavy	contains_hit	83 µs	62 µs	🟢 -25.3%
SizeLimitedCache(Memory,max=100)	compute_heavy	contains_miss	83 µs	62 µs	🟢 -25.3%
SizeLimitedCache(Memory,max=10)	data_heavy	contains_miss	83 µs	62 µs	🟢 -25.3%
Memory	lightweight	hit	160 µs	120 µs	🟢 -25.0%
SizeLimitedCache(Memory,max=100)	compute_heavy	hit	160 µs	120 µs	🟢 -25.0%
Memory	compute_heavy	hit	160 µs	120 µs	🟢 -25.0%
Memory(Raw)	data_heavy	hit	160 µs	120 µs	🟢 -25.0%
SizeLimitedCache(Memory,max=100)	lightweight	hit	160 µs	120 µs	🟢 -25.0%
SizeLimitedCache(Memory,max=100)	compute_heavy	contains_hit	81 µs	61 µs	🟢 -24.7%
SizeLimitedCache(Memory,max=100)	lightweight	contains_miss	81 µs	61 µs	🟢 -24.7%
Memory	lightweight	contains_hit	81 µs	61 µs	🟢 -24.7%
Memory	compute_heavy	contains_hit	81 µs	61 µs	🟢 -24.7%
SizeLimitedCache(Memory,max=10)	lightweight	contains_hit	82 µs	62 µs	🟢 -24.4%
Memory	compute_heavy	contains_miss	82 µs	62 µs	🟢 -24.4%
SizeLimitedCache(Memory,max=10)	lightweight	contains_miss	82 µs	62 µs	🟢 -24.4%
Memory(Raw)	lightweight	contains_hit	75 µs	57 µs	🟢 -24.0%
Memory	data_heavy	hit	170 µs	130 µs	🟢 -23.5%
SizeLimitedCache(Memory,max=100)	data_heavy	hit	170 µs	130 µs	🟢 -23.5%
SizeLimitedCache(Memory,max=10)	lightweight	hit	170 µs	130 µs	🟢 -23.5%
SizeLimitedCache(Memory,max=100)	lightweight	miss	350 µs	270 µs	🟢 -22.9%
SizeLimitedCache(Memory,max=10)	compute_heavy	hit	180 µs	140 µs	🟢 -22.2%
SizeLimitedCache(Memory,max=10)	lightweight	miss	370 µs	290 µs	🟢 -21.6%
H5+Sql	compute_heavy	contains_miss	380 µs	300 µs	🟢 -21.1%
Pickle+Sql	lightweight	contains_miss	390 µs	310 µs	🟢 -20.5%
Memory+Sqlite(:memory:)	data_heavy	contains_miss	360 µs	290 µs	🟢 -19.4%
Memory(Raw)	lightweight	miss	310 µs	250 µs	🟢 -19.4%
Memory+Sqlite(:memory:)	data_heavy	hit	990 µs	800 µs	🟢 -19.2%
Memory+Sqlite(:memory:)	compute_heavy	contains_miss	370 µs	300 µs	🟢 -18.9%
Memory+Sqlite(:memory:)	lightweight	contains_miss	370 µs	300 µs	🟢 -18.9%
Memory+Sqlite(:memory:)	data_heavy	contains_hit	370 µs	300 µs	🟢 -18.9%
H5+Sql	lightweight	contains_miss	380 µs	310 µs	🟢 -18.4%
H5+Sql	lightweight	contains_hit	380 µs	310 µs	🟢 -18.4%
Memory+Sqlite(:memory:)	lightweight	contains_hit	380 µs	310 µs	🟢 -18.4%
Pickle+Sql	compute_heavy	contains_miss	380 µs	310 µs	🟢 -18.4%
H5+Sql	data_heavy	contains_miss	380 µs	310 µs	🟢 -18.4%
H5+Sql	data_heavy	contains_hit	380 µs	310 µs	🟢 -18.4%
H5+Sql	compute_heavy	contains_hit	380 µs	310 µs	🟢 -18.4%
SizeLimitedCache(Memory,max=10)	data_heavy	miss	510 µs	420 µs	🟢 -17.6%
Memory+Sqlite(:memory:)	lightweight	hit	980 µs	820 µs	🟢 -16.3%
Memory+Sqlite(:memory:)	compute_heavy	contains_hit	370 µs	310 µs	🟢 -16.2%
SizeLimitedCache(Memory,max=100)	data_heavy	miss	500 µs	420 µs	🟢 -16.0%
Memory	data_heavy	miss	500 µs	420 µs	🟢 -16.0%
Pickle+Sql	compute_heavy	contains_hit	380 µs	320 µs	🟢 -15.8%
Pickle+Sql	data_heavy	contains_hit	380 µs	320 µs	🟢 -15.8%
Memory+Sqlite(:memory:)	compute_heavy	hit	950 µs	800 µs	🟢 -15.8%
Pickle+Sql	lightweight	contains_hit	390 µs	330 µs	🟢 -15.4%
Memory	compute_heavy	miss	1.3 ms	1.1 ms	🟢 -15.4%
Pickle+Sql	lightweight	hit	1.4 ms	1.2 ms	🟢 -14.3%
Pickle+Sql	data_heavy	contains_miss	370 µs	320 µs	🟢 -13.5%
Memory(Raw)	data_heavy	miss	450 µs	390 µs	🟢 -13.3%
H5+Sql	lightweight	hit	2.9 ms	2.6 ms	🟢 -10.3%
Memory+Sqlite(:memory:)	data_heavy	miss	2 ms	1.8 ms	🟢 -10.0%

Call Storage

configuration	workload	function	time @ `071d395`	time @ head	Δ vs base
SqlMemory	calls	contains_hit	260 µs	200 µs	🟢 -23.1%
SqlFile	calls	contains_hit	270 µs	210 µs	🟢 -22.2%
SqlFile	calls	load	760 µs	600 µs	🟢 -21.1%
SqlMemory	calls	contains_miss	240 µs	190 µs	🟢 -20.8%
SqlMemory	calls	evict	290 µs	230 µs	🟢 -20.7%
SqlFile	calls	contains_miss	250 µs	200 µs	🟢 -20.0%
SqlMemory	calls	load	730 µs	590 µs	🟢 -19.2%
SqlFile	calls	evict	460 µs	400 µs	🟢 -13.0%
SqlMemory	calls	save	2 ms	1.8 ms	🟢 -10.0%

Full results

Call Storage

calls

configuration	contains_hit	contains_miss	evict	load	save
🔴 SqlFile	🟩 210 µs	🟩 200 µs	🟩 400 µs	🟩 600 µs	🟥 2 ms
🔴 SqlMemory	🟩 200 µs	🟩 190 µs	🟩 230 µs	🟩 590 µs	🟧 1.8 ms

Digest

workload	digest
Dict (small)	🟩 1.4 ms
Float	🟩 260 µs
Integer	🟩 130 µs
List (integers, len<100)	🟩 920 µs
List (integers, len>100)	🟥 25 ms
Nested (Random Hypothesis)	🟩 3.4 ms
None	🟩 110 µs
Numpy (integers, len<100)	🟩 900 µs
Numpy (integers, len>100)	🟩 2.8 ms
String (len<100)	🟩 130 µs
String (len>100)	🟩 220 µs

Integration

compute_heavy

configuration	contains_hit	contains_miss	hit	miss
🟠 H5+Sql	🟩 310 µs	🟩 300 µs	🟨 2.7 ms	🟥 7.1 ms
🟢 Pickle+Sql	🟩 320 µs	🟩 310 µs	🟩 1.2 ms	🟨 4.1 ms
🟣 Memory	🟩 61 µs	🟩 62 µs	🟩 120 µs	🟩 1.1 ms
🟣 Memory(Raw)	🟩 56 µs	🟩 56 µs	🟩 110 µs	🟩 1.2 ms
🟣 SizeLimitedCache(Memory,max=10)	🟩 61 µs	🟩 61 µs	🟩 140 µs	🟩 1.2 ms
🟣 SizeLimitedCache(Memory,max=100)	🟩 61 µs	🟩 62 µs	🟩 120 µs	🟩 1.2 ms
🟤 Memory+Sqlite(:memory:)	🟩 310 µs	🟩 300 µs	🟩 800 µs	🟨 2.7 ms

data_heavy

configuration	contains_hit	contains_miss	hit	miss
🟠 H5+Sql	🟩 310 µs	🟩 310 µs	🟨 2.8 ms	🟥 6.6 ms
🟢 Pickle+Sql	🟩 320 µs	🟩 320 µs	🟩 1.3 ms	🟨 3.3 ms
🟣 Memory	🟩 61 µs	🟩 61 µs	🟩 130 µs	🟩 420 µs
🟣 Memory(Raw)	🟩 56 µs	🟩 56 µs	🟩 120 µs	🟩 390 µs
🟣 SizeLimitedCache(Memory,max=10)	🟩 62 µs	🟩 62 µs	🟩 140 µs	🟩 420 µs
🟣 SizeLimitedCache(Memory,max=100)	🟩 61 µs	🟩 62 µs	🟩 130 µs	🟩 420 µs
🟤 Memory+Sqlite(:memory:)	🟩 300 µs	🟩 290 µs	🟩 800 µs	🟩 1.8 ms

lightweight

configuration	contains_hit	contains_miss	hit	miss
🟠 H5+Sql	🟩 310 µs	🟩 310 µs	🟨 2.6 ms	🟥 6.2 ms
🟢 Pickle+Sql	🟩 330 µs	🟩 310 µs	🟩 1.2 ms	🟨 3.1 ms
🟣 Memory	🟩 61 µs	🟩 60 µs	🟩 120 µs	🟩 260 µs
🟣 Memory(Raw)	🟩 57 µs	🟩 56 µs	🟩 110 µs	🟩 250 µs
🟣 SizeLimitedCache(Memory,max=10)	🟩 62 µs	🟩 62 µs	🟩 130 µs	🟩 290 µs
🟣 SizeLimitedCache(Memory,max=100)	🟩 61 µs	🟩 61 µs	🟩 120 µs	🟩 270 µs
🟤 Memory+Sqlite(:memory:)	🟩 310 µs	🟩 300 µs	🟩 820 µs	🟩 1.7 ms

Value Storage

nested_structures

configuration	contains_hit	contains_miss	evict	load	save
🔵 CloudpickleFile	🟩 25 µs	🟩 25 µs	🟩 43 µs	🟩 210 µs	🟩 320 µs
🔵 CloudpickleFile_Signed	🟩 25 µs	🟩 25 µs	🟩 43 µs	🟩 230 µs	🟩 340 µs
🟠 BagOfHoldingH5File	🟩 46 µs	🟩 30 µs	🟩 220 µs	🟧 1 ms	🟥 1.4 ms
🟡 DillFile	🟩 25 µs	🟩 25 µs	🟩 43 µs	🟩 220 µs	🟩 360 µs
🟡 DillFile_Signed	🟩 25 µs	🟩 25 µs	🟩 43 µs	🟩 240 µs	🟩 360 µs
🟢 PickleFile	🟩 25 µs	🟩 25 µs	🟩 43 µs	🟩 220 µs	🟩 320 µs
🟢 PickleFile_Signed	🟩 25 µs	🟩 25 µs	🟩 43 µs	🟩 230 µs	🟩 330 µs
🟣 Memory	🟩 6.6 µs	🟩 6.5 µs	🟩 6.7 µs	🟩 8.2 µs	🟩 8.9 µs
🟣 Memory(Raw)	🟩 1.8 µs	🟩 1.7 µs	🟩 1.9 µs	🟩 3.2 µs	🟩 3.7 µs

numpy_arrays

configuration	contains_hit	contains_miss	evict	load	save
🔵 CloudpickleFile	🟩 25 µs	🟩 25 µs	🟩 84 µs	🟩 240 µs	🟩 430 µs
🔵 CloudpickleFile_Signed	🟩 25 µs	🟩 25 µs	🟩 83 µs	🟩 320 µs	🟩 510 µs
🟠 BagOfHoldingH5File	🟩 46 µs	🟩 30 µs	🟨 680 µs	🟧 1.6 ms	🟥 2 ms
🟡 DillFile	🟩 25 µs	🟩 25 µs	🟩 82 µs	🟩 260 µs	🟩 590 µs
🟡 DillFile_Signed	🟩 25 µs	🟩 24 µs	🟩 82 µs	🟩 340 µs	🟩 660 µs
🟢 PickleFile	🟩 25 µs	🟩 24 µs	🟩 83 µs	🟩 240 µs	🟩 420 µs
🟢 PickleFile_Signed	🟩 25 µs	🟩 25 µs	🟩 82 µs	🟩 320 µs	🟩 490 µs
🟣 Memory	🟩 6.5 µs	🟩 6.5 µs	🟩 6.9 µs	🟩 12 µs	🟩 13 µs
🟣 Memory(Raw)	🟩 1.7 µs	🟩 1.7 µs	🟩 2 µs	🟩 7 µs	🟩 7.6 µs

small_strings

configuration	contains_hit	contains_miss	evict	load	save
🔵 CloudpickleFile	🟩 25 µs	🟩 25 µs	🟩 78 µs	🟩 210 µs	🟩 330 µs
🔵 CloudpickleFile_Signed	🟩 25 µs	🟩 24 µs	🟩 78 µs	🟩 230 µs	🟩 350 µs
🟠 BagOfHoldingH5File	🟩 45 µs	🟩 30 µs	🟨 590 µs	🟥 1.5 ms	🟥 1.5 ms
🟡 DillFile	🟩 25 µs	🟩 25 µs	🟩 78 µs	🟩 220 µs	🟩 350 µs
🟡 DillFile_Signed	🟩 25 µs	🟩 24 µs	🟩 76 µs	🟩 240 µs	🟩 360 µs
🟢 PickleFile	🟩 25 µs	🟩 25 µs	🟩 78 µs	🟩 220 µs	🟩 320 µs
🟢 PickleFile_Signed	🟩 25 µs	🟩 25 µs	🟩 78 µs	🟩 230 µs	🟩 330 µs
🟣 Memory	🟩 6.5 µs	🟩 6.5 µs	🟩 6.5 µs	🟩 8 µs	🟩 8.7 µs
🟣 Memory(Raw)	🟩 1.8 µs	🟩 1.7 µs	🟩 1.8 µs	🟩 3.1 µs	🟩 3.6 µs

claude Bot added the benchmark Run benchmark action label Jul 23, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(storage): cache open HDF5 read handles for multi-bag bag files#786

fix(storage): cache open HDF5 read handles for multi-bag bag files#786
pmrv wants to merge 1 commit into
mainfrom
claude/bagofholding-perf-regression-8vshbo

pmrv commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

Value Storage

Integration

Call Storage

calls

compute_heavy

data_heavy

lightweight

nested_structures

numpy_arrays

small_strings

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pmrv commented Jul 23, 2026

What changed

Numbers

list() — remaining cost and options

Behavioral notes

Testing

Uh oh!

github-actions Bot commented Jul 23, 2026

Benchmark Results

Delta

Value Storage

Integration

Call Storage

Full results

calls

compute_heavy

data_heavy

lightweight

nested_structures

numpy_arrays

small_strings

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`list()` — remaining cost and options