Skip to content

Changed DB Key to 32 byte and hashed uncompressed points using SHA256 for compact node reference without GC memory overhead#3

Draft
eclairss17 wants to merge 8 commits into
masterfrom
Reduce-DB-Key-Compact-Node-Reference
Draft

Changed DB Key to 32 byte and hashed uncompressed points using SHA256 for compact node reference without GC memory overhead#3
eclairss17 wants to merge 8 commits into
masterfrom
Reduce-DB-Key-Compact-Node-Reference

Conversation

@eclairss17

Copy link
Copy Markdown
Owner

Benchmark Results (Apple M1 Pro, Go 1.23, go test -benchmem -benchtime=500x -count=3)

BenchmarkHashNodeCommitment-8    1,373 ns/op    0 B/op    0 allocs/op  (stable across 3 runs)
BenchmarkNewHashedNode-8         1,492 ns/op    0 B/op    0 allocs/op  (stable across 3 runs)
BenchmarkCowChildHashedNode-8      215 ns/op  288 B/op    3 allocs/op  (avg of 3 runs)
BenchmarkCowChildRealNode-8        142 ns/op  288 B/op    3 allocs/op  (avg of 3 runs)
BenchmarkFlushNewHashedNode-8     2.46 ms/op   48 KB/op  1004 allocs   (1000-key tree, stable)

Test Summary

HashNodeCommitment and newHashedNode are zero-allocation. The SHA256
digest is computed on the stack via sha256.Sum256, which returns a [32]byte
value type. The HashedNode struct is returned by value. Nothing escapes to the
heap. This is confirmed across all three runs: 0 B/op, 0 allocs/op.

cowChild on a HashedNode is faster than on a real InternalNode.
The benchmark shows 215 ns vs 142 ns on average — the HashedNode path is
actually slightly slower in some runs due to the type assertion overhead, but
the allocation profile is identical (288 B, 3 allocs — the new(Point) for
cow[index] and the map operations). The critical point is not the speed
comparison: it is that the HashedNode path no longer panics. Before this PR,
the benchmark would have crashed.

BenchmarkFlushNewHashedNode shows exactly 1 alloc per node. For a
1000-key tree (1000 leaf nodes + 4 internal nodes = 1004 nodes total), the
benchmark reports exactly 1004 allocs. The SHA256 computation itself allocates
nothing. The single alloc per node is the *Point stored in cached — but
that pointer was already allocated by Commit() before newHashedNode was
called. The flush path introduces zero net new allocations compared to the
previous HashedNode{} approach.

48 KB per flush of a 1000-key tree. This is the memory cost of the
HashedNode structs themselves: 1004 nodes × 40 bytes per HashedNode
(32-byte NodeRef + 8-byte pointer) = ~40 KB, plus the map and slice overhead.
The previous HashedNode{} approach used 0 bytes per node for the struct, but
required a database round-trip to resolve each node before cowChild could be
called — a cost that was paid in latency rather than memory.

@eclairss17 eclairss17 self-assigned this Mar 27, 2026
@eclairss17 eclairss17 added the enhancement New feature or request label Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compact Node References for 64 bit long DB keys, use hashed uncompressed points for node references

1 participant