Background
Summary
Profiling the block-stream write path (BlockStreamManagerImpl's ParallelTask/SequentialTask pipeline) turned up three
related inefficiencies in how block items are hashed and written to disk. This issue tracks the fix and the benchmarks used to
validate it.
Problem
- Leaf hashing was serialized unnecessarily.
SequentialTask.onExecute() did both the SHA-384 leaf hash (an O(item size)
operation with no dependency on other items) and the O(1) Merkle fold-up + writer append in the same strictly-ordered task
chain. Only the fold-up and the write actually need item order — the hash itself could run in parallel, same as serialization
already does.
- Every block item was copied 3x when 1 would do. In the standard file+gRPC config:
BlockItem.PROTOBUF.toBytes(item).toByteArray() threw away a zero-copy Bytes wrapper to force a full-array copy, and
FileAndGrpcBlockItemWriter/FileBlockItemWriter copied it again just to hand a byte[] to the writer — despite the PBJ
runtime already exposing zero-copy paths (Bytes.writeTo(MessageDigest), WritableStreamingData.writeBytes(RandomAccessData)).
IncrementalStreamingHasher used a LinkedList<byte[]> as a stack, allocating a Node per push and discarding two per
fold, on a hot path that runs once per leaf across up to 5 hashers per block item.
Proposed Changes
BlockStreamManagerImpl.java: moved SHA-384 leaf-hash computation into ParallelTask (parallel, per-item), leaving only
addNodeByHash (fold-up) and the writer append in SequentialTask. Added a ThreadLocal<MessageDigest> for per-thread reuse.
TRANSACTION_RESULT's running-hash update now reuses the same precomputed hash instead of hashing twice.
FileBlockItemWriter.java / FileAndGrpcBlockItemWriter.java: carry Bytes end-to-end instead of unwrapping to
byte[]; writeItem now calls writableStreamingData.writeBytes(bytes) directly (zero-copy).
IncrementalStreamingHasher.java: hashList changed from LinkedList<byte[]> to Deque<byte[]> (ArrayDeque);
computeRootHash() snapshots into a byte[][] for the indexed fold since ArrayDeque has no get(int).
- Added two JMH benchmarks (
hedera-app/src/jmh/.../blocks/) to validate and guard against regression:
SequentialTaskDesignBenchmark — current vs. proposed task-split design, with a correctness self-check that both produce
identical Merkle roots.
IncrementalStreamingHasherDesignBenchmark — current ArrayDeque vs. a reproduced LinkedList implementation, same
self-check.
Acceptance Criteria
See above
Dependencies
No response
Definition of Ready (DoR) Checklist
Definition of Done (DoD) Checklist
Background
Summary
Profiling the block-stream write path (
BlockStreamManagerImpl'sParallelTask/SequentialTaskpipeline) turned up threerelated inefficiencies in how block items are hashed and written to disk. This issue tracks the fix and the benchmarks used to
validate it.
Problem
SequentialTask.onExecute()did both the SHA-384 leaf hash (an O(item size)operation with no dependency on other items) and the O(1) Merkle fold-up + writer append in the same strictly-ordered task
chain. Only the fold-up and the write actually need item order — the hash itself could run in parallel, same as serialization
already does.
BlockItem.PROTOBUF.toBytes(item).toByteArray()threw away a zero-copyByteswrapper to force a full-array copy, andFileAndGrpcBlockItemWriter/FileBlockItemWritercopied it again just to hand abyte[]to the writer — despite the PBJruntime already exposing zero-copy paths (
Bytes.writeTo(MessageDigest),WritableStreamingData.writeBytes(RandomAccessData)).IncrementalStreamingHasherused aLinkedList<byte[]>as a stack, allocating aNodeper push and discarding two perfold, on a hot path that runs once per leaf across up to 5 hashers per block item.
Proposed Changes
BlockStreamManagerImpl.java: moved SHA-384 leaf-hash computation intoParallelTask(parallel, per-item), leaving onlyaddNodeByHash(fold-up) and the writer append inSequentialTask. Added aThreadLocal<MessageDigest>for per-thread reuse.TRANSACTION_RESULT's running-hash update now reuses the same precomputed hash instead of hashing twice.FileBlockItemWriter.java/FileAndGrpcBlockItemWriter.java: carryBytesend-to-end instead of unwrapping tobyte[];writeItemnow callswritableStreamingData.writeBytes(bytes)directly (zero-copy).IncrementalStreamingHasher.java:hashListchanged fromLinkedList<byte[]>toDeque<byte[]>(ArrayDeque);computeRootHash()snapshots into abyte[][]for the indexed fold sinceArrayDequehas noget(int).hedera-app/src/jmh/.../blocks/) to validate and guard against regression:SequentialTaskDesignBenchmark— current vs. proposed task-split design, with a correctness self-check that both produceidentical Merkle roots.
IncrementalStreamingHasherDesignBenchmark— currentArrayDequevs. a reproducedLinkedListimplementation, sameself-check.
Acceptance Criteria
See above
Dependencies
No response
Definition of Ready (DoR) Checklist
Definition of Done (DoD) Checklist