Skip to content

fix: introduce a max in progress block count that can trigger back pressure - #26993

Open
timfn-hg wants to merge 3 commits into
mainfrom
timfn/26949-incomplete-buffered-blocks
Open

fix: introduce a max in progress block count that can trigger back pressure#26993
timfn-hg wants to merge 3 commits into
mainfrom
timfn/26949-incomplete-buffered-blocks

Conversation

@timfn-hg

Copy link
Copy Markdown
Contributor

Description:
Create a new configuration that allows us to specify the maximum number of in-progress blocks within the block buffer to catch cases where upstream blocks are not able to be closed for some reason. In this case, we want to trigger back pressure sooner rather than later since it means there is a problem with the blocks and we should be more aggressive in applying back pressure to decrease the risk of data issues.

At any given moment there is always likely to be one block in progress, so applying this new configuration doesn't kick in unless at least 2 blocks are in progress. This means that aside from the latest block, there are older blocks that have yet to be closed. The threshold is currently 5 blocks.

Related issue(s):

Fixes #26949

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

…essure

Signed-off-by: Tim Farber-Newman <tim.farber-newman@swirldslabs.com>
@timfn-hg timfn-hg added this to the v0.79 milestone Aug 26, 2026
@timfn-hg timfn-hg self-assigned this Aug 26, 2026
@timfn-hg
timfn-hg requested a review from a team as a code owner August 26, 2026 21:27
@lfdt-bot

lfdt-bot commented Aug 26, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Signed-off-by: Tim Farber-Newman <tim.farber-newman@swirldslabs.com>
@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.18919% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
.../app/blocks/impl/streaming/BlockBufferService.java 89.18% 3 Missing and 1 partial ⚠️

Impacted file tree graph

@@             Coverage Diff              @@
##               main   #26993      +/-   ##
============================================
+ Coverage     70.55%   70.56%   +0.01%     
+ Complexity    11692    11691       -1     
============================================
  Files          2586     2586              
  Lines        108553   108570      +17     
  Branches      12165    12168       +3     
============================================
+ Hits          76586    76610      +24     
+ Misses        27970    27964       -6     
+ Partials       3997     3996       -1     
Files with missing lines Coverage Δ Complexity Δ
...com/hedera/node/config/data/BlockBufferConfig.java 100.00% <ø> (ø) 0.00 <0.00> (ø)
.../app/blocks/impl/streaming/BlockBufferService.java 80.40% <89.18%> (+0.70%) 0.00 <0.00> (ø)

... and 3 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ibankov ibankov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, ty @timfn-hg. I had a small comment but not blocking

@timfn-hg

timfn-hg commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

@testlens-app

testlens-app Bot commented Aug 27, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: 9261cae
▶️ Tests: 22785 executed
⚪️ Checks: 86/86 completed


Learn more about TestLens at testlens.app/docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include incomplete blocks when determinng block buffer saturation

3 participants