Skip to content

force: Vault NonceManagerKind::Safe for aggregator - #1152

Merged
ueco-jb merged 3 commits into
mainfrom
2.0.0-vault
May 25, 2026
Merged

force: Vault NonceManagerKind::Safe for aggregator#1152
ueco-jb merged 3 commits into
mainfrom
2.0.0-vault

Conversation

@reecepbcups

@reecepbcups reecepbcups commented May 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

The fast approach caused a retry condition which required a restart of WAVS. i think this is the 3rd time we have hit it in production.

WAVS is still on v2.0.0-rc.14, so building a specific vault image for us there. We can make this proper in the future (either removing or making it a config option or something)


ref: https://lay3r.slack.com/archives/C0A7JL2K337/p1779554921215579?thread_ts=1779532743.735499&cid=C0A7JL2K337

What happened: The aggregator went to push the rebalance last night @ 3am eastern. The aggregator did not have enough balance to do it. This pushed the tx into the re-try queue. This queue in the aggregator has a max of 1000 retries stored in memory.

This morning I funded the aggregator with $20 in ETH but it still did not submit on chain. There was an optimization in the aggregator to use an in memory nonce queue which while faster, can cause many issues when you get out of nonce sync - i.e. like retries. SO it would try to submit txs with an invalid nonce via the re-try queue

hacky fix: restarted wavs-1

@reecepbcups reecepbcups changed the title force: NonceManagerKind::Safe for aggregator force: Vault NonceManagerKind::Safe for aggregator May 23, 2026
@reecepbcups
reecepbcups marked this pull request as ready for review May 23, 2026 16:49

@ueco-jb ueco-jb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@ueco-jb
ueco-jb enabled auto-merge May 25, 2026 08:44
@ueco-jb
ueco-jb merged commit 37bbfc1 into main May 25, 2026
5 checks passed
@ueco-jb
ueco-jb deleted the 2.0.0-vault branch May 25, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants