Skip to content

Async raft sender - document in 5.26#3175

Open
yirensum wants to merge 3 commits into
neo4j:5.xfrom
yirensum:cluster-1827/document-async-raft-sender-in-5.26
Open

Async raft sender - document in 5.26#3175
yirensum wants to merge 3 commits into
neo4j:5.xfrom
yirensum:cluster-1827/document-async-raft-sender-in-5.26

Conversation

@yirensum

@yirensum yirensum commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Part of CLUSTER-1827

In this PR, we add troubleshooting for raft sender problems. This cause of this can be found in this internal document. However, customers in 5.26LTS are now asking for a more detailed write up of cause and mitigation, which is presented here.

I will also ask someone in clustering to review this.

@yirensum
yirensum force-pushed the cluster-1827/document-async-raft-sender-in-5.26 branch from 878c673 to 94f137e Compare July 13, 2026 15:45
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
Co-authored-by: Natalia Ivakina <82437520+NataliaIvakina@users.noreply.github.com>
Comment thread modules/ROOT/pages/clustering/troubleshooting.adoc Outdated
@neo4j-docops-agent

Copy link
Copy Markdown
Collaborator

This PR includes documentation updates
View the updated docs at https://neo4j-docs-operations-3175.surge.sh

New pages:

Updated pages:

@renetapopova renetapopova left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments and questions

[[clustering-troubleshooting-raft-sender-channel-acquisition]]
== Slow or unavailable writes when a cluster member is shut down

In Neo4j 5.26 LTS, a cluster can experience delayed or unavailable writes if the leader waits for a Raft sender connection to a cluster member that has stopped but whose network endpoint still accepts connection attempts.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we are on Neo4j 5.26, I don't think we need to say it explicitly.

Suggested change
In Neo4j 5.26 LTS, a cluster can experience delayed or unavailable writes if the leader waits for a Raft sender connection to a cluster member that has stopped but whose network endpoint still accepts connection attempts.
A cluster can experience delayed or unavailable writes if the leader waits for a Raft sender connection to a cluster member that has stopped but whose network endpoint still accepts connection attempts.

That case is less likely to cause the write latency described here because the leader can move on to another member without waiting for the connection timeout.

[[clustering-troubleshooting-raft-sender-channel-acquisition-mitigation]]
=== Mitigation in Neo4j 5.26 LTS

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Suggested change
=== Mitigation in Neo4j 5.26 LTS
=== Mitigation

[[clustering-troubleshooting-raft-sender-channel-acquisition-mitigation]]
=== Mitigation in Neo4j 5.26 LTS

In Neo4j 5.26 LTS, enable asynchronous acquisition of Raft sender channels by setting the following internal setting to `true` in _neo4j.conf_ on all cluster members:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here


[NOTE]
====
*Behavior from Neo4j 2025.02* +

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this note doesn't belong on this page. If I am on 5.26, why would I care about 2026.02? But if I'm on 2026.01, then I need to know how to fix it.
Maybe, instead of having this note here, explain a bit in the current version if we haven't already done that, and add a note that in previous version, "a cluster can experience delayed or unavailable writes if the leader waits for a Raft sender connection to a cluster member that has stopped but whose network endpoint still accepts connection attempts." and link to that page.

[[clustering-troubleshooting]]
= Troubleshooting

This page describes known cluster issues and their mitigations.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see only one issue. Do you know if we expect to add more?

This page describes known cluster issues and their mitigations.

[[clustering-troubleshooting-raft-sender-channel-acquisition]]
== Slow or unavailable writes when a cluster member is shut down

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have just one issue, maybe this could be the title instead of "Troubleshooting." Something like: "Troubleshooting slow or unavailable writes"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants