Raft-Based Distributed Key-Value Store

A distributed key-value store implemented in Node.js using a Raft-inspired consensus algorithm.

This project was built from scratch to explore the internals of distributed systems, including leader election, log replication, majority consensus, node recovery, and persistence.

Features

Consensus

• Leader election
• Heartbeats
• Term-based voting
• Majority quorum consensus
• Automatic leader failover

Replication

• Leader-based writes
• Log replication to followers
• Commit after majority acknowledgment
• Consistent state across nodes

Recovery

• Node restart recovery
• Log synchronization from leader
• Divergence detection
• Log repair and catch-up
• Automatic cache reconstruction from logs

Persistence

• Persistent log storage
• Persistent term and vote information
• Recovery after process restart

Key-Value Operations

• Set key-value pairs
• Delete keys
• Retrieve data from replicated state

---

Architecture

The cluster consists of multiple servers.

Client
   |
   v
Follower ----> Leader
                  |
                  v
            Replicated Log
                  |
         -----------------
         |               |
         v               v
     Follower       Follower

Only the leader accepts writes.

If a follower receives a write request, it forwards the request to the current leader.

The leader:

1. Appends the operation to its log.
2. Replicates the log entry to followers.
3. Waits for a majority of acknowledgments.
4. Commits the entry.
5. Applies the operation to its cache.
6. Notifies followers of the commit.

---

Endpoints

Write Operations

Set

POST /set

Body:

{
    "key":"goal",
    "value":"peace"
}

Delete

DELETE /delete/:key

---

Internal Replication Routes

Write

Used when followers forward client requests to the leader.

POST /write

Change

Used for log replication.

POST /change

Commit

Used to commit replicated entries.

POST /commit

Heartbeat

Used by the leader to maintain authority.

POST /heartbeat

Request Vote

Used during elections.

POST /requestVote

Sync

Used by recovering nodes to request missing log entries.

GET /sync/:lastIndex

---

Debug Routes

View Log

GET /log

View Cache

GET /cache

---

Log Replication

Each operation is recorded as a log entry:

{
    "term":10,
    "operation":"set",
    "key":"goal",
    "value":"peace"
}

Followers verify:

• Previous log index
• Previous log term
• Current term

before accepting entries.

If divergence is detected, the leader rewinds and repairs the follower's log before continuing replication.

---

Recovery

When a node rejoins the cluster:

1. It discovers the current leader through heartbeats.
2. It requests missing entries using /sync.
3. It appends the missing log entries.
4. It replays unapplied operations.
5. Its cache becomes consistent with the cluster.

---

Running Multiple Nodes

Example:

node server1.js
node server2.js
node server3.js

Each server should have:

• Unique ID
• Unique port
• Appropriate peer list

Example:

const myId = "server1";
const myAddress = "http://localhost:3001";

const servers = [
    "http://localhost:3002",
    "http://localhost:3003"
];

---

Learning Goals

This project was built to understand:

• Distributed systems
• Consensus algorithms
• Replication
• Leader election
• Failure recovery
• Eventual consistency
• State machine replication

Rather than relying on an existing implementation, the goal was to build the mechanisms manually and understand how they interact under failure and recovery scenarios.

---

Future Improvements

• Snapshotting and log compaction
• Dynamic cluster membership
• Read consistency guarantees
• Authentication and authorization
• Better persistence layer
• Network partition testing
• Automated integration tests

---

Disclaimer

This is an educational implementation inspired by Raft and distributed database design principles. It is intended for learning and experimentation rather than production use.