A background HTTP retry service built with Node.js, Express, and SQLite. Submit a request, and the worker retries it automatically on failure using exponential backoff with jitter and stopps when it succeeds, hits a 4xx, or exhausts all retries.
npm install
node index.jsThe server starts on port 3000. The background worker starts automatically.
In a separate terminal:
node test.jsThis spins up a mock server on port 3001 that fails 3 times then succeeds, registers a request with the retry engine, and polls for results every 1 second. The interval that has been set can be changed.
Register a new request for the worker to execute.
curl -X POST http://localhost:3000/request \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/api", "method": "GET"}'With all options:
curl -X POST http://localhost:3000/request \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/api", "method": "POST", "body": "{\"key\":\"value\"}", "maxRetries": 3, "backoffMs": 500}'Response:
{ "id": 1, "status": "pending" }Get a request and its full attempt history.
curl http://localhost:3000/requests/1Response:
{
"data": [
{
"id": 1,
"url": "http://localhost:3001/mock",
"method": "GET",
"status": "completed",
"attemptCount": 4,
"nextRetryAt": "2026-05-31 19:23:35",
"lastError": "{\"error\":\"Simulated failure #3\"}",
"result": "{\"message\":\"Success after retries!\"}",
"maxRetries": 5,
"backoffMs": 1000,
"createdAt": "2026-05-31 19:23:20",
"updatedAt": "2026-05-31 19:23:35",
"attemptStatus": "success",
"message": "{\"message\":\"Success after retries!\"}",
"attemptCreatedAt": "2026-05-31 19:23:35"
}
]
}Filter requests by status. Valid values: pending, retrying, completed, failed.
curl "http://localhost:3000/requests?status=failed"
curl "http://localhost:3000/requests?status=completed"
curl "http://localhost:3000/requests"Client
│
▼
POST /request ──► SQLite (status: pending)
│
▼
Background Worker (every 500ms)
polls: nextRetryAt <= now()
│
┌─────────┴──────────┐
│ │
success failure
│ │
status: completed ┌────┴─────┐
│ │
4xx 5xx / network
│ │
status: failed │
(no retry) │
attemptCount < maxRetries?
┌────┴────┐
yes no
│ │
schedule status: failed
next retry (dead-letter)
with backoff
When a server fails, it's usually under load or temporarily down. If 100 clients try a server and gets a server error, in the case where each of those clients try again especially in high traffic, what likely caused the issue almost never gets fixed in less than a second so the response is likely to fail again. Exponential backoff spreads retries out over time by increasing the time between retries exponentially. For example: waiting 1s, then 2s, then 4s, then 8s. In a case of probable network error or high traffic, this retry is more likely going to lead to something possible by reducing the eventual load.
Without jitter, all clients that started at the same time will retry at exactly the same time even with the exponential increases. If 20 ppl wait try once and wait 1s, then 2 then 4, the same issue persists. Jitter adds a small random multiplier (between 0.8 and 1.2) to each wait, so retries are spread out naturally.
A 4xx means the request itself is wrong probably a bad URL, missing auth, malformed body. Retrying won't fix it; the request will fail the same way every time. 5xx means the server had a problem (it crashed, timed out, was overloaded) so retrying might make sense. In a general sense, the goal of a retrying engine is to solve issues that occur due to overload or traffic mostly. If a request was wrong it makes no sense trying again.
The only struggle I had was in inserting the maxRetries and backoffMs since they were optional. My main issue was in writing different if-else blocks for the different types of request I would get. I just knew doing that was crazy. This led me to the params method of inserting data into SQLite databases. So far, I used positional methods. The params method enable me use objects where I could perform operations in that the positional methods wouldn't allow.
// Positional method — every ? must match the exact order of values
// Handling optional fields meant branching logic just to build the right array
const insert = db.prepare(
"INSERT INTO requests (url, method, maxRetries, backoffMs) \
VALUES (?, ?, ?, ?)",
);
insert.run(url, method, maxRetries ?? 5, backoffMs ?? 1000);
// Named params method — use @name placeholders and pass an object
// Now ?? defaults live right in the object, no branching needed
const insert = db.prepare(
"INSERT INTO requests (url, method, maxRetries, backoffMs) \
VALUES (@url, @method, @maxRetries, @backoffMs)",
);
insert.run({
url,
method,
maxRetries: maxRetries ?? 5,
backoffMs: backoffMs ?? 1000,
});I have never implemented background workers before and so this has helped me understand the concepts that are foundational to them. Secondly, in production scenario where external APIs are called, this could be foundational to preventing my backend services from being affected by downtimes or high traffics or rate limits too. If I were building a feature that sends OTP SMS via an external service, a better practice would be to route that through a retry engine instead of making the call inline in the request handler. That way, the external service outage doesn't fail the user's registration. There are libraries like BullMQ that handle this at scale, but now I understand the internals well enough to build a lightweight version myself when I don't need all that overhead.