Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 DanielCuevas1208

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
99 changes: 94 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A small durable background queue built with Go and SQLite.

The project shows leases, retries, idempotency, crash recovery, priority dispatch, priority aging, a dead-letter queue, Prometheus metrics, and an append-only event log.
The project shows leases, retries, idempotency, crash recovery, priority dispatch, priority aging, a dead-letter queue, Prometheus metrics, an append-only event log, and an HTML inspection dashboard.

## Value

Expand All @@ -21,6 +21,7 @@ The queue separates durable state from worker execution.
- `internal/fault` injects deterministic errors, panics, delays, and stalls.
- `internal/cli` renders commands, snapshots, history, and the demo.
- `internal/metrics` renders queue state in the Prometheus text format.
- `internal/web` serves the HTML inspection dashboard and its JSON API.
- `internal/fixture` provides repeatable sample workloads.

A job starts as `pending`.
Expand Down Expand Up @@ -67,6 +68,14 @@ go run . seed -db queue.db
go run . inspect -db queue.db
```

Serve the inspection dashboard with this command.

```text
go run . web -addr :8080 -db queue.db
```

Open http://localhost:8080 in a browser.

Inspect and requeue a dead-lettered job with these commands.

```text
Expand All @@ -89,7 +98,7 @@ jobqueue enqueue -kind <type> -payload <json> [-priority <n>] [-idempotency-key
### `work`

```text
jobqueue work -kind <type> [-concurrency <n>] [-lease <duration>] [-poll <duration>] [-aging <duration>] [-metrics-addr <addr>] [-db <path>]
jobqueue work -kind <type> [-concurrency <n>] [-lease <duration>] [-poll <duration>] [-aging <duration>] [-metrics-addr <addr>] [-web-addr <addr>] [-db <path>]
```

The worker recovers expired leases when it starts.
Expand All @@ -102,6 +111,8 @@ Use `-aging 0` to disable aging.

Use `-metrics-addr` to serve Prometheus metrics beside the worker.

Use `-web-addr` to serve the inspection dashboard beside the worker.

### `inspect`

```text
Expand Down Expand Up @@ -154,6 +165,30 @@ Scrape the endpoint with a Prometheus server.

Use `-once` to print one snapshot and exit.

### `web`

```text
jobqueue web [-addr <addr>] [-db <path>]
```

The command serves the HTML inspection dashboard.

The default address is `:8080`.

The dashboard shows state counts, jobs by kind, and a filterable job table.

Click any job to see its payload and complete event timeline.

The page refreshes itself every few seconds.

The JSON API lives under `/api/`.

Use `/api/snapshot` to fetch the full queue state.

Use `/api/jobs/<id>` to fetch one job and its events.

Use `POST /api/jobs/<id>/requeue` to return a dead-lettered job to pending.

### `demo`

```text
Expand Down Expand Up @@ -260,6 +295,22 @@ Use the `metrics` command for one snapshot or a live endpoint.

Use `work -metrics-addr` to serve the same endpoint beside a worker.

### Web dashboard

The `web` command serves an HTML page and a JSON API.

The page renders in the browser and needs no build step.

State counts, job tables, and the event feed update on a timer.

Click any job to inspect its payload and full timeline.

The dashboard reads the same SQLite store as the worker.

The JSON API is available to scripts without the HTML page.

Use `work -web-addr` to serve the dashboard beside a worker.

### Scheduling

A scheduled job stores its earliest lease time in `run_at`.
Expand Down Expand Up @@ -339,6 +390,32 @@ The demo uses generated job IDs and current timestamps.

The final counts depend on the scenario and run deadline.

### Dashboard

Run `jobqueue web -db queue.db` and open the root path.

The page shows the whole queue in one view.

```text
Local-first Durable Job Queue db: queue.db updated 18:13:10

pending 3 | leased 0 | completed 6 | dead letter 1 | failed 0

Jobs by kind
kind pending leased completed dead letter failed total
email 0 0 3 0 0 3
report 2 0 0 1 0 3

Jobs
status kind pri attempts created payload
dead_letter report 0 3/3 2026-08-03 18:13:05 {"task":"report"}
pending report 2 0/3 2026-08-03 18:13:05 {"task":"report"}
```

The page polls the JSON API every three seconds.

Click a job to open its detail panel and full timeline.

## Verification

Run the full test suite with this command.
Expand Down Expand Up @@ -376,7 +453,7 @@ A high-priority stream can delay lower-priority jobs until aging lifts them.

The worker is one process and does not coordinate across hosts.

The project does not provide a web interface.
The dashboard is read-only except for the requeue action.

## Roadmap

Expand All @@ -386,12 +463,24 @@ The project does not provide a web interface.
- [x] Priority aging to prevent starvation.
- [x] Dead-letter queue with requeue of permanently failed jobs.
- [x] Prometheus metrics for queue inspection.
- [ ] Web UI for queue inspection.
- [x] Web UI for queue inspection.
- [ ] Horizontal scaling with a shared SQLite file.

### Release notes

This release adds Prometheus metrics.
This release adds a web dashboard for queue inspection.

The new `web` command serves an HTML page and a JSON API.

The page shows state counts, jobs by kind, and a filterable job table.

Click a job to see its payload and complete event timeline.

The page refreshes itself every few seconds while a worker runs.

Use `work -web-addr` to serve the dashboard beside a worker.

The previous release added Prometheus metrics.

The new `metrics` command serves the exposition format over HTTP.

Expand Down
16 changes: 16 additions & 0 deletions cmd/web/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package main

import (
"log"
"os"

"github.com/local-first-job-queue/internal/cli"
)

func main() {
log.SetFlags(0)
if err := cli.Web(os.Args[1:]); err != nil {
log.Fatal(err)
os.Exit(1)
}
}
33 changes: 33 additions & 0 deletions internal/cli/web.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package cli

import (
"flag"
"fmt"
"log"
"net/http"

"github.com/local-first-job-queue/internal/queue"
"github.com/local-first-job-queue/internal/web"
)

// Web serves the HTML inspection dashboard and its JSON API. The dashboard
// reads the same SQLite store as the worker, so it shows jobs and events as
// they change. Point a browser at the listen address and open the root path.
func Web(args []string) error {
fs := flag.NewFlagSet("web", flag.ExitOnError)
addr := fs.String("addr", ":8080", "listen address for the dashboard")
dbPath := fs.String("db", "queue.db", "database path")
fs.Parse(args)

store, err := queue.NewSQLiteStore(*dbPath)
if err != nil {
return fmt.Errorf("open store: %w", err)
}
defer store.Close()

handler := web.New(store, web.WithDBPath(*dbPath)).Handler()

log.Printf("dashboard listening on %s (db=%s)", *addr, *dbPath)
log.Printf("open: http://localhost%s/", *addr)
return http.ListenAndServe(*addr, handler)
}
24 changes: 17 additions & 7 deletions internal/cli/work.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"github.com/local-first-job-queue/internal/fault"
"github.com/local-first-job-queue/internal/metrics"
"github.com/local-first-job-queue/internal/queue"
"github.com/local-first-job-queue/internal/web"
"github.com/local-first-job-queue/internal/worker"
)

Expand All @@ -37,6 +38,7 @@ func Work(args []string) error {
pollInterval := fs.Duration("poll", time.Second, "time between lease attempts")
aging := fs.Duration("aging", queue.DefaultAgingInterval, "priority aging interval; a job gains one priority point per interval it waits (0 disables)")
metricsAddr := fs.String("metrics-addr", "", "address to serve Prometheus metrics on, e.g. :9090 (empty disables)")
webAddr := fs.String("web-addr", "", "address to serve the inspection dashboard on, e.g. :8080 (empty disables)")
fs.Parse(args)

store, err := queue.NewSQLiteStore(*dbPath, queue.WithAgingInterval(*aging))
Expand All @@ -53,15 +55,23 @@ func Work(args []string) error {
worker.WithPollInterval(*pollInterval),
)

var srv *http.Server
if *metricsAddr != "" {
srv = &http.Server{Addr: *metricsAddr, Handler: metrics.Handler(store)}
var servers []*http.Server
serve := func(addr string, h http.Handler, label string) {
if addr == "" {
return
}
srv := &http.Server{Addr: addr, Handler: h}
servers = append(servers, srv)
go func() {
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Printf("metrics server: %v", err)
log.Printf("%s: %v", label, err)
}
}()
log.Printf("metrics listening on %s", *metricsAddr)
log.Printf("%s listening on %s", label, addr)
}
serve(*metricsAddr, metrics.Handler(store), "metrics")
if *webAddr != "" {
serve(*webAddr, web.New(store, web.WithDBPath(*dbPath)).Handler(), "dashboard")
}

ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
Expand All @@ -71,10 +81,10 @@ func Work(args []string) error {
*kind, *concurrency, *leaseDuration, *pollInterval, *aging)
err = w.Run(ctx)

if srv != nil {
for _, srv := range servers {
shutdownCtx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_ = srv.Shutdown(shutdownCtx)
cancel()
}
if errors.Is(err, context.Canceled) {
// A signal cancelled the run. This is a normal shutdown, not an error.
Expand Down
Loading
Loading