A fault-tolerant log ingestion and query service built in Go, designed to process structured logs asynchronously, persist them reliably, and provide operational visibility.
A lightweight logging service that accepts structured events, processes them through an asynchronous pipeline, persists them to SQLite, and exposes the information operators need to understand what's happening.
Strata-Log demonstrates practical Go backend and distributed-systems engineering patterns including concurrent processing, asynchronous pipelines, batching, durable persistence, retry and resilience mechanisms, HTTP API design, Prometheus observability, graceful shutdown, automated testing, and containerized deployment.
- Structured JSON log ingestion
- Asynchronous log processing
- Concurrent worker pipeline
- Bounded in-memory buffering
- Configurable batch processing
- SQLite persistent storage
- Atomic batch writes
- Retry with exponential backoff
- HTTP log query API
- Prometheus-compatible metrics
- Health-check endpoint
- Graceful shutdown
- Configurable runtime settings
- Docker and Docker Compose support
- Unit and integration tests
- Race-detector coverage
- Static analysis with
go vet - Benchmark suite
HTTP Client
│
▼
┌──────────────────┐
│ HTTP API │
│ │
│ POST /v1/logs │
│ GET /v1/logs │
│ GET /healthz │
│ GET /metrics │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Ingestion │
│ Pipeline │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Workers / Buffer │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Batcher │
│ │
│ Size + Interval │
└────────┬─────────┘
│
Retry + Backoff
│
▼
┌──────────────────┐
│ SQLite │
│ Storage │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Query Service │
└──────────────────┘
┌──────────────────┐
│ Telemetry │
│ /metrics │
└──────────────────┘
For a deeper explanation of the system design, see docs/architecture.md.
| Technology | Purpose |
|---|---|
| Go | Application runtime |
net/http |
HTTP server and API |
| SQLite | Persistent log storage |
modernc.org/sqlite |
Pure-Go SQLite driver |
| Docker | Containerization |
| Docker Compose | Local container deployment |
| Prometheus | Metrics collection |
log/slog |
Structured application logging |
- Go 1.26+
- Git
- Docker
- Docker Compose
Clone the repository:
git clone https://github.com/ikwukao/strata-log.git
cd strata-logRun the service:
go run ./cmd/strata-logStrata-Log listens on port 9090 by default.
curl http://localhost:9090/healthzExpected response:
{
"status": "ok"
}Send a structured log entry:
curl -X POST http://localhost:9090/v1/logs \
-H 'Content-Type: application/json' \
-d '{
"timestamp": "2026-08-25T19:00:00Z",
"level": "info",
"service": "api",
"message": "Strata-Log is alive",
"fields": {
"environment": "development"
}
}'The timestamp field is required.
Retrieve persisted logs:
curl http://localhost:9090/v1/logsFilter by level:
curl 'http://localhost:9090/v1/logs?level=error'Filter by service:
curl 'http://localhost:9090/v1/logs?service=api'Limit results:
curl 'http://localhost:9090/v1/logs?limit=20'Combine filters:
curl 'http://localhost:9090/v1/logs?level=error&service=api&limit=50'Example response:
{
"logs": [
{
"id": 1,
"timestamp": "2026-08-25T19:00:00Z",
"level": "info",
"service": "api",
"message": "Strata-Log is alive",
"fields": {
"environment": "development"
}
}
],
"count": 1
}See the complete API reference in docs/api.md.
Prometheus-compatible metrics are exposed at:
GET /metrics
View the metrics:
curl http://localhost:9090/metricsThe service exposes counters including:
strata_log_ingested_total
strata_log_stored_total
strata_log_errors_total
Strata-Log is configured through environment variables.
Common settings include:
STRATA_LOG_HOST
STRATA_LOG_PORT
STRATA_LOG_STORAGE_PATH
STRATA_LOG_RETRY_ATTEMPTS
STRATA_LOG_RETRY_BACKOFF
STRATA_LOG_BUFFER_CAPACITY
STRATA_LOG_BATCH_SIZE
STRATA_LOG_FLUSH_PERIOD
STRATA_LOG_READ_TIMEOUT
STRATA_LOG_WRITE_TIMEOUT
STRATA_LOG_IDLE_TIMEOUT
STRATA_LOG_SHUTDOWN_TIMEOUT
Default server configuration:
Host: 0.0.0.0
Port: 9090
Read timeout: 10s
Write timeout: 10s
Idle timeout: 60s
Shutdown timeout: 10s
See docs/configuration.md for the complete configuration reference.
Build the image:
make docker-buildStart Strata-Log:
make docker-upOr use Docker Compose directly:
docker compose -f deployments/docker-compose.yml up --buildRun in the background:
docker compose -f deployments/docker-compose.yml up --build -dStop the deployment:
make docker-downVerify the service:
curl http://localhost:9090/healthzSee docs/deployment.md for deployment instructions.
Strata-Log includes a Makefile for common development, testing, build, Docker, and operational workflows.
Display all available commands:
make helpFormat Go source files:
make fmtRun tests:
make testRun tests with the race detector:
make test-raceRun static analysis:
make vetBuild the Strata-Log binary:
make buildRun the complete validation and build workflow:
make checkBuild the Docker image:
make docker-buildStart the Docker Compose stack:
make upStop the stack:
make downRestart the stack:
make restartFollow Strata-Log container logs:
make logsCheck service health:
make healthDisplay Prometheus metrics:
make metricsRemove local build artifacts:
make cleanThe Makefile provides a consistent interface for local development and container-based workflows, while the underlying commands remain directly available when needed.
strata-log/
├── benchmarks/
│ ├── ingest_test.go
│ └── storage_test.go
│
├── cmd/
│ └── strata-log/
│ └── main.go
│
├── deployments/
│ ├── docker-compose.yml
│ ├── Dockerfile
│ └── prometheus.yml
│
├── docs/
│ ├── api.md
│ ├── architecture.md
│ ├── configuration.md
│ ├── deployment.md
│ └── development.md
│
├── internal/
│ ├── batcher/
│ ├── buffer/
│ ├── config/
│ ├── ingest/
│ ├── pipeline/
│ ├── query/
│ ├── resilience/
│ ├── storage/
│ └── telemetry/
│
├── tests/
│ └── integration_test.go
│
├── CONTRIBUTING.md
├── CHANGELOG.md
├── LICENSE
├── Makefile
├── ROADMAP.md
├── README.md
├── go.mod
└── go.sum
The persistence path is designed around several reliability mechanisms:
- Bounded ingestion buffering
- Concurrent processing
- Batch accumulation
- Atomic SQLite batch writes
- Configurable retry attempts
- Exponential retry backoff
- Context-aware cancellation
- Graceful shutdown
Transient storage failures can therefore be retried before being reported through the batcher's error channel.
Strata-Log handles:
SIGINT
SIGTERM
During shutdown, the service:
- Stops the HTTP server.
- Stops accepting new processing work.
- Drains pending pipeline work.
- Flushes pending batches.
- Closes the storage backend.
- Exits cleanly.
The shutdown timeout is configurable through:
STRATA_LOG_SHUTDOWN_TIMEOUT
Run the complete test suite:
go test ./...Run tests with the race detector:
go test -race ./...Run static analysis:
go vet ./...Run benchmarks:
go test -bench=. -benchmem ./benchmarks/...Or use:
make checkDevelopment documentation covers:
- local setup
- project structure
- testing
- concurrency
- storage development
- resilience development
- batcher development
- API development
- Docker workflows
See docs/development.md.
| Method | Endpoint | Purpose |
|---|---|---|
GET |
/healthz |
Health check |
POST |
/v1/logs |
Ingest a log |
GET |
/v1/logs |
Query logs |
GET |
/metrics |
Prometheus metrics |
See docs/api.md for the complete API reference.
Strata-Log is an MVP demonstrating practical Go backend engineering patterns, including:
- concurrent processing
- asynchronous ingestion
- bounded buffering
- batching
- persistent storage
- retry and resilience patterns
- HTTP API design
- observability
- graceful shutdown
- automated testing
- containerized deployment
The project is intended for local development, experimentation, learning, and demonstration of backend engineering practices.
Contributions are welcome.
Before submitting changes, run:
make checkPlease read CONTRIBUTING.md before contributing.
Strata-Log is licensed under the MIT License.
See LICENSE for the full license text.