Skip to content

miqdadyyy/reviewly

BranchesTags

Repository files navigation

Reviewly

Reviewly is a self-hostable, AI-powered code review platform. It connects to your Git providers (GitHub, GitLab, Bitbucket), listens for pull-request webhooks, runs the diff through a configurable LLM, and posts provider-native inline review comments, a summary comment, and a merge-gate check/status back on the pull request.

It runs as a single service that exposes an HTTP API, a background worker, and an embedded message queue together, plus a web dashboard served from the same server.

Table of Contents

What Reviewly Does

Reviewly automates pull-request reviews end-to-end:

  1. Connect a Git provider — register a VCS provider (GitHub App / OAuth / Personal Access Token) and connect the repositories you want reviewed.
  2. Configure an AI connector — register an LLM provider (OpenAI-compatible, Anthropic, or a custom proxy) and assign one per repository.
  3. Pick a review method — choose how diffs are analyzed (diff_only, file_by_file, two_pass, dependency_aware, semantic_chunk) and optionally define repository-specific review rules.
  4. Receive webhooks — when a PR is opened or updated, the Git provider notifies Reviewly.
  5. Review & report — Reviewly fetches the diff, sends it to the configured LLM, deduplicates findings, posts inline comments plus a summary, and sets a merge-gate check named reviewly. By default, high or critical findings fail the check; each repository can configure the blocking threshold from Review Rules.
Category Supported Providers
Version Control (VCS) GitHub, GitLab, Bitbucket
AI / LLM OpenAI (and any OpenAI-compatible endpoint: LiteLLM, vLLM, Ollama, Azure OpenAI), Anthropic, custom proxy
Project Management (planned) Jira, Linear, ClickUp

Provider credentials, LLM API keys, and OAuth secrets are never stored in plain config — they are encrypted at rest in the database.

How It Works

                        ┌──────────────────────────────────────────────┐
   Git provider ──────► │  HTTP API                                     │
   (PR webhook)         │  POST /api/v1/git/webhook/<provider>          │
                        │   1. verify the webhook signature             │
                        │   2. parse the normalized PR event            │
                        │   3. match an active connected repository     │
                        │      that has an AI connector assigned        │
                        │   4. create a "queued" merge-gate check       │
                        │   5. enqueue a review job onto the queue       │
                        └──────────────────────┬───────────────────────┘
                                               │  (NATS JetStream)
                                               ▼
                        ┌──────────────────────────────────────────────┐
                        │  Background Worker                            │
                        │   1. mark the review "analyzing"              │
                        │   2. update the check -> in progress          │
                        │   3. fetch the PR diff / changed files        │
                        │   4. review each chunk via the LLM            │
                        │      (with retry + backoff)                   │
                        │   5. deduplicate findings by severity         │
                        │   6. post inline comments + a summary         │
                        │   7. finalize the check                       │
│      (fails by configured severity threshold) │
                        │   8. persist the review + findings            │
                        └──────────────────────────────────────────────┘
  • The webhook endpoint is intentionally fast: it verifies, parses, and enqueues, then returns immediately.
  • The heavy lifting (diff fetch, LLM calls, comment posting) happens asynchronously in the worker, so slow provider/LLM APIs never block webhook delivery.
  • A review run is idempotent per pull request: re-opening/updating a PR resolves the previous Reviewly comments before posting fresh ones.

Architecture

Reviewly is a single deployable unit that internally runs three cooperating components in one process:

Component Responsibility
HTTP API Serves the REST API, receives webhooks, and serves the dashboard.
Worker Consumes review jobs from the message queue and performs the actual review.
Message Queue Decouples webhook intake from review processing (NATS JetStream; can run embedded inside the process). 
            ┌─────────────────────────────────────────────┐
            │                Reviewly (one process)        │
            │                                              │
   client ──┼─► HTTP API ──► Message Queue ──► Worker ──┐   │
   webhook  │      │              ▲                     │   │
            │      │              └─────── enqueue ──────┘   │
            │      ▼                                   ▼     │
            │   Dashboard                          Database  │
            │   (static web)              (SQLite or Postgres)│
            └─────────────────────────────────────────────┘
                         │                         │
                         ▼                         ▼
                  Git providers              LLM providers
              (GitHub/GitLab/Bitbucket)   (OpenAI/Anthropic/…)

Key design points

  • Single binary, three roles. The HTTP server, worker, and (optionally) an embedded NATS server boot together and shut down together. If any one fails on startup, the others stop gracefully.
  • Pluggable providers. Git and LLM integrations sit behind a common interface, so adding or swapping a provider does not change the review workflow.
  • Stateless workers. All durable state lives in the database and the message queue, so the service can be restarted safely.
  • Secrets encrypted at rest. Tokens and API keys are stored AES-encrypted using a configured key.
  • Database flexibility. Runs on SQLite out of the box (zero setup, great for single-node) or PostgreSQL (with optional read replicas) for larger deployments. The schema is created/updated automatically on startup.

Quick Start (Docker Compose)

This is the fastest way to get a running instance. It uses the bundled Dockerfile, SQLite, and an embedded message queue — no external database or NATS server required.

1. Create your environment file

Copy the sample and adjust the secrets:

cp .env.sample .env

A minimal but production-sane .env (based on .env.sample):

APP_ENV=production

# app
APP__NAME=Reviewly

# database (SQLite, stored on a mounted volume)
DATABASE__DRIVER=sqlite
DATABASE__SQLITE_PATH=/app/storage/reviewly.db

# http
HTTP_APP__HOST=0.0.0.0
HTTP_APP__PORT=8080
HTTP_APP__PUBLIC_URL=https://reviewly.example.com   # public URL providers call back to
HTTP_APP__DEBUG=false
HTTP_APP__FLAGGO=true

# message queue (embedded NATS JetStream)
WORKER_APP__PROVIDER=nats
WORKER_APP__NATS__URL=nats://127.0.0.1:4222
WORKER_APP__NATS__EMBEDDED__ENABLE=true
WORKER_APP__NATS__EMBEDDED__HOST=127.0.0.1
WORKER_APP__NATS__EMBEDDED__PORT=4222
WORKER_APP__NATS__EMBEDDED__STORE_DIR=/app/storage/nats

# feature flags dashboard auth — CHANGE THESE
FLAGGO__PROVIDER=file
FLAGGO__AUTH__ENABLED=true
FLAGGO__AUTH__USERNAME=admin
FLAGGO__AUTH__PASSWORD=change-me
FLAGGO__FILE__PATH=/app/storage/flags.json

# auth + secrets — CHANGE ALL OF THESE
JWT__SECRET_KEY=replace-with-a-long-random-string
JWT__REFRESH_SECRET_KEY=replace-with-another-long-random-string
JWT__EXPIRATION_IN_SECONDS=3600
ENCRYPTION__DATABASE_IV_KEY=replace-with-a-strong-encryption-key

# where the OAuth callback redirects after connecting a provider
OAUTH__DASHBOARD_URL=https://reviewly.example.com/web/git

Every value maps to a config key using double-underscore notation for nested keys (e.g. worker_app.nats.url → WORKER_APP__NATS__URL). See Configuration.

2. Wire the env file into Compose

The repo ships a minimal docker-compose.yaml. Point it at your .env and persist storage so your database and queue survive restarts:

services:
  app:
    container_name: reviewly
    build:
      context: .
      dockerfile: Dockerfile
    restart: on-failure
    env_file:
      - .env
    ports:
      - "8080:8080"
    volumes:
      - reviewly-storage:/app/storage

volumes:
  reviewly-storage:

3. Start it

docker compose up --build -d

Then open the dashboard at http://localhost:8080/web/ and start by registering a Git provider and an LLM connector.

To follow logs and stop:

docker compose logs -f
docker compose down          # stop (keeps the storage volume)
docker compose down -v       # stop and delete persisted data

Run with Docker

If you prefer plain Docker without Compose:

# build the image
docker build -t reviewly .

# run it, passing your env file and persisting storage
docker run -d \
  --name reviewly \
  --env-file .env \
  -p 8080:8080 \
  -v reviewly-storage:/app/storage \
  reviewly

The container starts the HTTP API, the worker, and the embedded message queue together. It listens on port 8080 and serves the dashboard at /web/.

Using PostgreSQL instead of SQLite

For multi-node or higher-throughput deployments, point Reviewly at PostgreSQL by changing the database section of your .env:

DATABASE__DRIVER=postgres
DATABASE__DSN=host=db user=postgres password=postgres dbname=reviewly port=5432 sslmode=disable
DATABASE__MAX_IDLE_CONNECTIONS=10
DATABASE__MAX_OPEN_CONNECTIONS=100

Example Compose snippet adding a Postgres service:

services:
  app:
    # ...as above...
    env_file:
      - .env
    depends_on:
      - db

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: reviewly
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    volumes:
      - reviewly-db:/var/lib/postgresql/data

volumes:
  reviewly-storage:
  reviewly-db:

The database schema is created and migrated automatically on first boot — no migration step is required.

Configuration

Reviewly is configured through environment variables (ideal for containers) or YAML files. Both map to the same set of keys.

How configuration is resolved

  • APP_ENV selects the base config profile (default local; use production in containers).
  • Environment variables override any value using double-underscore notation for nested keys:
    • app.name → APP__NAME
    • database.driver → DATABASE__DRIVER
    • worker_app.nats.url → WORKER_APP__NATS__URL
    • http_app.public_url → HTTP_APP__PUBLIC_URL

Configuration sections

Section Purpose
app Application name.
database Driver (sqlite / postgres), DSN or discrete connection fields, pool limits, SQLite path, optional read replicas.
http_app Bind host/port, public_url (the externally reachable URL providers call back to), debug toggle, feature-flag UI toggle.
worker_app Message-queue provider (nats) and NATS connection / embedded-server settings.
flaggo Feature-flag store (file / redis) and dashboard auth.
redis Optional Redis connection (cache / flag store).
storage S3-compatible object storage (endpoint, keys, bucket, region).
jwt Secret keys and token expiration.
encryption Key used to encrypt sensitive data (provider tokens, API keys) at rest. Set a strong value.
oauth dashboard_url the OAuth callback redirects to after connecting a provider.
log File logging and rotation.

Important values to set for any real deployment

  • ENCRYPTION__DATABASE_IV_KEY — encrypts all provider secrets; treat it like a master key and never rotate it casually.
  • JWT__SECRET_KEY / JWT__REFRESH_SECRET_KEY — long random strings.
  • HTTP_APP__PUBLIC_URL and OAUTH__DASHBOARD_URL — must be the real public URL so OAuth callbacks and webhooks resolve correctly.
  • FLAGGO__AUTH__USERNAME / FLAGGO__AUTH__PASSWORD — protects the feature-flag UI.

The full set of available variables lives in .env.sample.

Endpoints & Dashboard

Once running, the service exposes:

URL Purpose
/web/ Web dashboard (connect providers, configure connectors, browse reviews).
/api/v1 REST API base.
/api/v1/git/webhook/<provider> Inbound webhook endpoint for github / gitlab / bitbucket. Point your provider's webhook here.
/utility/flaggo Feature-flag UI (when enabled).

The merge-gate check posted on pull requests is named reviewly. By default it turns red when a high or critical issue is found; the per-repository Review Rules screen can raise, lower, or disable that blocking threshold.

When configuring a provider's webhook, use your public base URL plus the webhook path, e.g. https://reviewly.example.com/api/v1/git/webhook/github.

Operating the Service

Health & lifecycle

  • The process boots the API, worker, and queue together and shuts them down cleanly on SIGINT/SIGTERM (Docker stop).
  • Restarting is safe at any time — durable state is in the database and the queue's store directory.

Persistence

  • With SQLite + embedded NATS, persist /app/storage (the database file, NATS store, and flag file all live here). The Compose/Docker examples above mount a named volume for exactly this reason.
  • With PostgreSQL, persist the Postgres data volume; the queue store still lives under /app/storage.

Backups

  • Back up the database (SQLite file or Postgres dump) and the /app/storage directory.
  • The encryption key (ENCRYPTION__DATABASE_IV_KEY) is required to decrypt stored secrets — back it up securely and separately.

Production Notes

  • Run behind TLS. Terminate HTTPS at a reverse proxy (nginx, Caddy, Traefik) or your platform's load balancer, and set HTTP_APP__PUBLIC_URL to the public https:// URL.
  • Set APP_ENV=production to get production error handling (internal errors are not leaked to clients).
  • Turn off debug (HTTP_APP__DEBUG=false) and secure the feature-flag UI in production.
  • Scaling. For a single node, SQLite + embedded NATS is sufficient. To scale out, move to PostgreSQL and an external NATS server so multiple instances can share state and review jobs.
  • Secrets management. Inject secrets via your platform's secret store rather than committing them; the provided .env workflow is a convenient default for self-hosting.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages