Recruit Easy

A recurring outreach automation engine: describe a networking goal in one sentence, and it keeps finding new people, drafting personalized emails, and following up on replies on its own schedule, forever without anyone re-triggering it. Every outbound action still waits for a human approval tap on a phone before it happens.

The interesting part isn't the outreach idea. It's that this is a hand-built distributed job queue on top of Postgres, built to survive worker crashes, run multiple workers concurrently without double-processing, and recover automatically from a dead process, all backed by tests that actually prove those properties rather than assume them.

What it does

One goal sentence ("I want to network weekly with AI startup founders") creates one automation that spawns two independent recurring workflows:

Outreach (weekly)   source contacts → draft email → Telegram approval → send
Replies  (daily)    poll inbox → classify reply → Telegram approval → act

Both are shaped identically: produce something → recommend an action → wait for an approval → execute. Nothing reaches a real external person without an explicit Approve on Telegram.

flowchart LR
    subgraph W1["Outreach — runs weekly"]
        S[Source contacts<br/>SerpAPI + Claude agent] --> D[Draft email<br/>Claude]
        D --> A1["Telegram: approve draft?"]
        A1 -->|tap| SEND[Send via Gmail SMTP]
    end

    subgraph W2["Replies — runs daily"]
        P[Poll inbox<br/>Gmail IMAP] --> C[Classify reply<br/>Claude]
        C --> A2["Telegram: recommended action?"]
        A2 -->|tap| EXEC[Execute:<br/>propose times / close]
    end

Loading

System Design for the Job Queue

The recurring engine underneath the outreach feature is a real job queue, built from scratch on Postgres:

• Exactly-once claiming under concurrency, using SELECT ... FOR UPDATE SKIP LOCKED — the same primitive real job queues (Sidekiq, Celery+DB backends) use to let N workers poll one table without two of them ever grabbing the same row. Plain FOR UPDATE would quietly serialize a pool of workers into a queue of one; SKIP LOCKED is what makes horizontal scaling (docker compose up --scale worker=5) actually safe.
• Crash recovery with no dedicated recovery process. A job claimed but not completed within a stale-claim window is assumed orphaned and silently reclaimed by the next scheduler tick — no herd of watchers, one writer.
• At-least-once delivery, made safe by idempotent handlers. A worker can die after sending an email but before recording that it did. Every handler checks state before acting (send_approved_email only sends while an email is still approved), so a redelivered job is a no-op, not a duplicate email.
• Retry with exponential backoff and a real dead-letter state — failures ride a 5s-doubling backoff capped at 10 minutes, up to 5 attempts, then land in failed with the last error preserved. A queue that silently drops work it couldn't finish is worse than one that stops and leaves evidence.
• Verified, not assumed — two tiers. In the automated suite: a threaded test spins up 4 workers against 40 seeded jobs and asserts exactly one execution per job, and that the work was actually spread across workers, not done by one while the rest sat idle. The exactly-once guarantee is falsifiable, not just claimed — deliberately removing skip_locked=True and re-running reproduces 54 executions against those same 40 jobs. Separately, as a one-time manual check (not something CI re-runs): a live 3-container docker compose stack with 300 queued jobs, SIGKILLed mid-drain, came back with every job finished exactly once.

See SYSTEM_DESIGN.md for the full design writeup, including the exact claim query, the job catalog, and the reasoning behind every tradeoff.

Architecture

flowchart TB
    subgraph Stack["docker-compose"]
        API[api<br/>FastAPI — routes only]
        SCHED[scheduler<br/>single process, ticks every 5s]
        W1[worker × N<br/>horizontally scalable]
        PG[(postgres<br/>state + the job queue itself)]
    end
    FE[Next.js dashboard]
    TG[Telegram Bot API]
    GM[Gmail SMTP/IMAP]
    SRC[SerpAPI + Claude<br/>sourcing agent]
    HUNTER[Hunter.io<br/>email enrichment, optional]

    FE <--> API
    API <--> PG
    SCHED -->|enqueues due jobs| PG
    W1 -->|claims + runs jobs| PG
    W1 --> TG
    W1 --> GM
    W1 --> SRC
    W1 --> HUNTER

Loading

Postgres is the queue — one jobs table, claimed via SKIP LOCKED. The scheduler decides when work is due (reads each automation's cadence, inserts rows); workers decide what to do with a claimed row (dispatch by job_type, run the handler, mark done/failed). The two never talk to each other directly.

Real, working pieces

• Job engine: scheduler, N workers, retry/backoff/dead-letter, crash recovery — all tested against real Postgres, not mocks.
• Contact sourcing: a bounded search agent (SerpAPI, optionally wrapped in a Strands agent backed by Claude) that searches the public web, extracts structured candidates, dedupes against everyone already contacted, and validates output against a typed schema — raising typed exceptions on bad data rather than returning garbage. Capped at 5 searches per run so it can never hang unattended.
• Optional email enrichment via Hunter.io, confidence-gated.
• Two full approval-gated workflows, real Telegram bot, real Gmail SMTP/IMAP — verified live end-to-end, including real button taps on a real phone.
• Telegram UX that reflects real state: tapping Approve/Reject edits the message in place, removes the buttons (so a duplicate tap is structurally impossible, not just ignored), and the message updates again once the async send/action actually completes.
• Two safety nets, both on by default: every sourced contact's send address can be routed to a controlled inbox instead of the real person found, so real sourcing can run against the live internet without ever emailing a stranger until you deliberately turn that off — and a lifetime cap on how many contacts one automation will ever source stops a fast demo cadence left running unattended from compounding forever.

Deliberate Simplifications

• No message broker. The job queue is a Postgres table: a deliberate choice to keep the "distributed systems" story self-contained in one already-required piece of infrastructure.
• No Alembic migrations. Schema changes go through create_all; altering an existing table means wiping the dev volume. Fine pre-launch; the natural next step once real data exists.
• Long-polling, not Telegram webhooks. Avoids needing a public HTTPS URL (ngrok, ports) for local development. A webhook is the natural swap if this were ever deployed somewhere with a public URL.
• No cloud hosting. Runs as a full stack via docker-compose on one machine. The job-queue design is the centerpiece here, not where it's hosted.

Stack

Layer	Technology
Backend	Python, FastAPI, SQLAlchemy, Pydantic
Database / queue	PostgreSQL 16 (SELECT ... FOR UPDATE SKIP LOCKED)
Frontend	Next.js 16, React 19, Tailwind
LLM	Claude (Anthropic) — goal parsing, drafting, reply classification, candidate extraction
Sourcing	SerpAPI (web search), optional Strands Agents SDK, optional Hunter.io (email enrichment)
Messaging	Telegram Bot API (approvals), Gmail SMTP/IMAP (send + reply polling)
Infra	Docker Compose — api, scheduler, worker (scalable), postgres
Tests	pytest, 67 tests, run against a real Postgres instance — no mocked database

Running it

git clone <this repo>
cd recruit_easy
cp backend/.env.example backend/.env   # fill in Anthropic / Telegram / Gmail — see below
docker compose up --build              # api + scheduler + worker + postgres
docker compose up --build --scale worker=3   # or with a worker pool

GET http://localhost:8000/health → {"status": "ok"} once it's up. The frontend (cd frontend && npm install && npm run dev) talks to localhost:8000.

Every credential is optional in the sense that the system degrades safely without one: no ANTHROPIC_API_KEY falls back to rule-based drafting/classification; no SERPAPI_API_KEY falls back to 3 fixed demo contacts; no HUNTER_API_KEY just skips email enrichment. Telegram and Gmail credentials are required for the approval loop and sending to do anything real. See backend/.env.example for every variable and backend/README.md for the full setup walkthrough.

Tests

docker compose run --rm api pytest

67 tests, all against a real Postgres database (a separate recruit_easy_test DB, created automatically) — not SQLite, not a mock. The guarantees under test (SKIP LOCKED, crash recovery, idempotent retries) live in Postgres itself, so testing anywhere else would test nothing. Every outbound credential is blanked for the duration of the suite: no test can message a real phone, send a real email, or spend a real API dollar.

Documentation

• SYSTEM_DESIGN.md — the architecture, the exact claim query, the job catalog, and the reasoning behind every tradeoff.
• IMPLEMENTATION_PLAN.md — the build checklist and what's still open.
• backend/README.md — setup, layout, and the data model in detail.