Caliber — AI Resume Screening Agent

A self-contained full-stack AI application. Candidates upload a PDF resume plus a job title and job description; the system extracts the real text from the PDF, scores the candidate against the JD with an LLM, and stores both the resume and the screening result. HR admins review everything in a realtime dashboard, and send the candidate an interview or rejection email from there.

There are no external workflow dependencies — no n8n, no third-party webhook. Everything runs in the app and its backend.

---

Architecture

Browser (React + Vite + Tailwind + shadcn)
        │  supabase.functions.invoke('screen-resume')   [multipart/form-data]
        ▼
Edge function: screen-resume
        ├── rate limit (5 / IP / hour) + honeypot bot check + input validation
        ├── PDF text extraction  (unpdf)
        ├── resume upload        (private "resumes" storage bucket)
        ├── LLM scoring          (Lovable AI Gateway → google/gemini-2.5-flash, tool-call schema)
        └── insert into `candidates` with the service role  ← screening is never lost
        ▼
Postgres (RLS) ──realtime──▶ HR dashboard
                                   │  HR admin clicks "send" on a candidate
                                   ▼
                          Edge function: send-candidate-email (Resend)
                          — requires an authenticated hr_admin session

Notification is deliberately not automatic. send-candidate-email verifies the caller's session, so it cannot be driven by the anonymous public form; a human decides what the candidate is told, after seeing the score.

Data model

candidates — name, email, role (job title), score, verdict, confidence, summary, matched_skills, years_relevant_experience, short_reason, recommended_next_steps, email_draft, resume_path, email-sent flags.

user_roles + app_role enum (hr_admin, user) — roles live in their own table, never on a profile row.

Security model

• RLS everywhere. Candidate rows are readable only by hr_admin; updates and deletes are hr_admin-only.
• No public inserts. The anon key cannot write candidate rows any more; only the edge function (service role) inserts, after validation and rate limiting.
• has_role() is SECURITY DEFINER so role checks inside policies don't recurse.
• Resumes are private. The resumes bucket is not public; the dashboard opens files through short-lived signed URLs, and storage policies restrict reads to hr_admin.
• Admin bootstrap without SQL. claim_first_admin() grants hr_admin to the first signed-in user if — and only if — no HR admin exists yet. Every later sign-in is a no-op.
• Leaked-password protection (HIBP) is enabled on auth.

Local development

bun install
bun run dev

Backend secrets used by the edge functions: LOVABLE_API_KEY (AI Gateway), RESEND_API_KEY (candidate emails), plus the standard Supabase env vars.

Frontend variables live in .env: VITE_SUPABASE_URL, VITE_SUPABASE_PUBLISHABLE_KEY, VITE_SUPABASE_PROJECT_ID, and optionally VITE_CALENDAR_URL — a real booking link (Calendly or similar) shown on Interview verdicts. Leave it unset and the scheduling button is hidden.

First run

1. Create the resumes storage bucket — Storage → New bucket → name resumes, Public off. Required before resume files are retained; see the note below.
2. Sign up on /auth, then log in — you automatically become the HR admin.
3. Submit a resume on /.
4. Watch the row appear live on /dashboard, open the candidate, and reopen the stored PDF.

On step 1. Storage is intentionally non-blocking: if the bucket is missing, the upload failure is logged, resume_path stays null, and the response returns resume_stored: false — screening still completes and the candidate row is still written. Nothing downstream depends on the file. The upside is that a storage outage never costs a screening; the downside is that a missing bucket fails quietly, so verify resume_stored on your first submission. supabase/migrations/20260728110000_create_resumes_bucket.sql creates the bucket for a fresh clone.

Evaluation

eval/ contains a labelled evaluation set — twelve synthetic resumes against one job description, balanced 4/4/4 across Interview, Hold and Reject, including four deliberately borderline cases.

node eval/run-eval.mjs        # scores the set, writes eval/results.md

It reports exact agreement with the human labels, Cohen's kappa (accuracy corrected for chance — the number that actually matters with three balanced classes), per-verdict precision/recall/F1, score separation, and a written breakdown of every disagreement. See eval/README.md for the methodology and the limits of what twelve samples can show.

Design system

Editorial: warm paper, ink, and an ember accent; Libre Baskerville for display type over IBM Plex Sans for everything else. Colour, elevation and radius are HSL custom properties in src/index.css, so both themes are driven from one set of tokens.

Contrast is verified rather than eyeballed:

npm run check:contrast

That parses the tokens straight out of index.css and checks all 17 pairings the UI depends on, in both themes, against WCAG 2.1 AA — 4.5:1 for text, 3:1 for focus rings, 1.5:1 for structural borders. It exits non-zero on failure, so a restyle that breaks contrast is caught rather than shipped. All 34 checks currently pass.

Two decisions worth noting: --accent is deliberately not --primary, because shadcn uses accent for low-emphasis hover surfaces and pointing it at the brand colour turns every hover into a saturated block; and the ember primary lightens in dark mode while its label darkens, since white-on-ember only clears 4.5:1 when the ember is dark, and a dark ember on a dark page has too little separation. Motion respects prefers-reduced-motion.

Limitations

Known and deliberate, rather than overlooked:

• Rate limiting is per-instance. checkRateLimit holds an in-memory Map, so the 5/IP/hour ceiling applies per edge function instance, not globally. A durable limiter would key off a rate_limits table. Adequate for the traffic this handles; not a real abuse control.
• CORS differs between the two functions. screen-resume sends Access-Control-Allow-Origin: * so it can be called from preview deployments; send-candidate-email uses an explicit origin allowlist. The allowlist has to be updated when the published domain changes, or the browser blocks the response and email sending appears to fail for no visible reason.
• Email sending runs against Resend's shared sandbox sender (onboarding@resend.dev), which can only deliver to the address that owns the Resend account. Delivering to arbitrary candidates requires verifying a domain in Resend and changing the from address.
• No scheduling integration. An Interview verdict shows a booking link only when VITE_CALENDAR_URL is configured; there is no real calendar API behind it. Without that variable the button is hidden rather than pointing somewhere that does not resolve.
• Scanned resumes are rejected, not OCR'd. unpdf reads embedded text only, so an image-only PDF is refused with an explanatory message rather than silently scored on empty input.
• The evaluation set is small and synthetic (n = 12), which likely makes the reported agreement optimistic relative to real resumes.

Roadmap

• Explainable per-category scoring
• Bulk upload: 20 resumes against one JD → ranked shortlist
• Blind screening toggle with a measured bias delta
• Grow the evaluation set past 12 resumes and add a second job description
• pgvector semantic skill matching