Contract & COI Search Platform

An API-first backend for uploading, processing, and full-text-searching contracts and Certificates of Insurance (COIs). Documents are ingested as PDFs, text is extracted (with an OCR fallback for scanned pages), and every page is indexed into PostgreSQL's native full-text search so users can run exact-phrase and keyword queries with page-level results.

Originally built as a real-world document-search tool for a GRC (governance, risk & compliance) consultancy. The codebase is organized as a conventional layered service — routes → services → repositories → models — to keep business logic testable and the data-access layer swappable.

What it does

• PDF ingestion — content-type is verified with libmagic, uploads are size-capped, and each file is content-addressed by SHA-256 so duplicate uploads are rejected. Files are written under date-sharded directories with random, unguessable names, and every read path is checked against directory traversal.
• Asynchronous processing — uploads return immediately and a Celery worker extracts text page-by-page (pypdfium2, falling back to PyPDF2). Any page with almost no embedded text is routed through Tesseract OCR (pdf2image + pytesseract).
• Full-text search — each page is indexed into a Postgres tsvector column with a GIN index. Search supports exact-phrase matching (phraseto_tsquery) and keyword matching (to_tsquery), ranked with ts_rank, and returns a highlighted snippet plus the document name and page number.
• Auth & RBAC — JWT bearer auth (bcrypt password hashing via passlib), with admin and viewer roles. Admins get user-management, delete/reprocess, and reindex endpoints.
• Operational endpoints — health check, index statistics, per-document processing status, and an admin reindex job.

Architecture

Client ──▶ FastAPI (routes)
             │
             ├─ services/        business logic (auth, documents, search)
             ├─ repositories/    data access (SQLAlchemy + raw FTS SQL)
             ├─ models/          ORM entities (users, roles, documents, pages, search_index)
             │
             ├─ PostgreSQL 15    relational store + full-text index (tsvector / GIN)
             └─ Redis 7  ◀──▶  Celery workers
                                 ├─ document_processor  (extract text + OCR)
                                 └─ indexer             (build the search index)

Upload flow: POST /api/documents stores the file and enqueues a Celery task → the worker extracts/OCRs each page and writes pages rows → it enqueues the indexer → the indexer builds a tsvector per page. Search reads only from the index, so queries stay fast regardless of corpus size.

Tech stack

Layer	Choice
API	FastAPI, Uvicorn, Pydantic v2
Persistence	PostgreSQL 15, SQLAlchemy 2.0
Search	Postgres full-text search (tsvector, GIN)
Async processing	Celery + Redis 7
PDF / OCR	pypdfium2, PyPDF2, pdf2image, Tesseract
Auth	JWT (python-jose), bcrypt (passlib)
Packaging	Docker & Docker Compose

Quick start

# 1. Configure
cp backend/.env.example backend/.env      # then set SECRET_KEY and DB_PASSWORD

# 2. Launch the full stack (db, redis, backend, worker, static frontend)
docker-compose up --build -d

# 3. Verify
curl http://localhost:8000/health          # -> {"status":"healthy","version":"1.0.0"}

• API docs (Swagger UI): http://localhost:8000/docs
• Static status page: http://localhost:3000

First user & admin bootstrap

The app auto-creates the admin and viewer roles on startup but does not seed an admin account. Register the first user, then grant it the admin role directly:

# Register (created as a viewer by default)
curl -X POST http://localhost:8000/api/auth/register \
  -H 'Content-Type: application/json' \
  -d '{"email":"you@example.com","password":"changeme123","full_name":"You"}'

# Promote to admin (one-time, via the database)
docker-compose exec db psql -U app_user -d contract_search -c \
  "INSERT INTO user_roles (user_id, role_id)
   SELECT u.id, r.id FROM users u, roles r
   WHERE u.email='you@example.com' AND r.name='admin';"

Log in at POST /api/auth/login (OAuth2 password form) to receive a bearer token.

API overview

Area	Endpoint	Notes
Auth	POST /api/auth/login · /register · GET /me	JWT bearer
Documents	POST /api/documents	upload PDF, async processing
	GET /api/documents · /{id} · /{id}/status	list / detail / progress
	GET /api/documents/{id}/pdf	download original
	DELETE /api/documents/{id} · POST /{id}/reprocess	admin only
Search	POST /api/search · GET /api/search/stats	phrase & keyword search
Admin	GET /api/admin/stats · users CRUD · POST /reindex	admin only

Full request/response schemas are available in the interactive docs at /docs.

Security notes

• Parameterized queries throughout, including the raw full-text SQL (bind params, no string interpolation of user input).
• Passwords stored as bcrypt hashes; JWTs signed with a configurable SECRET_KEY.
• Upload path is content-type-validated, size-limited, and content-deduplicated; stored filenames are random and read paths are traversal-checked.
• SECRET_KEY and the database password come from the environment — the committed .env.example contains placeholders only.

Scope & status

This is a working, self-contained demo of the ingestion-and-search backend, not a finished product. Known scope boundaries:

• The frontend/ service is a single static status/landing page. The real interface is the API and its OpenAPI docs — this is an API-first project.
• Search is lexical (Postgres full-text), not semantic. Semantic/clause-extraction features would be a natural next step but are not implemented.
• An audit_log table is defined in the schema for future action tracking; it is not yet written to by the request handlers.
• No automated test suite ships yet, though the test tooling (pytest, httpx, faker) is wired into requirements.txt.

Roadmap

• Wire the audit-log table into request handlers for compliance trails.
• Add a pytest suite (repository + service layers, plus API smoke tests via httpx).
• Optional semantic search / clause extraction and enterprise SSO.