BasketIQ

A personal finance tool that ingests Portuguese supermarket receipts (PDF), parses every line item with an LLM, stores vectors in PostgreSQL, and lets you explore your spending through an analytics dashboard and a RAG chat interface — at zero inference cost, running entirely on local models.

Built with FastAPI · PostgreSQL + pgvector · Ollama · Groq · Streamlit

→ Architecture Decision Records — key design choices, alternatives considered, tradeoffs accepted

---

Demo

Receipts	Analytics dashboard

Sample input receipts

Pingo Doce	Continente

---

What it does

1. Upload a PDF receipt → text is extracted (pdfplumber, or LLaVA vision model for image-only PDFs)
2. LLM parses every line item → product name, category, quantity, unit price, total price
3. Embeddings stored in pgvector → each item is a searchable vector
4. Analytics dashboard → spend by category, monthly trends, top items, per-retailer breakdowns, repurchase cycles
5. RAG chat → ask natural language questions; the backend retrieves semantically similar items and answers with an LLM

---

Architecture

flowchart LR
    subgraph Ingestion
        A[PDF upload] --> B[pdfplumber]
        B -->|text ≥ 100 chars| D[Parser LLM]
        B -->|blank / image PDF| C[LLaVA vision model]
        C --> D
        D --> E[Embedder\nnomic-embed-text]
        E --> F[(PostgreSQL\n+ pgvector)]
    end

    subgraph API
        F --> G[FastAPI]
        G --> H["/receipts"]
        G --> I["/analytics"]
        G --> J["/chat"]
    end

    subgraph Frontend
        J --> K[RAG chat\nStreamlit]
        I --> L[Analytics charts\nPlotly]
        H --> M[Receipts table\nStreamlit]
    end

Loading

---

Key design decisions

1. Zero-cost inference stack

The original plan called for OpenAI (GPT-4o + text-embedding-3-small). During implementation, the entire LLM stack was replaced with free alternatives:

Role	Model	Provider
Receipt parsing	llama-3.1-8b-instant	Groq free tier (fast, primary)
Receipt parsing fallback	llama3.2	Ollama (local, offline)
Vision OCR (image PDFs)	llava	Ollama
Embeddings	nomic-embed-text (768 dims)	Ollama
Chat answers	llama3.2	Ollama

Groq is used first for parsing (low latency, free up to rate limit); Ollama is the fallback. Everything else runs locally via Ollama's OpenAI-compatible API, so no tokens are billed at any point.

2. Portuguese receipt specialization

Portuguese supermarket receipts have domain-specific formatting that generic prompts handle poorly:

• Decimal separator is a comma (1,99 = €1.99)
• QTD / QT = quantity, PVP / P.UNIT = unit price, IVA = VAT, DESCONTO = discount
• Multi-unit lines like 2UN 2,78 mean quantity 2, total €2.78 (unit price = €1.39) — a common source of LLM errors
• Store names appear with suffixes (Continente Online, Continente Modelo) that need normalizing

The parser system prompt encodes all of these rules explicitly.

3. Arithmetic expression repair

When parsing multi-unit items, smaller LLMs occasionally output arithmetic expressions inside JSON fields instead of computed values (e.g. "unit_price": 3.72 / 18). This makes the JSON unparseable. A regex-based repair pass evaluates any a / b pattern inside numeric JSON fields before the final parse attempt, recovering otherwise-lost receipts.

4. pgvector for semantic search

Each line item's name is embedded and stored as a 768-dimensional vector. The /chat endpoint embeds the user's query with the same model, then runs a cosine distance query:

SELECT * FROM line_items
ORDER BY embedding <=> $query_vector
LIMIT 5;

This finds semantically relevant items (e.g. "milk" matches "Leite Mimosa 1L") without requiring exact keyword matches.

5. Store name normalization

LLMs return store names inconsistently ("Continente", "CONTINENTE MODELO", "Continente Online Modelo"). A prefix-matching alias table in the analytics endpoint maps all variants to canonical names before aggregation, so per-retailer statistics are accurate.

Full rationale, alternatives considered, and tradeoffs accepted for each decision: docs/ADR.md

---

Analytics

Beyond simple totals, the analytics endpoint computes:

• Spend by category — Produce, Dairy, Meat, Bakery, Beverages, etc.
• Monthly spend + trip count — how many shopping trips per month and their total
• Average basket size — mean items per receipt
• Average days between trips — shopping frequency
• Per-retailer averages — mean spend at Continente vs Pingo Doce
• Top 10 most-purchased items (by frequency and by total spend)
• Repurchase cycles — items bought across multiple receipts with average days between purchases

---

Tech stack

Layer	Technology
API	FastAPI, SQLAlchemy 2 (sync), Alembic
Database	PostgreSQL 15 + pgvector
PDF extraction	pdfplumber, PyMuPDF (fitz)
LLM inference	Ollama (local), Groq API
Embeddings	nomic-embed-text via Ollama
Frontend	Streamlit, Plotly
Tests	pytest, httpx

---

Local setup

Prerequisites

• Python 3.11+
• PostgreSQL with pgvector extension
• Ollama with llama3.2, llava, and nomic-embed-text pulled
• (Optional) Groq API key for faster parsing

1. Pull Ollama models

ollama pull llama3.2
ollama pull llava
ollama pull nomic-embed-text

2. Configure environment

cp backend/.env.example backend/.env
# Edit backend/.env:
#   DATABASE_URL=postgresql://postgres:postgres@localhost:5432/receipts_db
#   GROQ_KEY=gsk_...   (optional)

3. Run migrations

cd backend
pip install -r requirements.txt
alembic upgrade head

4. Start the API

uvicorn app.main:app --reload
# → http://localhost:8000
# → http://localhost:8000/docs

5. Ingest receipts

# Place PDFs in receipts/<StoreName>/
python scripts/bulk_ingest.py

6. Start the dashboard

cd frontend
pip install -r requirements.txt
streamlit run app.py
# → http://localhost:8501

---

Project structure

supermarket-receipts/
├── backend/
│   ├── app/
│   │   ├── api/          # FastAPI routers (receipts, analytics, chat)
│   │   ├── pipeline/     # extractor → parser → embedder
│   │   ├── models.py     # SQLAlchemy ORM
│   │   ├── schemas.py    # Pydantic models
│   │   └── config.py     # pydantic-settings (.env)
│   ├── alembic/          # DB migrations
│   ├── scripts/          # bulk_ingest.py
│   └── tests/            # pytest (mocked LLM calls)
├── frontend/
│   ├── views/            # receipts.py · analytics.py · chat.py
│   ├── api_client.py     # HTTP client (testable, no Streamlit deps)
│   └── app.py            # entry point, sidebar routing
└── receipts/             # PDF source files (gitignored)

---

Running tests

# Backend
cd backend
pytest tests/ -v

# Frontend
cd frontend
pytest tests/ -v

---

Results on real receipts

Tested on Portuguese Continente receipts (standard text PDFs):

• pdfplumber extracts clean text on all digital receipts — vision fallback not needed for these
• Groq (llama-3.1-8b-instant) parses a 30-item receipt in ~2 seconds
• Arithmetic repair fires on ~10% of receipts with multi-unit items (e.g. 18UN packs)
• pgvector similarity search returns relevant items in <20 ms on a local database
• Analytics cover several months of shopping history once receipts are ingested

---

License

MIT