This repository contains full stack website (backend + frontend) for a "Lost & Found" website that helps list items found around campus/university. The backend uses YOLO for object detection and a vision-language model (BLIP) to auto-generate short titles and descriptions from item photos. Uploaded items are stored in the database with metadata (location, filename, generated captions (title + description), id) and can be listed, retrieved, or deleted. Click here to see it in action! (https://www.youtube.com/watch?v=YZbugWGhxXA)
Change Branch to using-Qwen2 to use Qwen2 model instead of Blip2! for a better and more faster generation of text!
File Structure:
lost-and-found/
│
├── backend/
│ ├── uploads/ # This folder will be created at the moment of uploading an image
│ │
│ ├── __init__.py
│ ├── categories.py # Categories for detection
│ ├── db.py # Database
│ ├── main_model.py # Loads BLIP-2 model
│ ├── main.py # Main FastAPI
│ ├── models.py # SQLAlchemy models
│ ├── schemas.py # Pydantic schemas
│ ├── yolo_detector.py # Loads YOLOv8 model
│ ├── yolov8n.pt # This will be created at the first run of backend
│ ├── lost_items.db # This will be created at the first run of backend (lost items local database)
│ └── requirements.txt # Requirements
│
├── frontend/
│ │
│ ├── public/ # Stores public assets/images
│ │
│ ├── src/
│ │ ├── app/
│ │ │ ├── page.tsx # Main landing page
│ │ │ ├── find-item/page.tsx # List all lost items page
│ │ │ └── report-item/page.tsx # Report lost item page
│ │ │
│ │ └── components/
│ │ └── Header.tsx
├── .gitignore
└── README.md
Requirements:
- Python 3.10–3.13, Node 16+ (or the version in frontend/package.json)
- Optional: NVIDIA GPU + CUDA for much faster model startup and inference
- Open backend:
cd backend
python -m venv venv
.\venv\Scripts\Activate
pip install -r requirements.txt
uvicorn main:app --reload --port 8000- (Optional) Open API docs to exercise endpoints: http://127.0.0.1:8000/docs
Run the frontend
- Open a new shell and from repo root:
cd frontend
npm install
npm run dev- Open the frontend: http://127.0.0.1:3000
- Models (BLIP-2 and YOLO weights) are large. Initial server start will be slow while models load (minutes on CPU).
- A NVIDIA GPU dramatically speeds startup and inference. On CPU-only machines expect much longer delays and higher memory usage.
-
POST /upload
- Purpose: accept a multipart/form-data image upload and save it to backend/uploads; returns filename.
-
POST /analyze-image
- Purpose: provide a saved filename (returned by /upload) to run YOLO detection + BLIP captioning; returns title, description, detected category, objectName, confidence and a filename or file_url.
-
POST /create-lost-item
- Purpose: create a DB entry for a lost item (title, description, category, foundLocation, filename). Payload follows CreateLostItem schema.
-
GET /lost-items
- Purpose: list lost items. Optional query params:
- category — filter by category name
- since — ISO datetime to filter items added after the timestamp
- Purpose: list lost items. Optional query params:
-
GET /lost-items/{item_id}
- Purpose: retrieve a single lost item by numeric id.
-
DELETE /delete-lost-item/{item_id}
- Purpose: remove an item from the DB (e.g., when claimed/found (not implemented yet)).
- Slow startup: loading BLIP-2 and YOLO on CPU can take several minutes; memory heavy.
- Better with NVIDIA GPU and proper CUDA drivers.
- Persistence: DB and model state are local.
Optimization plan: Probably change to a smaller BLIP variants to reduce memory and startup time: There is a new version using Qwen2, where caption generation is half the time with Blip2!
- Click on Branch and change it to Using-Qwen2 and follow the same instructions to use!
