Skip to content
BranchesTags

Repository files navigation

Handwriting Recognition

Upload a handwritten image (or a ZIP of them), have it checked as genuinely handwritten, and get the text back.

A Django REST API pairs two hosted models: a Hugging Face document classifier (microsoft/dit-base-finetuned-rvlcdip) screens each image, and Mistral OCR (mistral-ocr-latest) transcribes the ones that pass. A React single-page app provides the upload, preview and result UI.

License: AGPL v3

Requires third-party API keys. Recognition calls the Hugging Face and Mistral APIs. Without keys the app starts and serves normally, but the recognition endpoint returns 503. Both providers charge for usage.

Contents

Features

  • Single image or ZIP — upload one PNG/JPG, or a ZIP archive processed as a batch.
  • Handwriting screening — images classified as non-handwritten are reported as such instead of being sent to OCR.
  • Text extraction — Mistral OCR returns the transcription as Markdown.
  • Preview — the selected image, or a carousel across the images in a ZIP.
  • Copy and download — copy the result, or save it as a .txt file.
  • Contact form — reCAPTCHA-protected; stores submissions in the database and a CSV export. See Privacy.

Architecture

handwriting-recognition/
├── backend/                        Django 5.2 + DRF
│   ├── HandWrittenApp/
│   │   ├── exceptions.py           Domain errors carrying HTTP status codes
│   │   ├── models.py               ConnectUs submission model
│   │   ├── serializers.py          Request validation
│   │   ├── services.py             Business logic
│   │   ├── utils.py                Hugging Face / Mistral adapters
│   │   ├── views.py                Thin HTTP handlers
│   │   ├── urls.py
│   │   └── tests/                  pytest suite
│   ├── HandWrittenRecognition/     Project settings, URLs, WSGI/ASGI
│   ├── Dockerfile                  Multi-stage, non-root, gunicorn
│   └── requirements.txt
│
├── frontend/                       React 19 + Vite 7 + Tailwind 3
│   ├── src/
│   │   ├── components/             Presentational components
│   │   ├── constants/              Shared values
│   │   ├── hooks/                  Stateful logic
│   │   ├── pages/Home.jsx          Recognition page
│   │   └── services/               API layer
│   ├── Dockerfile                  Multi-stage, non-root, nginx
│   └── nginx.conf
│
└── docker-compose.yml

Layering: views handle transport only, services.py holds business logic, utils.py wraps the third-party providers, and serializers own validation. Provider failures raise typed exceptions that map to real HTTP status codes.

Stack: Python 3.12+, Django 5.2, Django REST Framework, Pillow, huggingface-hub, mistralai, gunicorn, WhiteNoise · React 19, Vite 7, Tailwind CSS 3, axios, react-dropzone, JSZip, react-toastify.

Quick start

Runs from a fresh clone with no database server — the backend defaults to SQLite.

Backend

cd backend
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env             # then set DEBUG=True and your API keys
python manage.py migrate
python manage.py runserver

The API is now on http://localhost:8000.

Frontend

cd frontend
npm install
cp .env.example .env             # VITE_API_URL defaults to the local backend
npm run dev

The app is now on http://localhost:5173.

Set HUGGINGFACE_API_KEY and MISTRAL_API_KEY in backend/.env before recognition will work, and RECAPTCHA_SECRET_KEY (plus the frontend's VITE_RECAPTCHA_SITE_KEY) before the contact form will accept submissions.

Configuration

Every variable below is read by the code. Copy backend/.env.example and frontend/.env.example and fill them in.

Backend (backend/.env)

Variable Default Description
DEBUG False Development mode. Never enable in production.
DJANGO_SECRET_KEY — Required when DEBUG=False; start-up aborts if unset.
DJANGO_ALLOWED_HOSTS localhost,127.0.0.1 (debug only) Comma-separated. Required when DEBUG=False.
CORS_ALLOWED_ORIGINS http://localhost:5173 (debug only) Comma-separated browser origins.
CSRF_TRUSTED_ORIGINS falls back to CORS_ALLOWED_ORIGINS Production only.
HUGGINGFACE_API_KEY empty Handwriting classifier. Endpoint returns 503 without it.
MISTRAL_API_KEY empty OCR provider. Endpoint returns 503 without it.
HANDWRITING_CLASSIFIER_MODEL microsoft/dit-base-finetuned-rvlcdip Classifier model id.
MISTRAL_OCR_MODEL mistral-ocr-latest OCR model id.
HANDWRITING_CONFIDENCE_THRESHOLD 0.7 Minimum classifier confidence (0.0–1.0).
RECAPTCHA_SECRET_KEY empty Server-side reCAPTCHA secret. Form rejects all submissions without it.
RECAPTCHA_VERIFY_URL Google siteverify endpoint Override only for testing.
DB_ENGINE django.db.backends.sqlite3 Set to django.db.backends.mysql for MySQL.
DB_NAME backend/db.sqlite3 Database name or path.
DB_USER / DB_PASSWORD / DB_HOST / DB_PORT empty Ignored under SQLite.
MAX_IMAGE_UPLOAD_BYTES 2097152 (2 MB) Per-image upload ceiling.
MAX_ZIP_UPLOAD_BYTES 10485760 (10 MB) Archive upload ceiling.
MAX_ZIP_ENTRIES 50 Maximum images processed per archive.
CSV_DIR backend/CSV Contact-form CSV export directory. Holds personal data.
LOG_LEVEL INFO Root logger level.
SECURE_SSL_REDIRECT True Production only. Set False when TLS terminates upstream.
SECURE_HSTS_SECONDS 31536000 Production only.

Using MySQL additionally requires pip install mysqlclient==2.2.7.

Frontend (frontend/.env)

Variable Default Description
VITE_API_URL http://localhost:8000/recognition Backend base URL, including the /recognition mount.
VITE_RECAPTCHA_SITE_KEY empty Public reCAPTCHA site key.

Vite inlines VITE_* values into the bundle at build time. They are publicly visible in shipped JavaScript — never put a secret there — and changing them requires a rebuild.

API reference

Base path: /recognition/.

GET /healthz/

Liveness probe. Not under the /recognition/ prefix.

{ "status": "ok" }

POST /recognition/handwriting_recognition/

multipart/form-data. Send either file (repeatable) or zip_file — not both.

Field Type Notes
file image image/png, image/jpeg, image/jpg. Max 2 MB each.
zip_file archive .zip. Max 10 MB, up to 50 images.

200 OK — one entry per image, in upload order:

{
  "results": [
    { "image_index": 1, "filename": "note.png", "recognized_text": "Meeting at 4pm" },
    { "image_index": 2, "filename": "logo.png", "recognized_text": "Rejected - Not handwritten" },
    { "image_index": 3, "filename": "broken.png", "recognized_text": null,
      "error": "Recognition provider is temporarily unavailable." }
  ]
}

recognized_text is "No text recognized" when the image is handwritten but yields no text. A per-image error field appears instead of text when that image failed; other images in the batch are unaffected.

Errors: 400 invalid/missing/oversized upload or corrupt archive · 502 provider unreachable · 503 provider API key not configured.

{ "error": "The uploaded archive is not a readable ZIP file." }

POST /recognition/submit_form/

Contact form. Accepts JSON or form data. All fields required.

Field Type
first_name, last_name, company, message string
email email
project array of strings, min length 1
recaptcha_token string

201 Created → { "message": "Form submitted successfully" } 400 Bad Request → DRF field errors, including failed reCAPTCHA.

/admin/

Django admin. No models are registered; the contact model is deliberately not exposed there.

Development

Backend

cd backend
pip install -r requirements.txt -r requirements-dev.txt

pytest                                          # test suite
pylint HandWrittenApp HandWrittenRecognition manage.py
python manage.py check --deploy                 # production config audit
pip-audit -r requirements.txt                   # dependency vulnerabilities

Frontend

cd frontend
npm install

npm run lint      # ESLint
npm run build     # production build
npm audit         # dependency vulnerabilities

These are exactly the commands CI runs — see CONTRIBUTING.md and .github/workflows/ci.yml.

Docker

Requires Docker Compose v2.24+. Starts with no .env present:

docker compose up --build

Both images are multi-stage and run as a non-root user. The backend serves through gunicorn and the frontend through nginx — neither uses a development server.

Because Vite inlines configuration at build time, changing VITE_API_URL or VITE_RECAPTCHA_SITE_KEY requires docker compose build frontend.

Before deploying, set a real DJANGO_SECRET_KEY and DJANGO_ALLOWED_HOSTS; the compose defaults are for local use only. See SECURITY.md.

Limitations

  • Not offline. Recognition requires reachable Hugging Face and Mistral APIs, and both are billed per use.
  • Screening is approximate. dit-base-finetuned-rvlcdip classifies document types; images are accepted on the labels handwritten or letter above the confidence threshold. Handwritten images can be rejected and some typed documents accepted. Tune HANDWRITING_CONFIDENCE_THRESHOLD for your inputs.
  • No authentication. Every endpoint is public. Put the deployment behind auth, a gateway or rate limiting before exposing it — nothing here prevents a visitor from spending your API budget.
  • No rate limiting is implemented.
  • PDFs are not supported — PNG/JPG only.
  • ZIP results are not paginated in the UI; the carousel steps one image at a time.
  • Batches are processed serially, so a large archive is slow.
  • No persistent storage of results. Transcriptions are returned and not saved.
  • SQLite by default is fine for evaluation, not for concurrent production use.
  • Frontend bundle is ~557 kB (~162 kB gzipped) and is not code-split.

Privacy

The contact form collects first name, last name, company, email address and message, and stores each submission in two places:

  1. the configured database, and
  2. a plaintext CSV at CSV_DIR/ConnectUs.csv (default backend/CSV/).

CSV/ is gitignored and excluded from Docker images, but on a live deployment it is unencrypted personal data on disk. Keep it off any web-served path, restrict filesystem access, and apply whatever retention and deletion policy your jurisdiction requires.

The form also loads Google reCAPTCHA, which shares visitor IP addresses and browser data with Google under Google's privacy policy. Uploaded images are transmitted to Hugging Face and Mistral for processing.

If you deploy this, say so in your own privacy notice.

License

Released under the GNU Affero General Public License v3.0 — see LICENSE.

AGPL-3.0 is network copyleft: if you run a modified version as a network service, you must offer its complete source to users of that service.

Security

Please report vulnerabilities privately — see SECURITY.md.

About

Digital solution designed to convert handwritten text into editable, searchable digital content using modern artificial intelligence and image processing techniques.

Topics

djangodockerhandwritinghandwriting-recognitionimage-processingmachine-learningocrpythonreacttext-extraction

Resources

Readme
AGPL-3.0 license

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Contributors

Languages