Human-in-the-Loop OCR for Degraded Ge'ez Manuscripts

An independent digital-preservation research project studying how OCR, multimodal models, error detection, and human review can produce trustworthy transcriptions of low-resource Ge'ez texts.

Why this project exists

Many Ge'ez liturgical works remain available only as printed books, scans, or manuscript images. Even when an AI-generated transcription looks fluent, visually similar Ethiopic characters can be confused in ways that change words or references.

The goal is not to claim perfect automatic transcription. The goal is to determine:

• what accuracy current systems actually achieve;
• which errors recur;
• which automated interventions help or hurt; and
• how a human reviewer can verify the remaining uncertain text efficiently.

Research question

What accuracy can current OCR and multimodal systems achieve on degraded Ge'ez microfilm, and which combination of preprocessing, error detection, and human review most efficiently produces a trustworthy transcription?

Study snapshot

Item	Result
Source	Dersana Mikael, vHMML COLL 00155
Surrogate	129 black-and-white microfilm images
Production output	462 columns and 6,839 transcribed lines
Human-verified benchmark	10 columns and 1,786 characters
Measured baseline	2.58% character error rate on the verified sample
Error-detection result	79.5% recall while flagging 22.3% of text, using three complementary detectors
Human-review alignment	Image strips available for 6,064 of 6,829 review lines (89%)
Experimental model cost	Approximately $14

The 2.58% result applies only to the 1,786-character verified sample. It is not a claim of full-manuscript accuracy.

System pipeline

registered source access
        |
        v
native-resolution page segmentation
        |
        v
column-level OCR
        |
        v
lexical / visual / consistency error flags
        |
        v
per-line human review against the manuscript ink
        |
        v
locally saved corrections and verified-text export
        |
        v
future Ge'ez Reader integration

Method

1. Segment each manuscript opening into native-resolution column crops.
2. Compare Tesseract, Claude, and Gemini outputs against manually constructed ground truth.
3. Normalize spacing for character-error-rate evaluation while preserving the original text separately.
4. Test alternative preprocessing, crop sizes, repeated reads, second-reader models, lexical review, proofreading, and corpus-vocabulary flags.
5. Preserve raw outputs, prompts, costs, malformed-response recovery flags, and per-column review data.
6. Route likely errors to a local proofreading interface instead of silently rewriting text.

Principal findings

• Native-resolution column crops outperformed whole-page and enlarged inputs.
• Seven attempted machine improvements were worse than or statistically flat with the baseline.
• The machine layer appears to reach an accuracy ceiling around 2.5% CER on this degraded material.
• A model was better at identifying a suspicious substring than at reproducing the full corrected line.
• Narrow suggested replacements were applicable to the source line 87.5% of the time.
• Three error detectors were complementary: each recalled about 41% of verified errors alone, while their union reached 79.5% recall.
• The result changed the project direction from automatic correction toward prioritized human verification.

Negative results are retained because they define what not to do and make the evaluation reproducible.

Human-review application

The local review interface:

• displays a crop of the actual manuscript line directly above editable text;
• ranks columns by salvage status and error flags;
• distinguishes actionable substring replacements from advisory warnings;
• saves corrections locally; and
• exports reviewed text for downstream reading tools.

The interface is designed to reduce eye movement between a page image and a separate transcription block.

Responsible research and data rights

The manuscript record is available through the vHMML Reading Room. The microfilm images are governed by vHMML's terms of use and are not included in this public repository.

This public case study should contain only:

• source code that contains no credentials;
• aggregate metrics and error categories;
• prompts with keys and private identifiers removed;
• synthetic, author-created, or separately permitted demonstration data; and
• screenshots that contain no restricted manuscript imagery or participant information.

The work is independent and has received informal domain-expert feedback. It should not be described as formally supervised university research unless a faculty mentor explicitly agrees to that relationship.

Public repository contents

The public case-study repository currently includes data-independent evaluation code and synthetic unit tests:

README.md
.github/workflows/test.yml
scripts/
  __init__.py
  metrics.py
  normalize_geez.py
tests/
  test_metrics.py
  test_normalize_geez.py

Run the tests from the repository root with:

python3 -m unittest discover -s tests -v

Future public additions can include methodology and results documents, a synthetic review demo, and independently permitted sample data.

The private manuscript images, raw authenticated downloads, API responses containing sensitive metadata, and unreviewed full transcription should remain outside the public repository.

Next phase

• Expand human verification from 10 to at least 25-30 representative columns.
• Compare the native-resolution baseline with the public Transkribus Ethiopic manuscript model.
• Obtain written guidance from HMML before sharing or integrating manuscript imagery.
• Ask a Ge'ez scholar to review the error taxonomy.
• Ask a computer-science faculty mentor to review the benchmark design and statistical claims.
• Publish a sanitized, reproducible technical report.

Researcher

Nahom Nigatu - Computer Science student at Kennesaw State University

GitHub: nahom8423