PDF/Image → Markdown Converter (Powered by Baidu Unlimited-OCR)

A Google Colab notebook that converts a PDF or image file into a clean Markdown (.md) file using baidu/Unlimited-OCR.

---

What this notebook does

• Takes a PDF or a single image as input.
• Runs each page through the Unlimited-OCR model (in high-accuracy "gundam" mode, with an automatic memory-safe fallback).
• Cleans up the raw OCR output (removes detection markers, broken image links, and repeated/degenerate text).
• Saves the result as a single .md file, named after the original file.
• Auto-downloads the .md file in Colab.

Supported input file types

Unlimited-OCR is an image-based OCR model, so it only supports:

Type	Extensions
PDF	.pdf
Images	.jpg, .jpeg, .png, .bmp, .tiff, .tif, .webp

It does not natively support Word, PowerPoint, or Excel files (.docx, .pptx, .xlsx). If you need to convert those, export them to PDF first, then feed the PDF into this notebook.

Requirements

• Google Colab with a GPU runtime (Runtime → Change runtime type → T4 GPU or better).
• No local installation needed — everything runs in Colab.

---

First-Time Setup (run once, in order)

Run these cells top to bottom, one at a time, waiting for each to finish before running the next.

Cell	Purpose	Notes
Cell 1	GPU check	Confirms a GPU runtime is attached. If it fails, go to Runtime → Change runtime type → select a GPU, then re-run.
Cell 2	Install dependencies	Installs torch, torchvision, transformers, pymupdf, and pins Pillow==11.3.0 (newer Pillow versions have a broken wheel that breaks torchvision). Takes ~2-3 minutes.
Cell 3	Imports	Imports core Python libraries (os, re, torch, fitz, etc.) and prints your GPU info.
Cell 4	Load model	Downloads and loads the Unlimited-OCR model (~7 GB on first run) and its tokenizer onto the GPU. Takes 5–15 minutes the first time; the model is cached afterward.
Cell 5	Define helper functions	Defines pdf_to_images(), remove_det(), strip_broken_images(), collapse_repetition(), and read_and_combine_outputs(). No output expected other than a success message.
Cell 6	Configuration	Set PDF_PATH to the file you want to convert — a PDF or an image (.pdf, .jpg, .jpeg, .png, .bmp, .tiff, .tif, .webp). This is the only cell you normally need to edit.
Cell 7	Define main processing function	Defines process_document_to_markdown() (aliased as process_pdf_to_markdown()). No file is processed yet — this just defines the function.
Cell 8	Run!	Actually processes the file set in Cell 6 and produces the .md file. This is where the OCR happens — can take a few minutes depending on page count.
Cell 9	Preview + Download	Prints a preview of the generated Markdown and automatically downloads the .md file to your computer.

Once Cell 9 finishes, your first file is done — check your browser's Downloads folder.

---

Converting Another File (after the first run)

You do not need to re-run everything. The GPU, dependencies, model, and function definitions all stay loaded in the Colab session's memory. Just repeat this short cycle:

Step	Cell to run
1	Cell 6 — Update PDF_PATH to the new file's path (upload the file to Colab first via the left sidebar, or mount Google Drive).
2	Cell 8 — Run to process the new file.
3	Cell 9 — Preview and download the new .md file.

That's it — just Cell 6 → Cell 8 → Cell 9, in that order, for every additional file.

When you DO need to re-run everything (Cells 1–9)

• If your Colab runtime disconnects or restarts (session timeout, manual restart, switching runtime type). You'll lose the loaded model and GPU state, so start again from Cell 1.
• If you edit the code in Cell 5 or Cell 7 (e.g., applying a bug fix) — only re-run that specific edited cell, not the whole notebook. The model itself does not need to reload.
• If you change MODEL_NAME in Cell 4 to a different model.

---

Output

For an input file named Report.pdf or Report.png, the output will be Report.md, saved in the same directory and auto-downloaded by Cell 9.

Each page's content is separated by --- in the final Markdown file.

---

Troubleshooting

ImportError: cannot import name '_Ink' from 'PIL._typing' Caused by a broken Pillow==12.1.1 wheel. Cell 2 already pins Pillow==11.3.0 to avoid this. If you still see it, restart the runtime (Runtime → Restart session) and re-run Cell 2 onward.

CUDA out of memory (OOM) The notebook automatically retries a failed page in a lower-memory "base" mode. If OOM still persists frequently:

• Lower DPI in Cell 6 (e.g., 300 → 200).
• Make sure no other notebook is using the GPU.
• Consider Colab Pro for a bigger GPU (A100/L4) if working with very large or dense documents.

Repeated/garbled text on a page (e.g., a phrase repeating dozens of times) This is a known OCR model degeneration issue on dense pages. collapse_repetition() in Cell 5 automatically detects and trims runs of repeated lines, leaving a note like …[N repeated/degenerate lines trimmed]…. This is a heuristic safety net, not a perfect fix.

Broken image links like ![](images/page_0_0.jpg) The notebook does not save extracted images, so these are automatically replaced with a placeholder note: *[Image on this page — not extracted]*.

Unsupported file type error Only .pdf, .jpg, .jpeg, .png, .bmp, .tiff, .tif, and .webp are supported. Convert other formats (Word, PowerPoint, Excel) to PDF first.

---

Credits

• Model: baidu/Unlimited-OCR
• PDF-to-image conversion: PyMuPDF