leaderboard.csv

Provider,Category,Overall,Tables,Charts,Content_Faithfulness,Semantic_Formatting,Visual_Grounding,Cost_Per_Page,Cost_Charts,Cost_Tables,Cost_Text,Cost_Layout,HF_Model_ID
LlamaParse Cost Effective,LlamaParse,71.89,73.16,66.66,88.02,73.04,58.56,0.375,,,,,
LlamaParse Agentic,LlamaParse,84.88,90.74,78.11,89.68,85.24,80.62,1.25,,,,,
OpenAI GPT-5 Mini (Reasoning Minimal),VLM - Proprietary,46.83,69.82,30.13,82.30,45.77,6.15,0.88,0.70,1.22,0.67,0.91,
OpenAI GPT-5 Mini (Reasoning Medium),VLM - Proprietary,51.52,74.60,38.96,85.68,45.35,13.03,3.05,2.93,3.43,2.67,3.15,
Anthropic Haiku 4.5 (Disable Thinking),VLM - Proprietary,45.17,77.21,13.77,78.74,49.39,6.72,1.64,0.97,2.46,1.53,1.60,
Anthropic Haiku 4.5 (Thinking),VLM - Proprietary,53.12,78.70,27.39,84.83,62.36,12.30,3.69,3.49,4.26,3.16,3.83,
Google Gemini 3 Flash (Thinking Minimal),VLM - Proprietary,71.04,89.85,64.83,86.19,58.35,55.97,0.65,0.49,0.84,0.56,0.69,
Google Gemini 3 Flash (Thinking High),VLM - Proprietary,75.05,91.50,64.79,90.87,68.31,59.77,2.41,2.38,2.29,2.25,2.70,
Google Gemini 3.5 Flash (Thinking Medium),VLM - Proprietary,69.92,91.12,44.01,90.19,59.14,65.14,4.28,3.50,5.12,3.57,4.94,
Google Gemini 3.5 Flash (Thinking Minimal),VLM - Proprietary,63.09,86.24,18.74,88.22,68.09,54.18,1.82,1.13,2.53,1.47,2.15,
Anthropic Opus 4.6,VLM - Proprietary,54.07,86.52,13.49,89.70,64.19,16.47,5.78,4.02,8.16,5.19,6.40,
Anthropic Opus 4.7,VLM - Proprietary,63.34,87.17,55.84,90.26,69.42,13.99,7.14,5.69,8.63,6.07,8.17,
Anthropic Opus 4.8,VLM - Proprietary,63.70,89.65,49.75,89.02,71.38,18.69,7.30,5.82,8.78,6.24,8.36,
Anthropic Fable 5,VLM - Proprietary,70.78,89.79,52.21,90.02,72.62,49.24,15.60,13.04,19.36,13.03,16.99,
Anthropic Sonnet 5,VLM - Proprietary,62.13,86.65,60.53,86.51,64.93,12.03,3.18,2.79,3.82,2.60,3.52,
Google Gemini 3.1 Pro,VLM - Proprietary,69.14,91.00,41.13,90.16,52.43,70.99,8.49,6.69,9.69,7.73,10.54,
Google Gemini 3.1 Flash Lite,VLM - Proprietary,58.32,85.48,9.92,89.46,58.38,48.38,0.29,0.19,0.37,0.27,0.31,
OpenAI GPT-5.4 (Reasoning None),VLM - Proprietary,62.23,83.89,65.22,85.57,59.52,16.95,2.90,2.55,4.04,2.13,3.00,
OpenAI GPT-5.5 (Reasoning Medium),VLM - Proprietary,67.76,90.05,65.53,86.81,60.12,36.28,13.09,13.22,13.93,11.30,13.90,
OpenAI GPT-5.5 (Reasoning None),VLM - Proprietary,64.39,89.31,59.11,87.17,64.46,21.90,5.93,4.87,8.12,4.40,6.33,
OpenAI GPT-5.4 Nano,VLM - Proprietary,43.35,60.16,17.57,78.05,54.56,6.41,0.25,0.17,0.35,0.19,0.28,
AWS Textract,Commercial - IDP,47.88,84.58,5.97,74.76,3.71,70.36,1.5,,,,,
Google Cloud Document AI,Commercial - IDP,50.39,55.10,1.44,83.65,50.51,61.26,1,,,,,
Azure Document Intelligence (Layout),Commercial - IDP,59.64,86.00,1.56,84.93,51.93,73.78,1,,,,,
Reducto (Agentic),Commercial - Startup APIs,72.97,80.42,73.4,86.37,57.6,67.07,4.76,6,3.4,4.2,5,
Datalab Accurate,Commercial - Startup APIs,69.95,90.29,62.40,83.87,40.79,72.38,1,,,,,
Datalab Balanced,Commercial - Startup APIs,69.51,89.10,62.05,83.63,39.74,73.05,0.4,,,,,
Datalab Fast,Commercial - Startup APIs,67.79,85.08,57.56,82.76,39.06,74.47,0.4,,,,,
Reducto,Commercial - Startup APIs,67.83,70.33,56.99,86.37,56.75,68.71,2.38,3,1.7,2.1,2.5,
Extend,Commercial - Startup APIs,55.75,85.05,1.59,84.08,47.36,60.67,2.5,,,,,
Extend (2.0),Commercial - Startup APIs,66.99,85.93,39.14,82.11,56.92,70.85,2.5,,,,,
LandingAI,Commercial - Startup APIs,45.23,73.72,10.88,88.60,27.87,25.08,3,,,,,
Firecrawl,Commercial - Startup APIs,31.08,55.88,0,74.37,25.16,0,0.9,,,,,
Pulse,Commercial - Startup APIs,54.3,86.0,1.5,86.5,29.1,77.6,1.5,,,,,
Mistral OCR 4 (Annotation),Commercial - IDP,68.23,73.94,40.11,89.55,66.37,71.17,0.5,,,,,
Mistral OCR 4,Commercial - IDP,60.68,73.94,2.35,89.55,66.37,71.17,0.4,,,,,
Databricks AI Parse,Commercial - IDP,60.68,83.67,0,88.25,55.25,76.23,6.06,,,,,
Databricks AI Parse (batch),Commercial - IDP,60.68,83.93,0,88.3,55.04,76.14,0.2,,,,,
Infinity-Parser2-Pro,VLM - Open Weight,74.28,86.4,61.3,89.7,59.1,74.9,,,,,,infly/Infinity-Parser2-Pro
Infinity-Parser2-Flash,VLM - Open Weight,73.25,82.88,55.56,89.52,57.7,80.61,,,,,,infly/Infinity-Parser2-Flash
Qwen3-VL-8B-Instruct,VLM - Open Weight,61.97,74.61,28.18,87.63,64.23,55.18,,,,,,Qwen/Qwen3-VL-8B-Instruct
Dots.mocr,VLM - Open Weight,55.79,85.15,0.95,90.03,46.99,55.81,,,,,,rednote-hilab/dots.mocr
Falcon-OCR,VLM - Open Weight,53.08,74.70,0.82,78.59,47.91,63.37,,,,,,tiiuae/Falcon-OCR
Docling-models,VLM - Open Weight,50.65,66.41,52.76,66.93,1.03,66.11,,,,,,docling-project/docling-models
Chandra-ocr-2,VLM - Open Weight,70.1,89.2,65.1,83.7,61.4,51.2,,,,,,datalab-to/chandra-ocr-2
PaddleOCR-VL-1.6,VLM - Open Weight,67.43,67.77,54.24,82.71,54.64,77.8,,,,,,PaddlePaddle/PaddleOCR-VL-1.6
PaddleOCR-VL-1.5,VLM - Open Weight,65.95,67.38,47.62,82.72,54.27,77.78,,,,,,PaddlePaddle/PaddleOCR-VL-1.5
Surya OCR 2,VLM - Open Weight,64.83,82.68,21.95,86.57,61.36,71.59,,,,,,datalab-to/surya-ocr-2
Gemma-4-31B-it,VLM - Open Weight,62.4,80.6,15,89.9,69.3,57.4,,,,,,google/gemma-4-31B-it
Gemma-4-26B-A4B-it,VLM - Open Weight,58.5,70,14.2,83.8,65.1,59.2,,,,,,google/gemma-4-26B-A4B-it
LightOnOCR-2-1B,VLM - Open Weight,48,75.5,13.5,87.8,63.2,0,,,,,,lightonai/LightOnOCR-2-1B
Qianfan-OCR,VLM - Open Weight,46.2,72.3,0.9,83.5,53.8,20.6,,,,,,baidu/Qianfan-OCR
MinerU2.5-Pro-2605-1.2B,VLM - Open Weight,72.78,77.59,61.64,87.88,57.49,79.30,,,,,,opendatalab/MinerU2.5-Pro-2605-1.2B
Unlimited-OCR,VLM - Open Weight,46.17,70.21,1.34,86.81,0.97,71.52,,,,,,baidu/Unlimited-OCR
MinerU2.5-2509-1.2B,VLM - Open Weight,45.9,71.7,1.1,80.8,4.5,71.5,,,,,,opendatalab/MinerU2.5-2509-1.2B
Qwen3.6-35B-A3B,VLM - Open Weight,44.1,19.1,5.1,90.7,58.3,47.4,,,,,,Qwen/Qwen3.6-35B-A3B
DeepSeek-OCR-2,VLM - Open Weight,41.2,61.7,1.1,82,54,7,,,,,,deepseek-ai/DeepSeek-OCR-2
PaddleOCR-VL,VLM - Open Weight,40.9,67.1,0.9,82.7,54,0,,,,,,PaddlePaddle/PaddleOCR-VL
Gemma-4-E4B-it,VLM - Open Weight,40.5,25,7.7,81.4,52.5,35.8,,,,,,google/gemma-4-E4B-it
Granite Vision 4.1 4B,VLM - Open Weight,39.45,63.81,47.47,64.43,21.52,0,,,,,,ibm-granite/granite-vision-4.1-4b
Qwen3.5-4B,VLM - Open Weight,35.4,8,2.5,88.9,57.8,19.7,,,,,,Qwen/Qwen3.5-4B
Qwen3.5-9B,VLM - Open Weight,31.9,9.9,5.4,90.5,22,31.8,,,,,,Qwen/Qwen3.5-9B
Qwen3.5-0.8B,VLM - Open Weight,28.4,1.5,0.4,82,43.1,15,,,,,,Qwen/Qwen3.5-0.8B
Qwen3.5-2B,VLM - Open Weight,27.3,0,0.1,87.2,31.1,18.3,,,,,,Qwen/Qwen3.5-2B
GLM-OCR,VLM - Open Weight,29.6,66.1,1.7,78,2.3,0,,,,,,zai-org/GLM-OCR
KDL-Frontier-Parser-nano,VLM - Open Weight,76.36,85.56,63.41,87.19,66.81,78.84,,,,,,KDLAI/KDL-Frontier-Parser-nano
MarkItDown,Open Source - Local,18.63,15.77,2.02,64.54,0.91,9.90,,,,,,
OpenDataLoader,Open Source - Local,29.40,35.18,0.92,66.06,34.07,10.77,,,,,,
PyMuPDF4LLM,Open Source - Local,30.88,36.68,1.58,60.85,44.63,10.68,,,,,,
PyMuPDF (Text),Open Source - Local,16.02,0.00,0.00,68.28,0.95,10.86,,,,,,
PyMuPDF (HTML),Open Source - Local,16.62,0.00,0.00,55.63,18.26,9.20,,,,,,
pypdf,Open Source - Local,14.87,0.00,0.00,62.50,0.91,10.92,,,,,,
pdf-inspector,Open Source - Local,26.59,26.64,5.29,56.06,35.07,9.91,,,,,,
LiteParse (no OCR),Open Source - Local,32.8,40.3,3.4,68.6,44.6,10.7,,,,,,