Add doceval to Toolkits

[dave8172]

Adds doceval to the Toolkits section.

doceval is an open-source eval harness for LLM-based document extraction pipelines. Point it at any extractor function (Claude, GPT, regex, rules) and a labeled dataset — it returns field-level accuracy, a failure-mode taxonomy (missed field / wrong value / wrong format), and optional per-document cost tracking.

Disclosure: I'm the author.

https://github.com/dave8172/doceval

[entropy2333]

Thanks for your contribution! doceval tackles a real pain point, and I appreciate your transparency about being the author.

After reviewing, I feel the current PR isn't yet ready for merging into the repo. My main concerns:

• The code is quite minimal and lacks extensibility (e.g., uniform data adapter, robust error handling, logging).
• Documentation and examples are sparse – a quick-start guide with a complete end-to-end run, input/output specs, and config explanation would greatly help users.
• No tests or sample datasets are included, making it hard to assess reliability.

This is not a rejection of the idea – I think it has potential. Could you please:

• Add a full worked example with sample data;
• Expand the docs with setup, usage, and common pitfalls;
• Include basic unit tests.

Once these are addressed, I'd be happy to re-review. Looking forward to your updated PR!

[dave8172]

Thanks for the detailed review — pushed updates addressing all three points:

• Tests: added coverage for the eval orchestration (harness.py), CLI, and report generation — 53 tests total, covering the happy path, extractor failures, cost tracking, unpaired docs/labels, and the failure-mode taxonomy. Fixture-based fake extractors, no API key needed to run the suite.
• Docs: restructured the README so the working example (20 real invoice PDFs + Claude extractor) leads, instead of sitting after the "bring your own extractor" section. Added a full CLI flag reference and an output/JSON schema reference.
• Extensibility: added CSV/JSONL label manifests as an alternative to one-JSON-file-per-document (matters once you're past a handful of docs), plus stdlib logging for unpaired docs/labels and extractor failures.

Repo: https://github.com/dave8172/doceval (v0.2.0). Happy to address anything else — appreciate you taking the second look.