Feature: Structured Table and Form Extraction Review Workspace

[hoangsonww]

Summary

Add a structured extraction workspace for tables, forms, and key-value fields so DocuThinker can turn semi-structured PDFs and documents into reviewable, exportable data.

Problem / Opportunity

Current document analysis focuses on summarization, chat, recommendations, analytics, and RAG. Many real document workflows also need structured outputs: invoice lines, contract metadata, form answers, tabular findings, and repeated sections. Without a reviewable extraction layer, users must manually transfer AI results into spreadsheets or downstream systems.

Proposed Feature

Create an extraction workflow where users choose or define a schema, run AI-assisted extraction against a document, review fields with confidence and source references, correct values, and export the approved result as CSV/JSON. The feature should support both table-like rows and named fields.

Scope

• Backend extraction endpoint that accepts document id/text plus a schema definition for fields, types, required flags, and optional row groups.
• AI/ML or orchestrator prompt path that returns validated structured JSON with confidence and source snippets/page references when available.
• Frontend review workspace for editing extracted values, marking fields as approved/rejected, and exporting approved results.
• Schema presets for common document types such as invoices, contracts, research papers, or meeting notes.
• Persistence for extraction runs, corrections, approval status, and export history.
• Documentation and examples for defining custom extraction schemas.

Acceptance Criteria

• Users can start an extraction run from an uploaded document using a preset or custom schema.
• Extraction output is validated against the requested schema before it is shown to the user.
• The review UI supports editing scalar fields and table rows without losing confidence/source metadata.
• Approved results can be exported as JSON and CSV with stable column names.
• Extraction run history is available per document, including status, schema name, timestamps, and reviewer state.
• Tests cover schema validation, malformed AI output handling, review persistence, and export formatting.

Non-Goals

• Guaranteed perfect extraction accuracy across all document layouts.
• Training a custom OCR or layout model in this issue.
• Replacing the existing summary/chat/report workflows.

Dependencies / Risks

• Needs strict JSON schema validation because AI-generated structured output can be malformed.
• Source references may depend on the quality of current text extraction and page mapping.
• CSV export must handle nested tables, missing values, and repeated row groups predictably.

Open Questions

• Which schema presets should ship first?
• Should corrections be fed into the future evaluation/golden-dataset workflow?
• Should exports be stored permanently or generated on demand from approved extraction runs?

[zeweihan]

Great feature request — this would be really useful for legal and compliance workflows where structured extraction from contracts and invoices is essential.

A few thoughts from experience building document AI tools:

1. Schema presets: For v1, I'd suggest shipping invoice and contract metadata schemas first — they have the most standardized field sets (parties, dates, amounts, clauses) and are the most requested in enterprise document workflows.

2. Confidence scoring + source refs: The approach of showing confidence scores with page/line references is spot-on. In our work on AI Workdeck (an open-source AI workspace for legal teams), we've found that letting users jump directly to the source location in the document significantly speeds up review and correction cycles.

3. JSON schema validation: Strongly recommend using something like jsonschema with strict mode and additionalProperties: false — AI-generated structured output can drift in field names, especially with complex nested schemas. Having a hard validation gate before showing results to the user saves a lot of downstream debugging.

4. Export format: For CSV export with nested/row-group data, consider flattening with dot-notation keys (e.g., line_items.0.description, line_items.0.amount) — this tends to be the most interoperable format for downstream tools.

Looking forward to seeing this land!