feat(batch-analysis): document grids with triage, verification, cited exports, and grid Q&A by joeyorlando · Pull Request #7299 · archestra-ai/archestra

joeyorlando · 2026-08-16T11:10:24Z

Intended for a dev-image build rather than immediate merge (label deploy-dev-platform-images; image tag = head SHA).

Ask the same questions across many documents and get a grid of answers back. An analysis is a set of columns (questions) run over a set of rows (sources); every cell is one model answer with a verbatim supporting quote from the source — plus the review workflow that makes the grid trustworthy at scale.

What this does

The grid

/batch-analyses: create an analysis (name, agent, columns, visibility) through a stepped wizard; list with search, visibility badges, and edit/delete actions; a detail grid that fills in live as a run progresses. Rows and columns are removable (cell records pruned honestly); any cell opens a detail sheet with the answer, the supporting quote, and a regenerate action.
Row sources: knowledge-base documents, repository files from Knowledge Files (bulk drag-and-drop upload creates the files, private by default, with 409 auto-suffixing), or pasted inline text. Source previews open in a dialog, PDFs included.
Scanned sources: a scanned repository file resolves through Document OCR when the organization has it configured — its transcription is what gets analysed, with the same per-document page caps and metered knowledge:ocr interactions.
Execution: a run dispatches one queue task per row on a dedicated analysis lane (ARCHESTRA_BATCH_ANALYSIS_WORKER_MAX_CONCURRENT); a row fetches and pays for its source once regardless of column count. Source text is capped at ARCHESTRA_BATCH_ANALYSIS_MAX_SOURCE_CHARS with truncation recorded on the cell, never silent.
Access control: a dedicated batchAnalysis RBAC resource (read/create/update/delete/execute) and org/team/personal visibility enforced on every path — read, edit, delete, add rows, run, retry — so an analysis someone cannot see is also one they cannot spend credentials on. Audited create/update with before/after diffs.

The review workflow

Column templates: 8 ready-made column sets for common contract reviews (NDA, services, lease, employment, SPA, credit, settlement, data protection), picked from a dropdown — the wizard replaces the draft columns, the edit dialog appends (existing answers stay attached to their columns).
Per-cell triage flags: a column can opt in to a model-assigned green/grey/yellow/red rating per answer; volunteered flags on non-opted columns are discarded, and opting out clears stored flags.
Human verification: per-cell "Mark verified" with reviewer and timestamp (transactional all-or-nothing bulk endpoint, done-cells-only); regeneration clears the sign-off; audited with a digest that moves even on count-neutral changes.
Filter & sort: text filter across answers, flag chips, "Verified only", and format-aware column sorting (number/date/boolean) with unknowns always last.
Cited exports: CSV/XLSX of the current view (filters + sort), carrying each answer's supporting text, flag, and verification state; formula-injection-neutralized CSV, XLSX with Review + Citations sheets.
Citation click-through: clicking an answer's supporting text opens the source with the quote highlighted.
Ask the grid: one-shot Q&A over the extracted table with reference chips that jump to the cited cells; references are validated against exactly the cells the model was shown, and provider failures relay as upstream-marked 502s.

Testing

Backend: visibility matrix across all action paths, source-resolution ACL tests (kb_document, kb_file incl. scanned-with-OCR via an MSW-mocked wire, inline text), runner/executor/prompt tests, verification/templates/grid-chat route tests (96 in the batch-analysis suites). Frontend: wizard, grid, dialogs, column-key stability, answer-view filter/sort, export neutralization, quote-highlight tests. Verified end to end in Chrome against a 50-document × 12-column run: sort, flag/verified filters, citation → highlighted quote, verification badges, Ask panel with clickable references, templates, exports.

Rebased onto main; the branch's migrations were regenerated as 0422_batch_analysis / 0423_eager_donald_blake after main's 0421_mcp_idle_hibernation.

…ity, indexing, and OCR for scans (archestra-ai#7321) Closes archestra-ai#7201. A repository for documents you upload to Archestra directly — a connector pulls from a source system; here you bring the file yourself. Split out of the combined branch so it can land on its own; batch analysis (archestra-ai#7299) stacks on top of this branch. ## What this does - **Knowledge > Files**: upload PDF / Word / Markdown / CSV / JSON / HTML / text (drag-and-drop batches), organize into flat directories, rename, delete, search. Bytes live in Postgres (`kb_files.data`), capped by `ARCHESTRA_KNOWLEDGE_FILES_MAX_UPLOAD_BYTES` (25 MB default). - **Per-item visibility** — Organization / Teams / Only me — on both files and directories, enforced in the repository *and* carried into retrieval as direct ACL tokens, so sharing a knowledge base never widens who can read a document. - **Indexing**: select files or whole directories → **Add to knowledge base** (existing or created from the selection). Each base gets one internal `file_upload` connector (never user-selectable, hidden from connector management); documents upsert on `(connector_id, source_id)` so re-indexing refreshes rather than duplicates. Chunking goes through the same `chunkAndStoreDocument` path connector syncs use. - **Save from chat**: promote a conversation attachment to the repository — from the attachment chip on a message or multi-selected in the Files panel — choosing name, directory, and visibility, with optional immediate indexing. Attachment/file preview (including PDFs, via a scoped CSP change) works from both chat and the repository. - **Scanned PDFs + Document OCR** (lands on top of archestra-ai#7318): a scanned PDF is *accepted* at upload when the organization has Document OCR configured and transcribed at indexing time through the same page-bounded pipeline connector syncs use (shared per-request page budget, per-document cap, metered `knowledge:ocr` interactions). Without OCR configured, the honest 400 at upload stays. A failed transcription fails that file's indexing with a reason naming the OCR outcome — never a silently empty document. - **Dead path removed**: `extractAndIngestDocuments` — the long-disabled chat auto-ingestion stub archestra-ai#7201 describes — is deleted along with its fire-and-forget call site and test mocks. Explicit promotion with user-chosen visibility is the deliberate answer to "what happens to chat uploads": nothing is silently copied into a knowledge base, and what the user promotes carries an ACL no wider than they chose. ## Testing - Backend: upload/visibility/directory/indexing route tests, promote-from-chat tests, extraction tests, and OCR-conjunction tests where the only fake is the Anthropic wire (MSW): scanned upload accepted → indexed → transcription retrievable; failed transcription → named per-file failure. Knowledge-base suite: 1446 tests green. - Frontend: repository page, dialogs, save-from-chat components, preview-kind tests. 1164 tests green across knowledge/chat suites. - Verified live in Chrome (see the walkthrough notes on the PR thread). Docs: Knowledge Files is a subsection on the Knowledge page (no separate page). Env vars documented in platform-deployment.md + `.env.example`. ---  <a href="https://archestra.ai/contributor-onboard" rel="nofollow noreferrer noopener" target="_blank"> <img alt="Archestra Contributor" src="https://raw.githubusercontent.com/archestra-ai/archestra/main/docs/assets/archestra-contributor-banner.webp"/> </a>

…nner, and UI Restores the batch-analysis half of the former combined branch on top of the knowledge-files base: grid analyses with per-cell citations, row sources (knowledge documents, repository files, inline text), the analysis task-queue lane, RBAC resource, and the /batch-analyses UI. Scanned repository files now resolve through Document OCR when the organization has it configured, so a corpus of scans can be analysed.

Swaps the hand-rolled <table> on the analysis detail page for react-data-grid (MIT, zero dependencies, React 19 peer): rows are virtualized so hundreds of sources stay smooth, the Source column stays frozen under horizontal scroll, columns are resizable, and the grid is arrow-key navigable like a spreadsheet — Enter or click opens the answer's detail sheet. Cell renderers (status, citations, file preview, hover delete) carry over unchanged; the grid's theme variables map onto the shadcn tokens so it follows the app palette and dark mode.

… the route The stylesheet moves from the component to the route file because the tailwind postcss config cannot process CSS imports under vitest. The new tests pin what the swap must preserve: statuses render per cell, an answer cell opens the detail sheet while the source column does not, and an uploaded source's label opens the file preview.

… closing it, and hand focus back to the grid Two keyboard-flow fixes found testing in the browser: the browser's default Enter activation clicked the sheet's just-focused close button (open, then closed 9ms later) — preventDefault stops it; and a controlled sheet has no trigger for Radix to return focus to, so closing it dropped focus on body and killed the grid's arrow-key navigation — onCloseAutoFocus now hands focus back to the active cell.

…urface Running an analysis with a GitHub Copilot codex model failed with 'model … is not accessible via the /chat/completions endpoint': those models are Responses-API-only, and which surface a model needs lives on the synced model row — the executor called createLLMModel without that lookup, so every Copilot model fell back to the chat-completions default. Same resolution createLLMModelForAgent does.

…add in-grid grow affordances Three gaps found using the sheet live: a pasted-text row was a dead label ('is Acme MSA a document?') — every source now carries a type icon and pasted text opens in a preview dialog, like uploaded files already did; removing a row was a single un-confirmed click on a hover-revealed trash can — it now goes through the standard delete confirmation naming the row; and growing the sheet meant reaching for the toolbar — a pinned bottom row offers a quiet 'Add row' and a slim trailing '+' header opens the column editor, spreadsheet-style.

…psis The grid's cells inherit nowrap + text-overflow: ellipsis; the 28px button plus the cell's inline padding and border overflowed the 44px stub column by a pixel, which painted a literal '…' beside the plus. Clip instead of ellipsize and give the stub enough width.

Answer cells copy their answer text; the source column copies the pasted text (or the row label for document-backed rows).

…a model call failed Two findings from a 50-row × 12-column run against claude-sonnet-5: The executor pinned temperature: 0 on every call, and Anthropic's thinking-by-default generations reject any explicit temperature ('temperature may only be set to 1 when thinking is enabled or in adaptive mode') — every row 400ed. The knob is now omitted for models that reject it (Anthropic thinking generations, OpenAI reasoning generations); elsewhere 0 stays for run-to-run comparability. Diagnosing that took longer than it should have: the cell error was a bare 'Model call failed: Bad Request' because the AI SDK falls back to the HTTP statusText when the error body is not in the provider's wire shape. The executor now surfaces the response body's own message alongside it.

Verification: a completed cell can be marked verified (transactional, done-cells-only, scoped through the analysis), stamped with the reviewer and shown with their name; regeneration of any kind clears the mark, and the sign-off survives reviewer deletion (verified_at is the predicate, mirroring the catalog approval columns). Audited through a dedicated registry entry whose snapshot digests the verified set, so even count-neutral changes produce a real diff. Flags: a column can opt in to triage — the same per-row model call then classifies each answer green/yellow/red/grey. Nullish in the row contract so a model omitting the flag never fails the row; volunteered flags on non-opted-in columns are discarded. Dots in the grid, badges in the cell sheet.

…emplate catalog Review follow-ups on verification/flags: the verification UPDATE re-asserts status='done' so a concurrent regeneration can't be signed off (zero rows rolls the batch back); retry refuses before resetting when a run is active, so a refused dispatch no longer wipes a cell's flag and sign-off; a column opting out of triage clears its stored flags and the UI renders flags only for opted-in columns; the audit digest includes verifiedAt so re-verification diffs. Templates: eight predefined column sets (NDA, services, lease, employment, share purchase, credit, settlement, data protection) served by GET /api/batch-analysis-templates — static constants validated against the column schema in tests; applying one only copies columns.

…d sorting Templates: the wizard's columns step offers the predefined sets (picking one replaces the draft, editable afterwards); the edit dialog appends a template's columns with collision-safe keys so existing answers stay attached — templates never replace configured columns there. Filter/sort: the grid's answer columns are sortable with format-aware comparators (numbers through currency noise, dates, yes/no), unknowns pinned last in either direction and the row's own order as tie-breaker so polling refetches don't reshuffle equals. A toolbar filters by substring across labels and answers, by triage flag, and to fully verified rows, showing x-of-N when active.

Export ships the current view (filtered and sorted) as CSV or Excel with provenance intact: every answer column is paired with its supporting text, flagged columns carry the triage rating, and verified answers name the reviewer. CSV is RFC 4180 with leading = + - @ neutralized so model output can never execute as a spreadsheet formula; Excel adds a Citations sheet and loads ExcelJS on demand, never with the page. Citations in the cell sheet are now click-through: for text-ish sources the preview marks and scrolls to the quote (same whitespace-normalized matching the grounding check uses); pasted-text rows highlight in their own dialog; PDF/markdown/html previews show the quote above the viewer with a find-in-page hint, since those renderers are browser-owned and citations carry no page metadata.

Ask a question of the table itself — not the source documents — and get an answer citing the cells it relied on; chips in the panel jump the grid to the cited cell and open its detail sheet. References are validated against exactly the cells serialized into the prompt, so a hallucinated or foreign coordinate never reaches the client, and the serialization stops at a size budget while saying how many rows it dropped. Deliberately unpersisted: the durable artifact is the grid. The agent-model resolution the row executor uses (key guard, multi-surface endpoint lookup, reasoning-model temperature omission) moves into a shared resolveBatchAnalysisModel so the chat path cannot drift from it. Gated by batchAnalysis:read plus chat:read, the standard key-spending gate.

Retry's cell reset now rides inside the run start, after its single-flight check — the check/reset/start sequence had a window where a concurrent run could win and the refused dispatch had already wiped the cell's answer state. Grid-chat marks cells citable only when their row actually fit the serialization budget, so the model cannot get an omitted cell 'validated'. Cell verification applies as two bulk UPDATEs (verify set, unverify set) instead of one round trip per entry, still all-or-nothing behind the done-only predicate. CSV formula neutralization tolerates leading whitespace and control characters, the same way spreadsheets do when deciding to execute a cell. The flag filter matches only columns that currently opt into triage, mirroring how stale flags are hidden everywhere else.

- Relay grid-chat provider failures as upstream-marked 502s using the shared describeModelCallError (moved to batch-analysis/llm.ts) - Tolerate duplicate entries in cell verification (dedupe keep-last) - Un-export internals (buildGridChatSystemPrompt), move CellVerificationError above the model class per module-order rules - Wrap highlight-split text siblings in spans (machine-translation rule) - Restore deleteCellsForRemovedColumns docblock placement

…xport in Batch Analysis

…on-guard regex

… horizontal card strip

…after rebase onto main

…audit) — transitive dep of exceljs

joeyorlando changed the title ~~feat(batch-analysis): run a set of questions across a set of documents~~ feat(knowledge-files, batch-analysis): a document repository, and asking the same questions across many documents Aug 16, 2026

joeyorlando force-pushed the joeyorlando/batch-analysis branch from 9225716 to 4363a58 Compare August 17, 2026 14:09

joeyorlando mentioned this pull request Aug 17, 2026

feat(knowledge): Knowledge Files — a document repository with visibility, indexing, and OCR for scans #7321

Merged

joeyorlando force-pushed the joeyorlando/batch-analysis branch from 4363a58 to 5641c7d Compare August 17, 2026 21:38

joeyorlando changed the title ~~feat(knowledge-files, batch-analysis): a document repository, and asking the same questions across many documents~~ feat(batch-analysis): ask the same questions across many documents Aug 17, 2026

joeyorlando changed the base branch from main to joeyorlando/knowledge-files August 17, 2026 21:39

joeyorlando force-pushed the joeyorlando/batch-analysis branch from 5641c7d to 4139b05 Compare August 17, 2026 22:05

joeyorlando force-pushed the joeyorlando/knowledge-files branch from 4d45d33 to 9bff928 Compare August 17, 2026 22:58

joeyorlando force-pushed the joeyorlando/batch-analysis branch 3 times, most recently from ab3cb9c to 9a657ae Compare August 17, 2026 23:21

Base automatically changed from joeyorlando/knowledge-files to main August 18, 2026 00:02

joeyorlando force-pushed the joeyorlando/batch-analysis branch from 9a657ae to becbb65 Compare August 18, 2026 00:15

joeyorlando added deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) and removed deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) labels Aug 18, 2026

joeyorlando added 13 commits August 18, 2026 13:24

chore(batch-analysis): sort the grid component's imports for biome

753ec67

feat(batch-analysis): Ctrl+C copies the focused cell, spreadsheet-style

ca31082

Answer cells copy their answer text; the source column copies the pasted text (or the row label for document-backed rows).

joeyorlando added 8 commits August 18, 2026 13:25

docs: templates, triage flags, verification, grid Q&A, filter/sort, e…

19e15b0

…xport in Batch Analysis

style(batch-analysis): biome formatting + suppression for the injecti…

7fcdad2

…on-guard regex

fix(batch-analysis): make the template picker a dropdown instead of a…

ab86afe

… horizontal card strip

chore: regenerate api client and restore migration-linter annotation …

88fff66

…after rebase onto main

joeyorlando force-pushed the joeyorlando/batch-analysis branch from 379c4ef to 88fff66 Compare August 18, 2026 17:27

joeyorlando changed the title ~~feat(batch-analysis): ask the same questions across many documents~~ feat(batch-analysis): document grids with triage, verification, cited exports, and grid Q&A Aug 18, 2026

joeyorlando added deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) and removed deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) labels Aug 18, 2026

style(batch-analysis): sort model imports for biome

ce5a822

joeyorlando added deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) and removed deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) labels Aug 18, 2026

chore(licenses): verify buffers@0.1.1 (MIT/X11, via Debian copyright …

d82e38b

…audit) — transitive dep of exceljs

joeyorlando added deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) and removed deploy-dev-platform-images add label to deploy docker images on demand on a PR (remove and re-add the label to retrigger) labels Aug 18, 2026

joeyorlando linked an issue Aug 23, 2026 that may be closed by this pull request

Knowledge: bulk document review — document sets and review tables #7200

Open

Navigation Menu

Uh oh!

feat(batch-analysis): document grids with triage, verification, cited exports, and grid Q&A - #7299

feat(batch-analysis): document grids with triage, verification, cited exports, and grid Q&A#7299
joeyorlando wants to merge 23 commits into
mainfrom
joeyorlando/batch-analysis

joeyorlando commented Aug 16, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

joeyorlando commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this does

The grid

The review workflow

Testing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

joeyorlando commented Aug 16, 2026 •

edited

Loading