Duct

Extract, chunk, embed, search, and ask — document intelligence in one command.

Duct is an open-source document intelligence pipeline. Point it at a PDF, DOCX, Markdown, image, HTML, or text file (or a whole directory), and it extracts the text, splits it into searchable chunks, and lets you query them instantly — with or without AI embeddings.

npx @docfide/duct index ./contracts/
npx @docfide/duct search "termination clauses"
npx @docfide/duct ask   "What are my obligations?"
npx @docfide/duct serve   # web UI at http://localhost:3456

Quickstart

npm install -g @docfide/duct

duct index ./docs
duct search "payment terms"
duct serve
# → http://localhost:3456

No API keys required. No configuration files. Works offline.

Features

Feature	Description
BM25 Search	Keyword search out of the box — no API keys, fully offline
Vector Search	Semantic search via OpenAI or Gemini embeddings
Hybrid Search	BM25 + Vector blended with Reciprocal Rank Fusion
Re-Ranking	Second-pass term-proximity scoring for precision
HyDE	Query expansion via hypothetical document embeddings
Q&A	Ask questions, get answers with source citations
Agentic Retrieval	Multi-hop search that decomposes complex questions
Watch Mode	Auto-index files as they're added or modified
Schema Extraction	Extract structured fields from documents via LLM
Diff Tracking	Line-level changes between document versions
Export API	Search results in JSON or CSV
Web UI	Tabbed interface for search, ask, upload, and settings
URL Indexing	Index web pages by URL
Table Extraction	Detects pipe and whitespace-separated tables
OCR	Tesseract.js + sharp for scanned PDFs and images
All Formats	PDF, DOCX, Markdown, HTML, plain text, images

Documentation

Guide	Contents
CLI Reference	All commands: index, search, ask, watch, extract, diff, serve
API Reference	REST endpoints for the web server
Library API	Programmatic usage in Node.js/TypeScript
Search	BM25, vector, hybrid, re-ranking, HyDE
Q&A	LLM providers, agentic retrieval, configuration

Quick Examples

# Index and search
duct index ./contracts/
duct search "indemnification clause"
duct search "termination" --search-mode hybrid --rerank

# Ask questions
duct ask "What is the governing law?"
duct ask "Compare all NDAs" --multi

# Watch a directory for changes
duct watch ./inbox --ocr

# Extract structured data
duct extract invoice_date:date:Issue date total:number:Amount --index ./invoices/

# Export results
curl "http://localhost:3456/api/export?q=termination&format=csv"

Environment

Variable	Required For
OPENAI_API_KEY	OpenAI embeddings (text-embedding-3-small / 3-large) and LLM (gpt-4o)
GEMINI_API_KEY	Google Gemini embeddings (text-embedding-004) and LLM (gemini-2.0-flash)
DUCT_AUTH_TOKEN	Server authentication (alternative to --auth-token)

Without any API key, Duct uses BM25 keyword search — still works, just no semantic understanding. For Q&A, Ollama is the default LLM provider and runs entirely locally.

Web UI

duct serve
# → http://localhost:3456

Four tabs:

• Search — search indexed documents with mode badge
• Ask — Q&A with configurable LLM, agentic mode toggle
• Upload — drag-and-drop files or index by URL
• Settings — configure LLM provider, API keys, search mode, chunking

duct serve --port 8080 --persist .duct-data --auth-token my-secret --llm ollama

Library

import { Duct } from '@docfide/duct'

const duct = new Duct({
  chunk: { strategy: 'by-heading', size: 1000 },
  embed: { provider: 'openai' },
  llm: { provider: 'ollama', model: 'llama3.2' },
  search: { mode: 'hybrid', alpha: 0.3, rerank: true },
})

await duct.index('./report.pdf')
const results = await duct.search('termination clause')
const answer = await duct.ask('What are my obligations?')

See the Library API for the full API.

Architecture

file.pdf ──┐
file.docx ─┤  extract() → chunk() → embed() → store() → search() → ask()
file.md ───┤           │          │         │          │
file.html ─┤        text      chunks   vectors    results   answer
file.png ───┤           │          │         │          │
file.txt ──┘        pdfjs-dist sliding  OpenAI    BM25     Ollama
                     mammoth    window  Gemini    hybrid   OpenAI
                     marked     by-               rerank   Gemini
                     cheerio    heading            HyDE
                     sharp +
                     tesseract

Deploy

docker build -t duct .
docker run -d -p 3456:3456 duct

See the Dockerfile for build details. Deploy on Railway, Fly.io, or any VPS.

Development

git clone https://github.com/docfide/duct
cd duct
npm install
npm run dev          # run CLI with tsx
npm test             # run tests
npm run build        # compile TypeScript
npm run typecheck    # type-check without emitting

License

MIT — see LICENSE.

---

Built by Docfide. We build contract software; Duct is our gift to developers who work with documents.