diff --git a/README.md b/README.md index b1acd59..7f26cb2 100644 --- a/README.md +++ b/README.md @@ -1,147 +1,77 @@ -# docket +# Docket -Research assistant over SEC EDGAR filings: retrieval, rerank, embeddings and -grounded generation all live in the **mettle** engine, consumed over its -OpenAI-compatible `/v1` HTTP API; docket owns the UI, the agent, the lineage -and the trust layer. +A self-hosted research assistant for SEC filings. Drop in 10-Ks and 10-Qs, ask questions in plain English, and get answers that cite the exact filing and page they came from. Everything runs on your own machine with a small local model, so no filing or question ever leaves it. -Self-contained. The release wheel ships the dashboard; no Node, no external -services required to start. The only supported engine is **mettle** — any -server that speaks the same OpenAI-compatible `/v1` contract (chat with -per-token logprobs, embeddings, rerank, collections) is a drop-in backend. +![Asking Docket about AMD's 2022 revenue](assets/screenshots/ask.png) -## Install +## What it does -Linux and macOS: +- **Answers with receipts.** Every answer points back to the filing and page it was drawn from, so you can check it in seconds. +- **Works across your whole coverage list.** Ask about one company or compare several; Docket searches every filing you have added. +- **Shows its work.** Each answer keeps a full trace: what was searched, which passages were picked, and the exact prompt the model saw. +- **Runs locally.** Retrieval and generation run on [mettle](https://github.com/adityam23/mettle), a local inference engine that fits on a 6 GB GPU. -```sh -curl -fsSL https://raw.githubusercontent.com/adityam23/docket/main/install.sh -o install.sh -sh install.sh +## How it works + +```mermaid +flowchart LR + A[Filing PDFs] -->|extract, chunk| B[Docket] + B -->|index| C[(mettle engine
search + rerank)] + Q[Your question] --> B + B -->|find best passages| C + C -->|top passages| B + B -->|grounded prompt| D[Local model] + D --> E[Answer with
filing + page citations] ``` -The installer downloads the latest release wheel, installs it as an isolated -[uv tool](https://docs.astral.sh/uv/concepts/tools/), and puts `dk` on your -PATH. State lands in XDG dirs (`~/.local/share/docket`, `~/.config/docket`). +Docket handles the app side: the dashboard, ingesting filings, the question-answering agent and the citations. The [mettle](https://github.com/adityam23/mettle) engine does the heavy lifting: embeddings, search, reranking and the language model itself. -Optional local engine: +## Getting started -```sh -sh install.sh --with-engine /path/to/mettle -``` +You need Python 3.11+, [uv](https://docs.astral.sh/uv/), an NVIDIA GPU with about 6 GB of memory, and a built [mettle](https://github.com/adityam23/mettle) engine. -This copies the binary next to `dk` as `docket-engine` and records it in the -manifest. It is not started or configured for you. +```sh +git clone https://github.com/adityam23/docket && cd docket +uv sync --extra ingest --extra agent -Uninstall: +# Let Docket start and manage the engine for you +export DK_ENGINE_BINARY=/path/to/mettled +export DK_METTLE_CONFIG=/path/to/mettle-config.toml -```sh -curl -fsSL https://raw.githubusercontent.com/adityam23/docket/main/uninstall.sh -o uninstall.sh -sh uninstall.sh # add --data to also delete the ingested index +uv run dk serve ``` -## Desktop app (AppImage / DMG) -Click-to-run builds are produced by CI (`.github/workflows/release-desktop.yml`, -on version tags): a thin Tauri shell spawns the frozen `docket-server` sidecar -on a free port, points it at the bundled `mettled` engine, and opens the OS -webview at the dashboard. Model weights are downloaded by the engine on first -run (mettle's own first-run flow). Artifacts are **unsigned** by design: -macOS shows a one-time right-click → Open warning; Linux AppImages need -`libfuse2` (or run with `--appimage-extract`) and the system -`webkit2gtk-4.1` webview. +Open http://127.0.0.1:8760, go to **filings** and add some PDFs, then ask away on the **ask** page. If you already run mettle yourself, set `DK_BACKEND_URL` to its `/v1` address instead of the two engine variables. + +Settings can also go in a `.env` file in the project folder. -Local Linux build (everything except the engine binary, which you point at via -`DK_ENGINE_BINARY`): +### From the terminal + +These talk to the engine directly, so keep `dk serve` (or your own mettle) running. ```sh -packaging/build-sidecar.sh # PyInstaller sidecar → dist/docket-server/ -cp -a dist/docket-server desktop/src-tauri/resources/server/ -mkdir -p desktop/src-tauri/resources/engine # put (or symlink) a mettled binary here -cd desktop/src-tauri && npx --yes @tauri-apps/cli build +uv run dk ingest ~/filings # add every PDF in a folder +uv run dk ask "What was AMD's revenue in 2022?" +uv run dk docs # list what's indexed ``` -The CLI/uv-tool install below remains the developer path. +## A quick tour -## Quickstart (dev) +**Home:** what's indexed and whether the engine is up. -```sh -git clone https://github.com/adityam23/docket && cd docket -uv sync # base deps only -uv run dk serve # dashboard + API on http://127.0.0.1:8760 -uv run dk chat "hello" # one-shot generation smoke test -uv run pytest # offline suite, no backend needed -``` +![Home](assets/screenshots/home.png) -Extras: `uv sync --extra ingest` (PDF text extraction), `--extra agent` -(LangGraph). The dashboard is a SvelteKit (Svelte 5) SPA: rebuild it with -`cd docket/web/frontend && npm ci && npm run build`; releases -ship it prebuilt. - -## Configuration - -Every knob is a `DK_`-prefixed env var or a line in `.env`. All retrieval -capability is engine-side: the engine serves chat with per-token logprobs, -embeddings, `/v1/rerank` and the collections surface on ONE base URL — there is -no separate embed or rerank endpoint. If the engine is unreachable, docket -features fail loudly (no silent local fallback). - -| Variable | Purpose | -|---|---| -| `DK_BACKEND_URL` | `/v1` base of the mettle engine. Default `http://127.0.0.1:11434/v1` | -| `DK_CHAT_MODEL` | Chat model id (or `role:` resolved from the engine's active profile) | -| `DK_EMBED_MODEL` | Embedding model id for the offline `dk embed` path only; ingest POSTs chunks to the engine's collection, which embeds with its own default model | -| `DK_RERANK_MODEL` | Cross-encoder reranker id. Default `qwen3-reranker-0.6b:latest` | -| `DK_ENGINE_BINARY` | Path to the `mettled` binary; set to let docket supervise (spawn + crash-restart) it | -| `DK_METTLE_CONFIG` | Passed to the engine as `METTLE_CONFIG` when spawning | -| `DK_ENGINE_KEY` | Bearer for the engine-management surface, when the engine enforces auth | -| `DK_ENGINE_COLLECTION` | Collection name filings are ingested into | +**Filings:** add PDFs and watch each one get indexed. -## How it works +![Filings](assets/screenshots/filings.png) -``` -ingest: pdf -> ocr -> chunk -> POST chunks to the engine collection -answer: query -> engine search (dense/keyword/hybrid) -> cross-encoder rerank - -> grounded LLM hops -> cited answer + per-token confidence label -``` +**Observability:** a trace for every answer, step by step. + +![Observability](assets/screenshots/observability.png) + +## Status -docket stores only chunk **lineage** locally (doc ids, pages, sources, filing -metadata) — chunk text and vectors live engine-side. Search, rerank, embed and -chat are engine capabilities reached through the one OpenAI-`/v1` client; the -offline test suite runs the full pipeline against a fake provider. - -## Benchmarks - -FinanceBench, 150 questions, production grounded prompt, k=10 context. The -full-RAG accuracy runs used `embed-gemma` + `qwen3-reranker-0.6b`; the shipped -default embedder is now `qwen3-embedding-0.6b` (higher recall@10 below), and the -full-RAG numbers have not yet been re-measured under it. They were also -measured on the direct grounded-QA path, before the aggregate router was part -of the harness; the harness now runs the production `answer()` path and a -re-run is pending. Two metrics, -never mixed: **answer accuracy** (numeric match within tolerance for metrics -questions, Claude-judged equivalence for free-form ones) and **gold-evidence -recall@k** (token overlap of retrieved chunks vs the gold evidence span). -Reproduction commands: -[`benchmarks/financebench/README.md`](benchmarks/financebench/README.md). - -Answer accuracy: - -| Generator | Oracle evidence | Full RAG @6 | Full RAG @10 (shipped) | -|---|---|---|---| -| Qwen3.5-4B Opus-distill (default chat model) | 80.7% | 68.0% | **82.0%** | -| gemma4:e2b | 55.3% | 40.0% | — | - -Retrieval recall@6 / recall@10 at threshold 0.5 (36,920 chunks): - -| Embedder | no reranker | + bge-reranker-v2-m3 | + qwen3-reranker-0.6b | -|---|---|---|---| -| embed-gemma (768d) | 0.527 / 0.613 | 0.773 / 0.840 | **0.840 / 0.900** | -| bge-m3 (1024d) | 0.513 / 0.600 | 0.747 / 0.807 | 0.793 / 0.847 | -| qwen3-embedding (1024d) | 0.540 / 0.667 | 0.793 / 0.860 | **0.840 / 0.913** | - -Raising the context from 6 to 10 chunks recovers +14 pt of answer accuracy, -closing the gap to the oracle ceiling; at k=10 retrieval is no longer the -distill's bottleneck. The cross-encoder reranker remains the dominant retrieval -lever (+31 pts recall@6 over the fused order). +Docket is an early, actively developed project. Next on the list are a per-answer confidence signal and an interpretability "lab mode" that shows which parts of the model's internals drove an answer. ## License diff --git a/assets/screenshots/ask.png b/assets/screenshots/ask.png new file mode 100644 index 0000000..5aeb6f1 Binary files /dev/null and b/assets/screenshots/ask.png differ diff --git a/assets/screenshots/filings.png b/assets/screenshots/filings.png new file mode 100644 index 0000000..9f4013d Binary files /dev/null and b/assets/screenshots/filings.png differ diff --git a/assets/screenshots/home.png b/assets/screenshots/home.png new file mode 100644 index 0000000..b58fa50 Binary files /dev/null and b/assets/screenshots/home.png differ diff --git a/assets/screenshots/observability.png b/assets/screenshots/observability.png new file mode 100644 index 0000000..d72833c Binary files /dev/null and b/assets/screenshots/observability.png differ diff --git a/desktop/src-tauri/src/main.rs b/desktop/src-tauri/src/main.rs index a9a7f11..e597e5d 100644 --- a/desktop/src-tauri/src/main.rs +++ b/desktop/src-tauri/src/main.rs @@ -250,7 +250,7 @@ fn main() { .parse() .expect("localhost URL"); tauri::WebviewWindowBuilder::new(app, "main", tauri::WebviewUrl::External(url)) - .title("docket") + .title("Docket") .inner_size(1280.0, 860.0) .build()?; Ok(()) diff --git a/desktop/src-tauri/tauri.conf.json b/desktop/src-tauri/tauri.conf.json index 569ffd8..35f3cdd 100644 --- a/desktop/src-tauri/tauri.conf.json +++ b/desktop/src-tauri/tauri.conf.json @@ -1,6 +1,6 @@ { "$schema": "https://schema.tauri.app/config/2", - "productName": "docket", + "productName": "Docket", "version": "0.2.0", "identifier": "app.docket.desktop", "build": { diff --git a/docket/cli.py b/docket/cli.py index f80ffa4..0fc39e6 100644 --- a/docket/cli.py +++ b/docket/cli.py @@ -11,7 +11,7 @@ app = typer.Typer( add_completion=False, pretty_exceptions_enable=False, # clean one-line errors, no rich traceback - help="docket — self-hostable agentic document engine", + help="Docket — a self-hosted research assistant for SEC filings: answers cited to the exact filing page, from a local model.", ) diff --git a/docket/web/app.py b/docket/web/app.py index e344a17..86c62e6 100644 --- a/docket/web/app.py +++ b/docket/web/app.py @@ -60,7 +60,7 @@ _MAX_FILE_BYTES = 50 * 1024 * 1024 _MAX_UPLOAD_BYTES = 200 * 1024 * 1024 -app = FastAPI(title="docket", version="0.2.0") +app = FastAPI(title="Docket", version="0.2.0") # --- Host/Origin protection (DNS rebinding + CSRF) --------------------------- diff --git a/docket/web/frontend/src/app.css b/docket/web/frontend/src/app.css index 4e8bced..7098ded 100644 --- a/docket/web/frontend/src/app.css +++ b/docket/web/frontend/src/app.css @@ -150,6 +150,14 @@ code, .mono { font-family: var(--mono); } } .page-sub { color: var(--text-dim); margin: 0; max-width: 68ch; font-size: var(--fs-normal); } +/* Numbered citation pill — answers, the Explain drawer's source list. */ +.cite-pill { + display: inline-grid; place-items: center; min-width: 19px; height: 19px; padding: 0 4px; + background: var(--accent-soft); color: var(--accent-bright); border-radius: 2px; + font: 600 0.72rem/1 var(--mono); vertical-align: baseline; margin: 0 1px; + cursor: default; +} + /* ALL-CAPS small utility label — the counterpoint to large lowercase display. */ .caps { font-size: 0.72rem; diff --git a/docket/web/frontend/src/app.html b/docket/web/frontend/src/app.html index adc529b..4cd21ba 100644 --- a/docket/web/frontend/src/app.html +++ b/docket/web/frontend/src/app.html @@ -4,8 +4,8 @@ - - docket + + Docket %sveltekit.head% diff --git a/docket/web/frontend/src/lib/components/Answer.svelte b/docket/web/frontend/src/lib/components/Answer.svelte new file mode 100644 index 0000000..12b1ae2 --- /dev/null +++ b/docket/web/frontend/src/lib/components/Answer.svelte @@ -0,0 +1,35 @@ + + + +
{@html renderAnswer(text)}
+ + diff --git a/docket/web/frontend/src/lib/components/ExplainDrawer.svelte b/docket/web/frontend/src/lib/components/ExplainDrawer.svelte index 6ff5f98..fa9bb7a 100644 --- a/docket/web/frontend/src/lib/components/ExplainDrawer.svelte +++ b/docket/web/frontend/src/lib/components/ExplainDrawer.svelte @@ -142,10 +142,6 @@ .source { display: flex; gap: 10px; padding: 9px 0; border-bottom: 1px solid var(--line); } .source:last-child { border-bottom: none; } .quote { color: var(--text-dim); font-size: 0.85rem; margin-top: 3px; } - .cite-pill { - display: inline-grid; place-items: center; min-width: 20px; height: 20px; padding: 0 5px; - background: var(--accent-soft); color: var(--accent-bright); - font: 600 0.74rem/1 var(--mono); flex-shrink: 0; - } + .cite-pill { flex-shrink: 0; } .hits { display: flex; flex-direction: column; gap: 8px; } diff --git a/docket/web/frontend/src/lib/components/TopNav.svelte b/docket/web/frontend/src/lib/components/TopNav.svelte index 872ec39..9e84976 100644 --- a/docket/web/frontend/src/lib/components/TopNav.svelte +++ b/docket/web/frontend/src/lib/components/TopNav.svelte @@ -64,7 +64,7 @@
- agent·assistant + Docket
{:else if t.res} {@const warn = caution(t.res.reliability)} - {@const parts = splitCitations(t.res.answer || '')}
answer
-
- {#each parts as p} - {#if p.t === 'cite'}{p.v}{:else}{p.v}{/if} - {/each} -
+ {#if warn}
@@ -249,13 +230,6 @@ padding-left: 12px; border-left: 3px solid var(--accent); } - .answer { font-size: var(--fs-normal); line-height: 1.7; white-space: pre-wrap; max-width: 78ch; } - .cite-pill { - display: inline-grid; place-items: center; min-width: 19px; height: 19px; padding: 0 4px; - background: var(--accent-soft); color: var(--accent-bright); border-radius: 2px; - font: 600 0.72rem/1 var(--mono); vertical-align: baseline; margin: 0 1px; - cursor: default; - } .glance { display: flex; gap: 9px; align-items: flex-start; margin-top: 14px; padding: 11px 13px; border-radius: var(--radius-sm); font-size: 0.88rem; max-width: 78ch; @@ -295,6 +269,6 @@ .new-chat { padding: 7px 12px; font-size: 0.8rem; } @media (max-width: 720px) { - .user-q, .answer { max-width: 100%; } + .user-q { max-width: 100%; } } diff --git a/docket/web/frontend/src/routes/chat/[id]/+page.svelte b/docket/web/frontend/src/routes/chat/[id]/+page.svelte index c56ab13..e12a54d 100644 --- a/docket/web/frontend/src/routes/chat/[id]/+page.svelte +++ b/docket/web/frontend/src/routes/chat/[id]/+page.svelte @@ -7,6 +7,7 @@ import { page } from '$app/stores'; import { api, errMsg } from '$lib/api.js'; import { when, reliability } from '$lib/format.js'; + import Answer from '$lib/components/Answer.svelte'; import Chip from '$lib/components/Chip.svelte'; import Icon from '$lib/components/Icon.svelte'; import Timeline from '$lib/components/Timeline.svelte'; @@ -55,7 +56,7 @@
{trace.question}
answer
-
{trace.answer}
+
{trace.reliability} @@ -84,6 +85,5 @@ font-size: var(--fs-ml); font-weight: var(--w-semibold); letter-spacing: -0.01em; padding-left: 12px; border-left: 3px solid var(--accent); } - .answer { white-space: pre-wrap; line-height: 1.7; max-width: 78ch; } .meta { margin-top: 20px; } diff --git a/pyproject.toml b/pyproject.toml index 82c7ec4..000777a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "docket" version = "0.0.1" -description = "Local RAG engine for long documents: hybrid retrieval, cross-encoder rerank, grounded generation, evaluated on FinanceBench." +description = "Self-hosted research assistant for SEC filings: answers cited to the exact filing page, from a local model." readme = "README.md" requires-python = ">=3.11" license = { text = "GPL-3.0-only" }