Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Turn local HTML — or any LLM's extraction output — into schema-valid JSON wh

`local-only` · `deterministic replay` · `no browser` · `no external service`

<img src="./docs/demo.svg" alt="grounded-json rejecting a hallucinated LLM field" width="760">

</div>

Most extractors stop at JSON. That is where review starts.
Expand Down Expand Up @@ -62,10 +64,10 @@ const receipt = ground({
});

receipt.coverage; // { grounded: 2, total: 3, invalid: 1, ratio: 0.67 }
receipt.issues; // [{ code: 'quote-not-found', pointer: '/color', ... }]
receipt.issues; // [{ code: 'quote-not-found', pointer: '/color', ... }]
```

Ask your model to return a quote per field, then let the gate decide. With `strict: true`, the hallucinated value never reaches your database.
Ask your model to return a quote per field, then let the gate decide. A runnable version with a recorded LLM response lives in [`examples/llm`](./examples/llm) (`node examples/llm/run.mjs`). With `strict: true`, the hallucinated value never reaches your database.

## Install

Expand Down
25 changes: 25 additions & 0 deletions docs/demo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 15 additions & 0 deletions examples/llm/llm-response.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"_note": "Recorded structured output from an LLM asked to return {value, quote} per field. 'remote' is hallucinated.",
"title": {
"value": "Senior Backend Engineer",
"quote": "Senior Backend Engineer"
},
"city": { "value": "Karlsruhe", "quote": "Karlsruhe" },
"salaryMin": {
"value": 68000,
"quote": "68,000",
"transforms": ["parse-number"]
},
"remote": { "value": "fully remote", "quote": "fully remote" },
"germanLevel": { "value": "B2", "quote": "B2" }
}
39 changes: 39 additions & 0 deletions examples/llm/run.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// Ground a recorded LLM extraction against its source.
// Run from the repo root after `npm run build:package`:
// node examples/llm/run.mjs
import { readFileSync } from 'node:fs';
import { ground } from 'grounded-json';
import { z } from 'zod';

const dir = new URL('./', import.meta.url);
const source = readFileSync(new URL('source.txt', dir), 'utf8');
const { _note, ...fields } = JSON.parse(
readFileSync(new URL('llm-response.json', dir), 'utf8'),
);

// Prompt pattern: "For every field return { value, quote } where quote is copied verbatim from the source."
const data = Object.fromEntries(
Object.entries(fields).map(([k, f]) => [k, f.value]),
);
const evidence = Object.entries(fields).map(([k, f]) => ({
pointer: `/${k}`,
quote: f.quote,
...(f.transforms && { transforms: f.transforms }),
}));

const receipt = ground({
data,
sources: [{ mediaType: 'text/plain', content: source }],
evidence,
schema: z.object({
title: z.string(),
city: z.string(),
salaryMin: z.number(),
remote: z.string(),
germanLevel: z.string(),
}),
});

console.log('coverage:', receipt.coverage);
for (const issue of receipt.issues)
console.log(`REJECT ${issue.pointer}: ${issue.code}`);
3 changes: 3 additions & 0 deletions examples/llm/source.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Senior Backend Engineer (m/w/d) – Karlsruhe
Full-time, hybrid (3 days on site). Salary range: 68,000–82,000 EUR per year.
Stack: TypeScript, Node.js, PostgreSQL. German B2 required.
Loading