AI agents confidently state facts every day. But when the user asks "where did you get that number?", the agent has no answer. Information comes from web pages, memory files, user conversations, tool output, or the model's training data, and the agent never tells you which is which.
Provenance Tracker fixes this. Every factual claim gets an inline citation. Every reply ends with a source list. The user can see exactly which parts of the answer are grounded in evidence versus which are the agent's inference or recall.
- Inline citations — Appends
[1],[2]markers to factual claims in agent replies - Source list — Generates a collapsible source list at the end of each substantive reply
- 12 source types — Tags every source:
[user],[web],[file],[exec],[api],[browser],[observation],[memory],[external],[calculation],[inference],[training] - Hallucination flags — Flags unverified claims that rely on training data only
- Session analysis — Python scripts to analyze session history and generate provenance reports
- Claim verification — Trace where any specific claim came from
User: What backend does sub-tracker use?
Agent: sub-tracker uses Supabase as the backend. The project ID
is atlcfajejgwfhealkztd. It's deployed on Vercel. Tech stack
is React + TypeScript + Tailwind CSS + Recharts.
Problem: the user has no way to tell which facts are verified and which might be outdated or fabricated.
User: What backend does sub-tracker use?
Agent: sub-tracker uses Supabase as the backend [1], with
project ID atlcfajejgwfhealkztd [1]. It's deployed on
Vercel [1]. The tech stack is React + TypeScript + Tailwind
CSS [1] + Recharts [2]. Last updated around April 2026 [1].
---
Sources:
[1] [memory] MEMORY.md, sub-tracker project section
[2] [inference] Recharts inferred from MEMORY.md tech stack
description
Every claim has a numbered source. The user can instantly check:
| Claim | Source | Confidence |
|---|---|---|
| Supabase project ID | MEMORY.md | High (directly verifiable) |
| Tech stack | MEMORY.md | High |
| Recharts usage | Inferred from memory | Medium (may be outdated) |
If the agent fabricates a number from training data, the source tag shows [training], confidence drops to lowest, and the user spots it immediately.
| Tag | Meaning | Verifiability |
|---|---|---|
[user] |
User stated it in conversation | Highest |
[web] |
Fetched live from a URL | High |
[file] |
Read from a file on disk | High |
[exec] |
Shell command output, reproducible | High |
[api] |
External API returned structured data | High |
[browser] |
Observed via browser snapshot/screenshot | Medium-high |
[observation] |
Media analysis (image, PDF, video) | Medium-high |
[memory] |
From workspace memory files | Medium (may be outdated) |
[external] |
Untrusted inbound content (email, chat) | Variable |
[calculation] |
Agent's own computation | Lower (may have errors) |
[inference] |
Agent derived it from other sources | Lower |
[training] |
Model parametric knowledge | Lowest (cannot verify) |
git clone https://github.com/Thomaszhou22/provenance-tracker.git ~/.openclaw/skills/provenance-trackermkdir -p ~/.openclaw/skills/provenance-tracker
cd ~/.openclaw/skills/provenance-tracker
# Copy SKILL.md, scripts/, references/, templates/ into this directoryRestart your gateway and ask your agent a factual question. It should append a Sources section.
openclaw gateway restartProvenance Tracker has three modes. Switch anytime in conversation:
| Mode | How to activate | Behavior |
|---|---|---|
| Off (default) | "provenance off" | Normal replies, no citations |
| Standard | "provenance on" | Citations on factual replies only |
| Research | "provenance research mode" | Full citations on everything, includes provenance logs |
Default is Off so it costs zero tokens until you explicitly turn it on.
User: provenance on
User: What Supabase project does sub-tracker use?
Agent: sub-tracker uses the Supabase project with ref
atlcfajejgwfhealkztd [1].
---
Sources:
[1] [memory] MEMORY.md, sub-tracker project section
Turn off anytime:
User: provenance off
Agent: Got it, citations disabled.
Generate a provenance report from session logs:
python3 scripts/provenance.py \
--session ~/.openclaw/agents/main/sessions/ \
--report markdownJSON format for programmatic use:
python3 scripts/provenance.py \
--session session.jsonl \
--report json \
-o report.jsonTrace where a specific claim came from:
python3 scripts/verify.py \
--claim "Supabase project atlcfajejgwfhealkztd" \
--sessions ~/.openclaw/agents/main/sessions/# Provenance Report
Generated: 2026-07-29T03:40:00
Total sources identified: 23
## Source Summary
| Type | Count |
|----------------|-------|
| user_statement | 5 |
| web | 8 |
| memory | 3 |
| file_read | 4 |
| exec | 3 |
## Quality Flags
- No issues detected. Source distribution looks healthy.
- Skill format: OpenClaw AgentSkill (SKILL.md)
- Analysis scripts: Python 3.8+ (no external dependencies)
- Compatibility: OpenClaw 2026.x+
provenance-tracker/
├── SKILL.md # Skill instructions (loaded by agent)
├── README.md # English documentation (this file)
├── README_CN.md # Chinese documentation
├── LICENSE # MIT
├── scripts/
│ ├── provenance.py # Session log analyzer and report generator
│ └── verify.py # Claim verification tool
├── references/
│ └── source-types.md # Source type specifications
└── templates/
└── provenance-log.md # Daily provenance log template
- Source tracking relies on the agent honestly self-reporting where information came from. The Python scripts verify tool call history but cannot detect claims the agent silently generated from training data.
- The analysis scripts parse OpenClaw session JSONL format. Other agent frameworks may need adapter code.
- Inline citations add tokens per reply (~50-100 in Standard mode, ~100-200 in Research mode). Default is Off, so there is no cost until the user explicitly enables it. Use
provenance offto disable anytime without removing the skill.
MIT