Does this paper exist?
Pure registry lookup. DOI, PMID, arXiv, NCT vs CrossRef, PubMed, OpenAlex, arXiv, ClinicalTrials.gov. No API key, no LLM, no hallucination of its own.
-npx evidentia check · claude mcp add evidentia -- npx -y evidentia-mcp
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index 14029b8..4fbfaba 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -3,7 +3,7 @@
"name": "evidentia",
"displayName": "Evidentia — Medical Evidence & Citation Audit",
"version": "1.1.0",
- "description": "Pre-publication evidence audit for medical and health content. Catches AI-fabricated DOIs, PMIDs, arXiv IDs, and trial registrations, then grades any health claim with a 15-criteria, physician-built rubric (A–F report).",
+ "description": "Pre-publication evidence audit for medical and health content. Catches AI-fabricated DOIs, PMIDs, arXiv IDs, and trial registrations; fills a claim ledger; then grades with a 15-criteria rubric and an adversarial KILL gate (A–F report).",
"author": {
"name": "Ken Okamoto, MD"
},
diff --git a/README.ja.md b/README.ja.md
index 8a6fa3d..f0c43ed 100644
--- a/README.ja.md
+++ b/README.ja.md
@@ -4,7 +4,7 @@
### AIが捏造した医学引用を、公開前に捕まえる。
-Evidentia は医学文章中のすべての引用を **CrossRef・PubMed・OpenAlex・arXiv・ClinicalTrials.gov** に照合し、4段階で分類します。さらにエージェントスキルでは、15項目のエビデンス評価を重ねて A〜F のレポートを生成します。作者は小児科専門医。
+Evidentia は医学文章中のすべての引用を **CrossRef・PubMed・OpenAlex・arXiv・ClinicalTrials.gov** に照合し、4段階で分類します。エージェントスキルはまず **主張台帳** を作り、エンジン / 意味 / 敵対 / 修正の4つのループを回してから A〜F を出します。作者は小児科専門医。
**サイト:** [https://kgraph57.github.io/evidentia/](https://kgraph57.github.io/evidentia/)
@@ -36,7 +36,7 @@ Evidentia は医学文章中のすべての引用を **CrossRef・PubMed・OpenA
npx evidentia check your-article.md
```
-**エージェントスキルとして**(15項目の完全評価。`SKILL.md` が `skills/medical-fact-check/SKILL.md` にあるので動きます):
+**エージェントスキルとして**(主張台帳 + 4つの検証ループ。`SKILL.md` が `skills/medical-fact-check/SKILL.md` にあるので動きます):
```bash
npx skills add kgraph57/evidentia
@@ -72,7 +72,7 @@ Evidentia: 4 citations — 1 verified, 1 mismatch, 2 hallucinated (75.0% fabrica
[MIS] doi:10.1002/14651858.cd012734 — 論文は実在するが、メタデータ(年)が不一致
```
-実在は1件。1件はDOIをでっち上げ。1件のPMIDは無関係な論文を指し、1件は年が誤り。**人間のレビュアーなら4件すべてを手で確認する必要があります。** Evidentia は数秒で終えました。
+実在は1件。1件はDOIをでっち上げ。1件のPMIDは無関係な論文を指し、1件は年が誤り。**人間のレビュアーなら4件すべてを手で確認する必要があります。** Evidentia は数秒で終えました。[エンジンの全文レポート](examples/reports/ai-generated-answer.report.md)。同じファイルをスキルのパイプライン(主張台帳 → 意味チェック → 敵対レビューで **KILL**)で歩いたものが [examples/case-studies/vitamin-d-adversarial.md](examples/case-studies/vitamin-d-adversarial.md) です。
## 4段階分類
@@ -91,36 +91,42 @@ Evidentia は「機械が完璧にできる部分」と「判断を要する部
**1. エンジン(CLI + MCPサーバー)** — 純粋に決定論的な引用検証。APIキー不要、LLM不要、それ自身がハルシネーションを起こすこともありません。「この引用論文は本当に実在し、識別子はそれを指しているか?」という1点を確実に答えます。ターミナル、CI、または任意のエージェントの MCP ツールとして使えます。
-**2. スキル(Claude Code)** — エンジンを **15項目の批判的評価ルーブリック** で包みます。エビデンスレベル、統計解釈(相対リスク vs 絶対リスク、NNT)、因果と相関、利益相反、誇張、対象集団の適合、倫理など — **A〜F のレポート** と具体的な修正案を生成。これはエンジン単独ではできない Tier 2「正しく使われているか」の層です。
+**2. スキル(Claude Code)** — エンジンを **主張台帳** と4つの名前付きループで包みます。エンジン、意味の誠実さ、敵対レビュー、修正(上限3回)。15項目のルーブリックはそのパイプラインの *中の一工程* であって、商品そのものではありません。判定は **KILL / MAJOR / MINOR / PASS**。KILL または MAJOR は A では出せません。エンジンだけではできない「正しく使われているか」の層です。
どちらも単独で使えます。組み合わせれば、引用の *存在*(決定論的)と引用の *誠実さ*(評価)の両方をカバーします。
-## MCPツールとして使う
+## スキルはどう検証するか
-任意のエージェントに引用検証能力を与えます:
+エンジンは「実在するか」に答える。スキルは「誠実に使っているか」に答える。最初に15個の箱を埋める作業ではない。
-```bash
-claude mcp add evidentia -- npx -y evidentia-mcp
-```
-
-JSON出力には、人間向けの4段階判定に加えて `lookupVerified` と `resolverOutcomes` が含まれます。どのレジストリに何を照合し、`matched / unmatched / unreachable / skipped` のどれだったかを機械的に追跡できます。
-
-## CIで使う
+**まず主張台帳。** 照合の前に、検証できる主張とその引用を全部抜き出す。見出しも主張のうち。書式は [`templates/claim-ledger.md`](skills/medical-fact-check/templates/claim-ledger.md)。
-捏造引用を混入させたプルリクエストをブロックします:
+**名前の付いたループは4つ**(それぞれ打ち切り条件あり。詳細は [`skills/medical-fact-check/references/verification-workflow.md`](skills/medical-fact-check/references/verification-workflow.md)):
-```yaml
-- run: npx evidentia check content/**/*.md --fail-on-fabrication
-```
+1. **エンジン** — `evidentia check`。届かなければ1回だけ再試行。それでもダメなら `unresolved`。推測のハルシネーションは付けない。存在についてのエンジン出力が正本。T4 を「たぶん実在」に書き換えない。
+2. **意味** — **Tier 1** の引用だけ、抄録を1回余分に取る。主張は主要アウトカム・対象集団・効果の向きと合うか。T3 や T4 をこのループで格上げしない。
+3. **敵対** — 5つのレンズと10行のチェックリスト。判定は **KILL / MAJOR / MINOR / PASS**。最大3パス。直したら採点からではなくエンジンから入り直す。
+4. **修正** — 著者が直したら、エンジン → 意味 → 敵対を再実行。上限3回。その後は止まって、残件を書く。
-`--fail-on-fabrication` は、いずれかの引用が不一致・ハルシネーションなら非ゼロ終了します。
+**KILL または MAJOR は A では出せない。** **KILL**(実在として出した T4、または従うと害が出る助言)は総合 ≤ D。捏造なら **F**。それが故障ではなく、仕組みが動いているということ。
-## 15項目の評価スキル(Claude Code)
+実例: [ビタミンD — 主張台帳 → エンジン → 意味 → KILL](examples/case-studies/vitamin-d-adversarial.md)。収録してある引用4件のデモを、スキルのパイプラインで歩いたもの。
-スキルとして呼び出すと、Evidentia は医学コンテンツを15の観点で評価し、メディアの種類(研究論文・ニュース記事・SNS投稿・患者向けリーフレット・学会スライド・診療ガイドライン・製薬マーケティング・AI生成テキスト)に応じて評価軸を調整します。
+```mermaid
+flowchart TD
+ A[原稿を取る] --> B[主張台帳を作る]
+ B --> C[エンジンを回す]
+ C --> D[意味の誠実さ]
+ D --> E[15項目評価]
+ E --> F[敵対レビュー]
+ F --> G[スコアとレポート]
+ G --> H{直す?}
+ H -->|はい・最大3回| C
+ H -->|いいえ / 上限| I[止める]
+```
15の評価項目
+15の評価項目(パイプラインの中の一工程であって、商品ではない)
1. エビデンスレベルと研究デザイン
2. 引用・出典の正確さ *(上記エンジンが担当)*
@@ -138,9 +144,29 @@ JSON出力には、人間向けの4段階判定に加えて `lookupVerified` と
14. 臨床的妥当性
15. 情報の完全性
-各項目を **Excellent / Good / Fair / Poor** で評価し、総合 **A〜F** スコアと **公衆衛生リスクレベル**(LOW / MEDIUM / HIGH)に集約します。詳細は [`skills/medical-fact-check/SKILL.md`](skills/medical-fact-check/SKILL.md)。
+各項目を **Excellent / Good / Fair / Poor** で評価し、総合 **A〜F** と **公衆衛生リスク**(LOW / MEDIUM / HIGH)にまとめたうえで、敵対判定でゲートする。詳細は [`skills/medical-fact-check/SKILL.md`](skills/medical-fact-check/SKILL.md)。
@@ -42,7 +42,7 @@ Homepage: [https://kgraph57.github.io/evidentia/](https://kgraph57.github.io/evi
npx evidentia check your-article.md
```
-**As an agent skill** (full 15-criteria appraisal; works because `SKILL.md` is at `skills/medical-fact-check/SKILL.md`):
+**As an agent skill** (claim ledger + four verification loops; works because `SKILL.md` is at `skills/medical-fact-check/SKILL.md`):
```bash
npx skills add kgraph57/evidentia
@@ -79,7 +79,7 @@ Evidentia: 4 citations — 1 verified, 1 mismatch, 2 hallucinated (75.0% fabrica
[MIS] doi:10.1002/14651858.cd012734 — Paper exists, but cited metadata disagrees with the record (year).
```
-One citation was real. One DOI was invented. One PMID pointed to an unrelated paper. One had the wrong year. **A human reviewer would have to check all four by hand.** Evidentia did it in seconds. See the [full report](examples/reports/ai-generated-answer.report.md).
+One citation was real. One DOI was invented. One PMID pointed to an unrelated paper. One had the wrong year. **A human reviewer would have to check all four by hand.** Evidentia did it in seconds. See the [full engine report](examples/reports/ai-generated-answer.report.md). The skill pipeline on this same file — claim ledger → semantic → adversarial **KILL** — is walked in [examples/case-studies/vitamin-d-adversarial.md](examples/case-studies/vitamin-d-adversarial.md).
> This is a deliberately tough example. Most carefully written articles score far lower — Evidentia's value is catching the handful that slip through, every time, without fatigue.
@@ -100,36 +100,42 @@ Evidentia is deliberately split into a part a computer can do perfectly and a pa
**1. The engine (CLI + MCP server)** — pure, deterministic citation verification. No API key, no LLM, no hallucination of its own. It answers one question with certainty: *does this cited paper actually exist, and does the identifier point to it?* Use it in a terminal, in CI, or as an MCP tool inside any agent.
-**2. The skill (Claude Code)** — wraps the engine in a full **15-criteria critical-appraisal rubric**: evidence level, statistical interpretation (relative vs. absolute risk, NNT), causation vs. correlation, conflicts of interest, exaggeration, population fit, ethics, and more — producing an **A–F report** with concrete fixes. This is the Tier-2 "is it used correctly?" layer the engine can't do alone.
+**2. The skill (Claude Code)** — wraps the engine in a **claim ledger** and four named loops: engine, semantic honesty, adversarial red-team, correction (cap 3). The 15-criteria rubric is one pass *inside* that pipeline, not the product. Verdicts are **KILL / MAJOR / MINOR / PASS**. KILL or MAJOR cannot ship as A. This is the "is it used honestly?" layer the engine can't do alone.
You can use either on its own. Together they cover citation *existence* (deterministic) and citation *honesty* (appraisal).
-## Use it as an MCP tool
+## How the skill verifies
-Give any agent the ability to verify citations:
+The engine answers existence. The skill answers honesty. It does not start by scoring 15 boxes.
-```bash
-claude mcp add evidentia -- npx -y evidentia-mcp
-```
-
-The server exposes one tool, `verify_citations(text)`, returning the tiered report as Markdown or JSON. JSON responses include both the public 4-tier verdict and a machine-readable `lookupVerified` / `resolverOutcomes` trace for agents and CI.
-
-## Use it in CI
+**Claim ledger first.** Extract every testable claim and its attached citation *before* any lookup. Headlines count. See [`templates/claim-ledger.md`](skills/medical-fact-check/templates/claim-ledger.md).
-Block a pull request that introduces a fabricated citation. Drop [`.github/workflows/evidentia.yml`](examples/ci/evidentia.yml) into any medical-content repo:
+**Four named loops** (each has a stop condition — full rules in [`skills/medical-fact-check/references/verification-workflow.md`](skills/medical-fact-check/references/verification-workflow.md)):
-```yaml
-- run: npx evidentia check content/**/*.md --fail-on-fabrication
-```
+1. **Engine** — `evidentia check`. Retry once if unreachable; then mark `unresolved`, never a guessed Hallucination. Engine output is ground truth for existence. Never override T4 to "probably real."
+2. **Semantic** — one extra abstract lookup per **Tier 1** cite. Does the claim match primary outcome, population, direction of effect? Do **not** use this loop to upgrade a T3 or T4.
+3. **Adversarial** — five lenses + a 10-line checklist. Verdict: **KILL / MAJOR / MINOR / PASS**. Max 3 passes. Re-enter from the engine, not from scoring.
+4. **Correction** — if the author revises, re-run engine → semantic → adversarial. Cap 3, then stop and report what is still open.
-`--fail-on-fabrication` exits non-zero if any citation is a mismatch or hallucination.
+**KILL or MAJOR cannot ship as A.** A **KILL** (any T4 presented as real, or advice that could cause harm if followed) forces overall score ≤ D — **F** if fabrication. That is the system working, not a failure of the tool.
-## The 15-criteria skill (Claude Code)
+Worked example: [vitamin D — claim ledger → engine → semantic → KILL](examples/case-studies/vitamin-d-adversarial.md) on the committed four-citation demo.
-When invoked as a skill, Evidentia evaluates medical content across 15 dimensions and adapts to the media type — research paper, news article, social post, patient leaflet, conference slide, guideline, pharma marketing, or AI-generated text.
+```mermaid
+flowchart TD
+ A[Acquire content] --> B[Extract claim ledger]
+ B --> C[Run evidentia engine]
+ C --> D[Semantic honesty check]
+ D --> E[15-criteria appraisal]
+ E --> F[Adversarial red-team]
+ F --> G[Score plus report]
+ G --> H{User revises?}
+ H -->|yes: max 3| C
+ H -->|no or cap| I[Stop]
+```
The 15 criteria
+The 15 criteria (one pass inside the pipeline, not the product)
1. Evidence level & study design
2. Citation & source accuracy *(powered by the engine above)*
@@ -147,9 +153,29 @@ When invoked as a skill, Evidentia evaluates medical content across 15 dimension
14. Clinical relevance
15. Information completeness
-Each item is rated **Excellent / Good / Fair / Poor**, aggregated into an overall **A–F** score with a **public-health risk level** (LOW / MEDIUM / HIGH). See [`skills/medical-fact-check/SKILL.md`](skills/medical-fact-check/SKILL.md).
+Each item is rated **Excellent / Good / Fair / Poor**, aggregated into an overall **A–F** score with a **public-health risk level** (LOW / MEDIUM / HIGH) — then gated by the adversarial verdict. See [`skills/medical-fact-check/SKILL.md`](skills/medical-fact-check/SKILL.md).
Pure registry lookup. DOI, PMID, arXiv, NCT vs CrossRef, PubMed, OpenAlex, arXiv, ClinicalTrials.gov. No API key, no LLM, no hallucination of its own.
-npx evidentia check · claude mcp add evidentia -- npx -y evidentia-mcp
npx evidentia check · claude mcp add evidentia -- npx -y evidentia-mcp
Wraps the engine in a 15-criteria appraisal: evidence level, statistics, causation vs correlation, conflicts, exaggeration, population fit, ethics — an A–F report with concrete fixes. This is the Tier 2 layer the engine cannot do alone.
+Claim ledger, then four named loops: engine, semantic, adversarial, correction. The 15-criteria rubric is one pass inside that pipeline. KILL or MAJOR cannot ship as A.
npx skills add kgraph57/evidentia
The 15-criteria rubric is one pass inside the pipeline, not the product. Existence is the engine. Honesty is the skill. A fabricated DOI presented as real is a KILL — overall ≤ D, F if fabrication. That is the system working.
+Registry lookup. Retry once if unreachable; then unresolved, never a guessed hallucination.
One extra abstract lookup per Tier 1 cite. Primary outcome, population, direction of effect. Does not upgrade a T3 or T4.
Five lenses. KILL / MAJOR / MINOR / PASS. KILL or MAJOR cannot ship as A. Max 3 passes.
Cap 3. Then stop and report what is still open. Do not raise the letter while a T4 remains.
Worked example: vitamin D — claim ledger → engine → semantic → KILL.
No API key. MIT. Pick the path that matches how you write.
@@ -70,5 +87,5 @@