A mechanism-aware language-model evidence layer for hereditary hearing-loss variant interpretation
The LLM never classifies a variant. It reads the literature, grades the evidence, and hands ACMG points to a Bayesian engine.
A Variant of Uncertain Significance is neither a diagnosis nor an all-clear. OtoVCE rescues them by using a large language model as a structured evidence extractor rather than a classifier: it retrieves literature from five sources, grades functional and case-level evidence under the Brnich 2019 rubric, and emits ACMG strength assignments that feed a Tavtigian Bayesian point sum. Every assignment traces back to a PubMed-verified citation.
| Discrimination | AUC 0.997, 93.6 % sensitivity at 1 % FPR on held-out post-2024 ClinVar (n = 1,885) |
| Mechanism conditioning | Paired A/B on 1,930 VUS: ΔAUC +0.178 [0.161, 0.196] vs. a generic prompt |
| Auditability | 3,185 / 3,185 cited PMIDs verified to exist — 0 hallucinations |
| Clinical yield | 211 of 2,389 VUS flagged easy-rescue at 98.1 % precision [95.2–99.3] |
| Cost | USD 0.0004 per variant (≈ $0.44 per 1,000) |
| Status | Accepted — Frontiers in Genetics (in production) |
Hereditary hearing loss is among the most genetically heterogeneous human conditions, with over 150 implicated genes, and its VUS burden is correspondingly large. Rule-based ACMG/AMP engines stall on missense variants for two reasons. First, the decisive evidence — functional assays, proband counts, segregation — sits scattered across thousands of papers that no rule engine can read. Second, disease mechanism varies dramatically gene to gene: what counts as a damaging assay result for a transmembrane K⁺ channel is not what counts for an unconventional myosin or a large tectin.
The obvious move — ask an LLM to classify the variant — is the wrong one. It produces unauditable free-text opinions, and it fails badly: on the same cohort, DeepSeek prompted directly reaches 50.6 % sensitivity and Claude 58.9 %, against OtoVCE's 80.3 % at a 0.00 % false-positive rate.
OtoVCE's architectural claim is that the LLM should occupy exactly one slot in the pipeline: reading papers and grading what it finds against a published rubric. Classification stays with a deterministic Bayesian engine.
flowchart TB
subgraph S1["① Enrichment — step1_enrich/"]
A["Variant split CSV"] --> B["myvariant.info live API<br/>REVEL · AlphaMissense · EVE<br/>ESM1b · PrimateAI · CADD · phyloP"]
A --> C["gnomAD v4 constraint<br/>UniProt domains · 164 HL genes"]
A --> D["HL-PLM predictions<br/>pathogenicity · mechanism · dB-gain"]
B & C & D --> E["enriched_split.csv<br/>60 columns"]
end
subgraph S2["② Rule engine — step2_rules/"]
E --> F["ClinGen HL-VCEP 2018<br/>17 rules · 4 strength levels"]
F --> G["Cofire safeguard<br/>BP4_HL_mech needs BS1/BP4"]
G --> H["net_pts · preliminary class"]
end
subgraph S3["③ Evidence layer — step3_llm/"]
H --> I{"Still VUS or<br/>borderline?"}
I -->|no| P["keep rule call<br/>saves 60–78% of LLM calls"]
I -->|yes| J["Tier 1: curated wet-lab lookup<br/>deterministic · zero cost"]
J -->|miss| K["5-source retrieval<br/>whitelist · DVD · LitVar2<br/>PubMed · PMC full text"]
K --> L["Brnich 2019 prompting<br/>dual-prompt ensemble · max-rank<br/>conditioned on PLM mechanism"]
L --> M["PS3 / BS3 / PS4 strengths<br/>+ rationale + PMIDs"]
M --> N["PS4_DVD from proband counts<br/>vetoed by any benign criterion"]
end
subgraph S4["④ Aggregation + report"]
N & P --> O["Tavtigian Bayesian sum<br/>prior 0.10 · OddsPath 350 per 8 pts"]
O --> Q["5-class ACMG/AMP call"]
Q --> R["Clinical report<br/>mechanism · dB-gain · AAV-OTOF tier"]
end
style S1 fill:#e8f4f8,stroke:#4a90a4
style S2 fill:#f0f4e8,stroke:#7a9a4a
style S3 fill:#f8e8f0,stroke:#a44a7a
style S4 fill:#fff4e0,stroke:#c49a3a
The LLM stage is where the design decisions live.
flowchart LR
A["Variant<br/>left VUS by rules"] --> B["HL-PLM mechanism<br/>LOF / DN / trafficking / transport"]
B --> C{"top prob ≥ 0.4<br/>and margin ≥ 0.10?"}
C -->|yes| D["Inject mechanism-specific<br/>assay-search guidance"]
C -->|no| E["Generic Brnich rubric<br/>ambiguity fallback"]
A --> F["5-source retrieval<br/>max 10 PMIDs, deduped<br/>title+abstract+methods+results<br/>±800 char variant context"]
D & E & F --> G["Prompt V1<br/>conservative"]
D & E & F --> H["Prompt V2<br/>broadened"]
G & H --> I["Max-rank merge<br/>per category"]
I --> J["JSON: PS3 / BS3 / PS4<br/>assay type · OR · n probands<br/>rationale · pmid_cited[]"]
J --> K["Label-leak regex stripper<br/>PMID existence audit"]
K --> L["ACMG points only —<br/>never a class label"]
style B fill:#e8f0f8,stroke:#4a7aa4
style I fill:#f8e8e8,stroke:#a44a4a
style L fill:#f0f8e8,stroke:#6a9a4a
Model. DeepSeek-V3 (deepseek-chat) at temperature 0.1 with enforced JSON output, 2 retries with exponential backoff.
Dual-prompt ensemble. A conservative prompt (numeric odds ratio required for Moderate or above) and a broadened prompt (qualitative clear loss-of-function may reach Moderate) run concurrently and merge by max rank per evidence category. A documented N=100 pilot justified this: V1 alone recovered 15.4 % of true-positive VUS, min-rank 9.6 %, max-rank 19.2 %, all at zero false positives on controls.
Mechanism conditioning. The HL-PLM's four-class mechanism distribution is injected into the prompt along with class-specific assay-search instructions — for a loss-of-function prediction, "Western blot for protein expression, voltage-clamp for I_Ks/I_Kr currents, immunofluorescence localisation." When the prediction is ambiguous (top class < 0.4, or margin < 0.10) the system falls back to the generic rubric rather than asserting a mechanism it does not have.
Brnich calibration is encoded explicitly, not left to the model's judgement: very strong at OR > 18.7; strong at OR ≥ 7 with ≥ 3 replicates; moderate at OR ≥ 3; supporting at OR ≥ 1.5. PS4 ladders on proband counts (≥ 15 very strong, 10–14 strong, 6–9 moderate, 2–5 supporting).
17 criteria implementing ClinGen Hearing Loss VCEP 2018 specifications — 14 standard ACMG plus three hearing-loss-specific (PP4_HL_mech, PP_PLM_supporting, BP4_HL_mech). PP2 and BP1 are hard-disabled per the VCEP specification. PS3, BS3 and PS4 are deliberately absent: those are the LLM's tier.
A representative design detail is the cofire safeguard. BP4_HL_mech is rewritten to not-fired unless BS1 or BP4 independently fires, because 41 false negatives on the DVD cohort traced to that criterion firing alone in dominant and X-linked genes. BA1 and BS3 are excluded as validators — BS3 is LLM-derived and must not be allowed to self-validate a PLM prior.
Implemented in src/rules/hl_vcep_engine.py and re-applied after the LLM stage in step3_llm/aggregate_step3.py:
P_post = sigmoid( logit(0.10) + net_pts × ln(350)/8 )
Prior 0.10, OddsPath 350× per Very Strong = 8 points, so ≈ 2.08× odds per point. Strengths map to points as very_strong 8, strong 4, moderate 2, supporting 1. Classes: BA1 forces Benign; net ≥ 10 Pathogenic; ≥ 6 Likely Pathogenic; ≤ −7 Benign; ≤ −1 Likely Benign; otherwise VUS.
A self-contained open-source release lives in plm/ with weights included.
| Backbone | facebook/esm2_t30_150M_UR50D, 640-d, frozen |
| Adaptation | LoRA r=8, α=16, dropout 0.1 on query/key/value |
| Representation | ±32 aa window; [h_mut − h_wt ; h_mut] = 1280-d |
| Fusion | 1280 + 9 hand features + 64-d gene embedding (164 genes) = 1353 → 512 → 256 |
| Heads | pathogenicity (1) · mechanism (4-class) · dB-gain (1) |
| Checkpoint | 6.8 MB — trainable tensors only |
The mechanism head reaches 87.6 % top-1 four-class accuracy (95 % CI 81.8–92.7, n = 137 wet-lab-labelled variants), and 97.8 % on the collapsed axis.
| Tool | n | AUC [95 % CI] | Sens @ 1 % FPR | Sens @ 5 % FPR |
|---|---|---|---|---|
| OtoVCE | 1,885 | 0.997 [0.996, 0.998] | 93.6 % | 97.0 % |
| HL-PLM alone | 1,869 | 0.993 [0.990, 0.995] | 85.1 % | 97.2 % |
| REVEL | 1,503 | 0.965 [0.957, 0.974] | 66.5 % | 84.8 % |
| AlphaMissense | 1,530 | 0.952 [0.940, 0.962] | 45.4 % | 78.5 % |
| CADD | 1,874 | 0.941 [0.932, 0.950] | 47.3 % | 69.4 % |
| EVE | 865 | 0.932 [0.916, 0.947] | 50.3 % | 67.2 % |
| ESM1b | 1,507 | 0.919 [0.903, 0.933] | 35.4 % | 67.8 % |
| PrimateAI | 1,521 | 0.859 [0.841, 0.879] | 14.7 % | 41.2 % |
+27.1 points of sensitivity over REVEL at 1 % FPR, with confidence intervals non-overlapping against HL-PLM alone. On the common-coverage subset where all eight tools return a score (n = 829): AUC 0.997, 92.0 % sensitivity.
At the actual P/LP decision threshold rather than a fixed FPR, OtoVCE reaches 80.3 % sensitivity at 0.00 % FPR [0, 0.4], versus DeepSeek prompted directly at 50.6 % / 1.31 % and Claude at 58.9 % / 12.78 %.
| Cohort | n | OtoVCE AUC | Purpose |
|---|---|---|---|
| Post-2024 ClinVar | 1,885 | 0.997 [0.996, 0.998] | Temporal hold-out |
| OTOF (gene withheld from PLM) | 205 | 0.993 [0.983, 0.999] | Gene-level generalisation |
| DVD strong-evidence | 2,835 | 0.917 [0.906, 0.929] | External curatorial provenance |
| PLM training set, PLM rules off | 8,096 | 0.976 | Memorisation defence |
The last row is the important one. Evaluated on the language model's own training set with all PLM-derived rules disabled, OtoVCE still reaches AUC 0.976 and 72.5 % sensitivity at 1 % FPR — +0.034 AUC and +16.7 points over the best baseline (REVEL 0.942 / 55.8 %). Whatever the system is doing, it is not merely recalling its training data.
Ablation on that same cohort: rule engine alone 0.9649 / 67.0 % → adding the LLM tier 0.9756 / 72.5 %, i.e. +0.011 AUC and +5.5 points from the evidence layer.
A paired A/B on 1,930 VUS (343 temporal, 62 OTOF, 1,525 DVD), mechanism-aware prompt versus generic prompt:
| Metric | Mechanism-aware | Generic | Ratio |
|---|---|---|---|
| PMIDs retrieved per variant | 1.22 | 0.17 | 7.1× |
| Quantitative evidence extracted | 13.8 % | 1.2 % | 11.6× |
| PS3 at ≥ moderate | 236 | 111 | 2.1× |
| PS4 at ≥ moderate | 105 | 12 | 8.8× |
| BS3 at ≥ moderate | 7 | 0 | — |
| Strength score as a classifier (AUC) | 0.715 [0.697, 0.734] | 0.537 [0.528, 0.547] | ΔAUC +0.178 [0.161, 0.196] |
An adversarial control — deliberately flipping the mechanism prediction — degrades AUC by at most 0.030, confirming the gain comes from correct conditioning rather than from prompt length.
Of a 2,389-variant VUS pool: 211 easy-rescue candidates, 207 of them true P/LP = 98.1 % precision [95.2–99.3]; 244 feasible; 1,331 needing deep curation; 603 unplaceable. Top-1,000 hit rate: OtoVCE 95.5 % vs REVEL 91.2 % vs AlphaMissense 59.1 %.
Against ClinGen HL-VCEP expert curation (n = 50 unique variants): Cohen's κ = 0.57 [0.34, 0.77], 76 % direction agreement, and zero expert-benign variants called pathogenic.
Calibration: Brier 0.029, ECE 0.041. Decision-curve analysis shows a maximum net-benefit gap over treat-all of +0.38 near threshold 0.49; every other predictor drops below treat-all somewhere in the clinically relevant range.
Across 4,925 variants there were 47 disagreements with the reference labels. Categorised: 26 likely label noise (≥ 75 % cross-tool consensus against ClinVar), 14 genuine over-calls, 5 predictor disagreement, 2 borderline. Externally re-checking the 9 highest-consensus label-noise candidates against ClinVar gave 3 corroborated, 3 contradicted, 3 no record — reported as such.
All inference in this work uses bootstrap and Wilson confidence intervals; there are no p-values.
OtoVCE/
├── step1_enrich/ ① Multi-source enrichment → enriched_<split>.csv (60 cols)
│ └── pipeline.py 7 resumable sub-steps with parquet checkpoints
├── step2_rules/ ② ClinGen HL-VCEP 2018 engine — 17 rules, cofire safeguard
├── step3_llm/ ③ Two-tier evidence layer
│ ├── tier1_lookup.py Curated wet-lab PS3/BS3, deterministic, zero cost
│ ├── dvd_pmid_rule.py PS4 from proband counts, vetoed by benign criteria
│ ├── aggregate_step3.py Tavtigian re-aggregation with LLM points
│ └── agent/debate.py Alternate branch: Prosecutor/Defender/Judge debate
├── step4_outputs/ ④ Clinical report — mechanism, dB-gain, AAV-OTOF eligibility
├── step5_experiments/ 27 validation scripts
├── step5_figures/ 6 main + 7 supplementary figures
├── src/
│ ├── rules/ 12 modules — per-criterion ACMG logic + Bayesian engine
│ ├── agent/ 10 modules — Brnich prompting, retrieval, verifier, adjudicator
│ └── scoring/ Production Tier-2 entry point
├── plm/ Open-source HL-PLM release — weights, CLI, CPU/CUDA/MPS
└── docs/ Design notes and engineering logs
git clone https://github.com/pipi0616/OtoVCE.git
cd OtoVCE
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
export DEEPSEEK_API_KEY="your_key_here" # required for the Tier-2 evidence layer
export NCBI_API_KEY="your_key_here" # optional: raises PubMed to 10 req/sPython 3.9+. The HL-PLM runs on CPU, CUDA or Apple MPS.
python -m step1_enrich.run # ① enrichment
python step2_rules/rules.py # ② rule engine
python step3_llm/tier1_lookup.py # ③ curated lookup (free)
bash scripts/03_run.sh # Tier-2 LLM batch (~$5 per 3,000 variants)
python step3_llm/dvd_pmid_rule.py # PS4 from proband counts
python step3_llm/aggregate_step3.py # Bayesian aggregation
python step4_outputs/consolidate.py # ④ clinical reportOnly variants left VUS or borderline by stage ② reach the LLM, which cuts 60–78 % of API calls. Per variant the LLM call itself takes ~2.4 s and retrieval ~57 s; end-to-end cost is USD 0.0004 per variant, halved in DeepSeek's off-peak window (16:30–00:30 UTC).
Self-contained — weights are in the repo, no external data needed:
pip install -r plm/requirements.txt
python plm/predict.py --gene SLC26A4 --protein-change Val163Ile
python plm/predict.py --csv variants.csv --out scored.csv --device mpsStage ① expects the enriched input corpus, which is not in this repository. The full data deposit is on Zenodo; upstream sources retain their own terms: ClinVar, gnomAD v4, Deafness Variation Database v9, UniProt, and myvariant.info.
Network calls are memoized in a WAL-mode SQLite cache with per-thread connections; every sub-step checkpoints to parquet so a killed run resumes where it stopped; import_legacy_tier2.py deliberately reuses already-paid-for LLM output rather than re-billing. Seeding is fixed at default_rng(42) and all intervals are bootstrap or Wilson.
Two integrity mechanisms are worth calling out. Temporal-leakage discipline propagates a maxdate through all five retrieval sources, re-checks DVD PMIDs by publication year, and rebuilds the PS1/PM5 reference table from pre-cutoff ClinVar — followed by a post-hoc audit that caught and reported its own single leak. Label-leak stripping removes sentences referencing ClinVar classifications from retrieved text before the model sees them.
A prior concordance-elevation bonus for co-firing PS3 and PS4 was removed after re-reading Brnich 2019 found it unsupported; the removal is documented in the code.
Ye S., Wang L., Chen P. OtoVCE: a mechanism-aware language-model evidence layer for hereditary hearing-loss variant interpretation. Frontiers in Genetics, 2026. (Accepted, in production)
@article{ye2026otovce,
title = {OtoVCE: a mechanism-aware language-model evidence layer for
hereditary hearing-loss variant interpretation},
author = {Ye, Shaopei and Wang, Lan and Chen, Peng},
journal = {Frontiers in Genetics},
year = {2026},
note = {Accepted, in production}
}Shaopei Ye† — Universitat de Barcelona, Spain · ORCID 0009-0006-3598-0469 Lan Wang† — Meishan People's Hospital, Sichuan, China Peng Chen* — Dept. of Otorhinolaryngology, Meishan People's Hospital · ORCID 0009-0001-1105-6638 · cpeng960@gmail.com
† equal contribution · * corresponding author
MIT — see LICENSE. Released data are CC BY 4.0.
- HEARS-LM — hearing-loss-specialized protein language model
- SeizeVar — mechanism-aware epilepsy variant interpretation
- QAFI — quantitative functional-impact prediction
"意义未明变异"（VUS）既不是诊断，也不是排除。OtoVCE 通过把大语言模型用作结构化证据抽取器而非分类器来解决这个问题：它从五个来源检索文献，按 Brnich 2019 规范对功能学和病例级证据定级，输出可直接进入 Tavtigian 贝叶斯计分的 ACMG 强度赋值。每一条赋值都能追溯到一个经 PubMed 验证的文献编号。
| 判别能力 | 在 2024 年后 ClinVar 留出集（n = 1,885）上 AUC 0.997，1% 假阳性率下灵敏度 93.6% |
| 机制条件化 | 1,930 个 VUS 的配对 A/B 实验：相比通用提示词 ΔAUC +0.178 [0.161, 0.196] |
| 可审计性 | 3,185 / 3,185 条引用文献编号全部核实存在——零幻觉 |
| 临床产出 | 2,389 个 VUS 中标记出 211 个"易挽救"候选，精确度 98.1% [95.2–99.3] |
| 成本 | 每个变异 0.0004 美元（约每千个 0.44 美元） |
| 状态 | 已被 Frontiers in Genetics 接收（制作中） |
遗传性听力损失是人类遗传异质性最强的疾病之一，已知致病基因超过 150 个，其 VUS 负担也相应巨大。基于规则的 ACMG/AMP 引擎在错义变异上卡壳有两个原因。第一，决定性证据——功能实验、先证者计数、家系共分离——散落在数千篇规则引擎读不了的文献里。第二，致病机制因基因而异：对跨膜钾通道而言"有害"的实验结果，放到非常规肌球蛋白或大型盖膜蛋白上就不再成立。
一个显而易见但错误的做法是：让大模型直接做分类判定。这会产出无法审计的自由文本意见，而且效果很差——在同一队列上，直接提示 DeepSeek 的灵敏度为 50.6%，Claude 为 58.9%，而 OtoVCE 在假阳性率 0.00% 时达到 80.3%。
OtoVCE 的架构主张是：大模型在流程中只应占据一个明确的位置——阅读文献，并依据已发表的规范为所读内容定级。分类判定始终交给确定性的贝叶斯引擎。
flowchart TB
subgraph S1["① 数据富集 — step1_enrich/"]
A["变异列表 CSV"] --> B["myvariant.info 实时 API<br/>REVEL · AlphaMissense · EVE<br/>ESM1b · PrimateAI · CADD · phyloP"]
A --> C["gnomAD v4 约束指标<br/>UniProt 结构域 · 164 个听力基因"]
A --> D["HL-PLM 预测<br/>致病性 · 机制 · 听阈增益"]
B & C & D --> E["enriched_split.csv<br/>60 列"]
end
subgraph S2["② 规则引擎 — step2_rules/"]
E --> F["ClinGen HL-VCEP 2018<br/>17 条规则 · 4 级强度"]
F --> G["共触发保护机制<br/>BP4_HL_mech 需 BS1/BP4 佐证"]
G --> H["净分值 · 初步分类"]
end
subgraph S3["③ 证据层 — step3_llm/"]
H --> I{"仍为 VUS<br/>或处于边界?"}
I -->|否| P["保留规则判定<br/>节省 60–78% 的大模型调用"]
I -->|是| J["第一层：人工审校湿实验查表<br/>确定性 · 零成本"]
J -->|未命中| K["五源文献检索<br/>白名单 · DVD · LitVar2<br/>PubMed · PMC 全文"]
K --> L["Brnich 2019 规范提示<br/>双提示词集成 · 取高档位<br/>以 PLM 机制预测为条件"]
L --> M["PS3 / BS3 / PS4 强度<br/>+ 推理依据 + 文献编号"]
M --> N["由先证者计数推 PS4_DVD<br/>任一良性判据触发即否决"]
end
subgraph S4["④ 汇总与报告"]
N & P --> O["Tavtigian 贝叶斯计分<br/>先验 0.10 · 每 8 分对应 350 倍似然比"]
O --> Q["ACMG/AMP 五分类判定"]
Q --> R["临床报告<br/>机制 · 听阈增益 · AAV-OTOF 适应性分层"]
end
style S1 fill:#e8f4f8,stroke:#4a90a4
style S2 fill:#f0f4e8,stroke:#7a9a4a
style S3 fill:#f8e8f0,stroke:#a44a7a
style S4 fill:#fff4e0,stroke:#c49a3a
大模型这一层是全部设计决策的所在。
flowchart LR
A["规则引擎<br/>未能定性的变异"] --> B["HL-PLM 机制预测<br/>功能缺失 / 显性负效 / 转运障碍 / 转运功能缺失"]
B --> C{"最高概率 ≥ 0.4<br/>且领先幅度 ≥ 0.10?"}
C -->|是| D["注入机制特异的<br/>实验检索指引"]
C -->|否| E["回退到通用 Brnich 规范<br/>歧义安全机制"]
A --> F["五源检索<br/>最多 10 篇去重文献<br/>标题+摘要+方法+结果<br/>变异上下文 ±800 字符"]
D & E & F --> G["提示词 V1<br/>保守版"]
D & E & F --> H["提示词 V2<br/>放宽版"]
G & H --> I["逐类别取较高档位<br/>合并"]
I --> J["JSON 输出：PS3 / BS3 / PS4<br/>实验类型 · 优势比 · 先证者数<br/>推理依据 · 引用文献编号"]
J --> K["标签泄漏正则清洗<br/>文献编号存在性审计"]
K --> L["只输出 ACMG 分值——<br/>绝不输出分类标签"]
style B fill:#e8f0f8,stroke:#4a7aa4
style I fill:#f8e8e8,stroke:#a44a4a
style L fill:#f0f8e8,stroke:#6a9a4a
模型。 DeepSeek-V3（deepseek-chat），温度 0.1，强制 JSON 输出，失败时指数退避重试 2 次。
双提示词集成。 一个保守提示词（Moderate 及以上必须有数值优势比）与一个放宽提示词（定性的明确功能缺失也可达 Moderate）并发运行，再按证据类别取较高档位合并。一项 N=100 的预实验为此提供了依据：仅用 V1 可召回 15.4% 的真阳性 VUS，取较低档位为 9.6%，取较高档位为 19.2%，且三者在对照集上的假阳性均为零。
机制条件化。 HL-PLM 的四分类机制概率分布连同该类别专属的实验检索指令一起注入提示词——例如对功能缺失预测，指令为"检索蛋白表达 Western blot、I_Ks/I_Kr 电流膜片钳、免疫荧光定位"。当预测本身存在歧义时（最高类别 < 0.4，或与次高类别差距 < 0.10），系统回退到通用规范，而不是硬套一个它并不确定的机制。
Brnich 定级标准以代码显式编码，而非交由模型自行判断：优势比 > 18.7 为 very strong；≥ 7 且重复次数 ≥ 3 为 strong；≥ 3 为 moderate；≥ 1.5 为 supporting。PS4 按先证者计数分档（≥ 15 very strong，10–14 strong，6–9 moderate，2–5 supporting）。
17 条判据，实现 ClinGen 听力损失 VCEP 2018 版规范——14 条标准 ACMG 判据，加 3 条听力损失特异判据（PP4_HL_mech、PP_PLM_supporting、BP4_HL_mech）。按 VCEP 规范，PP2 与 BP1 被硬性禁用。PS3、BS3、PS4 有意缺席：那是大模型层的职责。
一个有代表性的设计细节是共触发保护机制。除非 BS1 或 BP4 独立触发，否则 BP4_HL_mech 会被改写为未触发——因为 DVD 队列上有 41 个假阴性可追溯到该判据在显性和 X 连锁基因中单独触发。BA1 和 BS3 被排除在佐证判据之外：BS3 来自大模型，绝不能让它自我佐证一个来自 PLM 的先验。
实现于 src/rules/hl_vcep_engine.py，并在大模型层之后由 step3_llm/aggregate_step3.py 重新计算：
P_post = sigmoid( logit(0.10) + 净分值 × ln(350)/8 )
先验 0.10，每个 Very Strong（8 分）对应 350 倍似然比，即每分约 2.08 倍优势比。强度映射为 very_strong 8 分、strong 4 分、moderate 2 分、supporting 1 分。分类阈值：触发 BA1 直接判良性；净分 ≥ 10 致病；≥ 6 可能致病；≤ −7 良性；≤ −1 可能良性；其余为 VUS。
一个自包含的开源发布版位于 plm/ 目录，权重已随仓库提供。
| 骨干网络 | facebook/esm2_t30_150M_UR50D，640 维，冻结 |
| 适配方式 | LoRA r=8，α=16，dropout 0.1，作用于 query/key/value |
| 表示方式 | ±32 氨基酸窗口；[h_mut − h_wt ; h_mut] = 1280 维 |
| 融合 | 1280 + 9 个人工特征 + 64 维基因嵌入（164 个基因）= 1353 → 512 → 256 |
| 输出头 | 致病性（1）· 机制（4 分类）· 听阈增益（1） |
| 检查点 | 6.8 MB——仅含可训练张量 |
机制头达到 87.6% 的四分类 top-1 准确率（95% CI 81.8–92.7，n = 137 个有湿实验标注的变异），在合并轴上为 97.8%。
| 工具 | n | AUC [95% CI] | 1% 假阳性率下灵敏度 | 5% 假阳性率下灵敏度 |
|---|---|---|---|---|
| OtoVCE | 1,885 | 0.997 [0.996, 0.998] | 93.6% | 97.0% |
| 仅 HL-PLM | 1,869 | 0.993 [0.990, 0.995] | 85.1% | 97.2% |
| REVEL | 1,503 | 0.965 [0.957, 0.974] | 66.5% | 84.8% |
| AlphaMissense | 1,530 | 0.952 [0.940, 0.962] | 45.4% | 78.5% |
| CADD | 1,874 | 0.941 [0.932, 0.950] | 47.3% | 69.4% |
| EVE | 865 | 0.932 [0.916, 0.947] | 50.3% | 67.2% |
| ESM1b | 1,507 | 0.919 [0.903, 0.933] | 35.4% | 67.8% |
| PrimateAI | 1,521 | 0.859 [0.841, 0.879] | 14.7% | 41.2% |
在 1% 假阳性率下灵敏度比 REVEL 高 27.1 个百分点，且置信区间与"仅 HL-PLM"不重叠。在八个工具均能给出评分的共同覆盖子集上（n = 829）：AUC 0.997，灵敏度 92.0%。
若采用真实的 P/LP 判定阈值而非固定假阳性率，OtoVCE 达到假阳性率 0.00% 时灵敏度 80.3% [0, 0.4]，而直接提示 DeepSeek 为 50.6% / 1.31%，Claude 为 58.9% / 12.78%。
| 队列 | n | OtoVCE AUC | 目的 |
|---|---|---|---|
| 2024 年后 ClinVar | 1,885 | 0.997 [0.996, 0.998] | 时间留出 |
| OTOF（该基因从 PLM 训练中剔除） | 205 | 0.993 [0.983, 0.999] | 基因级泛化 |
| DVD 强证据集 | 2,835 | 0.917 [0.906, 0.929] | 外部人工审校来源 |
| PLM 训练集，PLM 相关规则关闭 | 8,096 | 0.976 | 记忆防御 |
最后一行才是关键。在语言模型自己的训练集上评估、并关闭全部 PLM 衍生规则后，OtoVCE 仍达到 AUC 0.976、1% 假阳性率下灵敏度 72.5%——比最佳基线（REVEL 0.942 / 55.8%）高 0.034 AUC 和 16.7 个百分点。无论这个系统在做什么，它都不只是在复述训练数据。
同一队列上的消融实验： 仅规则引擎 0.9649 / 67.0% → 加入大模型层后 0.9756 / 72.5%，即证据层贡献 +0.011 AUC 和 +5.5 个百分点。
在 1,930 个 VUS（343 个时间队列、62 个 OTOF、1,525 个 DVD）上做机制感知提示词 vs 通用提示词的配对对比：
| 指标 | 机制感知 | 通用 | 倍数 |
|---|---|---|---|
| 每个变异检索到的文献数 | 1.22 | 0.17 | 7.1× |
| 抽取到定量证据的比例 | 13.8% | 1.2% | 11.6× |
| PS3 达 ≥ moderate | 236 | 111 | 2.1× |
| PS4 达 ≥ moderate | 105 | 12 | 8.8× |
| BS3 达 ≥ moderate | 7 | 0 | — |
| 以强度分作为分类器（AUC） | 0.715 [0.697, 0.734] | 0.537 [0.528, 0.547] | ΔAUC +0.178 [0.161, 0.196] |
一项对抗性对照实验——故意翻转机制预测——使 AUC 最多下降 0.030，证实增益来自正确的机制条件化，而非提示词变长本身。
在 2,389 个 VUS 池中：标记出 211 个"易挽救"候选，其中 207 个为真正的 P/LP，精确度 98.1% [95.2–99.3]；244 个"可行"；1,331 个需深度审校；603 个无法定位。前 1,000 命中率：OtoVCE 95.5%，REVEL 91.2%，AlphaMissense 59.1%。
与 ClinGen HL-VCEP 专家审校对比（n = 50 个去重变异）：Cohen's κ = 0.57 [0.34, 0.77]，方向一致率 76%，且没有任何一个专家判定为良性的变异被判为致病。
校准度：Brier 0.029，ECE 0.041。决策曲线分析显示相对"全部处理"策略的最大净获益差为**+0.38**（阈值约 0.49）；其余所有预测器在临床相关区间内某处都会跌破"全部处理"基线。
在 4,925 个变异中，与参考标签存在 47 处分歧。分类归因为：26 处疑似标签噪声（≥ 75% 的跨工具共识与 ClinVar 相左）、14 处确属过度判定、5 处预测器分歧、2 处处于边界。将其中共识度最高的 9 个标签噪声候选拿到 ClinVar 外部复核，结果为 3 个得到佐证、3 个被反驳、3 个无记录——如实报告。
本工作所有统计推断均使用自助法和 Wilson 置信区间，不使用 p 值。
OtoVCE/
├── step1_enrich/ ① 多源数据富集 → enriched_<split>.csv（60 列）
│ └── pipeline.py 7 个可断点续跑的子步骤，parquet 检查点
├── step2_rules/ ② ClinGen HL-VCEP 2018 引擎——17 条规则，共触发保护
├── step3_llm/ ③ 两层证据抽取
│ ├── tier1_lookup.py 人工审校湿实验 PS3/BS3 查表，确定性，零成本
│ ├── dvd_pmid_rule.py 由先证者计数推 PS4，良性判据可否决
│ ├── aggregate_step3.py 纳入大模型分值后重新做 Tavtigian 汇总
│ └── agent/debate.py 备选分支：控方/辩方/法官三角色辩论
├── step4_outputs/ ④ 临床报告——机制、听阈增益、AAV-OTOF 适应性
├── step5_experiments/ 27 个验证脚本
├── step5_figures/ 6 张正文图 + 7 张补充图
├── src/
│ ├── rules/ 12 个模块——逐判据 ACMG 逻辑 + 贝叶斯引擎
│ ├── agent/ 10 个模块——Brnich 提示、检索、校验、裁决
│ └── scoring/ 生产环境第二层入口
├── plm/ HL-PLM 开源发布——权重、命令行、CPU/CUDA/MPS
└── docs/ 设计笔记与工程日志
git clone https://github.com/pipi0616/OtoVCE.git
cd OtoVCE
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
export DEEPSEEK_API_KEY="your_key_here" # 第二层证据抽取必需
export NCBI_API_KEY="your_key_here" # 可选：将 PubMed 限速提升到 10 次/秒需要 Python 3.9+。HL-PLM 可在 CPU、CUDA 或 Apple MPS 上运行。
python -m step1_enrich.run # ① 数据富集
python step2_rules/rules.py # ② 规则引擎
python step3_llm/tier1_lookup.py # ③ 查表（免费）
bash scripts/03_run.sh # 第二层大模型批处理（约 3,000 个变异 5 美元）
python step3_llm/dvd_pmid_rule.py # 由先证者计数推 PS4
python step3_llm/aggregate_step3.py # 贝叶斯汇总
python step4_outputs/consolidate.py # ④ 临床报告只有在第 ② 步后仍为 VUS 或处于边界的变异才会进入大模型层，这减少了 60–78% 的 API 调用。单个变异的大模型调用本身约需 2.4 秒，文献检索约 57 秒；端到端成本为每个变异 0.0004 美元，在 DeepSeek 的低峰时段（UTC 16:30–00:30）可再减半。
自包含——权重已在仓库中，无需任何外部数据：
pip install -r plm/requirements.txt
python plm/predict.py --gene SLC26A4 --protein-change Val163Ile
python plm/predict.py --csv variants.csv --out scored.csv --device mps第 ① 步需要富集后的输入语料，该数据不在本仓库内。完整数据集存放于 Zenodo；上游数据源各自保留其许可条款：ClinVar、gnomAD v4、Deafness Variation Database v9、UniProt 以及 myvariant.info。
所有网络请求都缓存在 WAL 模式的 SQLite 中（每线程独立连接）；每个子步骤都会写 parquet 检查点，中断后可从断点续跑；import_legacy_tier2.py 有意复用已付费的大模型输出，而不是重复计费。随机种子固定为 default_rng(42)，所有区间估计均为自助法或 Wilson 法。
有两项完整性机制值得特别说明。时间泄漏管控将 maxdate 参数贯穿全部五个检索源，按发表年份复核 DVD 文献编号，并用截止日期之前的 ClinVar 重建 PS1/PM5 参考表——随后的事后审计发现并主动报告了其自身唯一的一处泄漏。标签泄漏清洗在文本送入模型之前，先移除其中提及 ClinVar 分类结论的句子。
早期版本中，PS3 与 PS4 同时触发时曾有一项"一致性加成"，在重读 Brnich 2019 后确认其缺乏依据而被移除，该决定在代码中有记录。
Ye S., Wang L., Chen P. OtoVCE: a mechanism-aware language-model evidence layer for hereditary hearing-loss variant interpretation. Frontiers in Genetics, 2026.（已接收，制作中）
@article{ye2026otovce,
title = {OtoVCE: a mechanism-aware language-model evidence layer for
hereditary hearing-loss variant interpretation},
author = {Ye, Shaopei and Wang, Lan and Chen, Peng},
journal = {Frontiers in Genetics},
year = {2026},
note = {Accepted, in production}
}叶少培（Shaopei Ye）† — 巴塞罗那大学，西班牙 · ORCID 0009-0006-3598-0469 王岚（Lan Wang）† — 四川省眉山市人民医院 陈鹏（Peng Chen）* — 眉山市人民医院耳鼻咽喉科 · ORCID 0009-0001-1105-6638 · cpeng960@gmail.com
† 共同第一作者 · * 通讯作者
MIT——详见 LICENSE。发布的数据采用 CC BY 4.0。