feat(scholar): Phase 18 C2 tail — PMC references, ar5iv full text, Crossref

docs/CAPTURE_GUIDE.md

@@ -206,6 +206,23 @@ These "just work" without special handling:
   Substacks) — article body plus comments if you opt in.
 - **Any article page** — Readability extracts the main content. Works
   on most news sites, blogs, documentation, Wikipedia, etc.
+- **PubMed Central** (`pmc.ncbi.nlm.nih.gov/articles/PMC<id>/`, legacy
+  `www.ncbi.nlm.nih.gov/pmc/...` too) — plain server-rendered HTML, no
+  timing quirks. On top of the article body, the capture carries the
+  **reference list as structure** (year/DOI/PMID per entry, title and
+  authors only where the page marks them — never guessed), figure
+  captions, and the PMCID/PMID/DOI ids. References are a local capture
+  record; nothing new publishes.
+- **arXiv** (`arxiv.org/abs/<id>`) — the abs page shows only the
+  abstract, so the capture **prefers the ar5iv full-text rendition**
+  (`ar5iv.labs.arxiv.org`) when it's meaningfully fuller: the body is
+  swapped for the full text, `capture_url` records the ar5iv address
+  actually fetched, and the article's identity stays the `/abs/` URL.
+  Honesty note: ar5iv is a machine conversion of the LaTeX source, not
+  the PDF of record — figures/equations can differ; that's exactly why
+  the provenance is recorded. If ar5iv has no conversion (or it's
+  stub-short), you keep the abstract capture unchanged. `/pdf/` links
+  route through the PDF capture path instead.
 
 ---
 

docs/JOURNAL.md

@@ -19,6 +19,54 @@ or files, and the "so-what" for future readers.
 
 ---
 
+## 2026-07-17 — Phase 18 C2 tail: scholarly handlers, and what adversarial review caught
+
+Tags: `design`, `pattern`.
+
+The C2 deferred tail (PMC handler, ar5iv-preferring arXiv handler,
+Crossref) landed as pure modules + wiring. Decisions worth recording:
+
+**The scholar-meta hoist.** `enrichArticleForPlatform` ran the
+scholar-meta pass *after* handler dispatch, so any handler reading
+`article.scholar` read nothing. Harmless while no handler did; the
+arXiv handler's whole trigger is `scholar.arxiv_id`, so the pass now
+runs first. Safe: it is side-effect-free and substack (the only other
+enrich handler) never reads `scholar`.
+
+**References parse under an honesty rule** — wrong structure is worse
+than no structure — and the review pass proved the first cut violated
+it five ways, each with an executed repro: Google-Scholar *search*
+links landing in `entry.url` on essentially every modern-PMC reference
+(the `[Google Scholar]` decoration anchor); the year inside an
+Elsevier-style DOI (`10.1016/j.vaccine.2015…`) outranking the true
+year; SICI DOIs truncated at their literal `<` and emitted as wrong
+identifiers (now omitted — scholar-meta's private copy still truncates,
+unify later); italicized *journal names* adopted as titles (the journal
+is the segment the year follows directly — that positional check is the
+fix); and `'Lovelace, Ada'` mis-split into two authors (single-token
+comma parts now bail). Each fix is pinned by a regression test whose
+fixture IS the repro.
+
+**The arXiv body swap re-derives the wire-bound fields.** Adopting the
+ar5iv full text while keeping the abstract's `wordCount` and `links`
+would publish a `word_count` tag off by ~100× and `link` tags for a
+body no longer shipped — the same class of stale-derived-state bug as
+the archive-reload work, caught by the same review pattern.
+`extractFromHtmlString` extracts links from the adopted body so they
+ride along; absent that, `links` goes null ("not captured" ≠ "zero").
+
+**`xray:scholar:fetch` is host-allowlisted** (ar5iv/arxiv hosts only)
+and `xray:scholar:crossref` accepts a DOI, never a URL — the SW must
+not grow a generic fetch proxy one message at a time.
+
+**The Crossref reader trigger is deliberately deferred.** The obvious
+wiring is wrong twice: there is no scholar display in the reader to
+refresh, and persisting the patched article back to the session stash
+by writing a fresh record nulls `sourceTabId` — which silently reroutes
+NIP-07 publishes through the worker signer (`handleCapturePublish`'s
+tabless branch). It needs a read-modify-write and a surface; parked
+until there is one.
+
 ## 2026-07-17 — Load archive → republish drifted the x tag; the fix is a proof, not a flag
 
 Tags: `bug`, `design`.

docs/ROADMAP.md

@@ -1506,7 +1506,20 @@ grounding substrate*).
 - ✅ **C2 (light)** scholarly metadata — DOI/arXiv/journal/authors
   from citation meta tags on every capture (`platforms/scholar-meta.js`);
   additive `doi` + NIP-73 `i` + `arxiv` tags on 30023.
-  *(Deferred: ar5iv-preferring arXiv handler, PMC handler, Crossref.)*
+- ✅ **C2 (tail)** the deferred scholarly handlers — `platforms/pmc.js`
+  (references as structure via the shared `scholar-refs.js` parser —
+  title/authors only when DOM-marked, never guessed; figure captions;
+  PMCID/PMID ids) and `platforms/arxiv.js` (ar5iv full-text preference
+  on `/abs/` captures: body swap gated on a meaningfully-longer check,
+  `capture_url` records the rendition fetched, wire-bound
+  wordCount/links re-derived from the adopted body); `crossref.js`
+  (validated DOI → `api.crossref.org` request + fill-only-missing
+  mapping) behind the host-allowlisted `xray:scholar:fetch` /
+  `xray:scholar:crossref` SW messages. `references` stays a **local
+  capture record** (§7) — no wire change. *(Deferred: the reader-side
+  Crossref trigger — needs a scholar display surface and a careful
+  read-modify-write of the session stash; the naive write nulls
+  `sourceTabId` and breaks NIP-07 publish routing.)*
 - ✅ **C3** PDF routing + acquisition — background routes PDF tabs to
   the reader's `?pdf=` path (`shared/pdf-detect.js` + Content-Type
   sniff); Import-file fallback; IndexedDB v3 `source_documents` store

src/background/index.js

@@ -28,6 +28,7 @@ import { fetchSubstackPost, fetchSubstackComments } from '../shared/platforms/su
 import { handleScreenshotCapture } from '../shared/screenshot.js';
 import { runSuggestionPass, runAuditPass, runAuditModulePass, getLlmConfig, runLensPass, getLensConfig, runCorpusMapPass, runCorpusReducePass, runHypothesisEdgePass, getCorpusConfig } from '../shared/llm-client.js';
 import { pdfDocumentUrl } from '../shared/pdf-detect.js';
+import { crossrefRequestFor, mapCrossrefWork } from '../shared/crossref.js';
 import { articleAnswersTo } from '../shared/url-identity.js';
 import { Signer } from '../shared/signer.js';
 import { loadFlags, isEnabled } from '../shared/metadata/feature-flags.js';
@@ -763,6 +764,59 @@ chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
         return true; // async
     }
 
+    // Content script → worker: fetch a scholarly rendition's HTML on
+    // the capture pipeline's behalf (Phase 18 C2 tail — the arXiv
+    // handler's ar5iv full-text preference). Lives here because MV3
+    // content scripts have no cross-origin fetch; the SW does, via
+    // host_permissions. HOST-ALLOWLISTED: this message must never
+    // become a generic fetch proxy — only the scholarly hosts the C2
+    // handlers actually consume. credentials:'omit' — public pages,
+    // no cookie riding.
+    if (message.type === 'xray:scholar:fetch') {
+        const SCHOLAR_FETCH_HOSTS = new Set([
+            'ar5iv.labs.arxiv.org', 'ar5iv.org', 'arxiv.org'
+        ]);
+        let target = null;
+        try { target = new URL(String(message.url || '')); } catch (_) { /* rejected below */ }
+        if (!target || target.protocol !== 'https:'
+                || !SCHOLAR_FETCH_HOSTS.has(target.hostname.toLowerCase().replace(/^www\./, ''))) {
+            sendResponse({ ok: false, error: 'host not allowed' });
+            return false;
+        }
+        (async () => {
+            try {
+                const resp = await fetch(target.href, { credentials: 'omit' });
+                if (!resp.ok) { sendResponse({ ok: false, error: 'HTTP ' + resp.status }); return; }
+                sendResponse({ ok: true, html: await resp.text() });
+            } catch (err) {
+                sendResponse({ ok: false, error: err && err.message ? err.message : String(err) });
+            }
+        })();
+        return true; // async
+    }
+
+    // Reader → worker: Crossref DOI lookup (Phase 18 C2 tail).
+    // Accepts a DOI ONLY — the URL is built exclusively by
+    // crossrefRequestFor, which hard-validates the shape; a
+    // caller-supplied URL would make this an open proxy. The patch is
+    // metadata-only and applyCrossref (shared/crossref.js) fills only
+    // fields the page itself did not provide.
+    if (message.type === 'xray:scholar:crossref') {
+        const req = crossrefRequestFor(message.doi);
+        if (!req) { sendResponse({ ok: false, error: 'invalid doi' }); return false; }
+        (async () => {
+            try {
+                const resp = await fetch(req.url, { headers: { Accept: 'application/json' } });
+                if (!resp.ok) { sendResponse({ ok: false, error: 'crossref HTTP ' + resp.status }); return; }
+                const patch = mapCrossrefWork(await resp.json());
+                sendResponse(patch ? { ok: true, patch } : { ok: false, error: 'unmappable response' });
+            } catch (err) {
+                sendResponse({ ok: false, error: err && err.message ? err.message : String(err) });
+            }
+        })();
+        return true; // async
+    }
+
     // Reader → worker: archive-reader flow. Given a URL, query the
     // configured relay pool for kind-30023 events tagged with that
     // URL, pick the most recent, reconstruct the article, and hand

src/shared/content-detector.js

@@ -1,3 +1,5 @@
+import { isPmcPage } from './platforms/pmc.js';
+
 export const ContentDetector = {
     /**
      * Analyze current page and return content type info
@@ -26,6 +28,18 @@ export const ContentDetector = {
         if (hostname.includes('substack.com') || isSubstack()) {
             return { type: 'article', platform: 'substack', confidence: 0.9, metadata: {} };
         }
+        // Phase 18 C2 tail — scholarly platforms. arXiv is exact-host
+        // on purpose: .includes('arxiv.org') would also claim
+        // ar5iv.labs.arxiv.org tabs (a rendition, not the platform).
+        // PMC goes through isPmcPage(url), not a bare hostname check —
+        // gene/pubmed/other NCBI pages must fall through to generic
+        // article detection (wrong platform label otherwise).
+        if (hostname === 'arxiv.org' || hostname === 'www.arxiv.org') {
+            return { type: 'article', platform: 'arxiv', confidence: 0.9, metadata: {} };
+        }
+        if (isPmcPage(url)) {
+            return { type: 'article', platform: 'pmc', confidence: 1.0, metadata: {} };
+        }
         if (hostname.includes('reddit.com')) {
             return { type: 'social_post', platform: 'reddit', confidence: 1.0, metadata: {} };
         }

src/shared/content-extractor.js

@@ -357,6 +357,47 @@ export const ContentExtractor = {
     }
   },
 
+  // Extraction over a FETCHED HTML string (Phase 18 C2 tail) — the
+  // seam the arXiv handler's ar5iv full-text preference runs through.
+  // Same Readability contract as the live-DOM path: `content` stays
+  // HTML at capture time (markdown happens downstream in
+  // event-builder). A <base> element makes the fetched page's relative
+  // image/link URLs resolve against its own origin, and outbound links
+  // are extracted from the SAME parsed body so the `link` tags a later
+  // publish emits describe the adopted text, not the page it replaced
+  // (wire-bound — see arxiv.js's adopt block). `ownHost` is the fetched
+  // URL's host, so a rendition's internal anchors mark `internal` and
+  // never publish. Null on any failure — callers fail open.
+  extractFromHtmlString: (html, baseUrl) => {
+    try {
+      const parsedDoc = new DOMParser().parseFromString(String(html || ''), 'text/html');
+      if (baseUrl && parsedDoc.head) {
+        const base = parsedDoc.createElement('base');
+        base.setAttribute('href', baseUrl);
+        parsedDoc.head.insertBefore(base, parsedDoc.head.firstChild);
+      }
+      const parsed = new Readability(parsedDoc).parse();
+      if (!parsed || !parsed.content || !parsed.textContent) return null;
+      const out = {
+        content: parsed.content,
+        textContent: parsed.textContent,
+        title: parsed.title || ''
+      };
+      try {
+        const bodyDiv = parsedDoc.createElement('div');
+        bodyDiv.innerHTML = parsed.content;
+        let ownHost = '';
+        try { ownHost = new URL(baseUrl).hostname; } catch (_) { /* no host */ }
+        const outbound = ContentExtractor.extractOutboundLinks(bodyDiv, baseUrl, ownHost);
+        out.links = outbound.links;
+        if (outbound.truncated) out.links_truncated = true;
+      } catch (_) { /* links stay absent — arxiv.js nulls them honestly */ }
+      return out;
+    } catch (_) {
+      return null;
+    }
+  },
+
   // Outbound links (docs/NIP_DRAFT.md `link` tags): every
   // http(s) anchor in the extracted body, deduped through the unified
   // normalizer so a link and its tracking-param variant count as ONE