diff --git a/.claude/skills/xray-capture/SKILL.md b/.claude/skills/xray-capture/SKILL.md
index faf6e9e..183ac7a 100644
--- a/.claude/skills/xray-capture/SKILL.md
+++ b/.claude/skills/xray-capture/SKILL.md
@@ -1,6 +1,6 @@
---
name: xray-capture
-description: Capture articles into X-Ray by URL, enumerate an article's outbound-link frontier for corpus expansion, and run a scouting suggest pass. Use when the user asks to capture a URL into X-Ray, capture the links from an article, expand a case corpus, or scout what an article would yield. Requires the claude-in-chrome connector and the X-Ray extension loaded.
+description: Capture articles into X-Ray by URL and enumerate an article's outbound-link frontier for corpus expansion, driving the loaded extension through the claude-in-chrome connector. Use when the user asks to capture a URL into X-Ray, capture the links from an article, or expand a case corpus. Requires the claude-in-chrome connector, the X-Ray extension loaded, and the Capture-automation flag ON.
---
# X-Ray capture — drive the extension, expand the corpus
@@ -12,165 +12,129 @@ never judge, and **nothing model-produced becomes durable X-Ray data
through you** — proposals are accepted by the human in the reader,
or not at all.
+## Connector limits (verified 2026-07-16 — design around these, not against them)
+
+- The connector **cannot navigate to `chrome-extension://` URLs** (it
+ forces an `https://` prefix) and **cannot attach the debugger to
+ extension pages** ("Cannot attach debugger to chrome-extension://
+ pages"). Everything that needed extension-page JS — IDB reads,
+ `chrome.tabs.sendMessage`, `xray:llm:suggest` — is unreachable.
+- Synthetic keys do not fire extension command shortcuts, and OS-level
+ keystrokes into browsers are blocked by the computer-use tier.
+- Therefore the ONLY verb you have against the extension is
+ **navigation of ordinary pages** — which is exactly what the
+ `#xray:capture` marker (Phase 27 K.4, flag-gated) exists for.
+
## Hard rules (the project constitution — these bind every step)
1. **No auto-accept, ever.** Never write entities, claims, links,
- facts, findings, or baselines into `chrome.storage.local` or the
- extension's IndexedDB. The ONLY writes you may cause are the ones
- the extension itself performs in response to its own capture flow
- (`xray:capture` → reader auto-archive). If a suggest pass returns
- proposals, you REPORT them; the human accepts in the reader.
+ facts, findings, or baselines anywhere. The ONLY writes you may
+ cause are the extension's own capture flow (the marker → reader
+ auto-archive).
2. **Never touch publish paths.** No relay messages, no signing.
-3. **The extension's API key stays in its service worker.** Talk to
- the LLM only via the extension's own `xray:llm:*` messages (which
- gate on the user's flags + key). Never read `xray:llm:key`.
-4. **Pace yourself.** Captures run sequentially — one tab, one URL at
- a time, waiting for each load. Suggest passes are paid API calls:
- state the count and get the user's go-ahead in chat before running
- more than one.
-5. **Capture only where directed.** The user names the seed URL(s) or
+3. **Pace yourself.** Captures run sequentially — one URL at a time,
+ waiting for each load and verification before the next.
+4. **Capture only where directed.** The user names the seed URL(s) or
approves the frontier list before you fan out. Skip login-walled or
paywalled pages you cannot render; report them as uncapturable
rather than working around access controls.
-6. **Forensic findings stay out of scope** for suggestion scouting
- (maintainer decision 2026-07-16: not until the attribution fixes
- are proven on real articles).
+5. **Suggest-pass scouting is currently NOT runnable by this skill**
+ (it needs extension-context messaging the connector cannot reach).
+ The reader's own Suggest button is the human's path; recommend it
+ per-article instead of simulating it.
## Preflight (once per session)
-1. Confirm the claude-in-chrome tools are available (load via
- ToolSearch if deferred).
-2. Find the X-Ray extension origin: call `tabs_context`. If no
- `chrome-extension://` tab is open, ask the user to open the X-Ray
- portal (extension icon → or `chrome://extensions` → X-Ray →
- Details → open `src/portal/index.html`), then re-read
- `tabs_context` and note the origin `chrome-extension://`.
- Keep that portal tab open — it is your command surface.
-3. Capability probe: run `javascript_tool` on the portal tab with
- `typeof chrome.tabs.query` — expect `"function"`. If the tool
- cannot execute JS on extension pages, STOP and tell the user this
- skill needs that capability; do not improvise another write path.
-
-## Capture one URL (K.1)
-
-1. Open/navigate a NON-portal tab to the article URL. Wait for load
- (screenshot or `get_page_text` sanity check — a real article body,
- not a consent wall or error page).
-2. From the **portal tab**, run `javascript_tool`:
-
- ```js
- (async () => {
- const tabs = await chrome.tabs.query({ url: '' });
- if (!tabs.length) return 'no-tab';
- const resp = await chrome.tabs.sendMessage(tabs[0].id, { type: 'xray:capture' });
- return resp && resp.ok ? 'capture-sent' : ('failed: ' + JSON.stringify(resp));
- })()
- ```
-
- (`url` patterns need a trailing `*` for query params; the message
- lands in the content script — `src/content/index.js` — which opens
- the reader, and the reader auto-archives on open.)
-3. **PDFs:** skip steps 1–2 and instead open a tab at
+1. Confirm the claude-in-chrome tools are loaded (ToolSearch if
+ deferred), then `tabs_context_mcp {createIfEmpty: true}`.
+2. Ask the user to flip **Options → Advanced → Capture automation** ON
+ (the `#xray:capture` marker is default-off; it gates only the
+ marker). Remind them to turn it back off when the session ends.
+3. Probe once: navigate a group tab to a harmless captureable page
+ with `#xray:capture` appended, wait ~3s, then `javascript_tool`:
+ `document.documentElement.dataset.xrayCaptured` —
+ - `"ok"` → capture fired; a reader tab opened (outside your group —
+ you won't see it; that's expected).
+ - `"flag-off"` → the flag isn't on; ask again.
+ - `undefined` → the extension may not be loaded, or the page blocked
+ the content script; try one reload, then report.
+
+## Capture one URL
+
+1. Navigate a group tab to `#xray:capture` (if the URL already
+ has a fragment, the marker replaces it — note that to the user).
+2. Wait for load + ~5s (the marker waits 1.5s after init to let
+ dynamic pages settle, then opens the reader, which auto-archives).
+ **SPAs (Medium, some news sites) need ~10s** — a capture that fires
+ before the body renders archives an empty shell.
+3. Verify on the SAME tab (ordinary page — attachable):
+ `document.documentElement.dataset.xrayCaptured` → expect `"ok"`.
+ `"error"`/`undefined` → report the URL as failed (paywall? consent
+ wall? script-blocked?); retry at most once.
+4. **A stamp of `"ok"` means the capture FIRED, not that the content
+ was good.** Check `document.title` in the same probe: a title like
+ `"archive.is"`, `"Just a moment…"`, `"Medium"`, or a bare hostname
+ means an interstitial/loading state was live when the capture ran —
+ wait for the real title and re-capture (see the re-capture note
+ below). Batch the title into the verification expression:
+ `({stamp: document.documentElement.dataset.xrayCaptured, title: document.title, len: document.body.innerText.length})`
+ — `len` under ~2000 on a supposed article is another tell.
+5. **Re-capturing:** navigate somewhere else first (`https://example.com`)
+ and back. Going from `` to `#xray:capture` used to be a
+ silent no-op (same-document navigation → no re-init); a `hashchange`
+ listener now covers it, but a full navigation is still the reliable
+ path, and re-capture is safe — the archive keys by URL and keeps the
+ prior version.
+6. Report title (from the tab) and move to the next URL.
+5. **Tab pileup is real:** every capture opens a reader tab the
+ connector cannot close (they're outside the group). Tell the user
+ at the end how many reader tabs they'll find and that each shows a
+ captured article ready for claim extraction.
+6. **PDFs:** the reader's `?pdf=` path is an extension-page URL the
+ connector cannot open. Give the user the exact
`chrome-extension:///src/reader/index.html?pdf=`
- — the reader's tabless PDF path captures directly.
-4. Verify the archive landed — from the portal tab:
-
- ```js
- (async () => {
- const db = await new Promise((res, rej) => {
- const r = indexedDB.open('xray-archive');
- r.onsuccess = () => res(r.result); r.onerror = () => rej(r.error);
- });
- const recs = await new Promise((res, rej) => {
- const tx = db.transaction('articles').objectStore('articles').getAll();
- tx.onsuccess = () => res(tx.result); tx.onerror = () => rej(tx.error);
- });
- db.close();
- const hit = recs.find((r) => r.url && r.url.includes(''));
- return hit ? { ok: true, title: hit.article && hit.article.title,
- links: (hit.article && hit.article.links || []).length }
- : { ok: false, captured: recs.length };
- })()
- ```
-
- READ-ONLY: this is verification, never a write. If verification
- fails, report it (paywall? consent wall? CSP?) and move on — do not
- retry more than once.
-5. Close the article tab if you opened it; report `title` + link
- count to the user.
-
-## Enumerate the frontier (K.1)
-
-From the portal tab, read the seed article's outbound links and
-subtract what's already captured:
-
-```js
-(async () => {
- const db = await new Promise((res, rej) => {
- const r = indexedDB.open('xray-archive');
- r.onsuccess = () => res(r.result); r.onerror = () => rej(r.error);
- });
- const recs = await new Promise((res, rej) => {
- const tx = db.transaction('articles').objectStore('articles').getAll();
- tx.onsuccess = () => res(tx.result); tx.onerror = () => rej(tx.error);
- });
- db.close();
- const seed = recs.find((r) => r.url && r.url.includes(''));
- if (!seed) return 'seed-not-captured';
- const have = new Set(recs.map((r) => r.url));
- const links = ((seed.article && seed.article.links) || [])
- .filter((l) => l && l.url && !l.internal)
- .filter((l) => !have.has(l.url));
- return { frontier: links.map((l) => ({ url: l.url, text: l.text, count: l.count })),
- truncated: !!(seed.article && seed.article.links_truncated) };
-})()
-```
-
-Present the frontier as a numbered list (url + anchor text + cite
-count). **Wait for the user to pick** which to capture (or confirm
-"all"), then loop the capture procedure over the picks, one at a
-time, reporting progress. Notes: `internal` is same-host-approximate;
-`links_truncated` means the capture capped at 100 links — say so.
-
-## Scouting suggest pass (K.2 — optional, costs API tokens)
-
-Purpose: tell the researcher what a captured article would yield
-BEFORE they invest reader time. It does NOT feed the reader's review
-panel — accepting still happens there (and the reader's own Suggest
-run is a second paid pass; say so when recommending it).
-
-1. The pass scopes itself to the USER'S stored suggestion kinds
- (`xray:llm:suggest_kinds` — the worker reads it; the request cannot
- override it, by design). Read it first (portal tab,
- `chrome.storage.local.get`) so you can tell the user what the pass
- will cover. If `findings` or `baselines` are enabled there, note
- that per rule 6 you will not elaborate those proposals in your
- summary — review them in the reader.
-2. Get the article text from the archive record
- (`record.article.markdown` — fall back to `textContent` if absent).
-3. From the portal tab:
-
- ```js
- (async () => new Promise((res) => chrome.runtime.sendMessage({
- type: 'xray:llm:suggest',
- request: { articleText: , articleUrl: , articleTitle: }
- }, res)))()
- ```
-
- Gates (llmAssist flag + key + at-least-one-kind) are enforced
- worker-side; if the response is `{ok:false}`, relay its error
- verbatim (it names the Options toggle to flip).
-4. Report a scouting summary in chat: counts per kind + the two or
- three most load-bearing claims verbatim (skip findings/baselines
- per rule 6). Recommend which articles deserve a reader pass. Do
- not store anything.
+ link to open by hand instead.
+
+## Enumerate the frontier
+
+Extension IDB is unreachable, so the frontier comes from a **workspace
+backup export** the user hands you:
+
+1. Ask the user: Options → Advanced → Workspace → download backup
+ (source bytes not needed), and give you the file path (usually
+ `~/Downloads/…json`).
+2. Parse it in node (it is one JSON object):
+ `databases['xray-archive'].stores.articles` (explore the exact
+ nesting — `format` and `exportedAt` sit at the top level) → each
+ record has `url`, `articleHash`, and `article.links` (shape
+ `{url, text, count, internal}`, capped at 100 with
+ `article.links_truncated`).
+3. Frontier = every non-`internal` link URL across the seed records,
+ minus every `url` already in the archive, deduped, with citation
+ counts and which seeds cite it. Disclose `links_truncated` seeds.
+4. Present the frontier as a numbered pick list sorted by citation
+ count. **Wait for the user to pick** (or confirm "all"), then loop
+ the capture procedure sequentially over the picks.
+5. The export is a snapshot — captures made after it won't be in it.
+ Re-request an export if you need a fresh frontier; don't guess.
## Failure modes
| Symptom | Meaning | Do |
|---|---|---|
-| `javascript_tool` refuses extension pages | capability missing | Stop; tell the user (preflight step 3) |
-| `no-tab` from capture | URL pattern mismatch | Add `*` for query strings; re-query |
-| `sendMessage` throws "no receiving end" | content script not injected (chrome:// page, PDF viewer, race) | PDFs → the `?pdf=` reader path; else reload the tab once |
-| Verification finds no record | paywall/consent wall/CSP ate the capture | Report as uncapturable; do not bypass |
-| Suggest returns `ok:false` | flag off or no key | Relay the error; the user flips it in Options |
+| `dataset.xrayCaptured === "flag-off"` | Capture automation off | Ask the user to enable it in Options → Advanced |
+| `dataset.xrayCaptured` undefined after reload | content script absent (blocked page, chrome://, PDF viewer) | Report uncapturable; PDFs → hand the user the `?pdf=` reader link |
+| `"error"` stamp | the marker branch threw | Read the console (`pattern: "X-Ray"`) — it logs the reason verbatim |
+| Stamp `"ok"` but the title is `archive.is` / `Just a moment…` / `Medium` | captured an interstitial, not the article | Wait for the real title, then re-capture via a full navigation |
+| Stamp never appears on one specific host | the content script may be throwing before the marker runs | Console first, guess never — this is how the archive.is `` bug was found (JOURNAL 2026-07-17) |
+| Navigate mangles the URL | you passed a `chrome-extension://` URL | Don't — ordinary pages only (see Connector limits) |
+| Frontier stale | export predates recent captures | Ask for a fresh export |
+
+## Reporting back
+
+Give the user a per-URL table (title + outcome), not just a count.
+State plainly: how many reader tabs are now open (one per capture, all
+outside the group — you cannot close them), which URLs failed and why,
+and which captured content you are UNSURE about. An honest "this one
+may have caught a loading page" is worth more than a green checkmark
+they later discover was empty.
diff --git a/docs/JOURNAL.md b/docs/JOURNAL.md
index 486103c..56468bc 100644
--- a/docs/JOURNAL.md
+++ b/docs/JOURNAL.md
@@ -19,6 +19,46 @@ or files, and the "so-what" for future readers.
---
+## 2026-07-17 — A cosmetic replaceState aborted the capture it preceded (27 K.4)
+
+Tags: `bug`, `external`.
+
+Found in the first live agent-driven capture run, on `archive.is`
+(several COVID-corpus frontier links are archive.is snapshots of
+paywalled reporting). The `#xray:capture` marker fired once, then
+never again on that host:
+
+```
+SecurityError: Failed to execute 'replaceState' on 'History':
+A history state object with URL 'https://d8w6fmknl1l6hl.archive.is/7gYpy'
+cannot be created in a document with origin 'https://archive.is'
+```
+
+archive.is serves snapshot content under a **randomized subdomain** and
+sets a `` to it. `history.replaceState(null, '', pathname +
+search)` resolves its RELATIVE argument against the document's **base
+URI**, not its origin — so the rewrite targeted another host and threw.
+The marker-strip sat before `UI.openReader()` inside the same `try`, so
+a cosmetic address-bar tidy killed the capture. It "worked" on the very
+first hit only because the base tag hadn't been applied yet — which is
+also why that capture archived the interstitial rather than the article.
+
+Two fixes, both worth keeping in mind for any content-script URL work:
+build the replacement **absolute from `location.origin`** (never
+relative — `` is another site's to control), and give
+best-effort cosmetics **their own try/catch** so they can never abort
+the load-bearing work behind them. Also fixed alongside: a hash-only
+navigation (`` → `#xray:capture`) is a same-document
+navigation, so `init()` never re-runs and the marker silently no-oped;
+a `hashchange` listener now covers it.
+
+No unit test would have caught either — both need a real page with a
+cross-origin base tag. The lesson generalizes past this slice: a
+try-block that wraps a nice-to-have around a must-have inherits the
+nice-to-have's failure modes.
+
+---
+
## 2026-07-16 — The scope question never reached the LLM; proposals were never asked for (27 S.1/S.2)
Tags: `bug`, `design`.
diff --git a/src/content/index.js b/src/content/index.js
index deedc71..c6faa85 100644
--- a/src/content/index.js
+++ b/src/content/index.js
@@ -13,6 +13,7 @@ import { Signer } from '../shared/signer.js';
import { NIP07Client } from './nip07-client.js';
import { UI } from './ui.js';
import { installBufferListener, configureInterceptor } from '../shared/api-hook-buffer.js';
+import { loadFlags, isEnabled } from '../shared/metadata/feature-flags.js';
async function init() {
// Initialize storage (migrates from any legacy GM storage if present).
@@ -130,6 +131,78 @@ async function init() {
}
Utils.log('Initialization complete');
+
+ // Phase 27 K.4 — the automation capture marker. A driving agent
+ // (the xray-capture skill) can neither reach extension pages nor
+ // fire the command shortcut through the browser connector, so
+ // NAVIGATION is its only verb: opening `#xray:capture`
+ // triggers the same capture the toolbar button would. Flag-gated,
+ // default OFF (Options → Advanced → Capture automation). The
+ // marker is navigation plumbing, not page identity — it is
+ // stripped before capture (the URL normalizer would drop it from
+ // stored URLs regardless), and the outcome is stamped on the DOM
+ // (`data-xray-captured`) so the driver can verify from ordinary
+ // page context. Captures the page, nothing more.
+ await maybeAutomationCapture();
+ // A hash-only navigation (same page, marker appended) does NOT
+ // re-run init — the driver's capture would silently no-op. Listen
+ // so the marker works from an already-loaded page too.
+ window.addEventListener('hashchange', () => {
+ maybeAutomationCapture().catch((err) =>
+ Utils.error('Automation capture marker failed:', err));
+ });
+}
+
+/**
+ * The `#xray:capture` automation marker (Phase 27 K.4). Flag-gated,
+ * default OFF. Triggers the same capture the toolbar button does, and
+ * stamps the outcome on the DOM so a driving agent can verify from
+ * ordinary page context.
+ */
+async function maybeAutomationCapture() {
+ if (window.location.hash !== '#xray:capture') return;
+ try {
+ await loadFlags();
+ if (!isEnabled('captureAutomation')) {
+ document.documentElement.dataset.xrayCaptured = 'flag-off';
+ Utils.log('#xray:capture marker present but captureAutomation is off');
+ return;
+ }
+ stripCaptureMarker();
+ // Give late-loading pages the settle time a human clicking the
+ // toolbar would naturally allow.
+ setTimeout(() => {
+ try {
+ UI.openReader();
+ document.documentElement.dataset.xrayCaptured = 'ok';
+ } catch (err) {
+ document.documentElement.dataset.xrayCaptured = 'error';
+ Utils.error('Automation capture failed:', err);
+ }
+ }, 1500);
+ } catch (err) {
+ document.documentElement.dataset.xrayCaptured = 'error';
+ Utils.error('Automation capture marker failed:', err);
+ }
+}
+
+/**
+ * Drop the marker from the address bar. Cosmetic ONLY — the URL
+ * normalizer strips fragments from stored URLs regardless — so it must
+ * never block the capture it precedes. The URL is rebuilt ABSOLUTE
+ * from the origin: a relative argument resolves against the document's
+ * base URI, and a page whose `` points at another host
+ * (archive.is serves content under a randomized subdomain) makes
+ * replaceState throw SecurityError — which previously aborted the
+ * whole capture.
+ */
+function stripCaptureMarker() {
+ try {
+ history.replaceState(null, '',
+ window.location.origin + window.location.pathname + window.location.search);
+ } catch (err) {
+ Utils.log('Capture marker left in the address bar (harmless):', err && err.message);
+ }
}
/**
diff --git a/src/options/index.js b/src/options/index.js
index e91f1f0..00b9edb 100644
--- a/src/options/index.js
+++ b/src/options/index.js
@@ -757,6 +757,7 @@ async function loadAdvanced() {
// Case synthesis (Phase 20.4) — requires llmAssist + the key on top.
document.getElementById('pref-case-synthesis').checked = isEnabled('caseSynthesis');
+ document.getElementById('pref-capture-automation').checked = isEnabled('captureAutomation');
// LLM assist (Phase 14.5). The flag lives in feature-flags; the key
// + model live under their own chrome.storage.local keys. We never
@@ -871,6 +872,10 @@ async function saveAdvanced() {
const synthOn = document.getElementById('pref-case-synthesis').checked;
await setOverride('caseSynthesis', synthOn ? true : null);
+ // Capture automation (Phase 27 K.4).
+ const captureAutoOn = document.getElementById('pref-capture-automation').checked;
+ await setOverride('captureAutomation', captureAutoOn ? true : null);
+
// LLM assist: flag + model preference always; the key only when the
// user typed a new one (blank leaves the saved key untouched).
const llmOn = document.getElementById('pref-llm-assist').checked;
diff --git a/src/options/options.html b/src/options/options.html
index 795ec32..ba99ba3 100644
--- a/src/options/options.html
+++ b/src/options/options.html
@@ -476,6 +476,23 @@
Moral lens
browser session.
+
+
Capture automation
+
+
+ Off by default. When on, opening any page whose address ends in
+ #xray:capture captures it and opens the reader —
+ exactly as the toolbar button would. This exists so a driving
+ agent (the xray-capture skill) can queue captures by
+ navigation alone; the marker is stripped before capture and never
+ reaches the stored URL. It captures pages, nothing more: no
+ accepting, no publishing. Leave off unless an agent session is
+ driving.
+