fix(ofac): stop downloading 267 MB per sweep; source always timed out

[YAMRAJ13y]

Summary

The OFAC source downloads 267 MB per sweep — about 25.7 GB/day at the default 15-minute cadence — discards all but 500 characters of it, and never completes inside its 30-second budget. This makes it fetch only the ~128 KB it actually reads.

Why

briefing() issues three full-body requests:

Request	Size	Purpose
SDN.XML	27.5 MB	metadata
SDN_ADVANCED.XML	120.0 MB	metadata
SDN_ADVANCED.XML	120.0 MB	again, for sample entries
	267.5 MB	per sweep

Sizes are from the origin's own Content-Range headers:

$ curl -sIL -r 0-127 '.../exports/SDN.XML'          # -> bytes 0-127/28809039
$ curl -sIL -r 0-127 '.../exports/SDN_ADVANCED.XML' # -> bytes 0-127/125868278

None of it is kept. safeFetch reads the whole body with res.text() and only then truncates: catch { return { rawText: text.slice(0, 500) } }. The comment at the call site —

// The full SDN XML is large; safeFetch will get the first 500 chars

— describes an optimisation that doesn't exist. There is no ranged request and no early abort; the full 267 MB is buffered in memory and thrown away.

So the source never completes. apis/briefing.mjs allows 30s per source. This path needs two parallel 20s fetches followed by a sequential 25s fetch. Every sweep logs:

[Crucix] Sweep complete in 30120ms — 27/29 sources returned data
  errors: [ { name: 'OFAC', error: 'Source OFAC timed out after 30s' } ]

And two of the three downloads were never going to work anyway. SDN_ADVANCED.XML uses a different schema from SDN.XML — no <sdnEntry>, no <Publish_Date>, no <Record_Count>:

<Sanctions ... Version="3">
  <DateOfIssue CalendarTypeID="1"><Year>2026</Year><Month>8</Month><Day>7</Day></DateOfIssue>

so advancedList was always {publishDate: null, entryCount: null, recordCount: null} and parseRecentEntries on it always returned []. 240 MB of the 267 MB was downloaded twice to produce nothing.

Fix

• Request only the first 64 KB with Range: bytes=0-65535. The origin redirects to S3, which advertises Accept-Ranges: bytes and answers 206 Partial Content.
• Bound the read with a streaming reader as well, so an intermediary that ignores Range and starts streaming 120 MB still can't pull it into memory.
• Fetch each list once and reuse the buffer for both metadata and sampling.
• Take sample entries from SDN.XML, which actually contains <sdnEntry>.
• Parse the advanced export's <DateOfIssue> block so its date populates.

Everything the briefing reports lives in the first few KB — the header block carries <Publish_Date>08/07/2026</Publish_Date> and <Record_Count>19199</Record_Count> within the first 300 bytes.

Measured before / after

	before	after
requests	3	2
transferred	267.5 MB	128.0 KB (~2,140× less)
duration	timed out at 30s	5.0 s
sampleEntries	[]	10 entries
advancedList.publishDate	null	2026-08-07
sdnList.recordCount	19199	19199

// node apis/sources/ofac.mjs — on this branch
{
  "source": "OFAC Sanctions",
  "lastUpdated": "08/07/2026",
  "sdnList":      { "publishDate": "08/07/2026", "recordCount": 19199, "dataAvailable": true },
  "advancedList": { "publishDate": "2026-08-07", "dataAvailable": true },
  "sampleEntries": [
    { "uid": "36",  "name": "AEROCARIBBEAN AIRLINES",   "type": "Entity", "programs": ["CUBA"] },
    { "uid": "306", "name": "BANCO NACIONAL DE CUBA",   "type": "Entity", "programs": ["CUBA"] },
    ...
  ]
}

Notes

• No conflict with fix(safeFetch): exponential backoff, HTTP 429 awareness, Retry-After support #121. This does not touch apis/utils/fetch.mjs. safeFetch can't express "read at most N bytes", and adding that to the shared helper would collide with the retry/backoff work in flight, so the bounded read lives in ofac.mjs.
• entryCount now counts entries in the sampled window rather than the whole file, and is commented as such — recordCount (19199, straight from the file header) remains the authoritative total. Previously entryCount counted within a 500-char string, so it was 0 or 1.
• Nothing in dashboard/inject.mjs reads data.sources.OFAC, so no dashboard surface changes.

Scope

• Focused bug fix
• Small UX improvement
• New source
• Dashboard change
• Docs/config change

Validation

# 0 -> 10 sample entries, 30s timeout -> 5s
node apis/sources/ofac.mjs

# instrumented byte count: 2 calls / 131072 bytes / 5.0s
node --input-type=module -e "
const orig = globalThis.fetch; let calls = 0, bytes = 0;
globalThis.fetch = async (u, o) => { calls++; const r = await orig(u, o);
  const rd = r.body.getReader();
  return new Response(new ReadableStream({ async pull(c) {
    const { done, value } = await rd.read();
    if (done) { c.close(); return; } bytes += value.length; c.enqueue(value);
  }}), { status: r.status, headers: r.headers }); };
const { briefing } = await import('./apis/sources/ofac.mjs');
await briefing(); console.log(calls, bytes);
"

# network-failure path returns rather than throwing
node --input-type=module -e "
globalThis.fetch = async () => { throw new Error('boom'); };
const { briefing } = await import('./apis/sources/ofac.mjs');
console.log((await briefing()).lastUpdated);   // -> 'unknown'
"

node --check apis/sources/ofac.mjs
node --test test/*.test.mjs     # 45 pass, 1 skipped, 0 fail — unchanged

Config and Docs

• No new environment variables
• .env.example unchanged — OFAC needs no key
• README.md unchanged