feat(photo-exif): swap face stack from tfjs-node to onnxruntime-node

connectors/photo-exif/.env.example

@@ -35,13 +35,19 @@ PHOTO_ROOT=/path/to/your/photos
 # PHOTO_EXIF_CAPTION_STATE_PATH=/home/you/.life-context/photo-exif-captions.json
 
 # --- face-worker.js (#53): local, offline face detection -> `pictured` entity hints ---
-# Directory holding the face-api model weights (ssdMobilenetv1 + faceLandmark68Net +
-# faceRecognitionNet). One-time download, fully offline after. Required to run `node face-worker.js`
-# (scan). See README "Face detection" for the download step.
+# Directory holding the ONNX face models (#268 — SCRFD detection + ArcFace recognition, buffalo_l
+# set: det_10g.onnx + w600k_r50.onnx). One-time download, fully offline after. Required to run
+# `node face-worker.js` (scan). See README "Face detection" for the download step.
 # FACE_MODELS_PATH=/home/you/.life-context/face-models
+# Model filenames within FACE_MODELS_PATH, if you use different ones than buffalo_l's defaults.
+# FACE_DET_MODEL=det_10g.onnx
+# FACE_REC_MODEL=w600k_r50.onnx
 
 # Euclidean distance below which two face descriptors are treated as the same person (cluster).
-# FACE_MATCH_THRESHOLD=0.6
+# Descriptors are L2-normalized 512-d ArcFace embeddings (#268): for unit vectors,
+# euclidean_d = sqrt(2*(1-cosine)), so raise this to merge more aggressively, lower to split more.
+# NOT comparable to the pre-#268 face-api value (0.6 there was a 128-d, non-normalized space).
+# FACE_MATCH_THRESHOLD=1.0
 # Confidence on emitted `pictured` hints (core caps name-type aliases at 0.9 regardless).
 # FACE_HINT_CONFIDENCE=0.6
 # Delay between photos during a scan pass (ms); 0 = no throttle.
@@ -53,9 +59,10 @@ PHOTO_ROOT=/path/to/your/photos
 # `node face-worker.js suggest-labels` (#84): distance threshold for matching a contact's own
 # preserved photo (core's CONTACTS_RAW_DIR, from vCard PHOTO import) against an unlabeled
 # cluster centroid. Separate from FACE_MATCH_THRESHOLD — a posed contact photo and a candid
-# camera-roll photo differ enough that cross-source matching may need a different threshold.
-# Defaults to FACE_MATCH_THRESHOLD if unset. Only ever PRINTS suggestions — never writes a
-# label. Requires this connector's process to be able to read the raw_path LifeContext returns
-# (same machine or a shared/synced volume as core's CONTACTS_RAW_DIR — there is no endpoint to
-# fetch the bytes over HTTP). See README "Face detection" for the suggest-labels workflow.
-# FACE_SEED_THRESHOLD=0.6
+# camera-roll photo differ enough that cross-source matching may need a different (looser)
+# threshold; defaults to 1.15 (~= cosine 0.34) rather than FACE_MATCH_THRESHOLD. Only ever PRINTS
+# suggestions — never writes a label. Requires this connector's process to be able to read the
+# raw_path LifeContext returns (same machine or a shared/synced volume as core's
+# CONTACTS_RAW_DIR — there is no endpoint to fetch the bytes over HTTP). See README "Face
+# detection" for the suggest-labels workflow.
+# FACE_SEED_THRESHOLD=1.15

connectors/photo-exif/README.md

@@ -40,8 +40,8 @@ The roadmap flags this as an open question ("worker lives with core or alongside
 
 A third, local-first enrichment pass that links photos to *who is in them* via the contract's `pictured` `entity_hints` role (doc 04 §4). Detection runs entirely on-device — no cloud face API (Prime Directive: local-first).
 
-1. Detects faces in each photo (local `@vladmandic/face-api` + `@tensorflow/tfjs-node`, models loaded from `FACE_MODELS_PATH`).
-2. Clusters the 128-d face descriptors into anonymous groups by nearest-centroid (`FACE_MATCH_THRESHOLD`). **Descriptors never leave the machine** — they live only in the local clusters file; doc 04 §11 forbids connectors sending embeddings.
+1. Detects faces in each photo (local `onnxruntime-node` + `sharp` — SCRFD detection + ArcFace recognition, buffalo_l ONNX models loaded from `FACE_MODELS_PATH`; #268 swapped this off `@tensorflow/tfjs-node`/`@vladmandic/face-api`/`canvas`, which have no prebuilt Windows binary for current Node and need a VS C++ workload to build from source).
+2. Clusters the 512-d, L2-normalized ArcFace face descriptors into anonymous groups by nearest-centroid (`FACE_MATCH_THRESHOLD`). **Descriptors never leave the machine** — they live only in the local clusters file; doc 04 §11 forbids connectors sending embeddings.
 3. Records `extra.faces_detected` on every scanned photo. A photo whose faces are all in *unlabeled* clusters gets **no** `pictured` hint (its `text_repr` is re-sent as base + caption, reconstructed from the caption cache, so captioning is preserved, not extended).
 4. Once you name a cluster (`label`), every photo containing it upserts `entity_hints: [{alias, alias_type:"name", role:"pictured", confidence}]` and a `text_repr` with a "Pictured: …" sentence appended to the base + caption text.
 
@@ -65,15 +65,15 @@ node face-worker.js suggest-labels
 # photo-exif: suggest-labels — checked 18 contact photo(s), 1 cluster(s) suggested
 ```
 
-**Requires this connector's process to be able to read the file path LifeContext core returns** (`raw_path`, under core's `CONTACTS_RAW_DIR`) — i.e., this connector and core must share a filesystem (same machine, or a mounted/synced volume). There is no endpoint to fetch the raw bytes over HTTP; a `raw_path` this process can't read is skipped and logged, not fatal to the run. If `CONTACTS_RAW_DIR` is a relative path in core's `.env`, set it to an **absolute** path there for reliable resolution — a relative path resolves against whatever directory core's own process happened to start from, which this connector has no way to know. Already-labeled clusters are never re-suggested, and a reference photo with zero or multiple detected faces is skipped as ambiguous. Tune the match distance with `FACE_SEED_THRESHOLD` (defaults to `FACE_MATCH_THRESHOLD`) — see `.env.example`.
+**Requires this connector's process to be able to read the file path LifeContext core returns** (`raw_path`, under core's `CONTACTS_RAW_DIR`) — i.e., this connector and core must share a filesystem (same machine, or a mounted/synced volume). There is no endpoint to fetch the raw bytes over HTTP; a `raw_path` this process can't read is skipped and logged, not fatal to the run. If `CONTACTS_RAW_DIR` is a relative path in core's `.env`, set it to an **absolute** path there for reliable resolution — a relative path resolves against whatever directory core's own process happened to start from, which this connector has no way to know. Already-labeled clusters are never re-suggested, and a reference photo with zero or multiple detected faces is skipped as ambiguous. Tune the match distance with `FACE_SEED_THRESHOLD` (defaults to `1.15`, looser than `FACE_MATCH_THRESHOLD`) — see `.env.example`.
 
 ## Setup
 
 1. `cp .env.example .env` and fill in `LIFECONTEXT_URL` / `LIFECONTEXT_API_KEY` / `PHOTO_ROOT`.
-2. `npm install` (real dependency: `exifr`; the face worker additionally needs `@vladmandic/face-api`, `@tensorflow/tfjs-node`, and `canvas` — native modules, only required if you run `face-worker.js`).
+2. `npm install` (real dependency: `exifr`; the face worker additionally needs `onnxruntime-node` and `sharp` — both ship prebuilt binaries, no native compile step, only required if you run `face-worker.js`).
 3. Backfill: `node scan.js`.
 4. Optionally, once you have a vision model pulled in Ollama (`ollama pull llava`): `node caption-worker.js`.
-5. Optionally, for face → contact linking: download the face-api model weights into `FACE_MODELS_PATH` (the `ssdMobilenetv1`, `faceLandmark68Net`, and `faceRecognitionNet` weight files from the `@vladmandic/face-api` model repo — one-time, offline after), then `node face-worker.js`, browse clusters with `export-thumbnails`, and `label` the ones you recognize.
+5. Optionally, for face → contact linking: download the buffalo_l ONNX model pair into `FACE_MODELS_PATH` — `det_10g.onnx` (SCRFD detection) and `w600k_r50.onnx` (ArcFace recognition), from the InsightFace model zoo, one-time and fully offline after — then `node face-worker.js`, browse clusters with `export-thumbnails`, and `label` the ones you recognize.
 
 ### Prep a Takeout download (Windows) — `prep-takeout.ps1`
 
@@ -141,8 +141,8 @@ On Windows, use Task Scheduler with a "Daily, 1:00 AM" trigger running `node cap
 
 - **Clustering is approximate** — nearest-centroid with a fixed Euclidean threshold, not a trained recognizer. Expect occasional split clusters (same person, two buckets) or, rarely, a merged one; `export-thumbnails` + re-`label` is the correction path.
 - **`export-thumbnails` writes the sample *image*, not a tight face crop** — a real crop would pull in the native image-processing stack at export time. Per-face bounding boxes aren't persisted today (the clusters file stores only centroid/count/label/sample), so a future cropped version would need to re-detect the sample image or start persisting boxes.
-- **The ML stack is unverified in this repo's CI** — `test.mjs` covers the full clustering/label/ingest pipeline with an injected fixture detector (no models), so the wire behavior is tested, but real `face-api` detection quality/latency is a manual, on-device concern (same posture as the VLM caption worker).
-- **Native dependencies** — `@tensorflow/tfjs-node` and `canvas` are native modules; they're only loaded (via dynamic import) when you actually run a scan, so the other two scripts and the test suite need none of them.
+- **The ML stack is unverified in this repo's CI** — `test.mjs` covers the full clustering/label/ingest pipeline with an injected fixture detector (no models), so the wire behavior is tested, but real ONNX detection/recognition quality/latency is a manual, on-device concern (same posture as the VLM caption worker). The pure decode/alignment math (`lib/face-align.js`) — SCRFD box/keypoint decode, NMS, and the Umeyama similarity + bilinear warp used to align each face crop — **is** unit-tested without any model.
+- **Native dependencies** — `onnxruntime-node` and `sharp` ship prebuilt binaries for current Node/Windows (no VS C++ workload, unlike the tfjs-node stack this replaced, #268); they're only loaded (via dynamic import) when you actually run a scan, so the other two scripts and the test suite need none of them.
 - **`suggest-labels` requires a shared filesystem with core (#84)** — it reads `raw_path` values LifeContext core returns; there's no HTTP endpoint to fetch those bytes, so this connector must be able to read the same disk (or a mounted/synced volume) core wrote contact photos to. Cross-machine setups (this connector on a different host than core, e.g. the Mac Mini/Windows-server iMessage topology) aren't supported for this command specifically — everything else in this connector works unchanged in that topology.
 
 ## Files
@@ -155,6 +155,7 @@ On Windows, use Task Scheduler with a "Daily, 1:00 AM" trigger running `node cap
 - `lib/describe.js` — shared EXIF + Takeout-sidecar description logic (`describePhoto`/`readSidecar`/`sidecarPathFor`), used by every script so they can never drift
 - `lib/caption-cache.js` — caption state (relPath→text map) + `currentTextRepr`, shared by caption + face workers
 - `lib/face-cluster.js` — pure, IO-free descriptor clustering (euclidean, nearest-centroid, (de)serialization)
+- `lib/face-align.js` — pure, IO-free SCRFD decode (distance2bbox/kps, anchor centers, NMS) + Umeyama similarity/bilinear warp used to align each 112x112 ArcFace input crop (#268)
 - `lib/face-detect.js` — lazy ML-model detector + the test fixture detector
 - `test.mjs` — `node --test` suite
 - `.env.example` — copy to `.env`

connectors/photo-exif/face-worker.js

@@ -38,8 +38,11 @@ const FACE_STATE_PATH = process.env.PHOTO_EXIF_FACE_STATE_PATH || HOME_STATE('ph
 const CLUSTERS_PATH = process.env.PHOTO_EXIF_FACE_CLUSTERS_PATH || HOME_STATE('photo-exif-face-clusters.json');
 const CAPTION_STATE_PATH = process.env.PHOTO_EXIF_CAPTION_STATE_PATH || HOME_STATE('photo-exif-captions.json');
 // Number()-with-isFinite so an explicit 0 isn't overridden by a `|| default` (0 is falsy).
+// Euclidean thresholds for L2-normalized 512-d ArcFace descriptors (#268) — NOT comparable to the
+// old 128-d face-api value (0.6). For unit vectors, euclidean d = sqrt(2*(1-cos)), so raising the
+// threshold merges more aggressively, lowering it splits more aggressively.
 const matchRaw = Number(process.env.FACE_MATCH_THRESHOLD);
-const FACE_MATCH_THRESHOLD = Number.isFinite(matchRaw) ? matchRaw : 0.6;
+const FACE_MATCH_THRESHOLD = Number.isFinite(matchRaw) ? matchRaw : 1.0; // ~= cosine 0.50
 const confRaw = Number(process.env.FACE_HINT_CONFIDENCE);
 const FACE_HINT_CONFIDENCE = Number.isFinite(confRaw) ? confRaw : 0.6;
 const throttleRaw = Number(process.env.FACE_THROTTLE_MS);
@@ -50,7 +53,7 @@ const FACE_THROTTLE_MS = Number.isFinite(throttleRaw) ? throttleRaw : 0;
 // cross-source matching may warrant a different threshold. Defaults to the same value so
 // behavior is a pure addition until tuned.
 const seedRaw = Number(process.env.FACE_SEED_THRESHOLD);
-const FACE_SEED_THRESHOLD = Number.isFinite(seedRaw) ? seedRaw : FACE_MATCH_THRESHOLD;
+const FACE_SEED_THRESHOLD = Number.isFinite(seedRaw) ? seedRaw : 1.15; // ~= cosine 0.34, looser than match
 
 const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
 const requireApiKey = () => {