Skip to content

VERIFY.md

Latest commit

 

History

History
206 lines (158 loc) · 8.89 KB

VERIFY.md

File metadata and controls

206 lines (158 loc) · 8.89 KB
Raw

Verify irlume's claims yourself

irlume makes measurable claims: anti-spoof numbers, encrypted storage, error rates. None of them ask for your trust: each maps to something you can run on your own machine and check against the docs. This page collects the reproductions, easiest first.

If a claim in the README or docs has no reproduction path here and you think it should, open an issue or a discussion.

1. Your face is stored encrypted, never as an image · ~2 min

Claim: templates are 512-D embeddings (never images); the primary store is AES-256-GCM encrypted under a TPM-sealed key, root-only at rest.

After enrolling a face, look at the stored profile:

sudo head -c 200 /var/lib/irlume/*.json
sudo stat -c '%a %U:%G' /var/lib/irlume/*.json
sudo ls /var/lib/irlume/cameras/ 2>/dev/null || true
sudo head -c 200 /var/lib/irlume/cameras/*.json 2>/dev/null || true

New encrypted writes use version 3, with a public key_id and an enc ciphertext field; legacy encrypted version 2 remains readable. Expect mode 600 root:root. The biometric data is an encrypted blob, not readable embeddings, and no image is ever written. If you enrolled a secondary camera (0.13.0+), its cameras/<user>.json store uses the same encryption as the primary (upgraded on its next write after updating from 0.13.0), so it also reads as an opaque enc blob - never as fields and floats.

On a machine without a TPM the daemon stores the same embeddings root-only but unencrypted. The TUI says so on the Keyring tab, and the cross-machine disk-theft test (copy the files to a second machine with its own TPM: they do not decrypt) is written up in SECURITY_AT_REST.md.

2. The per-camera anti-spoof numbers are real readings · ~2 min

Claim: the moiré values in cross-distro/2026-07-01-arch-ubuntu-survey.md (≈9–13 on one camera, ≈18–27 on another) are real readings from the code.

Turn on the daemon's diagnostic tracing:

sudo irlume logs debug on

If your camera is an IR (Windows Hello) one, also force the RGB path the moiré cue lives on (it drops to the RGB / convenience tier for the test):

sudo systemctl edit irlumed     # add:  [Service]  Environment=IRLUME_FORCE_NO_IR=1
sudo systemctl restart irlumed

Run a check with a lit face:

irlume identify
irlume logs | grep moire

You will see your own camera's score, e.g. rgb-only cues: ... moire 10 .... Different camera modules read different values; that per-camera spread is the whole reason the threshold is tunable per camera. Put things back when done:

sudo systemctl revert irlumed && sudo systemctl restart irlumed

3. It builds and passes its tests · ~5 min

Claim: the workspace is real, tested Rust.

git clone https://github.com/archledger/irlume
cd irlume && bash scripts/fetch-models.sh
cargo test --workspace

Around 2,700 tests pass; the ones that need camera or TPM hardware are marked ignored.

4. The liveness gate is self-tested against ISO/IEC 30107-3 · deeper (needs your own spoofs)

Claim: the presentation-attack self-test methodology and results in PAD_SELFTEST.md.

The tooling is dev-gated (IRLUME_DEV=1) and opens the camera directly. A live bona-fide capture works with no extra configuration:

IRLUME_DEV=1 irlume padcapture --species live --kind bonafide \
  --det /usr/share/irlume/models/face_detection_yunet_2023mar.onnx \
  --out pad.jsonl --n 5

To measure attack resistance you make the spoofs yourself (a matte print, a glossy print, a phone or tablet replay) and capture each as its own species:

IRLUME_DEV=1 irlume padcapture --species phone_replay --kind attack \
  --det /usr/share/irlume/models/face_detection_yunet_2023mar.onnx --out pad.jsonl
IRLUME_DEV=1 irlume padreport --in pad.jsonl --md report.md

The report gives per-species APCER / BPCER with exact-binomial confidence intervals; a 0% point estimate on a small sample does not prove 0% (read the upper CI bound). This is the same tooling that caught the life-size glossy-vinyl breach documented in the threat model, including the failures, not just the wins.

5. The demographic FAR numbers · deeper (needs a face dataset)

Claim: the real-face False Accept Rate in FAIRNESS.md.

Download LFW (Labeled Faces in the Wild), for example the Kaggle jessicali9530/lfw-dataset, the source used here. Any copy with the standard Person_Name_NNNN.jpg filenames works; the figure is stable to the variant. Then run the impostor benchmark against the bundled recognizer:

IRLUME_DEV=1 irlume irbench --lfw --impostor-only --dir <lfw-image-dir> \
  --det /usr/share/irlume/models/face_detection_yunet_2023mar.onnx \
  --model /usr/share/irlume/models/glintr100.onnx

On the full set (13,233 images, ~87M PAIRS) you get an all-pairs false-match rate of 2.3×10⁻³ at threshold 0.50. Read that as an UPPER BOUND, not an impostor rate: the mode that produces it pairs every image with every other and does not group by identity, so LFW's several-images-per-person means genuine pairs are counted in the numerator too. A true impostor rate would be lower by an amount this measurement does not establish. irbench now refuses --impostor-only --lfw together for exactly this reason. Embedding all 13k images takes roughly an hour on a CPU; --max-images N bounds it for a quick look, at the cost of a noisier estimate. FAIRNESS.md has the full protocol and the per-group FairFace table.

6. The model-accuracy numbers · benchmarks/

Claim: the recognition, detection-cascade, and landmark figures in the README, models/README.md, the CHANGELOG, and the release notes (LFW 99.03%, the IR-adapter overfit that justified its removal, the cascade's 76.9% → 98.5% outdoor rescue, the 478-point mesh's 28% eye-NME gain).

The scripts that produced every one of these live in ../benchmarks/, and the raw result files are committed beside them (results-*.json / .log); read them directly, or reproduce from scratch on the public datasets (LFW, CBSR NIR, Oulu-CASIA NIR, Tufts Face); where each copy was obtained is in ../benchmarks/README.md, and it is not always the canonical page. Datasets, exact protocols, the runtime, and the honest caveats (small outdoor sample; InsightFace's non-commercial recognizer beats the permissive one irlume ships) are all in ../benchmarks/README.md.

7. The release you downloaded is the one that was published · ~2 min

Claim: release packages match the maintainer's signed checksum manifest. The release key is F350 5339 8E3C 80FE 2089 1B82 C10B 8492 BD7F 30C6, committed at .github/release-signing-key.asc. From a reviewed checkout, verify a directory containing all downloaded assets:

python3 scripts/verify-release-assets.py /path/to/downloaded-assets

This requires Python 3.11+, GnuPG, dpkg-deb, tar and zstd (also rpm when an RPM is present). It verifies the signer, exact checksums, coverage and package structure without installing packages. Both Debian and Arch packages are required; missing assets fail. For an explicitly selected historical Debian-only release, use --require deb; that exception is not used by the current release workflows.

The provenance workflow separately attests these verified package digests. It includes .deb, .pkg.tar.zst, and any signed irlume-selinux-*.rpm assets, and uses the stable filename multiple.intoto.jsonl. Check each package with slsa-verifier:

slsa-verifier verify-artifact irlume_*.deb irlume-*.pkg.tar.zst \
  --provenance-path multiple.intoto.jsonl \
  --source-uri github.com/archledger/irlume

If a SELinux RPM is published, include its actual filename too. Older releases may have per-package provenance names: v0.11.3 has irlume_0.11.3_amd64.deb.intoto.jsonl, covering only the Debian package. Do not infer Arch coverage from a successful Debian verification.

The packages are built outside CI. This attestation records digests supplied to the generator; it does not prove a reproducible or isolated package build. .github/workflows/verify-release.yml strictly checks version releases, and .github/workflows/slsa-provenance.yml checks the same complete signed inputs before generating provenance and confirming its upload. Neither workflow treats an incomplete release as verified. Follow release finalization.

A note on effort. Some of these reproductions are easy (1–3, 6, 7) and some take real effort (4–5). The point is that every claim can be checked, against code and data that are in this repo.