Skip to content

Embed face regions in videos instead of API re-linking (issue #30) - #32

Merged
Majorfi merged 2 commits into
mainfrom
feature/embed-video-face-regions
Jul 29, 2026
Merged

Embed face regions in videos instead of API re-linking (issue #30)#32
Majorfi merged 2 commits into
mainfrom
feature/embed-video-face-regions

Conversation

@Majorfi

@Majorfi Majorfi commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Follow-up to #30. Addresses @Nurgak's report that video faces are "still not embedded in videos" on v1.0.4.

Why

v1.0.4 preserved video people by re-creating them over POST /faces after re-upload (recreateFaces). That is Immich-internal only — it never writes anything to the file. So Nurgak's observation is literally correct: nothing lands in the video file, and there's no portable, transfer-surviving metadata (unlike the image path).

It turns out that assumption — "MWG regions are image-only" — was wrong. Immich's metadata face importer reads RegionInfo from a video container just as it does from an image (getExifTags runs readTags on all asset types; applyTaggedFaces has no image gate). Verified end-to-end against a live Immich v3.0.1 server: a re-uploaded .mp4 with an embedded region comes back with a sourceType: exif face linked to the person.

What

Videos now embed the same MWG regions as images and re-import them when "Import faces from metadata" is on — the same round trip photos already rely on, and portable (survives sharing, readable by digiKam).

  • HasFaceRegionsToEmbed accepts mp4/mov/m4v; other containers are skipped.
  • A video has no EXIF Orientation, so regionOrientation reads the QuickTime Rotation tag and maps it to the equivalent orientation (videoRotationToOrientation: 0°→1, 90°→6, 270°→8), feeding the existing rasterRegion inverse — no new coordinate math.
  • 180° / non-cardinal rotation is left un-embedded. Live testing showed Immich re-orients 90°/270° video regions on import but not 180°, so writing a 180° region would misplace the box. Landscape and both portrait orientations are covered.
  • Removes the API re-link path entirely (recreateFaces, CreateFace, the finalizeReplacement hook, the Faces uploader field). Embedding + re-link together would double-create faces (one exif, one manual) when the import setting is on; embedding alone is consistent with how images already behave.

Net −369 lines.

Validation

End-to-end on a live Immich v3.0.1: ran -faces on a real video → the file gained Facu, Gui, Karlen regions → the re-uploaded asset shows 3 sourceType: exif faces for those people. Rotation mapping (0/90/270 import correctly; 180 anomalous) confirmed by uploading tagged videos and reading back the imported boxes. go vet, gofmt, full suite green; new unit tests cover the rotation mapping and the video embeddability gate.

Note

This relies on the server's "Import faces from metadata" setting, exactly as the image path already does. A server with it off keeps the regions in the file (portable / other tools) but won't surface them in Immich — same behavior as photos.

Capture d’écran 2026-07-29 à 11 49 41

Majorfi added 2 commits July 29, 2026 10:26
The v1.0.4 fix preserved video people by re-creating them over POST /faces
after re-upload. That is Immich-internal only and never touches the file.
Nurgak's follow-up ("faces still not embedded in videos") is exactly that:
nothing lands in the video file.

Immich's metadata face importer actually reads RegionInfo from a video
container (verified live), so videos can embed the same MWG regions as
images and re-import them when "Import faces from metadata" is on — the
same round trip photos already rely on, and portable (survives sharing,
readable by digiKam).

- HasFaceRegionsToEmbed now accepts mp4/mov/m4v; other containers skip
- regionOrientation reads the video Rotation tag (no EXIF Orientation) and
  maps 0/90/270 to orientation 1/6/8, feeding the existing rasterRegion
  inverse; 180/non-cardinal is left un-embedded (Immich does not re-orient
  180 video regions, verified live) rather than misplace a box
- removes the API re-link path (recreateFaces, CreateFace, the
  finalizeReplacement hook, the Faces uploader field) so the two
  mechanisms cannot double-create faces
- videoRotationToOrientation: exiftool always reports Rotation normalized
  to 0/90/180/270, so the ((r%360)+360)%360 guard was dead; plain switch
- add pipeline tests: a 90° video embeds RegionInfo through the mapped
  orientation (swapped coords), and a 180° video embeds nothing and never
  fetches faces
@Majorfi
Majorfi merged commit 4d6c1f6 into main Jul 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant