Embed face regions in videos instead of API re-linking (issue #30) - #32
Merged
Conversation
The v1.0.4 fix preserved video people by re-creating them over POST /faces
after re-upload. That is Immich-internal only and never touches the file.
Nurgak's follow-up ("faces still not embedded in videos") is exactly that:
nothing lands in the video file.
Immich's metadata face importer actually reads RegionInfo from a video
container (verified live), so videos can embed the same MWG regions as
images and re-import them when "Import faces from metadata" is on — the
same round trip photos already rely on, and portable (survives sharing,
readable by digiKam).
- HasFaceRegionsToEmbed now accepts mp4/mov/m4v; other containers skip
- regionOrientation reads the video Rotation tag (no EXIF Orientation) and
maps 0/90/270 to orientation 1/6/8, feeding the existing rasterRegion
inverse; 180/non-cardinal is left un-embedded (Immich does not re-orient
180 video regions, verified live) rather than misplace a box
- removes the API re-link path (recreateFaces, CreateFace, the
finalizeReplacement hook, the Faces uploader field) so the two
mechanisms cannot double-create faces
- videoRotationToOrientation: exiftool always reports Rotation normalized to 0/90/180/270, so the ((r%360)+360)%360 guard was dead; plain switch - add pipeline tests: a 90° video embeds RegionInfo through the mapped orientation (swapped coords), and a 180° video embeds nothing and never fetches faces
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #30. Addresses @Nurgak's report that video faces are "still not embedded in videos" on v1.0.4.
Why
v1.0.4 preserved video people by re-creating them over
POST /facesafter re-upload (recreateFaces). That is Immich-internal only — it never writes anything to the file. So Nurgak's observation is literally correct: nothing lands in the video file, and there's no portable, transfer-surviving metadata (unlike the image path).It turns out that assumption — "MWG regions are image-only" — was wrong. Immich's metadata face importer reads
RegionInfofrom a video container just as it does from an image (getExifTagsrunsreadTagson all asset types;applyTaggedFaceshas no image gate). Verified end-to-end against a live Immich v3.0.1 server: a re-uploaded.mp4with an embedded region comes back with asourceType: exifface linked to the person.What
Videos now embed the same MWG regions as images and re-import them when "Import faces from metadata" is on — the same round trip photos already rely on, and portable (survives sharing, readable by digiKam).
HasFaceRegionsToEmbedacceptsmp4/mov/m4v; other containers are skipped.Orientation, soregionOrientationreads the QuickTimeRotationtag and maps it to the equivalent orientation (videoRotationToOrientation: 0°→1, 90°→6, 270°→8), feeding the existingrasterRegioninverse — no new coordinate math.recreateFaces,CreateFace, thefinalizeReplacementhook, theFacesuploader field). Embedding + re-link together would double-create faces (oneexif, onemanual) when the import setting is on; embedding alone is consistent with how images already behave.Net −369 lines.
Validation
End-to-end on a live Immich v3.0.1: ran
-faceson a real video → the file gainedFacu, Gui, Karlenregions → the re-uploaded asset shows 3sourceType: exiffaces for those people. Rotation mapping (0/90/270 import correctly; 180 anomalous) confirmed by uploading tagged videos and reading back the imported boxes.go vet,gofmt, full suite green; new unit tests cover the rotation mapping and the video embeddability gate.Note
This relies on the server's "Import faces from metadata" setting, exactly as the image path already does. A server with it off keeps the regions in the file (portable / other tools) but won't surface them in Immich — same behavior as photos.