Add iOS Live Photo support and another format for guess_extractor by BobTheSoftwareDeveloper · Pull Request #224 · TheLastGimbus/GooglePhotosTakeoutHelper

BobTheSoftwareDeveloper · 2023-07-16T12:00:10Z

First of all, great tool! It has made my process of organising the Google Photos export so much nicer for when I upload the backup to S3.

I have tested the changes locally and it was able to add the EXIF date data to the video files successfully.

BobTheSoftwareDeveloper · 2023-07-16T12:01:02Z

        r'(?<date>(20|19|18)\d{2}_(01|02|03|04|05|06|07|08|09|10|11|12)_[0-3]\d_\d{2}_\d{2}_\d{2})'),
    'YYYY_MM_DD_hh_mm_ss',
  ],
+  // example: 20190620_184109.jpg


I found this format of photos when going through my old photos 20190620_184109.jpg, it seems to be used by older cameras.

BobTheSoftwareDeveloper · 2023-07-16T12:04:09Z

+  final fileNameWithoutExtension = p.basenameWithoutExtension(file.path);
+
+  if (fileNameWithoutExtension.startsWith("IMG") &&
+      p.extension(file.path).toLowerCase() == ".mp4") {


Photo from iPhone with Live Photo enabled when uploading to Google Photos will create three files:

IMG_1845.HEIC

IMG_1845.MP4

IMG_1845.HEIC.json

The .MP4 file always comes in a pair with the .HEIC file when the live photo is enabled, however, there is no JSON file for the video. We can also use the JSON file for the image to set the date for the video. This will ensure that the image and the video are stored in the same directory.

I noticed that it doesn't always start with IMG, for example, one of mine is like this:

97E2514D-F5BB-4314-B099-9A5FAE5EEA22.JPG 97E2514D-F5BB-4314-B099-9A5FAE5EEA22.JPG.json 97E2514D-F5BB-4314-B099-9A5FAE5EEA22.MP4

I too see other files. Perhaps instead try to probe whether the img file exists next to it by doing an exists check on the "possibleFileExtensions".

+1 to andrewchen5678's comment. I had a single file named 51750CEF-B61E-43C7-B4C9-00C19ED09E55 that fit this criteria, so the strict rule of IMG prefix needs rework

BobTheSoftwareDeveloper · 2023-07-16T12:08:12Z

+    "jpg",
+    "JPEG",
+    "jpeg"
+  ];


I have come across a few situations when the original image in the live photo pair is saved as a .jpg or .jpeg, they seem to use both upper and lower case. I suspect this is either because the HEIC format is disabled or the photo with the live photo feature was sent over iMessage which converted the image into a .jpeg format.

Sample directory of the files:

IMG_3984.jpeg

IMG_3984.MP4

IMG_3984.jpeg.json

and...

IMG_7334.JPG

IMG_7334.MP4

IMG_7334.JPG.json

BobTheSoftwareDeveloper

Done adding comments, requesting for your review :) @TheLastGimbus

TheLastGimbus · 2023-08-13T22:59:31Z

Nice! Tho i will need to think more about it, how we should handle it etc, so give me some time to merge it...

TheLastGimbus · 2023-09-23T00:57:27Z

So, for now, i'm leaving this waiting for further advancements in metioned discussion...

But, for anyway that wants this NOW, here are nightlies :>

https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/6280579316

dbrennand · 2023-09-25T14:45:19Z

Hey @TheLastGimbus @BobTheSoftwareDeveloper

Thanks for putting this PR together, I just tested the nightly build and wow! What a difference!

Using the latest version (v3.4.3) gpth shows loads of errors like:

Can't get date on /Volumes/Macintosh HD/Users/user/Downloads/Takeout/Google Photos/.../IMG_8354.MP4

And the summary output shows gpth failing to find metadata for many files:

Couldn't find date for 1997 photos/videos :/

Whereas, using the nightly build with this PR included:

Finding duplicates...
Guessing dates from files : █████████████████....................... 5285/12187  
Can't get date on /Volumes/Macintosh HD/Users/user/Downloads/Takeout/Google Photos/Photos from 2019/moments_09_19_8_20_51 pm.MP4
Guessing dates from files : ███████████████████████████████████████. 12186/12187  
Finding albums (this may take some time, dont worry :) ...
Moving photos to output folder : ████████████████████████████████████████ 12187/12187  
========================================
DONE! FREEEEEDOOOOM!!!
Skipped 3 duplicates
Couldn't find date for 1 photos/videos :/

Only 1 file failed 👍🏻 😃

TheLastGimbus · 2023-09-25T15:21:37Z

Only 1 file failed

We're leaving not even one! @dbrennand Could you try to find other files related to this one (og photo and json)?

dbrennand · 2023-09-25T15:26:47Z

Only 1 file failed

We're leaving not even one! @dbrennand Could you try to find other files related to this one (og photo and json)?

I can have a look, what am I looking for exactly in the JSON?

TheLastGimbus · 2023-09-25T21:12:18Z

You're just looking for the json, and it's filename 👍👍

I just want to figure out why gpth didn't find it - it does so by filenames purely

dbrennand · 2023-09-26T21:30:08Z

You're just looking for the json, and it's filename 👍👍

I just want to figure out why gpth didn't find it - it does so by filenames purely

@TheLastGimbus - I'm not sure why it failed... 🤔 So for this live photo, the three files are named:

moments_09_19_8_20_51 pm.JPG
moments_09_19_8_20_51 pm.MP4
moments_09_19_8_20_51 pm.JPG.json

title value in the JSON is:

{
  "title": "moments_09_19_8_20_51 pm.JPG",
}

zxsleebu · 2023-12-06T19:33:26Z

can you please look into it?

TheLastGimbus · 2023-12-07T08:44:54Z

@zxsleebu I'm gonna be honest - right now, no.

And I can't really point when, but like a month from now at least. If you need your takeout with this, use @BobTheSoftwareDeveloper fork - nightlies are still here: https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/6280579316

It will even help to test it if you used it. I just don't have the time right now to make 10000% sure it all works (and potentially add all cool features above) and list it on the official repo

Alkanov · 2023-12-25T20:11:09Z

can you please re-build https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/6280579316

Edit: nvm I forked it and ran it myself. I can confirm I went from 3.5k "Can't get date on...." to only 40 - everything else worked fine. Thanks

TheLastGimbus · 2023-12-29T16:30:58Z

https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/7358237528

pzh200707 · 2024-04-01T08:59:09Z

https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/7358237528

I have met the same problem. Latest release is on Sep 23, 2023. The rebuild has expired. Can we have a new release?

TheLastGimbus · 2024-04-01T15:45:07Z

Sure guys

https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/8510134630

janhoy · 2024-04-17T10:56:26Z

    jsonExtractor,
    exifExtractor,
    if (args['guess-from-name']) guessExtractor,
+    heicVideoExtractor,


Should this extractor perhaps come before the guessExtractor? If it exits quickly when conditions are not met, it should be safe.

mguzman011 · 2024-05-19T21:42:46Z

Any chance of being able to add RAW file support? Seems like some .dng and .cr2 files I had weren't transferred over.

Not the worst, since it seems like the EXIF data has been kept intact, but adds another step to the process.

Edit: Seems like .cr2 files were transferred over fine. What caused me issues were ~3mb "CR2" files that were actually jpgs, but labeled CR2 thanks to Google Photos. Renaming the files as JPG brought back all of their EXIF data and I could organize accordingly.

TheLastGimbus · 2024-05-20T11:11:16Z

Seems like some .dng and .cr2 files I had weren't transferred over.

I will look at it when i will take care of this project some day 🌈

safareli · 2025-01-07T14:27:48Z

I tried to use this branch changes and it worked very well. the only thing you user should be aware of is to do merge of the takeout properly. as it could be that for example video part is in another takeout and photo is in another. or photo is in one takeout while json file in another. what I did to make sure takeouts are merged properly is to use rsync on macos

rsync -a ./Takeout/ merged/
rsync -a ./Takeout\ 2/ merged/
rsync -a ./Takeout\ 3/ merged/
rsync -a ./Takeout\ 4/ merged/

and with this way of merging there were no more unknown dates folder in the output

brandisf · 2025-01-26T20:32:18Z

Hi, thanks for the great tool. Could you build a new release for this branch? Ran into errors with 8k out of 30k Photos ending in the date-unknown folder. Seems like it is due to the iOS live Photo Problem with MP4's not having corresponding jsons. Thank you

mureev · 2025-02-08T07:59:23Z

Sure guys

https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper/actions/runs/8510134630

Can you trigger new nightly build? ty

trmfreitas · 2025-05-12T23:30:03Z

I added support for live photos MP4 file.
https://github.com/trmfreitas/GooglePhotosTakeoutHelper

Tomred214 · 2025-05-24T15:26:14Z

I added support for live photos MP4 file. https://github.com/trmfreitas/GooglePhotosTakeoutHelper

Hi! Is it possible for you to build a release for this? It's totally ok if not
Thank you!

trmfreitas · 2025-05-26T12:05:41Z

I added support for live photos MP4 file. https://github.com/trmfreitas/GooglePhotosTakeoutHelper

Hi! Is it possible for you to build a release for this? It's totally ok if not Thank you!

Ok, I will do it.
Does this help?
https://github.com/trmfreitas/GooglePhotosTakeoutHelper/actions/runs/15254627571

aiastia · 2025-06-24T02:35:46Z

I added support for live photos MP4 file. https://github.com/trmfreitas/GooglePhotosTakeoutHelper

Hi! Is it possible for you to build a release for this? It's totally ok if not Thank you!

Ok, I will do it. Does this help? https://github.com/trmfreitas/GooglePhotosTakeoutHelper/actions/runs/15254627571

Hi, he worked, but he still said that there were some anomalies, but for example, the Wacheee version is much better. The Wacheee version has more than 8,000. The date cannot be obtained. At present, these with (1) have some problems. ,


** Guessing dates from files: ██........................................ 2302/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0013(1).MP4
Guessing dates from files: ██........................................ 2338/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0028(1).MP4
Guessing dates from files : ██........................................ 2348/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0023(1).MP4
Guessing dates from files: ██........................................ 2351/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0018(1).MP4
Guessing dates from files : ██........................................ 2355/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0020(1).MP4
Guessing dates from files: ██........................................ 2361/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0019(1).MP4
Guessing dates from files : ██........................................ 2722/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0031(1).MP4
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_0030(1).MP4
Guessing dates from files: ███........................................ 3541/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1116(1).MP4
Guessing dates from files: ███........................................ 3563/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1109(1).MP4
Guessing dates from files : ███........................................ 3565/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1108(1).MP4
Guessing dates from files: ███........................................ 3647/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1034(1).MP4
Guessing dates from files: ███........................................ 3654/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1013(1).MP4
Guessing dates from files: ███........................................ 3663/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1012(1).MP4
Guessing dates from files: ███........................................ 3880/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2019/IMG_1011(1).MP4
Guessing dates from files : ████........................ 4952/41415
Can't get date on /volume5/macOS/photo/Takeout/Google Photos/Photos from 2021/IMG_0108(1).MP4

BobTheSoftwareDeveloper added 2 commits July 16, 2023 23:51

Add iOS Live Photo support

3944b11

Add another format for photo guess_extractor

33a3756

BobTheSoftwareDeveloper commented Jul 16, 2023

View reviewed changes

BobTheSoftwareDeveloper commented Jul 18, 2023

View reviewed changes

BobTheSoftwareDeveloper mentioned this pull request Jul 21, 2023

Live Photos / HEICs #225

Closed

TheLastGimbus mentioned this pull request Sep 11, 2023

Handle "moving photo" video files #180

Open

janhoy reviewed Apr 17, 2024

View reviewed changes

kushagharahi mentioned this pull request May 4, 2025

Add Support for iPhone Raw and Live Photos #336

Open

Uh oh!

Conversation

BobTheSoftwareDeveloper commented Jul 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BobTheSoftwareDeveloper Jul 16, 2023

Choose a reason for hiding this comment

Uh oh!

BobTheSoftwareDeveloper Jul 16, 2023

Choose a reason for hiding this comment

Uh oh!

andrewchen5678 Sep 4, 2023

Choose a reason for hiding this comment

Uh oh!

janhoy Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

kushagharahi May 4, 2025

Choose a reason for hiding this comment

Uh oh!

BobTheSoftwareDeveloper Jul 16, 2023

Choose a reason for hiding this comment

Uh oh!

BobTheSoftwareDeveloper left a comment

Choose a reason for hiding this comment

Uh oh!

TheLastGimbus commented Aug 13, 2023

Uh oh!

TheLastGimbus commented Sep 23, 2023

Uh oh!

dbrennand commented Sep 25, 2023

Uh oh!

TheLastGimbus commented Sep 25, 2023

Uh oh!

dbrennand commented Sep 25, 2023

Uh oh!

TheLastGimbus commented Sep 25, 2023

Uh oh!

dbrennand commented Sep 26, 2023

Uh oh!

zxsleebu commented Dec 6, 2023

Uh oh!

TheLastGimbus commented Dec 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Alkanov commented Dec 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheLastGimbus commented Dec 29, 2023

Uh oh!

pzh200707 commented Apr 1, 2024

Uh oh!

TheLastGimbus commented Apr 1, 2024

Uh oh!

janhoy Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

mguzman011 commented May 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheLastGimbus commented May 20, 2024

Uh oh!

safareli commented Jan 7, 2025

Uh oh!

brandisf commented Jan 26, 2025

Uh oh!

mureev commented Feb 8, 2025

Uh oh!

trmfreitas commented May 12, 2025

Uh oh!

Tomred214 commented May 24, 2025

Uh oh!

trmfreitas commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aiastia commented Jun 24, 2025

Uh oh!

Reviewers

Assignees

Labels

BobTheSoftwareDeveloper commented Jul 16, 2023 •

edited

Loading

TheLastGimbus commented Dec 7, 2023 •

edited

Loading

Alkanov commented Dec 25, 2023 •

edited

Loading

mguzman011 commented May 19, 2024 •

edited

Loading

trmfreitas commented May 26, 2025 •

edited

Loading