[{"id":"cG9zdDo3Ng==","slug":"cleanup-downloaded-google-photos-takeout-archives","title":"Cleanup downloaded Google Photos Takeout archives","content":"\n

Recently I’ve been taking my tin foil hat a bit more seriously and since I mostly live in the Apple ecosystem (yes, you can judge me) the iCloud Photos felt like a pretty good alternative. Yes, it’s still a cloud but the content is encrypted and most importantly Apple has no real economical incentive on data mining you data. They are far ahead in terms of privacy. With that out of the way let’s go! 🚀

\n\n\n\n

TLDR: I wrote this cleaning script (tested only on macOS) to remove all duplicates from the Google Takeout folders.

\n\n\n\n

\"\" — Photo by Rayan Almuslem on Unsplash

\n\n\n\n

The process seemed easy at first. Google offers an awesome tools for exporting data out of their servers. It’s called Takeout. So basically you select the Google Photos service and let them create the archives. Then after a few hours you can download them.

\n\n\n\n

Now at the time I had ~40gb worth of pictures and videos saved in Googles cloud, however the archives I downloaded where about ~90gb. I started looking into it and a lot of photos where duplicates and edited versions that google was keeping. In addition the folders where full of JSON metadata.

\n\n\n\n

time for cleanup 🧹

\n\n\n\n

Fortunately for us there is the awesome find command that will save our lives. Removing metadata and duplicates from more than ~50k files is impossible by hand.

\n\n\n\n

First we need to remove all .json files:

\n\n\n\n

find ./my_takeout_folder -name \"*.json\" -type f -delete

\n\n\n\n

Then all the duplicates that contain a (1) at the end of the file.

\n\n\n\n

# macOS\nfind -E ./my_takeout_folder -regex \".*\\([0-9]+\\).*\" -type f -delete\n\n# Unix (Thanks to Stravos F. for pointing that out ❤️)\nfind ./my_takeout_folder -regextype posix-extended -regex \".*\\([0-9]+\\).*\" -type f -delete

\n\n\n\n

All the edited photos by google

\n\n\n\n

find ./my_takeout_folder -name \"*edited*\" -type f -delete

\n\n\n\n

And lastly remove all the empty folders.

\n\n\n\n

find ./my_takeout_folder -type d -empty -delete

\n\n\n\n

You probably will have multiple folders because you will have to download multiple archives. Simply unpack them all into one folder and run the scripts on to that folder.

\n\n\n\n

If you are to lazy to run them manually just get this script I wrote

\n\n\n\n\n\n\n\n

Then…

\n\n\n\n

chmod +x ./clean\n./clean my_folder_with_all_the_google_takouts

\n\n\n\n

Finally just drag and drop into the Photos app.

\n","status":"publish","date":"2019-05-04T21:53:32","modified":"2020-10-04T08:54:01","tags":{"nodes":[{"id":"dGVybToyMw==","slug":"google-photos","name":"Google Photos","count":1},{"id":"dGVybToyNA==","slug":"google-takeout","name":"Google Takeout","count":1},{"id":"dGVybToyOA==","slug":"icloud-photos","name":"iCloud Photos","count":1},{"id":"dGVybTozOQ==","slug":"migration","name":"migration","count":2}]},"post":{"featured":{"srcSet":"https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-300x200.jpg 300w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-1024x683.jpg 1024w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-150x100.jpg 150w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-768x512.jpg 768w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-1536x1024.jpg 1536w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-2048x1365.jpg 2048w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-500x333.jpg 500w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-800x533.jpg 800w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-1200x800.jpg 1200w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-1500x1000.jpg 1500w, https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-2000x1333.jpg 2000w","altText":"","sourceUrl":"https://api.nicco.io/wp-content/uploads/2019/05/rayan-almuslem-1302778-unsplash-scaled.jpg"}}}]