Apple Photos to Lightroom Classic CC: Step 2

Here’s where I left things in March:

• I had a catalog of my exported Apple Photos
• I was hoping to merge them into my Lightroom master catalog.

One little problem: importing from catalog didn’t recognize any of the photos in the temporary catalog full of photos from Apple as duplicates of those already in the master catalog. So my bright idea of using this system to update the metadata of photos in the master catalog came crashing to the ground.

I figure it has something to do with the hash Lightroom uses to identify photos. Anyway… off to find another idea.

My next hope had to do with using the Syncomatic plugin to sync metadata between files with similar names or capture times. A little scary to run on the whole catalog, and I haven’t managed to get it working predictably enough to trust it. Might still come, thought?

So I ended up looking at Photosweeper and LR/Transporter. There was my solution, though it was tedious, and the result is not perfect.

Here’s a bunch of thing I did, best I remember.

I used Photosweeper to weed out duplicates from my Apple Photos. I tried various settings and experimented quite a bit. This also allowed me to get rid of “Edited in Apple Photos” photos which were so close to the original photos it wasn’t worth importing a duplicate bloated JPG just for fun.

LR/Transporter allows you to import metadata to a photo based on a filename or a capture date.

That meant that if I could export a list of keywords of my Apple Photos (with LR/Transporter) and import it onto photos that were already in the catalog, matching the keywords to files with the same name, I could avoid importing duplicate files and having to hunt them down afterwards.

I would have to find a way, however, to flag the photos in the Apple Photos folders from which I had extracted and imported the metadata — because those that remained, “non-duplicates”, would need to be moved to the main folder. Thankfully, LR/Transporter flags photos that have last been modified by it. So once I had imported the keywords from Apple Photos, I had only to:

• use the metadata filter on my master folders to select all photos that had just been modified by LR/Transporter
• export a list of those filenames
• edit it in Google Sheets and add a “delete” flag column
• import that metadata back to the Apple Photos folders, matching the delete flag column on the “job identifier” field (for example)
• select those photos through a filter or smart collection and delete them

I did this on the Apple “original” photos I had in my collection.

You’ll not that it’s totally uninteresting to go through this exercise with Apple Photos that don’t have keywords. As I had previously keyworded all apple photos to identify if they were master photos, edited photos, master photo of an edited version, HDR, panorama, etc, I couldn’t easily filter out photos with no “real” keywords. I started out by replacing all those “status” keywords by flags in the “job identifier” field. For exemple

• I selected all apple edited photos
• added “apple edited” in the job identifier field
• removed the “apple edited” keyword
• used the “job identifier” filter to display those photos again
• now I could see which ones had no keywords
• selected those and gave them a slightly different job identifier, etc.

So when doing the LR/Tranporter export above, I was able to select only the photos that actually had keywords to work with.

What about those that had a different name? I used a similar process, counting on the capture time. Her’s what I did, year by year (to avoid having unending tables to open in Google Sheets — max 10k photos, roughly).

• I selected a year of photos with keywords both in the “master” and “Apple Photos” folders
• I exported them with LR/Transporter. Fields: size, filename, path, keywords, capture time
• I opened the CSV in Google Sheets and added four columns to help me “sort” them and generate for the correct file in each matching pair either the updated list of keywords or a delete flag (see example spreadsheet here)
• I replaced the path for all the files in the Apple Photos part of my directory structure with “apple” for readability’s sake (useful when scanning to check things)
• Ordered the photos by capture date (with Apple folder photos first, so sort them A-Z by that column first) to let the formulas do their magic
• Duplicate the sheet, then copy-”paste special” (values only) the columns containing the formulas
• I went through the list checking that the formulas worked correctly (this is where filename and size come in handy, as well as conditional formatting in the spreadsheet to create a visual pattern that is easy to scan for anomalies), corrected manually when necessary
• Once that was done, used some A-Z sorting to delete the lines of all the photos that did not match anything
• Duplicated the sheet once more (trying to be safe) and removed all unnecessary columns
• Duplicated it one last time so I could have to clean “export” sheets: one to update the keywords, one to set the delete flags
• Import the “keywords” metadata sheet onto the photos in the master folder
• Import the delete flags metadata sheet onto the apple photos folder photos.

Once I had done that for each year, it meant that all the Apple Photo original photos with keywords had been processed to see if they had a “twin” already in my Lightroom photos, and keywords imported accordingly (and source file removed).

I then was able to import the remaining Apple Photos files into my main folders, knowing there should be no obvious “same time” or “same name” duplicates — except of course for edited photos. (For that, I deleted them from the main catalog and imported/moved them using the standard “import”, so that they would sit in the right monthly folders.)

The saga continues, as my reduced list (12k from 20k!) of Apple Photos is now integrated with my main catalog, but there are still duplicates in there.

Ideally, I’d get syncomatic to work how I want it to, sync keywords, then use Find Duplicates 2 to delete duplicates. But that’s not going well for the moment.

In addition to this, somewhere in the process a pile of my photos have lost their capture date, or seen their capture date replaced by “today’s” date (the day I was obviously doing whatever caused them to lose their capture date). So I have about 2k badly dated (or undated) photos I need to find a solution for. Many of my videos seem not to be read correctly by Lightroom anymore, and have metadata/date issues. My catalog is also sprouting metadata conflicts, and Lightroom very helpfully (not) asks you whether you want to import metadata from the file or write Lightroom metadata to the file without showing you what the conflict is made of. Not much chance of troubleshooting what is going on in there.

So, what’s left to do?

1. Identify duplicates with FD2 and figure out a way to be systematic about which versions I keep (I’m doing a test on a month of photos to see how it goes). The key seems to be smart collections, and using color labels like I remember doing in March: for example, color all the “most edited” duplicates identified by the plugin green, then go and look at the full list and figure out if that is a good criteria or not.
   But for that to work I need Syncomatic to work, or to mess around with LR/Transporter again, because many of these duplicates do not have synced keywords. *sigh*
2. See what’s going on with those videos. Not too sure where to start, but I was pointed to this, and need to dig into it. Reminder, there are three problems: systematic metadata being flagged as up-to-date though in a smart collection it’s indicated as changed; mess-ups with capture times; unreadable videos.
3. Look at the photos with a bad capture date (or none) and see what I can salvage. One idea is to find a back up of my master catalog before all this mess started happening, use LR/Transporter to match those “lost” photos to their “ancestors” in the old vers of the catalog, and use a process similar to the one described above to fix their capture times. But I’d still like to know what created this situation so I can avoid it in future.
4. I’m not sure how understandable all this is, these are mainly notes for myself for when I pick this up again, but if it’s useful for you, all the better! Feel free to ask questions if certain parts are unclear and you would like explanations.

Originally published at Climb to the Stars.