Photo Ingestion: From Chaos thru Workflow and back to Ad-Hoc

Over the years, my photo ingestion process, jargon in digital photo world for uploading your photos say to a photo managment system, has evolved from simple to semi-chaotic, shaped by changing gear and software quirks. I started out loading JPEGs straight from the SD card using camera-branded software, then shifted to Lightroom Classic’s folder-based system. It was slow, and I quickly ditched metadata tagging in favour of a manual copying of the files and folder structure, yearly, monthly, daily, based on the camera’s default naming.

As RAW formats became standard and file sizes ballooned, I had to juggle multiple SD cards and software that didn’t always play nice. Initially my Canon EOS 1100D required its own viewer for RAW and MOV files. Then in 2017, I won a copy of Photo Mechanic, a game changer. It mimicked old-school contact sheets, let me tag and sort on the fly, and even added GPS data, which was perfect for my sun-chasing road trips around Australia. I also had two Pentax DLSR cameras, a K200 and a bigger K20, both nice cameras and I really liked Photo Mechanic. It was a very useful little programme when loading photos particularly when you’re travelling because it significantly reduced the amount of time and fiddling.

I had become interested in mirrorless cameras and got the little OMD-10. Unfortunately, Photo Mechanic didn’t support Olympus RAW, so I pivoted to Olympus, now OM Workspace. It was free, handled most formats, and even updated camera firmware. Slowly, it became my go-to for the OMD-10 and later the OMD-5, which became my main camera.

Now, with just a single smartphone and mainly Olympus gear, I can upload in a mixture of USB cables, card reader, Wi-Fi transfers, and Microsoft’s Phone Link. It works, but it’s again ad hoc. 

The challenge now is keeping everything backed up and organized across formats, folders, and devices.

Revisiting & Extending my Photographic Punctuation Marks

Its been a while since I wrote about the idea of including pictorial punctuation marks in my camera’s photostream. It much handier (pun intended) than taking notes or trusting memory. As I’m taking more and more group image sets this is a very easy way to find and group like images just in the thumbnail view (eg PhotoMechanic as I load the photos where I can colour code them accordingly or in OnOne Photo RAW where I like to group Stitched Panoramas, HDR and Focus Stacked sets in their own sub–directories). This avoids a lot of wasted time churning thorough a mountain of similar images, and keeps things a little better organised.

Think of these first the hand signals as visual brackets

Panoramic Sequence, and direction of shooting. I still don’t use this all the time, because a panoramic series is usually pretty self explanatory. When I have used this I will add this marker photo before the panoramic sequence is shot. I further identify the stitched panorama series with a Green Colour Class

 HDR Bracketed Sequence, I use this to indicate I am taking 3 or 5 a photo sequence. This is another punctuation marker that I normally use to precede a sequence. I usually don’t bother put this in front of every HDR set in a continuous series (such as combined HDR and Stitch Panorama because it is relatively easy to find the groupings). I then use the Yellow Colour Class to easily identify the HDR sequences.

Focus Staking. or other multi-bracket techniques. This is a new marker because it can be very difficult to discern the beginning and/or end of focus bracketed sequences when more that two or three shots are involved. I have therefore adopted the approach of adding this at the end of each sequence. I use the Blue Colour Class to identify this type of bracketing.

I like to think of the next hand sigalns as simple punctuation marks  ; (=battery) . (=end)  ? (=cut)  ! (=good)

Battery Change, still my way of recording when I changes my battery, looking back through your photos and finding the previous battery change photo, then just subtracting the new photo number from the previous gives you the battery life in terms of number of shot per battery charge. On the Pentax DSLRs this can be weeks to months worth of shooting but on the mirrorless Olympus this is generally days to a week. Doing this let me work out I get around 750 photos from a single battery charge from my new Olympus OMD EM10iii, which is about double the claimed 330 shots.

Full Stop, End is a marker to end one series or starts another. I used this often for a variety of breaks or just simple visual markers to aid looking through a large collection.

Cut, not something I use much, but I have used this simple bit of sign language to remind myself that the previous photos should be deleted. I never just flag photos to be deleted I immediately delete them as soon as they are loaded onto my computer, I also delete this marker.

Good, another new punctuation mark just to remind me that close to this marker is a photo I liked at the time, maybe good light, composition or just that perfect moment.

Whilst I normally take both Jpeg & RAW, I usually delete the larger RAW version of any punctuation photos, as I never intend to post process them.

Can you find the Kookaburra?

Taken on the Jells Park Photowalk :: Seeing as the Camera Sees.

This is a good example of my new approach to handling a (big) batches of photos without wasting time and waiting for (slow) software. (ie bringing the fun back into photography). Taken from the camera card with Photo Mechanic (180 images, 90 pairs of jpeg+raw), adding metadata, select and rating the best (under 3 minutes) then pumping 40 raw files out to photolemur 3.0 to see what is possible (another 2 minutes for that to happen, time to make an espresso) Then I sat down (5 minutes later) and choose half a dozen photos to work on.  All I had to do with this one was crop the photolemur output. Voilà.

Then I used Photo Mechanic Upload tools to send this cropped image to both flickr and twitter (in less than 2 additional minutes)

For a couple of photos I will go back and take them into On1 Photo RAW and three sets of 5 images each I will take into AuroraHDR 2019. Even though I’m not under any time constraint taking the chore aspect out of culling ranking and caring for my photos before I can even start the fun of enhancing things, give to time to reflect and hopefully lift my creativity

Matching GPS Log Files to Photos

When first trying out the GPS match up in photo mechanic from GPS wayfiles and my photos I was out of internet accessible areas and could not look at the result on a map. When I finally did get a chance to check things where out all over the place. A real mess. Since I had had massive issues trying to synch my phone and camera I figure that was the issue. And it was … just not a matter of synching the current time.

The real issue was the fact the google my tracks app which I use on my phones, actually stored the time stamp in UTC (univeral coordinated time) which most people know as Greenwich Mean Time or GMT. This is for the simple reason that satellite position is reference to this time standard. At the moment its 11 hours behind our local time here on the eastern side of Australia. Once I realise this (its not documented anywhere I could find) everything else about loading the GPS wayfile (in gpx format) and merging the coordinates to my photos in photo mechanic was easy.

I’m doing the gps merge straight after the ingest step. I have to export the gps wayfile from My Tracks app in .gpx format (and despite what the app says it does not reliably export to my google drive anymore), so this is now a two step dance, export to the card in the phone, connect the phone to my computer and up load the .gpx file into the same directory as the photos. I then run File/Import GPS coordinates… from the main drop down menu. Which starts the dialogue show above. There are just four things to do/check.

1) Press the little add button in the GPS log file dialogue up on the top left, and use the normal file browse to select the .gpx file. Then press the plot GPS logs (you can have several files being processed together) and this will display a map of your path on the right hand panel.

2) the next important place to set up and/or check is the GMT Offset of photos panel on the right below the Map panel. First time you use it either enter the difference in your local time to GMT (11 hours in my case). You can also try using the Auto, but it seems to match your first photo with the first way point, and often I start the gps tracking well before I photograph. So there can be a mismatch (as I discovered the hard way)

3) The small panel on the right will show the first photo and coordinates. If the photo is miss located because of time difference you can make those changes easily by clicking on the small + and – button over under the map. Each time you press the button it moves onto the next way point (the ++ give you bigger steps). You just need to move the blue pin along the way path till it reaches the correct spot. you will also notice that the time offset is being changed in accordance with the shift. You can also step through the photos using the -> arrow keys below the photo in the panel on the lower right. Despite my best efforts to synchronise everything one camera is 4 minutes 30 slow the other 1 minute 25 seconds fast. These time difference make a difference even at m walking speeds. The time differences are “sticky” and show up next time you import more coordinates. So after a few ingest the small shifts are usually not required.

4) The last step is to click the Import GPS coordinates button in the lower right hand panel.

The whole process usually only takes less than a minute.

In the past I found picasa a reasonable way to review photo location for .jpeg files. Even Microsoft photo will display a map for an individual photo once it has lat and log coordinates. However I have had very mixed results trying to use Lightroom (it will read the coordinates from the way files and do a merge with photos but very often refuses to display map backgrounds so it a bit impossible to check and got it wrong more often than not. I had used a special apps such as geotag and geo tracker to do the merge but that was also tedious and error prone, so I had got used to only occasionally bothering to geotag a sample batch. Just manually locating a few photo in Flickr, when appropriate.

The new approach with photo mechanic is so quick and easy I am doing it routinely again. I’m even going back and redoing a few of those batches of the photos in the past for which I still do have gps wayfiles. Geotagging is back on my agenda

The Social Media Diet, that isn’t going so well

Alvin (aka my Social Media Diet Coach) is very disappointed with me. My social media posting hasn’t really reduced and I was travelling. He is particularly exasperated as I was off line for 10 days and still managed lots of posts. He claims I have become a social media glutton, that I might need to detox. He is threatening to change the passwords, or worse install apps that limit the time I can waste.

Secretly I enjoyed the silence, He is right but I won’t tell him that.

I dare not mention that rather than deleting software this month I’ve taken on two more packages. Photo Mechanic, which I won in a give away contest on Instagram (and I already like) and a public beta of Luminar for Windows (I also like it). Maybe I am truly a glutton? So ssssh!

Photo Mechanic Unplugged

There are certain disadvantages to learning new software away from the net, especially when the help system is on-line. Quickly realizing that I downloaded the Flat version of the help system as a pdf. It not exactly the same but I was able to find and answer to most queries. The next and probably the biggest draw back was I was out most of the days either travelling around on the boat’s tender, visiting remote island, or scrambling over rocks to visit cave painting site, walking along remote beaches or out spotting birds, crocodiles, whales, sharks, turtles and not to mention getting some photos of sunrise and sunsets. In other words I was doing a lot and when back on the cruise vessel I was pretty much eating or sleeping which left precious little time to review and cull photos.

Still I had three cameras to upload (aka ingest in photo mechanic jargon). My favourite pentax, my old faithfully canon rebel, which I tended to use for wet beach landing and rock scrambling outing, just in case it got dropped, wet etc (which it didn’t) and finally the camera in my mobile phone. I must admit I could not get Photomechanic to ingest directly from my phone (a HTC UPlay) either with a cable or via wifi. There are generally only a few photos so copy copying them via a USB cable was fine. Actually I had two smartphones, even though there was no phone signal most of the trip. The oldest was used just simply as a GPS to collect wayfiles, I’d set it in the morning and put it in my camera bag (or backpack), The new phone I used used both with My Tracks to save a wayfiles and also with the default camera and location set to include geocode in the EXIF header, mainly for checking purposes. GPS logging is a pretty heavy battery drain om the phones so they often ran out of puff. So I went over to different morning and afternoon shifts on each phone for GPS tracking. I usually managed to upload every thing at least every second or third day and made a point of also running a backup to one of two external drives each time anything new was loaded, all simple to set up in Photomechanic.

Photo mechanic did make culling a lot quicker, but I couldn’t find an auto-advance setting (such as lightrooms) as I rated photos, I kept using my favourite 1, 2 Or 3 stars. However I could not use the press X for flag images to reject. These star rating are written to a .xmp sidecar files, so all the rating where useful in lightroom and AfterShotPro. Photomosaic tagging system works a bit different so I actually just used the keyboard delete key instead of a shartcut (it does also mean also doing a confirm delete each time). I will also admit that I had trouble doing selective transfer of files into lightroom if it was running. I tried both drag and drop and setting up lightroom as the external editor. However both attempts to work on the images in lightroom starts the import from disk function but without and files selected and as I had already culled I could just click on select all (and go have a coffee).

Whilst Photo mechanic process is fast, the step of just copying the photo files to the computer is actually not as fast as a straight system copy file, or Picasa or Aftershot Pro imports. They all loaded all the photos faster than the limited comparity Photomechanic testing I did. However what photo mechanic lets you do was start working on the first files loaded straight away, so you could begin your rating and culling quickly and didn’t have to wait for all the photos to be copied or loaded before beginning. This is a big deal if you have a lot of images to load (I was averaging 800 a day but had a 2,000 day in there). You can easily halve the time to load and rate your photos.

Other than the ingesting process and culling I didn’t really get to exercise photo mechanic a lot. I also have beta version of Luminar for windows on my PC but was not successful in opening an image directly from photo mechanic into Luminar, again I did have access to the on-line help for Luminar. The transfer to edit in AfterShot Pro worked beautifully but there is so much overlap in what the browser mode in AfterShop and Photomechanic do I probably wouldn’t be using them together.

I was also very keen to get the GPS tagging running but ran into a series if unexpected issues. The synchronization of timing sounds simple but because the phones had no mobile signal they kept defaulting to different time zones (not “ship” time) and the camera let me change times (both have dual cloaks) but they too often just reset themselves to the base, my home, time. So figuring out time shift was a confusing and annoyingly repeated problem. A second contributing issue was the tools I had at my disposal to show the photos on a map (ie Photo mechanic, Picasa, lightroom and google earth all require and on-line connection) This meant even when I did merge the wayfiles and photos I could not check the location on a map. Recognizing this could lead to a lot of frustration I just gave up.

In the process of getting off the boat in Darwin the screen of my beloved little HP Spectre got cracked (no idea how) and the touch screen and pen functions have stopped working. So this has put an end to my investigation for the moment.