Refining my "Air Gapped" Backup/Archive, rather than a New Year’s Resolution

I’ve not been so diligent; my "air gapped" archive/backup system has been sitting on the shelf for the past two years, gathering dust. You know how it goes. I had good intentions about checking those old hard drives every six months (they need to be "exercised" to stay healthy), but eye issues and life got in the way.

When I finally fired it up again, I accidentally left the Wi-Fi on, and Ubuntu immediately reminded me about updates. At first, I'd been avoiding updates thinking "I barely use any features, why bother?" But now seemed like the perfect time to upgrade and add some new tasks into the system.

Enhancing the 3-2-1 Backup Strategy

This whole approach really came out of the ransomware era (still very much a security problem). The idea is simple: keep an offline backup so you can recover if something goes wrong, and make sure you're actually verifying that your stored data is correct zero errors.

 Just looking at a photo or document isn't enough to know if it's been corrupted or tampered with. How do you really check a file? It depends on the type of file and format, and there’s a growing miriad of different ones to worry about.

My Practical Approach

For photos, I'm using MD5 checksums, basically digital fingerprints for each file. Adobe has its own proprietary checksum system in Lightroom and new content credential tools, but I'd rather stick with the commonly used standard.

Raw files are trickier. They vary between camera manufacturers and have already changed over time and camera models.. Some photographers convert everything to lossless TIFF files (which can become huge), others swear by Adobe's DNG format, but I'm not seeing widespread adoption outside the Adobe ecosystem.

Video files are a more difficult area, they rapidly gobble up space and I've got formats from software I have no idea about. For now I will just have to stay with VLC video viewer, which seems to cover most video formats.

The Reality Check

I'm being realistic here, I’m getting older, and I have half a million photos. Nobody wants to wade through all of that when I'm gone. So instead of keeping everything organised just by date and time, I need to actually curate the mess and my legacy. Pull out the family photos, separate events and places, get rid of the junk. I'm planning to explore best ways to identify the high-value stuff worth preserving properly.

The plan is to do this reorganisation on my main computers first, then pass over a well-organised set of archives to the air-gapped system. It's a bit scary because I'll be deleting files, so I'm keeping the full backup for at least a year as a safety net. Currently sitting at 5.5 terabytes and hoping to cut my archive at least in half, but we'll see how realistic that is, same time next year.

Why I started using an Air Gapped Archive

I'm not paranoid or afraid of the NSA. I'm just trying to avoid the risk of ransom-ware or the corruption of a disk via a virus, mechanical malfunction or a range of other things that might affect an active network and hard disk always spinning. I realise my approach is somewhat hybrid and would not satisfy Edward Snowden or the security gurus but It does give me confidence that I have addresses the long term safe storage of my source photos.
I am using an old Toshiba netbook (it served me well as a photography & email connection to the world as I traveled around Europe in 2011). However it was always slow and it was the first computer I tried to update to windows 10 and basically windows 10 killed it! I ran out of patience and installed Linux, and a little photo software but it sat unused for the past year-ish. Then it struck me this would be ideal for the Air Gapped Archive. I leave it turned off most of the time, turn off the WiFi & Bluetooth and just use it to manage my photo archives on an External Drive. It has 3 USB slots (unfortunately USB 2.0 not 3.0, ie file transfer will be slow) so I can do my transfer of files to the archive on this little Linux computer. I can also preview the files with XnView when necessary

The Air Gap isn't perfect because I have to get the photos onto the external hard drive (ie connect it to a network somewhere) and at regular intervals the archive must be updated, again I do this for convenience by creating a new copy of the archive and updating that. This is the Generational approach. The new copy is the SON and this become the new air gapped Archive. The old copy is the FATHER and can be put away till next cycle when It is rewritten as the new SON. In the old TAPE backup days this was often taken to a third GRANDFATHER generation. At present I only have enough spare hard disks to do two generations and I only plan to cycle them over every three months.
I plan to use small memory sticks (only used for that purpose) to extract files if necessary between generation updates and this in theory reduced the likelihood of large scale malware, and particularly ransom-wear but does not avoid it altogether (eg stuxnet). The fact than I'm also crossing operating systems significantly reduces the possibilities of viruses being transmitted.
The Linux computer and the hard drive  containing the Primary Archive are turned off and stored together in my studio rather than my office where the on-line collection of my photos is stored. Hopefully I will only need them every 3 months or so as I cycle and then update my primary archive.