Frustration Central

I lost some of my working computer files earlier in the year and confidence in Microsoft.. My office computer died right after a Windows update, and it caught me between backup cycles. About a month of photos was only partially saved. Not great!

Firstly I stepped back and looked at what I need to do to make sure it doesn't happen again. This lead to going back through all my backups and archives. It wasn't a happy task many of the archive sets were missing a significant number of files and several were of zero length. It took a while to track what was happening.

The problem turned out to be a short-coming of the copy function in Microsoft windows when copy multiple nested directories in an unsupervised way. The copy function can hit a file it can't copy and it then moves onto the next directory leaving the rest of the directory or directories not copied. Occasionally it sems it just created an empty file header and moved onto the next file. These missing files only showed up with detailed review. The number of files in the affect directories would become less in subsequent copies or the amount of disk pace required would become slightly lass. Empty files where easy enough to find once I knew What I was looking for. My adoption of creating at list of the MD5 checksums a few years ago really helped zero in on problems in recent year. 

For a short time I was side-tracked by the unnoticed update in windows 11 that copied my working directories of photo into a subdirectory under OneDrive and then started uploading them to the cloud, and worst still starting to remove older files from my computer (all unknow to me).

Turning off the link to OneDrive, was the simple solution. But microsoft seems to be able to turn on the "private" OneDrive after it updates (now manditory in window 11 home. Arrggghh!

This is unacceptable behaviour microsoft

I searched through every backup or archive I had and saw the pattern where files disappeared during times where I bulk copied many directories. Looking in forum and targeted web searches I found I was not the only one who noticed this and found most highlighted the bulk copies of directories although there was a variety of reasons suggested as to why.  Not checking the copy I had left myself open to both such problems,  The other likely cause was "data rot", from earlier backup sets that used DVD and CDs as the backup media. I also have been using old hard disk from computers that died from update in my air-gapped backups/archives, and t was possible that these might suffer from  a process known as magnetic decay or "bit rot", but I could not find any evidence of this even in disks tat had not been fred up in 3 years. Solving both these issues lead me to Teracopy because it does have the  option to verify files after copy to ensure they are identical.. It is nice and simple and works,

It was complex figuring out how best to compare a range of "incomplete" backup set I already had and just find one good copy of each photo. The crux of the problem is; I had no simple way to know what was missing and/or what might be corrupted. I realized that my "air gapped" copies on old harddrives would be a nice starting point. It was

So in the end I returned to the 3-2-1 with 1-0 extension as the best method of backup and keeping an archive of my photos. I've modified the process a little, I don't keep my off-site backup in the cloud and I use Teracopy, XNview MP to look at photos as images and the MD5 checksums to ensure that my photo collection is free of errors.