DNG and the Month of Living Dangerously

Well i’m glad September is over, I can reset the dial of my beloved Pentax back to RAW/PEF not RAW/DNG. This story began several months ago when I installed the new version of OnOne 10.5, and suddenly I started having problems with OnOne Browse not display RAW files. Initially it wasn’t every RAW file but as I tried reinstalling the update and specifically once I had installed version 10.5.1 I was not able to view any .pef (the Pentax native RAW file) other than for a fleeting moment when the file was first viewed. Also .tif files and .cr2 first would intermittently disappear. This made trying to use OnOne software standalone tedious and frustrating. My continued request for support were just answered with trite rubbish about updating drivers and re-installing the software. I got frustrated and stopped using OnOne for a long time.

I got used to using Aftershot Pro mainly for my initial review and perhaps quick processing in the field. Then loading up lightroom back home and finishing images. For a few photos I wanted to do extra post processing I used the plug-ins linked to either OnOne Enhance, OnOne Effects or Nik’s Software.There is really nothing wrong with Aftershot Pro, in fact despite OnOne’s claims to have the fastest RAW rendering I suspect aftershot might be faster. However I quiet liked being able to flag just the images I wanted to further post process and load only those into lightroom or just finish them in the other OnOne modules, as I had been able to in version 10.0.1.  This was ok and seldom more than two steps. usinf aftershot, then lightroom and then a plg-in was three and meant having to generate an intermediate exchange file. Then I remember that my Pentax is one of the very few cameras that can capture DNG files in camera. I had previously looked at using that format but the files where substantially bigger so I couldn’t see any benefit. However if I avoided the intermediate copies I felt I could still be ahead in terms of spaced consumed so I bravely decided to only take DNG files during September.

Where did all the space go?

 

In september I created 1420 photo files. Most RAW plus jpeg pairs and most of them RAW DNG with a few .cr2 files. They took up a whopping 85 GB (excluding any .tiff post processing versions)
compared with 3710 files taking up only 40gb for my  month long candian month long trip earlier.

Exploding DNG  Grenades

The first hint of an issue had showed up before September but I assumed it was an isolated accident. Occasionally, and I still can not figure out why or which piece of software is to blame, some of the DNG files will suddenly double in size. Typically it is my better shot that I am interested in. Then sometimes they double in size again. as an example most DNG file from the camera are around 23 MB each, then when they first explode they will reach around 54MB then the second explosion will see them at 120mb plus. It is only when I try to view one of these exploded files that the havoc begins. Aftershot nicely reports it has stopped working, and trying to restart it report that it shut down unexpectedly and is set to read that directory again but it nicely reports it is stopping again, and so on. Lightroom seems to disappear for a while and eventually just doesn’t acknowledge that these bigger DNG files exist. OnOne browser however goes into suspended animation and my computer totally locks up. If I kill the process, restart the computer OnOne will refuse toit refuses to restart.

I eventually worked out if I renamed the last directory I had used before such crashes I could restart the misbehaving software. However if I tried to re-access that directory with OnOne browse, it was back to sleep, and crashes again. Lightroom just kept ignoring the big files after long contemplation and aftershot was ok unless I tried to look more closely at those files. Picasa would still view these files but took a long time to load them, perhaps understandably because they were large. These problems occurred on both my new HP Spectre laptop and my HP desktop, but less so on tests on an old Toshiba laptop.

I soon came to the conclusion that these exploded DNG files were not good, possibly corrupted some how. Which left me with a risk, if I tried to view these files before I had a backup I could loose the RAW version (since I take RAW and jpeg I would at least still have the jpeg version). Unfortunately I realised this AFTER I lost a couple of rather nice photos. I do nothave decent DNG version of these now.

As the month progressed this just became a bigger and more frequent problem and I am no closer to knowing why. I just have a mass of DGN files I am reluctant to even veiw.

Toxic Tiffs

When I am working with plug-ins like OnOne or Nik’s software I had adopted the idea of using .tiff files which give me more colour bit depth and preserve the illumination dynamic range compared with Jpeg and are way smaller than using the .png format (photoshop file format). Again occasionally to start with these .tiff files derived from DNG files began to refuse to load. I could work on the file to begin with, but once i quit the plug-in concerned the file would not open. Worst once this occurred within a session all other tiff files I tried to access would suffer the same fate. Exiting the software and starting a new session sometimes fixed the problem till I hit another toxic .tiff. the file sized does not seem to be altered in the case of the .tif files, so there is no easyway to figure out which fie will or won’t open. So opening any .tif file has a risk!

At this stage I also started to suspect a photo/dng targeting virus or malware, but I can not findany reports of such a beast and my virus and malware scans do not report any problems,

So I have stopped recording in DNG for four days now and guess what? no problems for two days!

 

Is the problem in the Pentax version of DNG, or is it in the software, or even combination of software I’m trying to use?

 

The fact that the new apple IPhone 7 will save optionally save in DNG format could see others finding similar problems. I’ll let the do the debugging. I’ve had enough for now no more DNG for me!