Linux Digital Photography Workflow Redux - Is Low Tech better?

Almost a month ago, I posted about how I was attempting to deal with my digital photo workflow under Linux. Since then, a couple of things have happened which have made me re-think my workflow... and probably in a way you might find surprising.

Here are the two things that have happened:

A) I've become enamoured with the linux command line and the linux philosophy. While no guru, I am becoming reacquainted with the simplicity and power of the linux file system, the command line and the Unix philosophy of 'writing small tools that do one thing and do it well' and utilizing those tools together to create a powerful solution.

B) Adobe recently released the Beta version of it's Lightroom program for Windows (previously it was only available for the Mac). I downloaded it and gave it a quick test spin.

In case you didn't already guess, B) is at odds with A).In fact, after running the Lightroom Beta, I immediately came to the realization that it was NOT what I needed.

In fact, I soon came to the realization that most photo management applications I've tried - and I've tried A LOT - are more frustrating than useful to me. I'm not so sure an all-in-one solution is what I'm after at all. In the past I've tried: Paintshop Pro Album, ACDSee, Photoshop Album, Picasa, and now F-Spot on linux among several others.

Here are some the main reasons they frustrate me:

1. References to my photos and metadata (including any tagging or rating systems etc.) are kept in a hard-to-access database. I realize that this is probably the fastest/most stable/best way to do it, but I don't like it. I can't see the data, move the data or copy the data without really getting my hands dirty.

2. I find that some of these programs have trouble when I move images around on my hard drive. I've sometimes got multiple copies of the same image in the database and it's hard for me to distinguish one copy from another. I don't feel safe moving image files around or reorganizing my folder structure without wiping things out or screwing things up.

3. Many times these apps rely on locking you into their way of doing things. They work much better (in terms of built-in versioning schemes etc) if you do all your editing within them. This is not what I'm looking for. I want to use the best tool for the job without worrying about how it affects another program's functionality.

4. What happens to all that metadata (ratings, tagging, etc.) that I spent hours and hours entering when I find a super-duper new application from a completely different company 5 years from now? Maybe F-Spot will do an honest job of exporting this info for others, but could you say the same about the proprietary solutions?

So what am I thinking about in terms of a digital photography workflow?

What about a relatively low-tech DIY photomanagement approach?

"Huh...Whuh?.." you might ask. Read on and I'll explain what I'm thinking about. It's by no means a complete solution (hence I said 'approach') but it may provide some food for thought. And if at the end of it you say 'screw off.. you're nuts! - I'm using Lightroom/Aperture/Iphoto', then go right ahead...

And also bear in mind that this was based on *my* workflow needs. Yours of course might be totally different.

The first thing I asked myself was 'what do I really do with the photos on my CF card?'. In it's most basic form, here's what I do:

1. Bring them into the computer.
2. Back them up to other media (usually DVD).
3. Make adjustments to them (crop, resize, colour correct, etc..).
4. Tag and/or rate them to make future searches easier - I've been hesitant at this.
5. Publish them to Flickr, and/or
6. Publish them to my blog directly, and/or
7. Print them

And what more specifically are the problems I have with the systems I've used in the past?

1. Many apps take a data-silo/lock-in approach to my photos and metadata (as I outlined at the start).

2. I find that my system gets 'photo-bloat'. I end up with 10 copies of the same image, all of different sizes or different cropping for different purposes and of course, none of those 10 is the one I want.

3. I have a hell of a time distinguishing one version of an image with another without examining each one in some kind of image browser.

I intend to try and let the OS do part of the work. I want to leverage the power of the ubiquitous 'filename'. A large amount of command line work can be done (nevermind scripting) when the file naming system is made somewhat useful and more importantly, consistent.

Some ground rules:

1. Make sure your camera is set to keep image numbering continuous. So if you empty and format your card or swap in a new card, it doesn't reset the file name to IMG001.jpg again. That rootname (or even the number portion alone) is key to my proposed filenaming system. This name will anchor all the versions of a given image.

2. Subjective or verbose filenames are Satan's work:

eg. UncleBills_birthday_Cousin_Huey_drunk_05.jpg

Don't even go there. This type of thing does nothing to help searching 6 months for now. Avoid it at all costs.

3. Exif data is your friend.
Make sure any editing you do (and any editing software you use) keeps the Exif data intact. Try not to duplicate any of the Exif data in your filename unless it's really necessary.


Folder structure:

On my linux system, I use F-spot to import my images. I shoot mostly in RAW, so a typical image filename would be 'img_2345.cr2'. F-spot creates a nice directory structure where the /photos folder contains a /2006 folder which itself contains monthly folders. Those monthly folders contain date folders which are created based on the Exif information in the images. So if the img_2345.cr2 file was shot on July 21, 2006, the path to the file would be:

/photos/2006/7/21/img_2345.cr2

Many systems do a similar thing. The Canon software on my XP system would create something like:

\MyPhotos\2006_7_21\img_2345.cr2

In any case, make sure you set up a workable date-based folder structure. Unless you're a professional photographer and have to separate things by project, a simple date-based folder structure is a good way to go.

I would NOT create separate subfolders for different photo tasks unless it makes sense. So I want my cr2 file, my jpeg master and any subsequent versions of that jpeg all in the same folder.The naming system I came up with makes better sense having all the versions of a file in one folder.

A Necessary Next Step (for RAW shooters):

Next I would create a duplicate set of jpegs from your RAW files. These would be colour-corrected and tuned with a raw conversion program. These would NOT be scaled, resized, or otherwise modified in any way. You're trying to create a master copy of jpegs here. If of course you're shooting jpeg anyway, then this step is unnecessary (other than doing some general colour correction if required).

Now the fun part.

Here's where you come up with a file naming system that will perform a kind of 'version tracking' with the filename itself. Here are a few simple principles I tried to follow when coming up with it:

- All naming to be lowercase (in windows this doesn't matter, but linux is case sensitive - so it's important)

- All modifications to the filename to be made using suffixes and not prefixes. I want the original image name ('img_2345') to remain intact in all versions. This way when you list these files they are all grouped together.

- Demarcation between suffixes to be done with a period. I think it looks a lot cleaner than using underscores, but the choice was arbitrary.

- An indicator of image size in the filename would be helpful but wouldn't be a requirement. So I considered it optional.

- The order of suffixes should match the actual order of processing if possible. This just gives you some clues as to how the photo was processed just by looking at the filename.


So what would my system look like:

Based on what I do with my photos, here's my file naming system. Yours of course might be different to suit your own needs:

rootname.purpose.modifications.sizeindicator.jpg

rootname - This is the original filename minus extension (eg. img_2345)

purpose - This is a short (2 or 3 character) suffix indicating the purpose of the image file. For me this would be either 'bp'-blog post, 'fl'-flickr post, or 'pr'-for printing

modifications - I do 3 general things to my images: crop, resize, and correct ('correct' meaning colour correction, levels, enhancement, noise-reduction etc..). So I will assign a short suffix indicating what type of basic correction was made to that specific image. 'cr'-indicates cropping, 'rz'-indicates resizing, 'co'-indicates corrected.

sizeindicator - An optional suffix, but when it is used I will likely use a '700w' OR '600h' suffix to indicate 700px wide or 600px high respectively. I would describe the width or the height but likely not both unless absolutely required. This gives a quick indication of the size of the image without having to open it up in an image browser.

So looking at a directory listing of an image with several versions might yield (path omitted for clarity):

img_2345.bp.cc.rz.500w.jpg
img_2345.fl.jpg
img_2345.pr.cc.rz.1500w.jpg
img_2345.cr2
img_2345.jpg


A couple of key things to remember:

- Keep It Simple Stupid: or you won't remember the system or use it!

- Document it. If you come up with a system, jot it down and stick it to your monitor. Don't give yourself excuses to let it fall apart.


Some discussion points:

- A system like this allows for a quick search for all versions of an image. So simply searching for 'img_2345*' will yield all the versions of the image.

- You don't have to create endless subfolders for each type or use of an image. Subfolders are good... to a point, after which I find they become a nuisance.

- You won't always need a thumbnailing program to tell versions of an image apart.

- A simple file naming scheme like this doesn't do anything about metadata. And seeing how that was one of my key gripes (way up at the start of this post) that is a big problem. I think metadata is useful. I'm not saying give up on F-Spot, Picasa etc.. I intend to still use them for metadata and visual searches on images. However they don't give me a flexible versioning or naming system. I will still use them, but for more limited uses.

- Maybe a system wide file tagging system is in order. Not just image files, but all files.

- Applying a system like this retroactively to existing images is of course near impossible. All I can say is that if you think it's a good idea, then start doing it now.

- Overwriting of a parent file with a child file is always a risk (for instance say you resize a file and save it out without changing the filename, overwriting your jpeg master copy). I have two ideas about this:

1. Perhaps using file permissions might be an answer, at least to protect your master files. Maybe after importing the RAW images and creating the jpeg master files, you set all of those files to be 'read-only' in the file permissions. This would prevent overwriting of the master files later on.

2. Getting into the habit of creating a file with the 'purpose' suffix prior to doing any editing. So if you know that you're going to publish a given file to Flickr and print it, you could create the img_2345.fl.jpg and img_2345.pr.jpg files PRIOR to doing any editing.

So that's it for now. I really didn't think it would take so long to explain a system so simple, but I think the reasons behind wanting to do it are valid ones. I'll probably continue to post about my workflow as I fine-tune it further.

Of course if you disagree, or have some something to add, correct, or improve by all means do so in the comments below.