Bad things happen to good photographers; be prepared

I got a call from a fiend's office the other day. Their archive had been on 2 mirrored Drobo drives. They bought a new mac pro and suddenly the entire archive was unreadable. That is a very scary place to be. Years of work can be wiped out instantly.

I've written about archiving and backing up before, but since all most photographers produce are 1s and 0s it bears repeating.

the system protects against 3 things

• Mechanical failure
• File corruption and deletion
• Physical catastrophe

There are 3 things I'm after in my office storage system, live -spinning- files that I can access instantly, backups of those files and an offsite archive of them incase something terrible happens.

• Mechanical failure

The live files are the image files that I need access to every day. Primarily they are all the RAW files I've ever shot and the 16 bit tif files that have been converted from those RAWs. In my case that is a lot of RAW files, close to 2 terabytes a year. Some people are surprised that I keep them all, but I've found it is far easier to keep them all than to make decisions about what to discard. A lot of people use a DROBO for this primary storage, but I have always found the DROBO to be a bit too slow. 

For RAW storage I run a RAID 5 array off a High Point Technologies 2522 SATA card and a Raidon port multiplier box. This combination gives me lots of speed and near infinite expandability. RAID 5 protects against a single hard drive failure in the array and lets me get to the files even when the RAID is rebuilding.

The tif files are on 2 internal, mirrored 2 terabyte drives. I have them mirrored because I really need that information live all the time. If one drive goes down the files are still accessible while the array rebuilds itself. My business is based on those files being accessible so having them live is essential. Between the RAID 5 and Mirrored drives I am pretty well protected against mechanical hard drive failure. 

• File corruption and deletion

Mechanical failure is only 1 of the things that can shut me down. When I've lost pictures in the past it has been through file corruption. A mirrored drive or RAID array won't help there. If a file is corrupt on one drive it will be corrupt on the mirror. 

To guard against corruption I do daily, incremental backups. My system uses time machine server. Basically the software copies all the files on the first backup to another hard drive. On subsequent backups it only copies changed and new files but does not over write anything. It keeps all previous versions of a file. That way if something becomes corrupted I can go back in time to an uncorrupt version. This is also true if I have changed a file in a way I don't like or -um- accidentally deleted one. 

Time Machine backups get huge. Mine is on a Drobo 4-Bay. The Drobo's pokey FireWire 800 interface is no problem here as Time Machine runs in the background.

 

• Physical catastrophe

This is probably the scariest issue. What if all my computers are stolen, or there is a fire or a flood? Lets face it, bad things do happen.

To deal with that I have an archive offsite. My RAW files are arranged by year, so at an offsite location there are drives for 2004, 2005, 2006... For 2010 there are 2 drives A and B. On alternate weeks either the A or B drive is in the office being copied with the newest files. The other drive acts as the back up. I use Carbon Copy Cloner and it runs practically in the background.

I do the exact same thing for the Tif files that I do for the 2010 RAWs. A and B drives, CCC backups. The only thing I have to do here is remember to physically switch the drives. 

So that is how I keep my files safe. I am CERTAIN that I've missed something so please weigh in with your own ideas in the comments.

 

Stephen Alvarez

Sewanee, TN

Comments

[Stephen Alvarez]

@Fred,
I do check them every now and then, probably not often enough. They are on standard esata drives.

[Fred]

Hi Stephen,
and are you sure that your offsite archive drives (2004, 2005, 2006,..) will still be readable when you will need them ? do you do check them regularly like the A & B drives?
I strongly recommend to scan them once in a while.
Fred

[Stephen Alvarez]

Bernard,
RAW data doesn't really compress so the Time Machine back up ends up being larger than the original. Remember it is keeping all the changed versions of a file. My RAW catalog is currently around 5 terabytes, the back up is close to 6.

[Bernard van Dierendonck]

Stephen

that's great! thanks for your information.

After a nights sleep and rethinking my system I figured that file corruption hasn't been an issue because I don't use RAID (so far). My structure is very archaic and manually organized. But now, confronted with a fast growing archive it's good to think about reorganization. Your strategy seems like a very good solution.

A question about your time machine backup: is time machine compressing the data? If you have 8 TB of photographs, does the time machine backup have the same size or is it bigger/smaler?

Best regards
Bernard

[Stephen Alvarez]

Bernard,
Thanks for the links.
The HP card is actually a raid controller that is doing the raid calculations so that your os is freed up to do other things. The drives in each array have to be identical, but different arrays can have different drive/PM box combinations. I suggest this route if you really need the speed. A RAID 5 array run off one of these SATA cards is very very quick. While the array is outside the machine, it appears internal.
There are a bunch of different raid controllers some of them are reviewed here
http://www.hptesata.com/review_contents.html

[Bernard van Dierendonck]

Dear Stephen

thanks! Now it starts to make a lot of sense (before I startet feeling a bit like a digital analphabete ;-)): So the HP 2522 is managing the PM and acts like a hub for more units. with array you say: in your case all have to be PM and inside those PMs the drives have to be identical too? Or only identical inside the PM and you can add other Raids to it?

I'll do some research to see if I can find this or a similar system in Switzerland. What I really like that you have a fast connection to your RAWs, so you can work directly of your external Drives. (also my MacPro is filling with data)
I guess the rest of your system (like mails, letters, texts...) goes also with time machine backup.

It's interesting because some other photographers also blogged about their backup strategies, like
http://blog.chasejarvis.com/blog/2010/06/workflow-and-backup-for-photo-video/

and vincent laforet

or this one (Drew Gardener) who had lots of issues with his Drobo Pro:
http://photography-thedarkart.blogspot.com/2010/09/end-of-my-tether-with-drobo-iscsi.html

But so far I missed the notion of the data corruption

So thanks again - it's been great to follow you on your blog and on twitter and I'm looking forward to the Paris article in a future ngm

Best regards

Bernard

[Stephen Alvarez]

Bernard,

Time Machine runs across the entire picture library. The first backup took the better part of a week. But subsequent ones do not slow down the machine noticeably as each incremental backup is rather small.

The way my RAID array works is the HP 2522 card is in a PCI slot in my MacPro. It is just above the graphics card. There are 8 cables that come out of the back of the card and each one will plug into a Port Multiplier enclosure. The Radon enclosure that I use holds 5 x 2TB drives. At RAID 5 it is formatted to just under 8 TB. I say it is expandable because I can continue to add enclosures (up to 8 off of the same card). The downside of RAID 5 is that all the drives in each array have to be the same manufacturer, size and model.

let me know if this makes sense.

-Stephen

[Bernard van Dierendonck]

Dear Stephen
thank you very much for your information! Very helpfu. First time I noticed the three dangers mechanical, physical and file corruption.
After looking up the links to the High Points 2522 and Radon Port multiplier I still don't understand this completely. If time allows could you explain this system or show a picture of it. Thanks!
And the time machine is running over your complete picture library? Is this not slowing your computer down a lot? Or is time machine only running over night?

Best regards from Switzerland

Bernard