Now that I have a baby, I’m taking a lot more pictures and recording a lot more video than I ever have before. And as you can imagine, I’d hate to lose any of it. So I spent some time over the last few months researching backup options for Mac OS X. I’ve read many articles and reviews, talked with a number of colleagues (including some who write backup software and others who run datacenters), and checked out lots of software and gear (software like Time Machine, SuperDuper!, and Carbon Copy Cloner, external hard drives like those made by Seagate, LaCie, Western Digital, and also “smart” hardware like the ones made by Iomega, ReadyNAS, and Drobo).
Past Lives
I take file backups and redundancy seriously. You might think it’s because, in a past life, I was a systems and network engineer for an aerospace company, in charge of the backups for all of our corporate data, everything from payroll and HR to user-created files. Nobody ever forgets those middle-of-the-night visits to the server room, bleary-eyed, standing at the VT220 hard-wired console, using commands like dumpfs and restore, working through stack after stack of tapes from the fire-safe, so the VP of Something can get that file he deleted last week (or was it last month?), for that meeting in whatever town tomorrow morning.
But it started further back in time, when I was in my very early teens. I had a Commodore 64, and was using the GEOS operating system) and geoWrite, one of the very first WYSIWYG text editors, to create a paper for school. I’d saved up enough to add a 5¼" floppy drive, and although saving files was much faster than when the cassette tape, it was still slow, and generally I didn’t save very often, except, when I was taking a break or done for the time being.
If you have good file-saving practices, you’re probably shaking your head right now, but remember, I was about 10 or 11 years old at the time and I’d never lost a whole day of work before, especially not the day before it was due. So when the printer jammed and GEOS crashed and I lost most of everything I’d typed up, I was actually a bit surprised. Until that time, I’d always thought of computers and their hardware as being completely reliable.
Ever since then, I’ve been very careful to keep multiple copies of my work. In the old days, that just meant buying a second floppy disk (my entire allowance). Today, it’s not quite so simple. The choices we have both in software and hardware can be overwhelming.
The Short Answer
I’m now relying primarily on a combination of a Drobo, Subversion, automatic .Mac syncing, and occasional drive-cloning. Keep reading for the details.
The Data
As with most aspects of my life, my goal is to simplify. The fewer tools, applications, and devices I need, the better. In the past, I used to backup everything, but now, although I do backup my entire system, I’m mainly interested in a backup of what I consider to be critical data. This includes:
- Code and Writing
- Bookmarks, Address Book, Serials, and Passwords
- Photos
- Video
- Music
- Other Stuff
What about the applications themselves? What about preferences? What about system-specific settings? Those are all handled by the system level backup/clone. But you know what? After many years of preserving those files, I’ve found that when it comes to actually restoring my machine or setting up a new one, I rarely took those things along with me, preferring to install only what I needed as I needed it. I’ve really been able to keep a minimalist setup in the process, shedding things here and there as I move from system to system. Still, it’s nice to know they’re there when I need them.
I’ve found that a multi-pronged approach works best for me, and I use a few different technologies to make it all work. Reading this, you might think I’m a bit retentive when it comes to data redundancy. And you’re probably correct … but then again, I’ve seen many drives fail in my time, and lots of data get lost. I just don’t want to take that chance with the stuff that matters.
Code and Writing
All of my code lives in a Subversion repository which is offsite and fully backed up, and I always have a relatively local copy ready to go, should something odd happen to the repository. The merits of using a revision control system are many, and well beyond the scope of this article, but if you’re not using something like Subversion, Git, or heck, even CVS, you really should be.
Another nice benefit to keeping code and writing in a repository, aside from the revision history, is that you automatically have an easy path to synchronized data on any machine you use.
Bookmarks, Address Book, Serial Numbers, and Passwords
I use .Mac to sync data between my Mac Pro and my Macbook, and .Mac keeps these things backed up in the process. It’s true that .Mac is a bit expensive, and this data is backed up elsewhere (see below), but I’m hoping that one day, Apple will do something really amazing with .Mac and I’ll be glad I’m signed up for it.
I use Yojimbo, from Bare Bones, to manage all of my serial numbers and passwords. It’s a great utility, and it can even use .Mac to sync (and therefore backup) your data.
Get a nice, reliable host like Mailtrust (formerly Webmail.us), and store your mail there in IMAP folders. They keep good backups, and you can go from machine to machine and everything will be right where you left it. Unfortunately, Mailtrust has recently recently raised their prices, so you might want to check out Google Apps for a free (but possibly less robust) solution. The main thing is IMAP folders and a company that backs things up reliably.
Intermittently, I will copy the contents of the important IMAP folders to my local system within Mail.app, as further redundancy, but that’s probably overkill.
System Cloning
Although I’m doing this much less than I used to (see The Solution below), I sometimes still like to make a complete clone of my system every once in a while using SuperDuper! and a Seagate FreeAgent Pro external hard drive.
Many people use SuperDuper! clones as their primary form of backup, as I did for a long time. I love the idea that at any moment, I have a full clone of my system ready to roll in case of a severe emergency. When it comes to cloning your Mac’s hard drive to a spare disk (external or internal), SuperDuper! is the best. It uses a technique called Smart Update to only update files that have changed, keeping the cloned drive identical to what’s on your Mac’s primary hard drive.
What you get is a bootable clone of your hard drive, as up to date as the last time you’ve run a backup. You can take this drive, plug it in to any Mac, and boot up from it. What you’ll get is your very own system, just the way you left it, ready to roll. From there, you can continue working right where you left off. This is really useful if your primary Mac needs a repair or had a total drive failure, and if you have access to a spare machine.
You can also use SuperDuper! to clone data onto a replacement Mac from a Mac booted into Target Disk Mode or from a cloned external hard drive. For people who like to preserve everything, this is often easier than setting up a new machine from scratch and can be faster than using Apple’s Migration wizard or a Time Machine restore.
What you don’t get is an incremental backup, where changes to files are preserved, or files you’ve deleted are still available. This is where Time Machine comes in.
Time Machine
Apple’s Time Machine is a really nice backup solution that ships as part of Leopard. For people who don’t want to think at all about backups, Time Machine is amazing. It just works, maintaining a copy of your entire system and updating things automatically as you go. It’s even smart enough to pick up where it left off if you have a laptop and disconnect from your backup drive, too. For most people, Time Machine is all you’ll probably need.
I threw an old SATA drive into one of the spare bays of my Mac Pro, and let Time Machine update it as it pleases. I don’t really need to have multiple copies of a single file, saving each revision, except for my code and writing, which are in Subversion, but I had the drive and the bay, so I figured why not? I have used it to restore a few files here and there, and it worked as I expected it to.
I wouldn’t want to use Time Machine to restore my data to a new computer if I was upgrading or replacing current hardware because I’d be forced to do it a certain way, and because it seems like it might take a while. I’d still prefer a clone with SuperDuper! or a manual copy. And while Apple’s previous backup solutions were sometimes problematic, I’ve heard enough good things and positive restore stories to believe that Time Machine might just work as described. My friend Duncan tried a Time Machine restore and it seemed to work just fine.
For my wife and parents, Time Machine is just right, and I can relax knowing that should something happen, they have a good way to get their data back … and I don’t need to babysit the process. I can just plug-in an external drive, let Mac OS X use it for Time Machine when it prompts, and walk away (fingers crossed).
Documents
I have a small collection of documents that I like to keep around. There isn’t much, these days, but there are a few things I wouldn’t want to lose. I rarely add to this collection of files, but I need to keep them around. If I were a designer, architect, or something like that, and I wasn’t using subversion, I bet I’d have a lot more of these.
Photos, Video, and Music
Like any modern parent, I take several hundred pictures of my new baby boy every single day. Each picture takes several megabytes of storage, and none of them, not even the poorly framed, out-of-focus ones, will ever be deleted. If you’re a parent, especially a new parent, you know what I mean. And don’t even talk about the video.
What I wanted was a solution that would grow as my library grows, but it had to be simple.
The Solution
Enter the Drobo. After talking about it at length with a few friends, most notably John Gruber and Andy Ihnatko on Episode 19 of The Talk Show, I decided I’d give it a try. On the company website, the Drobo is described as “fully automated data storage that ensures your data is always protected, your capacity is unlimited and requires no special knowledge or expertise.”
Sounds too good to be true, right?
Well, in fact, it really is just that great. Unlike the RAID systems I’ve looked at (and forget about the dozens of RAID servers I’ve configured and built over the years) which require some degree of configuration and often come with a heavy price tag, the Drobo offers truly redundant storage and is simple to use and easy to setup. And it’s affordable.
Just turn add a few drives, between two and four of them, turn it on, plug it in to your computer, and away you go. The Drobo intelligently spreads your data across all of the available drives. If a drive fails (or if you remove it), the Drobo reconfigures itself to keep your data safe. If you add a drive, your storage instantly increases. In fact, this is how you upgrade your storage capacity. It’s unlike other RAID devices because you don’t need to add drives of the same capacity. Just give the Drobo whatever drives you happen to have handy, say, a 750GB Hitachi, a 500GB Western Digital, and a 250GB Seagate, and it will use them all intelligently, giving you the most redundancy and the most space it can. And as prices on larger drives fall, you can pick one up, remove an old drive, add a new drive, and behold: an instant increase in storage space. And you can do all of this without any interruption. You can continue to work, creating, moving, or deleting files as you please as the Drobo works in the background.
It’s quiet, runs cool, and although it’s USB, I can’t really tell a difference in speed when compared with the same files stored on my primary, internal drive. Maybe high-end media people doing video editing or drive-intensive media work might be able to tell, but for me, it’s just fine. It just sits on top of the Mac Pro and does its thing. I can easily connect the Drobo to any other Mac, or use the DroboShare to make the Drobo a stand-alone NAS device.
So how am I using it?
I’ve moved my iPhoto library, Lightroom catalog, iTunes library, and my main Documents folder to the Drobo. I suppose I could get fancy and mount the Drobo as my home folder if I wanted, but that’s not without its issues. I could probably also create aliases (or symlinks) for the folders in my home folder too. But I’m content to just leave things as they are, keep things in folders right on the Drobo, and access them directly.
Reader Request: If anybody can tell me the best way to easily force iMovie 8 to store everything somewhere other than the default locations, please let me know in the comments.
I’m still cloning things to my external drive once in a while, just to be on the safe side, and letting Time Machine do its thing to that internal drive. Even though I trust the Drobo, having a backup I could easily take offsite is pretty handy. Many people use the Drobo as their Time Machine drive. And it could be used as a SuperDuper! clone too. Mac OS X treats it just like a normal USB drive, and that means it’s bootable.
Final Thoughts
I’d be curious to hear about how you’re backing things up, so be sure to leave some comments and share your thoughts and experiences.
Truth be told, as much as I really do trust the Drobo and my other drives, there’s still a part of me that wants to create DVD copies of everything for storage in an ETL Verified 1-hour fire protection (up to 1700°F), waterproof safe. I’ll be sure to let you know how that goes.
Commenting is not available in this weblog entry.
Subscribe via RSS
Travis Cripps
29 April 2008 at 7:07 pm
I use SuperDuper to back up my computer every night to an external Raid-5 NAS volume. Works like a charm.
Koz
29 April 2008 at 7:12 pm
I personally use TimeMachine as my primary backup. It’s saved my ass a few times by tracking down files I accidentally deleted, but I’ve yet to do a full restore.
I also store copies of my critical stuff (tax records etc) on s3, manually copying them up using transmit.
Finally, my server backups are handled either by my host, or by using duplicity (again, to s3). This came in damned handy not too long ago when a cheap-and-nasty server lost a harddrive and we were back up and running in a few minutes.
Scott M.
29 April 2008 at 7:32 pm
re: Drobo
Just a not to say that a friend of mine who was an early Drobo user lost all of the data on his Drobo after a firmware update:
http://ihnatko.com/index.php/2008/02/18/sun-times-drobo-and-droboshare/#comment-5121
Natalie
29 April 2008 at 7:32 pm
I’m not currently backing anything up! I have an external HD, about 250GB, that worked great when I was on a PC but it’s crap now that I’m on a mac. I can _read_ SOME files (about half are invisible) but I can’t _write_ anything at all.
I’d like to have something I can take back and forth from PC to Mac because I don’t know when I’ll be able to get Photoshop/Fireworks on my Mac so I still use my PC quite a bit for graphic work. Do you know if your solution would work for me? Could I plug it in to my Mac and PC interchangeably? Is there anything that would do that?
Thanks for doing all the research on this! For the same reason as you, I don’t want to lose all the photos and video of the kiddos (not to mention client work)!
Robert
29 April 2008 at 7:43 pm
For Natalie: The HD was probably formatted as an NTFS drive. Backup the data to your PC, and reformat the external drive as FAT32. Transfer the data back, and you now have a drive that will work with either operating system. Your external drive probably has the steps in the manual.
Marc
29 April 2008 at 7:45 pm
I’ve been using S3 via JungleDisk and it’s been pretty good so far.
Dan Benjamin
29 April 2008 at 8:02 pm
@Scott - I realize that there’s always the possibility of data loss, even with Drobos and RAID. Hence the SuperDuper! cloning and the desire for DVD archival.
That said, firmware updates are *always* scary, and a good backup of your data (and for me, Drobo is live data, not backup) is especially necessary before you run one, in my own experience.
I feel for your friend, though.
James Duncan Davidson
29 April 2008 at 8:07 pm
Indeed, multipronged approaches work best. As cool as the drobo is, I’d consider it one device that needs to be backed up to a secondary.
Sean O'Steen
29 April 2008 at 8:33 pm
I too have a drobo and was even a part of the Drobosled (NAS attachment for Drobo) beta test group. I think it’s a great product, but it is certainly not quiet. Once the ambient air temperature in my office goes above about 75 degrees F, the Drobo’s cooling fan spins up to medium and can be heard by callers on the other end of my phone calls. If the fan speed goes to its max, It’s hard to carry on a conversation. Even though it’s a pretty device, I’ve had to move it over to the bookshelf across the room to keep the fan noise down.
Greg
29 April 2008 at 9:30 pm
I’ve been using Amazon S3/JungleDisk as well, but mainly for smaller work documents that I want stored offsite (in case of fire/burglary/aliens/etc.) and soon for photo backups. I also use an additional internal drive with Time Machine in case I have an “uh oh” moment. I actually hosed my entire desktop folder (don’t ask) and got it back no problem—but I was sweating the whole time.
I’ve considered buying one and I hope you’ll give more feedback as you continue to use the Drobo. Is it well constructed? Is the power supply internal, or an external brick?
On a side note: in my tech support past, most external drive weirdness (aside from clicking drives) tended to be bad/dying power supplies. If anyone reading this has a bad external, always try the drive in another enclosure or internally before giving up on it.
One last thing, and it may just be me, but I have absolutely nothing good to say about Maxtor drives (I’m not sure if LaCie uses them exclusively, but they do use them). Western Digital drives have always worked best for me (knock on wood).
Alderete
29 April 2008 at 9:43 pm
I back my wife’s and my laptops up to an Infrant ReadyNAS server once a night, using SuperDuper. This backs up the entire drive, to disk images stored on the server.
I *also* back up both machines nightly using CrashPlan, http://crashplan.com/, to the CrashPlan Central service. This backs up only our Home folders, minus our iTunes music libraries. But it’s off-site, and versioned. $5/month, for up to 50 gigabytes, and $1/10 gigs after that.
I’ll eventually expand the ReadyNAS and start using it for Time Machine networked backups, but that’s low priority. It’ll be more convenient than CrashPlan for accidentally deleted files, but otherwise I don’t expect it to provide much *new* in the way of protection.
Natalie
29 April 2008 at 10:08 pm
@Robert, thanks! I’ll definitely look into that. Would be so much easier - not to mention more frugal!
scott
29 April 2008 at 10:27 pm
” If anybody can tell me the best way to easily force iMovie 8 to store everything somewhere other than the default locations, please let me know in the comments.”
I haven’t tried this with iMovie, but, in the early days of iTunes, I accomplished what you’re asking for by creating an aliased folder that pointed to an external drive folder and gave it the same name as the one iTunes was looking for and put it in the expected directory and it worked like a charm. iTunes would follow the alias and look-for/store music in the external folder. No need to do that today with iTunes, but perhaps you can fool iMovie into doing the same thing.
Rahul Gonsalves
29 April 2008 at 11:38 pm
I take a Zen approach to backup. At the end of a large project, (say, just before uploading a site), I delete all the files and change my name to Trudy.
Chris Bowler
29 April 2008 at 11:40 pm
I prefer multiple devices as well. Time Machine with my Time Capsule and weekly clones with SuperDuper to an external drive are enough for me to feel secure.
Mike West
30 April 2008 at 2:35 am
Regarding DVDs and data loss: If you’re truly paranoid about losing data, and want to verify that no blurry shot of your loved ones are lost in the creation and subsequent storage of not-so-trustworthy DVDs, take a hint from the practices that have sprung up around Usenet file-swapping. Specifically, generate parity files that enable recovery from Bad Things like accidentally flipped bits.
I’ve written up the process I’ve started using for my Aperture masters: http://mikewest.org/archive/safegarding-your-data-with-parchive It’s hopefully far more paranoid than I’ll ever actually need to be. :)
Martin Westin
30 April 2008 at 8:58 am
There is for me the important distinction between backup and archive.
Backup is to me what can keep my current digital data from being lost by a mechanical failure and/or user error.
Archive is the long-term backup. The - keep this safe in case I need it in 5 years - kind of thing.
Since I have only ever deleted something important by mistake twise in my life (yes I am a pretty careful guy), I have made the decision that mechanical failure is what I primarily need protection from. I, too, use a combination of svn and cloning of my internal drive. I did try TimeMachine a while but it did not work too well on my MBP (some cpu-usage related problem).
I have a really big problem finding reliable media for archiving. These are my experiences… most of them bad ones actually.
CDs and DVDs are pretty useless IMHO. They do not archive for very long before parts of the data start going bad. Rewritable once are no good for backup since they are too slow and even less reliable. I would trust a DVD-backup for absolutely not more than 3 years.
HDs that are stored are not too good either I have found. Use a drive for a year and then stick it in a closet for 5 and you may already have problem retrieving data. I am starting to think that I would have to “touch” och copy the data at least once a year or so to keep it “fresh”. Is there an application that can rewrite data on a drive to “refresh” the magnetism so to speak?
I have not tried DAT-tapes much. They are a bit out of the pricerange of freelancer and home user.
How do you guys store your DVD-backups if they are still intact after 5-7years?
Mark Newhouse
30 April 2008 at 12:18 pm
Other than DVDs and the smaller bits that are stored on servers off site, how are you handling off site copies of your data? That always seems to be the bottleneck for me.
Anthony
30 April 2008 at 2:29 pm
I am curious myself about what to do for off-site backup of large files like photos and video. I don’t shoot nearly as much as I should, but would not want to pay the monthly bill for what I have as it is for Amazon or other services.
I am currently using Super Duper to do nightly changes and a once a week erase and copy from one internal drive to another.
I think I probably need another level of backup in that solution to mitigate data loss migrating from the main drive to the copied drive before I notice it.
Part of the problem is that Super Duper is so seamless, unless I go out of my way to check, I assume it is working fine and everything is good.
James
30 April 2008 at 7:13 pm
Time machine saved by bacon just about a month go. Older PowerMacG4 MDD - Original hard disk that came with it, about six years old. Still pretty usable system, amazed at how each subsequent MacOS upgrade gets faster on older hardware!
Upgraded to Leopard as soon as it shipped and about 4 weeks later… The wife screams for me and the hard disk was making those sick mechanical noises of death. The spinning beach ball hung the system and a subsequent reboot resulted in the infamous blinking question mark.
Went to Circuit City the next day and bought a new hard disk. It took longer to figure out how to pull out the disk caddy in the PowerMacG4 then it did to get the Time Machine restore running. Once the new disk was in, booted to the Leopard DVD and ran Disk Utility, partitioned and quick formatted the disk. Rebooted to the Leopard DVD again and chose restore from backup and it prompted for the source and destination drives and about 2.5 hours later, all was well with the world. Zero problems, it just worked and I was back to an hour prior to the failure!
Here’s an article to ensure Time Machine runs even if you are logged out - http://systemsboy.blogspot.com/2008/04/time-machine-after-logout.html
Just have to be careful with that option… i.e. dismount the backup drive, manually, etc.
Jim Parsons
30 April 2008 at 11:03 pm
Best. Advice. Ever! Hard drive storage is so inexpensive now that it doesn’t make sense NOT to be hard core about backing up.
El Aura
30 April 2008 at 11:14 pm
- Time Machine: first level of redundancy, only about one hour lag time, multiple versions, fully automatic
- Subversion: adds multiple levels of redundancy (offsite; redundancy of remote server; frequent, labelled, documented revisions), collaborative work, for stuff that really matters
- two clones using different software (SuperDuper and Synk Pro), one always offsite, each gets updated about once per week
Critical is always the weakest link (don’t delete images from your memory card until you have verified the images are good on disk AND back-uped at least once.)
Joshua Ochs
30 April 2008 at 11:18 pm
A very good read. I particularly like you you have specific backup strategies for different types of data, and different failure scenarios. The one that you haven’t covered is off-site; I’d really urge you to do that with as much of your photos and video as possible - they’ll ultimately mean a lot more than your SVN repo (although I gather from the post you already know that!).
I used to use SuperDuper to external FireWire drives. These days I use Time Machine to a Time Capsule, partially because I’ve done a full restore (and trust it) and partially because it requires no thought whatsoever. I do exclude certain large chunks of data that I don’t need temporal backups up (video, virtual machines, etc) and run automatic SuperDuper backups (via some launchd-foo) of that content nightly.
The great thing about Time Machine/Time Capsule is its simplicity. It just works, requires no effort, and doesn’t require me to have a drive array hooked up to run my backups. They just happen. For those who have laptops, using a NAS (network-attached storage) is a necessity. You don’t want to even have to bother to hook up your backup drive; it should just happen.
Another nice thing about the Time Capsule is the USB server. In the future should I need more backup space than 1TB, I’ll probably get a Drobo and hook it up to the Time Capsule. All the advantages of the Time Capsule and the Drobo, without any more fuss.
Dave McCaskill
30 April 2008 at 11:25 pm
I’ve been putting together a backup system for my family over the last couple of months, prompted by the many Gb of music and photos accumulating on my Powerbook G4. In addition to my Mac, my wife has a laptop running Vista, daughter has a desktop running XP, and we have an old Dell desktop (with an additional 750 Gb drive dropped into it) that I’m turning into a server for storing everything onsite.
Because it’s a mixed environment with an XP system acting as the main onsite backup system, I’m using Syncback (http://www.2brightsparks.com/) running on the XP server to connect to each system and automatically copy any changed files over to the 750 Gb drive nightly around 3 am. If it encounters a problem with any of the backups, it emails me that there was a problem, other than that, it runs unattended. At the same time, I’m backing up from the 750 Gb drive to mozy (http://mozy.com/) to take care of off-site storage. I know I just don’t have the self-discipline to physically swap drives and take them off-site for storage on a regular basis. I’m only backing up files that can’t be replaced, programs, system software, system preferences can all be replaced, family photos can’t.
Once I’m convinced that everything is running reliably (it’s been running about 2 weeks so far without a hitch), I’ll stuff the server in a closet and use Timbuktu from my Mac to keep an eye on it.
Chris
30 April 2008 at 11:27 pm
I use large USB sticks in a safe deposit box.
I have two sets and I switch them about once a month.
I only have about 10GB that I store but if you have a lot more you could use hard disks.
Jason Green
30 April 2008 at 11:28 pm
I’ve been backing up by Time Machine, but I’ve always been afraid of a break in where my mac and my Time Capsule would be taken.
Just today I sent my Time Capsule into a data center so I can back up to there via the net using time machine. (I used the service from Macminicolo.net)
Noah
30 April 2008 at 11:31 pm
I use Time Machine running on two hard drives in a RAID 1 to back up my main system (which is on a RAID 0). I use it mostly to back up my documents, movies, photos and songs. Anything else on my system is either downloadable from elsewhere on the internet, or of no importance. If my hard drives crash, or I reinstall, I don’t particularly care about my Applications, I’d rather redownload as I need to reduce clutter.
Personally, I find that being so anal about backups is pointless, since most of our data is already stored redundantly across the internet.
Sean O'Steen
30 April 2008 at 11:41 pm
This is a follow up to a comment I posted on April 29th. As a result of that comment, someone from Data Robotics did contact me regarding my fan noise gripe, and they are looking into it. Let’s hear it for judicious use of google alerts and a group who seems to stand behind their product! CustomerInteraction++
elmimmo
01 May 2008 at 12:10 am
I cannot use Time Machine, unfortunately, since I have my portable’s account FileVault’ed (I know I can still use it, but to restore the full home folder at once, instead of on a per-file basis).
My concerns about security (hence FileVault) are only when I am on the go, i.e. I am fine with having a non-encripted backup at home. What I use, then, is manual (one a week more or less) execution of rsnapshot (<http://rsnapshot.org>, something like Time Machine but a bit a pain to configure and definitely not so neat looking) of both the full system and my home folder. It is not bootable, but not that hard to do a full restore either.
One thing I regret is that while this is somewhat of a valid substitute for restoring of lost/deleted/corrupted files, I miss the back-in-time support of certain application such as Address Book.
Sam Hill
01 May 2008 at 1:23 am
I back up with rsync to a firewire drive and to my dreamhost account (300GB available). I code in SVN or Git on a remote repository, but anything else I don’t care enough about, or I keep versioned files xyz_001.ai, xyz_002.ai, etc.
Charlie Hartman
01 May 2008 at 1:38 am
I’ve been researching this for a while, and I am planning to set up a spare G4 with a big cheap drive and store it offsite at a friend’s house.
Carbon Copy Cloner will sync over the internet using SSH, and by doing the sync nightly (and the initial sync locally) it generally shouldn’t be too much data each night. It isn’t a true incremental backup, but CCC will at least put anything that has been deleted on the local system into folders on the remote system. I’ll just need to keep an eye on the space, or figure out a way to get rid of the deleted folders whenever space is an issue. This would be nicely complimented by using time machine locally.
Personally, I’m pretty big on offsite-ing anything I consider really important, and as much automation for the process as possible. A safe is nice, but isn’t necessarily going to help for a flood or tornado (unless it’s a bank vault). Even if it is a bank vault, it’s still reliant on the data getting there somehow.
My might be overdoing things a little, but I figure it’s cheap and simple enough, so why not…
charles Parnot
01 May 2008 at 2:09 am
I am surprised nobody mentioned Flickr as an offsite backup for pictures. I still have a real hard-drive backup + DVDs + CDs spread around the family, but basically all my pictures are also on Flickr. All for $25 / year. I hope Microsoft won’t put its greasy hands on them, though.
Stephen Darlington
01 May 2008 at 7:09 am
The long, pre-Leopard answer is on my website: http://www.zx81.org.uk/computing/opinion/backup.html
The short, current version: a Time Capsule for my day-to-day backups and SuperDuper! once a week or so. I keep meaning to take my LaCie off-site but I’ve not quite got around to it yet…
I like the fact that both Time Machine and SuperDuper create their backups as just a regular file system, which means I don’t need any unreliable, difficult to use software to restore stuff.
David Avraamides
01 May 2008 at 8:06 am
I used to use SuperDuper! but when I upgraded to Leopard and started using Time Machine, I stopped. While the idea of having a complete bootable clone is nice, the odds of my internal disk crashing are low enough that if/when it does happen, I don’t mind reinstalling OS X from scratch and then restoring data from Time Machine.
But I’m surprised you don’t have any sort of automatic off-site backup. As other people have mentioned, I use JungleDisk (Plus) with Amazon’s S3 service and use an rsync script to efficiently backup my key files (encrypted, of course). That way if my house burns down while I’m away, I still haven’t lost anything.
Jay
01 May 2008 at 8:07 am
My backup routine:
Continous Backup
Mozy.com account for critical files backed up offsite (~8gig at the moment)
Daily
Chronosync of the Home Folder to an external FW drive (scheduled to automatically run at lunch)
Weekly
SuperDuper clone to a dedicated FW drive
Mozy for the mac has been great and saved me a few times even thought it’s still in beta.
Mark
01 May 2008 at 8:45 am
(1) SuperDuper to two onsite hard drives, with a biweekly third offsite.
(2) Time Machine to a larger onsite hard drive.
(3) No #3, but I’d feel safer with a Mozy-like option at a less flaky company (like Google), and our server is backed up with svn and MySQL dumps downloaded to my local Mac.
In sum: My internal HD has EVERYTHING and can be duplicated from any of three SuperDuper drives. Unless you do so much video that you can’t fit everything on internal, this is the way to go. SuperDuper and TM are completely no-brainer automatic solutions. Offsite is not. If you are completely anal, then a solution that requires you to do something to back up that isn’t automatic is best.
Using your internal as your primary backup mean you NEVER have obsolete media you can’t read or you have to transfer to newer formats. No SCSI problems and the like. You always migrate your internal to the newest format as you upgrade your computer. That is the best approach.
Most common problem: Internal crashes (1 year MTB for HD), SuperDuper solves it in minutees.
Second: “Where’s that file?”: Time Machine solves it if it didn’t fall between the cracks of when TM backs up.
Third: House burns down: Off site SuperDuper, ready to go in an extremely pressure-filled time when you don’t want to think about recovery of data.
That’s all there is.
aninterestedparty
01 May 2008 at 9:14 am
The problem with backing up to Flickr is there’s currently no easy, supported(!) way to get all of your data back—especially not all the metadata you’ve put effort into creating.
It’d do in a pinch, but getting thousands of bits of data from Flickr does not appeal—until Flickr implements a fully-fledged ‘Here’s a ZIP file with all your data’ or ‘Mail me all my data here’, it’s not a true backup solution.
@HivelogicAuthor: you didn’t mention encryption in your post at all; do you do entirely unencrypted off-site backups? Are you worried about corruption in, say, a .DMG, file? Is the time not worth the benefit? Do you just not care? Or do you do it but just didn’t mention it? :)—I’d be interested to hear something on this from you.
Thanks for your time!
Richard
01 May 2008 at 9:18 am
I’ve been using SuperDuper! since it was released and have never had an issue with it.
I use three small external firewire drives to backup my MacBook Pro, one each successive day with one going out to my truck weekly and the one there being rotated back in.
Even though I’ve never had an issue with SD I recommend occasionally starting up from a clone just to make sure it works and to make sure, if you’re using Intel Macs that you’ve got it formatted using GUID.
Rossco
01 May 2008 at 9:28 am
You’re really going to trust your stuff to SuperDuper?! I don’t advise it, they screwed customers hard with Leopard, in two ways. First, by being so terribly slow to update, but worse by continuing to offer the Tiger version during that period when they knew, full well, that their “smart” copy routines corrupted your data when used on Leopard. It’s fixed now, apparently, but I can’t trust a company that had a data corruption bug and did nothing, literally, to advise or protect customers.
Shamino
01 May 2008 at 9:29 am
My iBook isn’t backed up. But I don’t care because it only contains copies of files from my home/desktop Mac. When its drive dies, the only thing I’ll lose will be some time.
My home Mac (a PowerMac G4) is backed up this way:
- Weekly (friday night) backups to FireWire hard drive, via Retrospect. I have two 300G disks which, presently, can hold three full backups of my system each. My weekly backup rotates through these six backup sets. This is my primary backup.
- Less frequently (every two weeks, typically), I make a tape backup (to a FireWire VXA-1 drive) of all but my iTunes collection. I have three sets of backup tapes which these rotate through. The most recent set is stored off-site (in my desk at work)
- I also make tape backups of my iTunes collection, but that is only done when I add new music, which usually about once a month. I have two sets of tapes for this. The most recent backup set is stored off-site.
- Finally, I make an archival DVD backup (via Toast) of all Finder-accessible files just before a major system upgrade. I’m not concerned with recovering from a botched upgrade (the hard drive/Retrospect backup is for that), but with the ability to fetch some useful system file that was obsoleted. (For example, Apple sometimes removes sounds, fonts and screen savers that I like. I sometimes don’t notice until months later.) I used to make this backup with Retrospect, but Retrospect’s support for optical media has become virtually non-existent, so I now perform that backup using Toast.
- I plan on buying/building a 1TB FireWire drive for use as Time Machine storage, but I have not yet done so.
This system isn’t perfect, but I figure that any catastrophe capable of destroying both my home and office at once will be sufficiently large to render the issue moot.
Note that I do not use hard drives for off-site storage. I don’t think any media that can be damaged by dropping it on the floor is appropriate for off-site storage. Tape is slower and more expensive, but I think it’s best for this purpose.
Josh
01 May 2008 at 11:19 am
The D-Link DNS-323 is an inexpensive 2 bay SATA NAS that supports raid1 out of the box:
http://www.dlink.com/products/?pid=509
Its also got a healthy hacking community:
http://wiki.dns323.info/
Lastly, I have mine set up not to do raid1 (since its technically provides high availability, but not true backup), however it mirrors volume A to volume B nightly:
http://forum.dsmg600.info/t1150-Tutorial:-Backup-Everything-from-once-night.html
The advantage is that you can pull a mirrored drive (for offsite storage) and replace it with a new drive.
DVDR or soon to be affordable 50gb blu ray DVD backups stored in a firesafe or safety deposit box are also a good plan B, especially for photos/video
dansays
01 May 2008 at 1:11 pm
Very topical, as I just recently re-evaluated my backup strategy.
Between Kathryn and myself, we have three laptops and a Mac mini. Each of these computers is backed up to a Time Capsule which, compared to the ease of configuring Time Machine with a FireWire drive, has been a royal pain in the behind. It seems to be running fine now, though.
We also have an Infrant ReadyNAS NV+ that we use to archive music, photos, and video. The ReadyNAS will mimick an iTunes share, so I find myself keeping very little music on my laptops.
Once a month, I take a naked 1TB drive and connect it to the ReadyNAS via a CoolDrives USB-to-SATA adapter, and archive its contents. I then connect it to the Time Capsule, and do the same.
I also have a backup job that runs in our NOC in Milwaukee that compresses down the absolutely critical, we’re-dead-without-this-data stuff once a week, and copies it over the network to my office in New York; this gets copied to the drive as well. (This is only a small part of a far more complicated enterprise backup system we have in place.)
Then, the naked drive gets carefully placed back within its anti-static bag, and I walk it over to our safe deposit box at Chase.
I’ve thought about using Super Duper or Carbon Copy Cloner, but managing that across four computers is just too much.
Rich Trouton
01 May 2008 at 3:19 pm
At home, I’ve got a mix of backup solutions as I’ve got two laptops (my wife’s, running 10.4.11 and mine, running 10.5.2), two G4 towers running Mac OS X Server 10.4.11, another G4 tower running Mac OS X Server 10.5.2 and a Mac Mini home theater system running Mac OS X 10.5.2.
The laptops use rsync to back up to one of the 10.4 Server towers, the 10.4 towers use Carbon Copy Cloner to back up their boot drive to another internal hard drive, and the two 10.5 Macs use Time Machine to back up to attached external drives. I’ve had a boot drive fail on me once, so I know that the CCC backups in the 10.4 Servers work the way they’re supposed to. I’ve also tested the Time Machine backups with a complete restore of my Mini’s boot drive and, aside from my having to recreate a /var/log/apache2 folder (so that Apache would run again), the restore worked as advertised.
Alex
01 May 2008 at 3:57 pm
First, it is curious to me that only only Shamino and Richard have commented on the source of their data (and Dan’s brief mention of his macbook).
I think the desktops and portables need different strategies. There simply are some things that you can take for granted with desktops (network configuration, peripherals, etc), that you cannot with portables. Even when I am home, my MacBook Pro could finish the data in any of four places.
** Dan, how do you backup the data on your MacBook OTHER than “Bookmarks, Address Book, Serial Numbers, and Passwords”? Is it all Subversion? **
Those of us who just use portables—rather than having a portable as a secondary machine—have different backup needs. Furthermore, those with multiple computers (i.e. the spouse’s macbook) need to take that into consideration.
So, what do I do? Well, in the past have done weekly archives of my entire documents + desktop folders to an external drive, eventually pruned back to monthlies. But this was manual.
Now, I use TimeMachine from two machines to an external drive attached to TimeCapsule. In short order I will replace that external drive with a Drobo. I also have a Vault for my Aperture library. Unfortunately, TimeMachine does not play nicely with Aperture—it’s a TM bug—and so Aperture needs its own backup. However, vaults don’t go over networks. So, I use a spare 2.5” drive for that.
The Drobo will be the backup, as all the working files will continue to exist on the internal drives of my MBP and my wife’s MB.
I do not have an offsite strategy at this point.
Brian Hutchison
01 May 2008 at 8:27 pm
Great post. I use Time Machine + Super Duper myself, and am going to a policy where I have one set at the office and one at home (separate external HDs).
I also carry a smaller external drive with me and keep my photos on. That way, my photos are available to edit anywhere. Master RAW files (CR2) also get backed up to DVDs (two sets, one off-site).
To the person who suggested Flickr for backup: if only they offered RAW/CR2 file support! :) No one has yet been crazy enough to do this that I can find, at least at all-you-can-eat pricing. And with DSLRs, I eat a lot of GBs.
El Aura
01 May 2008 at 8:50 pm
A lot of interesting points made here. A few of my own:
- almost everything can be done over a network connection (local or remote) by using disk images
- mounting the disk images automatically might be the difficult part, particularly if they are encrypted (TM can mount an unencrypted image in case of Time Capsule but probably not an encrypted one)
- corruption is a potential issue with disk images (particularly encrypted ones), even if Time Capsule uses a disk image for TM ...
- MacFuse and MacFusion allow mounting of SSH connected file systems this could allow encrypted offsite backups, potentially without using disk images, IPSec connections are an alternative (but more difficult to set up) and probably still requires images (?)
What comes up as design goals is:
- automation
- offsite
- hardware redundancy
- software redundancy (don’t rely on one software alone)
- versioning
- bootability
Versioning and bootability are rarely available from same the software (Synk offers bootability plus automated archiving, which is a sort of versioning).
Ideally, I’d like to have both local and remote versioning and bootability (this might mean four different systems, or three, if Synk or similar is one of them). Locally I would not need encryption, on the remote location it depends (and bootability and encryption are tricky to combine as well).
Tony Requist
01 May 2008 at 11:18 pm
As the author of geoWrite for Commodore GEOS (oh so long ago), I will claim a bit of ignominious credit for starting you down the path of never trusting any hardware or software :)
Matt Henderson
02 May 2008 at 1:26 pm
A few years back, I switched from a desktop/laptop, to a single (maxed out) MacBook that I carry between home and work each day. It has a 250 GB internal drive, that’s far insufficient to hold all my media. Here’s my backup scheme and still-open issues:
* At both office and home, I have a USB hub connected to the MacBook, of which hang small 250GB USB drives. I use SuperDuper! to keep both drives mirrored daily to my MacBook drive. (I use Knox to encrypt the entire drive at work. That’s great—I can leave the drive at work, and not worry about data security if it’s ever stolen.)
* At home, I have a small server (a Mac mini) with three drives attached: two 1TB drives, and one 500GB drive. I use the 500GB drive as a Time Machine backup target (over the network).
* I keep my iTunes library (master) on one of the 1TB drives. At the moment, there’s about 300GB left. I use something like SuperDuper! (rsync) to keep the second 1TB drive mirrored to the first (to provide redundancy). (I decided to write a Ruby script, which uses rsync, to keep the drives mirrored, so that I can get an email if the master drive starts getting dangerously full.)
* Last but not least, I’m experimenting with S3/JungleDisk, which would presumably replace Time Machine. S3 provides endless storage, offsite, and JungleDisk provides intelligent backup with archiving. I’m VERY happy with the reliability and implementation of JungleDisk, and version 2.0 looks like it’s going to bring a lot of enhancement.
* The obvious problem is the upload bandwidth. It’s been running for a week or so, and has backed up 3% of my MacBook’s drive. For that reason, I took a good hard look (and refactoring) of my drive’s folder structure.
I’m pretty happy with this overall backup strategy. It’s probably a bit over-redundant, but it’s dead simple. Once setup, I never do anything; it’s all automatic.
For me, the biggest open issue still is management of my photo, music and video libraries, given that my MacBook won’t hold them all, but I *do* want to be able to access at least a portion of this data at all times with my MacBook, wherever I am.
I have my iTunes library (as mentioned above) on a server drive at home. I pretty much add videos and music to the MacBook wherever (usually purchased through iTunes Music Store), and then occasionally do a “Consolidate Library” at home, and then use FileBuddy to find and delete big files from the local MacBook drive. This feels like an unsatisfactory hack.
I have my entire Aperture library on my MacBook. It’s 30GB, and growing. Hopefully laptop drive capacities will keep ahead of my photo shooting. (I still have to learn about Aperture Vaults, to understand if that’s a feature that would provide some off-laptop archiving options.)
Videos. Big Mess. I have two categories of videos—those I could stand to lose (basically movies and TV shows purchased through iTunes) and those I couldn’t (basically family movies). At the moment, the former dominates the latter (in storage space), and it’s all sitting at home on the server. Recently, though, I bought a Flip Video and the yFlicks management tool, and I could easily see myself shooting much more video now and wanting to have much of it locally accessible on my MacBook. So the whole topic of video storage/management is still a big open issue for me.
Riddler
02 May 2008 at 2:24 pm
I’m in kind of the same boat as Matt Henderson. I actually recently sold my mac mini server and moved to a Time Capsule/Airport Disk setup. The Time Capsule works fabulously for Time Machine backups of two notebooks, but all of my media (music/video/photos) outgrew the portable’s drives a while ago. I have a 1TB drive hooked up as an Airport Disk off of the Time Capsule, which works very well for the music but doesn’t work with video and is rather painful for the iPhoto library. Plus I have no off-site method for this and to make things worse the 1TB drive isn’t being backed up at all right now.
I suppose the best solution would be a Drobo NAS for the media and two 1TB drives to back up the Drobo and Time Capsule to take offsite. Then I’d still need a drive to clone my MBP to since I have a very optimized audio production setup on it. Oh, and a drive to backup my production raid.
It’s simply endless… and overwealming when you think about it all.
Bob
02 May 2008 at 5:28 pm
Most of my bits are in the family photo/video category too, and my iBook is my only computer now. I don’t use iPhoto, but keep the media files in a yearly/monthly/dated-event hierarchy of folders. Here’s my current (but always looking for improvement) strategy.
1. Weekly manual backup to DVD, of Documents folder plus as many recent months of photos & videos will fill the disk. That way I have redundant copies of any particular month as time goes on. I store the most recent disks at work, and vary the DVD brands to have less risk of a bad batch of disks. Our safe deposit box contains a 1-DVD-per-year archive of the photos and videos. (I really ought to re-burn new copies of those periodically.)
2. Monthly Super Duper clone to 1 of 2 alternating firewire external drives, with the older one stored at work. This is what I depend on for iTunes music backup, but most of those are CD rips or DVD audio Wiretaps anyway. For apps I’ve installed, I keep the (other than physical disk) installers in a subfolder of Shared.
My email is all IMAP at work or Gmail, and I keep serial numbers & passwords on an encrypted disk image in Docs. I’m not using Leopard yet, so Time Machine’s not on my menu. I’m glad to see so many ideas here, and I’ll check a few of them out.
Charles Parnot
02 May 2008 at 5:49 pm
About Flickr: I mentioned it because it does give me some extra peace of mind, as a last resort. Once thing I did not realize when posting is that unless the files are in jpeg, you end up with a different picture on the web site (that and some lost in metadata). Thanks for the comments on that.
Pictures are one of the things that matter most to me, and it is nice to know they are there too. But of course, I don’t consider it a real backup, and I would not want to have to retrieve all my pictures from it. I also have a Time Capsule setup, and a SuperDuper! cloning every week :-)
Jeff Judge
06 May 2008 at 2:42 pm
Hey Dan,
I’ve been using Google Apps for my personal and office email for over a year now and I’m very happy with it. I think the IMAP support works fine (not sure how it could be less optimal per your comment).
I’ve also been using DropBox (http://getdropbox.com) for all storing and syncing documents across machine, plus sharing with others and it works fantastically. DropBox is the perfect solution for sharing stuff with my wife and office folk and keeping everything in sync.
I’ll check out Drobo, thanks for the recommendation.