Scary, but worth it

Backup and recovery is like auto insurance. You never know how good your policy and agent really are until you crash and actually test them. Given the massive wave of recent Ransomware attacks and a nagging feeling I didn’t know exactly how my backup and recovery plan would work in the event of a loss, I decided to put my backup and recovery solution through the paces. I backed everything up three times (Cloud and two external drives) to three separate locations in case things blew up, then erased my harddrive to see how long it took to restore back to normal.

Things didn’t go according to plan.

UPDATE: Aug 28, 2017. Crashplan no longer supports Crashplan for Home consumer products and is focusing on enterprise customers. Use Carbonite instead.

What I wanted to test

Simply put, I wanted to test if my backup and recovery system worked the way it should. I wanted to test three things in particular:

  1. Could I recover to full performance if my computer (and data) was lost, stolen, compromised or corrupted?
  2. How fast could I recover?
  3. How hard would it be?

In other words, could I simulate losing a machine, buying a brand new one and reinstalling apps, data, photos, email and other business-critical work assets to get back up to speed as soon as possible? 

I use Crashplan and backed up my entire harddrive so the backup included photos, documents, music, video, system setting and apps. I was looking for a one button solution, i.e.  ‘hit restore and everything magically comes back to life.’ Unrealistic? Maybe. But when I can order an Uber from my phone in three swipes why can’t restoring a bunch of files be just as easy?

What happened

First of all, it’s really unnerving to willingly delete all your files. No matter how many backup copies I make, it just feels weird.

I first tried to restore from the harddrive linked to Crashplan and had to click Destinations, then select the location of the external backup that matched my computer ID (a crazy 15 or seventeen digital number), then click OK.

It didn’t work.

I had to reference multiple help articles on to match destinations. Had trouble locating the drive and the backup archive. Got a message: “Unable to connect. Check your network.”

Not a promising start.

In the end I couldn’t restore from the external drive due to the above errors so I tried the cloud solution, CrashPlan Central.

When I hit “Restore” for the cloud backup, I got an “Archive I/O error.” What is an archive I/O error? This is as far from simple as you can get. I spent over an hour reading FAQs and support articles, to no avail.

Tried customer service live chat at 4 PM PT.  They are only open 9-5 Central Time.

Next morning, 16 hours after the backup event started, I chatted with T. from Code42. T. said he had to “rebuild the maintenance archive and that will take a few hours, at the least, so check back tonight.” That afternoon T. emailed me and said the maintenance archive was fixed and to restart restore. Upon restart, 56,000 out of 350,000 files were errors. Since it was 2 PM PT on a Friday I didn’t want to be stranded all weekend with no data so I picked up the phone and called D. at Tier 2 support. He took over the screen and increased the memory Crashplan could use. The restore process was drastically faster as a result.

D. recommended we select just the user folder which contained working files such as docs and photos because restoring system files and apps from a backup is not ideal. Crashplan and other small business backup solutions (Crashplan does have an enterprise product but freelancers aren’t enterprise-level so that doesn’t count) are better suited for file backup (i.e. music, photos and documents). This makes sense but negatively impacts my goal of a magic restore button… D. recommended I restore apps from their original source. I get it but don’t like it since it’s a hassle.

So after about 76 hours the restore began and only took about 6 hours. It took another hour or so to get core apps installed.

lessons learned

Focus on files

Until we find, and test, an all-in-one backup solution that works, focus on backing up your core files that contain critical assets such as photos, client work, personal files, etc.

Embrace escalation

When a restore event hits, don’t be afraid to pick up the phone and get tier 2 support to do a screenshare. Crashplan had great support and D. fixed the problem in one call.

Test your solution

Hate to say it but you really do have to test your solution to know it works. You don’t have to test the whole enchilada but test a subset of files to go through the process and make sure it works.

Note about Crashplan:

Crashplan had great customer care despite a less than intuitive user interface. I initially used their live chat which worked well but it became apparent I needed to escalate with a sense of urgency to get my backup fixed and restored. They have extensive documentation to help with a restore but I’d rather see an easier-to-use solution that didn’t rely so heavily on how-to and FAQ articles.