After an ugly laptop's HDD failure and all the hassles associated with reinstalling and recovering I decided that I need to do better backups to protect my mental wellbeing in the future. In this post I describe my personal storage and backup solution and introduce the backup script that I've created.

A Bit of History and Context

In 2005 I had a Windows XP laptop with HDD split into Windows C:\ and Data D:. I used Acronis to perform "online" backup of system C:\ while Windows was running and used rsync to backup non-system partition D:\ to home PC's backup drive.

In 2010 I got a new laptop with Windows 7 and split HDD into C:\ and D:\ to separate OS from the rest of the data. There are free tools like gparted that can resize Windows C:\ partition to ~100GB without OS reinstall. I stopped using Acronis due to instabilities on x64 Windows in favor of built-in Windows 7 Backup for system C:\ and kept using rsync for the rest of the data.

In 2011 I created 4x1TB RAID5 using a PCI-X raid card on home PC for archives and added 3TB drive for backups. This remains my primary storage and backup machine. Inside PC

Backup Motivations

I have multiple devices: a laptop, a home PC, a blackberry an android phone, tablets and kindles with a few terabytes (and counting) that I want to keep safe from hardware failure and loss.

I enjoy photography and have a lot of RAW photos and converted JPEG images. I store all JPEGs and recent RAWs (~150GB) on my laptop for quick access and editing in Lightroom. I store the archive of all RAW photos on home PC (~350GB). Unfortunately my laptop drive(s) are never large enough to accommodate everything at once and I have to use home PC for archives which complicates my backup a bit.

I prefer to store my "active" data, archives and backups under my control on a home PC. So far I don't use any cloud storage for backups, because of privacy concerns, high cost and inability to support all of my devices and scenarios.

Backup and Archive Scenarios

Over time I have defined a few backup scenarios that formed my backup habits:

  1. (weekly) Windows backup for "bare metal" recovery
    Content of System drive C:\ using Windows Backup for quick recovery after physical disk failure, as I don't want to reinstall Windows and applications from scratch. This is configured to run automatically using Windows Backup schedule. Backup Control Panel
  2. (daily and conditional) Active content backup
    All changed files from laptop, phone, other devices. I backup laptop every night and after significant changes in content, like import of the recent photo RAWs from SD card to laptop.
  3. (weekly and conditional) Handheld devices backup
    I use rsync for devices that can be mounted as USB disk drive in Windows (Blackberry, Kindle) and Bittorrent Sync for Android, iOS to copy all changed files to my laptop.
  4. (conditional) Archived content backup
    When I move files from laptop to home PC archive, I explicitly ensure that updated archive is backed up. This happens when I run out of disk space on my laptop and move RAW photos to home PC's RAID5.

I recommend everyone to adopt at least 1. and 2. and configure some sort of periodic backup of all data. You can use my approach with rsync, Acronis or Windows Backup.

How to Use Rsync for Backup in Windows

To take care of my backup scenarios and needs I created a script rsync_backup_win.sh that you're welcome to use. Once it's configured, all you need is to press enter to initiate the backup (or use task scheduler):

$ sh ~/rsync_backup_win.sh netshare
== Using Home Samba/Network profile
== Mount network share to use as backup destination: /cygdrive/y

== Start rsync full D:\
Thu, Mar 25, 2014 16:21:28
== Running cmd:
rsync -rltDHh /cygdrive/d/ /cygdrive/y/D --exclude /Windows/ --exclude pagefile.sys --exclude \*RECYCLE.BIN\* --exclude "System Volume Information" --exclude "Lightroom 5 Catalog Previews.lrdata" --exclude /VMs/

Details on how to use the backup script are provided in README and in the comments within rsync_backup_win.sh.