Backing up to Backblaze with Rclone and Restic

Backblaze Logo

I talk about how I switched from Backlaze personal to Backblaze B2 with restic and rclone, and draw some comparisons between them.

If you use a computer and care about the data stored on it, follow best practice and keep backups:

  • The live data
  • A local backup
  • An offsite backup

But there are so many options available for how you manage these.

XKCD 1718

 Backblaze Personal

Backblaze Logo

For the past couple of years I’ve been using Backblaze as my offsite backup solution. It works really well, and it’s saved me a couple of times so in my mind it was worth it. That is until I considered just how much I was spending on it.

I had in the order of 100GB being backed up at any one time, and was paying $110 for 2 years. I found Backblaze’s cloud storage (B2) calculator, and figured that for this kind of data I should be paying nowhere near as much if I was managing the backups myself. Cloud storage is pretty cheap these days, and unless you have terabytes of data to backup, going DIY is probably cheapest.

So I decided to make the switch from Backblaze’s managed personal backup to a self-managed backup hosted on Backblaze’s B2 cloud storage platform. The following may also be of interest to Linux users, as Backblaze doesn’t offer a personal backup system for linux OS.

 Restic

Restic

The first solution I decided to try and use was called restic. It’s a backup system that comes with encryption capability out of the box and packs everything into a repository, not dissimilar to how git works.

The command line interface is fairly easy to get started with, but you need to manage storing your passwords and so on yourself. Either as files or environment variables. Because it uses a repository system, it can intelligently avoid duplicating data that are the same, cutting down on storage costs.

The first thing I noticed with restic was how slow it was. Even backing up to a local repository took time. I tried to speed that up by setting up an exclude file, which did wonders, but restic still wasn’t quite fast enough. A problem that was compounded when I started connecting to B2 over the network.

restic also offers a snapshot system. I’ve read the documentation many times over and can’t for the life of me figure out how it works. I’m used to managing repositories and file histories with git, and that’s fine, but restic just seems a bit too complicated for me.

Given that you want a backup system to be easy to use, to avoid making mistakes, I decided to look for something else.

 Rclone

Rclone logo

The most important difference with rclone is that it does away with repositories altogether. It’s primarily aimed at keeping things cloned and in sync. Luckily though, the B2 integration can use B2’s bucket history to keep track of changed files so it’s still possible to use it to store versions of backups or even files that you’ve deleted.

When you first run rclone config, the interface seems a lot more complex. But actually, the nice walk through menus it gives are very helpful in getting things set up. B2 is a supported backend, with its own documentation and options.

But where it all started to fall apart for me was with the encryption. rclone does no encryption at all by default. It has a whole separate system for that which, somewhat confusingly, is treated as a separate “backend” on top of the one you currently use. Which means If I want to securely use Backblaze I have to make a B2 backend, then a crypt backend pointing to that B2 backend.

The concept of managing multiple backends also took a bit of getting used to once I started trying to figure out how to make buckets for B2. Especially with all the subtle warnings it gives about being careful when using buckets and crypt together.

Eventually figured it out. To use a crypted B2 backend I needed to:

  • Make a new backend ‘b2backend‘ pointing to B2 using my ID and Key.
  • Make a new backend ‘crypted‘ pointing to ‘b2backend:unique-bucket-name
  • When backing things up with rclone sync, point to ‘crypted:directoryname

And it is much faster than restic, especially when using the --fast-list option. One thing I did note was hat though the excluded files list could be re-used between restic and rclone, rclone needed a slash after names if it was to apply the rules to directories as well. For example ‘Test’ would block the file ‘Test’, and also any folder called ‘Test’ in restic, but in rclone it will only block files called ‘Test’. In rclone ‘Test/’ would block a folder called ‘Test’.

Comparison

Here’s a summary of my experiences using the different software I’ve discussed:

Software

Backblaze Personal

Restic

Rclone

OS

Windows, Mac

All

All

Price

$60/year

Software is free

Software is free

Storage

Free unlimited storage included in price

Your storage may vary in costs

Your storage may vary in costs

Storage Locations

Only Backblaze personal servers

Many storage options, including local and B2

Many storage options, including local and B2

Storage Optimisation

Default exclusions for some files, GUI to manage

Must manually define exclusions, avoids duplicating data

Must manually define exclusions

History

Keeps last 30 days of files

Managed repository system lets you track snapshots of files

Doesn’t keep histories by default, some backends do support it

Difficulty of use

Easy

Medium

Medium-Difficult

Encryption

Off by default

On by default

Off by default

Backup frequency

Automatic in background

Needs manual run

Needs manual run

Speed

Fast

Slow

Fast with –fast-list option

I think for now I’ll stick with rclone, once you get the hang of configuring it, it’s actually quite nice to use. It’s also fast. And cheap if you find a good storage provider, which Backblaze does nicely with B2.

Join the Conversation

  1. I think you should give restic another look. Restic has advanced block level deduplication, which is why it needs to scan and index all your files on the initial backup to – probably what you’re experiencing as “slow”. Subsequent backups should be *much* faster as it doesn’t need to re-index all the existing files.

    The snapshots are simply that “snapshots” of your file tree at the exact time that the backup was ran. This is how most backup software works – by creating “views” of your files at the time of the backup such that full file trees can be restored to working order should something bad happen. restic handles this very efficiently using deduplication that only transfers new and modified blobs of data. This works even for individual files – if only a subset of the data inside the file has changed, restic only transfers this changed sub-data.

    rclone is a fantastic piece of software, but in comparison to restic rclone it is “just” a file transfer tool. In fact restic can use rclone as a backend to backup data to any of the destinations supported by rclone.

    restic is still fairly new software, but it’s used and trusted by many people (as can be witness by following the development on GitHub). I agree the documentation still have to ways to go before being completely accessible.

    In any case, you should use what tool is best for you. I just wanted to clear up some possible misunderstandings about the differences between rclone (a file synchronization tool) and restic (a backup tool). Thanks for writing about your experiences. Cheers!

    1. Thanks for the clarifications. That’s quite helpful. I’ve got a setup that works well for now with rclone, but I’ll keep an eye on restic as it matures and develops and may try it out again later.

Comment

Vivaldi