Build a Nearly Bulletproof Backup Strategy with Btrfs & Snapshots

[00:00:00]Now, backups are boring, incredibly boring. It's not a subject that people like to talk about. A bit like the dentist. You just don't want to talk about it. You don't want to go. You're not interested. But, when disaster strikes, then it becomes very useful.

[00:00:16]So, backups are something that we really should worry about, but we don't. I don't know how many times I've had people come to me and they say, "This hard drive failed." Or these things got deleted. And I say, "Well, what have you got on backup?" And they say, "I don't have any backup." So, anything we can do to make backups easier and more manageable is a good thing.

[00:00:36]Now, in this video I want to talk about Btrfs and snapshots and how they can be incredibly useful in managing your backups. It should make your backups more easy and maybe even a little bit interesting. So, if you want to find out more, please let me explain.

[00:00:58]So, the first thing to do is look at Btrfs stands for the B-tree file system.

[00:01:05]I normally call it butterfs. Some people call it betterfs.

[00:01:09]So, butterfs is a modern Linux file system designed to scale. It focuses on maximizing data reliability, storage efficiency, and simplified administration. Now, it has lots of built-in things including snapshots, which is the main focus [music] of this video, and also checksums along with native RAID. Now, it uses butterfs copy-on-write as its architecture and we're going to dive into that in a second. So, the key thing about butterfs is you've got snapshots, you've got checksums, and you've got copy-on-write.

[00:01:43]And these things combined together give us an excellent tool in our toolkit for managing backups. So, what is copy-on-write? Well, let's say we here is a very very simplified overview of file system. If you have a file like readme.md, it has to occupy some space on the disk, and the disk is divided into a thing called blocks. So, there's a kind of a table that says this there's a file called readme.md, and it uses this block. If you've got a longer file, let's say like a photo, then you've got photo one, and it goes it uses this block, and then this block, and then this block. Sometimes the blocks can be consecutive. So, look, 3377833779, and sometimes the block can be somewhere else on the disk. Obviously, part of the thing about making disks efficient is how you arrange these blocks, but that's more than what we're going to talk about today. And again, a video file would have even more blocks. So, basically, you have a list of files, and then you have a list of blocks associated with each file. Now, this is obviously an oversimplification, but that's the basic concept. So, what is copy-on-write?

[00:02:45]Well, let's say I've got a file readme.md, and it takes up, uh, you know, one block on the disk. And a block might be, let's say, 4K, something like that. Now, when I want to change that file, rather than actually modify that block, write into this block, all the data gets written into a new block. So, let's say I've modified the file, and now a new block is written with the changed version of the file. And then finally, the file readme.md is pointed to this new block. So, on copy-on-write, you copy any existing data that you need that hasn't been changed, and you write out new blocks. You don't ever go back and modify existing blocks.

[00:03:25]And this block, of course, now 22, uh, 23993, because it's no longer used, it gets put back into the pool of empty blocks that can be used for other files.

[00:03:34]Now, with a snapshot, what happens is I've got readme.md, it occupies that block, and then I say, "I want to make a snapshot." So, I've now got a second list of all the files on the file system, and now in this one, I kind of here, the snapshot version of readme.md also points to here. Now, the first thing to notice is that they're both pointing to the same block. So, I'm not using any extra disk space because all I'm doing is just keeping a copy of the list, which is quite relatively small, you know, compared to an MPEG-4 file or a picture or whatever. This is just a list of all the files. And they both point to this same block. And so, if I now modify it readme.md, the same thing happens before. I've got this new block that I'm going to use instead. It's got the changes written into it. And now, I link readme.md to the new block. But, the important thing is the snapshot version readme.md still points to the old block. So, now this block remains used. It doesn't become free like it did before. So, on the current file system, if I go and look at the file, it points to this new block and it gives me my new file. But, if I go back into the snapshot and say, "What did this file look like then? 5 minutes ago? 5 days ago? 5 months ago?" It goes, "Well, here it is here. This is the copy that you used to have historically." So, the great thing about snapshots is they don't use any extra disk space. They just keep a copy of how things were. And then, when you change files, when you add files, then the disk space grows because now you're changing blocks as the files get added and get changed. So, I'm going to give you two demos now. One I'm going to give you a PC demo. So, I'm running openSUSE running on a PC with a Btrfs root filing system. And then, I'm going to show you a TerraMaster NAS running a version 7 of their operating system with a Btrfs formatted disk. And both of them allow you to do snapshots based on the underlying technology that you get from Btrfs. Now, Btrfs is a Linux filing system. Now, it is supported by most of the major distributions. It just depends on what level of integration you get. Now, distributions like openSUSE allow you to install the root partition as Btrfs and it will just do the full install. In fact, the default is Btrfs and you get it for free as you install it. Other operating systems, let's say like Ubuntu, support it, but you manually have to take a disk, format it, uh you know, partition it, format it and so on and say this is a Btrfs uh disk. It's not something you get uh as part of the standard installation. You're probably going to get something else like EXT4 or something like that. So, I'm using open SUSE because it will give me Btrfs out of the box. Now, one thing to note when using the Snapper tool is that SUSE doesn't include your home directory by default. So, you run this command, snapper -c home create config/home, and that will actually add your home directory to the Snapper configuration tool, which is what I'll show you now.

[00:06:36]Okay, so here I am on the open SUSE desktop. I have opened up a file browser here on the right-hand side showing two files I've got in my documents folder.

[00:06:45]Here, I've got the Snapper tool that you can just uh run down here from the start menu.

[00:06:51]And the thing to note here is it's talking about the root partition here at the top. So, I'm going to click on that and go down to home. As I said, that adds it when you run that command I showed you a minute ago, that adds that to the Snapper tool. Now, here is a list of all of the snapshots I've got. So, what we can do now is we're going to say create a snapshot. Give it a name, snapshot week one. I don't know, whatever. Okay, I'll go ahead and it's now created that snapshot there like that. Now, over here in the uh document window, let's create ourselves a new text file.

[00:07:22]>> [snorts] >> Okay, so we'll call it uh work one {dot} text, okay? And we can edit that and just put some stuff in it.

[00:07:30]Okay, it doesn't matter what it is right now. Okay, so there we go. There's work one and our text files. Let's also imagine that we deleted text file two, okay? By mistake maybe, whatever, it's gone now. So, we've got text file and work one. So, things have changed in my document folder. So, now I can go ahead and I can create another snapshot. So, that's midweek one. Okay? So, I've done one and I've done some days of work and I'm creating another snapshot. Okay. So, now let's go back into snapshot week one and I can do show changes.

[00:08:04]Okay? And that's going to show me the changes with my actual current filing system. And if I go into documents here, you can see that text file two is in here, which it wasn't before, and I can actually go down here and click restore to restore it. And work one has now been created, new file that was created. So, I can actually see at any point what the difference is between a previous snapshot and my current snapshot. And then I can actually, you know, restore those files if I want to. Now, I won't see in here text file txt cuz nothing's changed with that. So, it's the same that I've currently got and what was historically there. But this old file, I can get it back. Let's try that. If I hit restore, okay, "Do you want to restore that?" Yes, I do. So, it's going to go ahead and do that. And now if we go over here to the file browser, look, text file two has come back. So, I've been able to get a file that I've deleted by just going back into an old snapshot, which is a snapshot in time.

[00:09:06]It's also worth mentioning at this point that snapshots are always read-only. So, you can't actually delete things in a previous snapshot, which is great because once they're there, it means that they are they're kind of they stay there and they're not going to get accidentally deleted. Now, you can also see these files from the command line.

[00:09:22]Here, I've opened up a console. I have got full root access. And so, if I go to {slash} home, because that's the area that I added into the snapper there with that command, I can see that there's a directory called {dot} snapshots. So, I can go into dot snapshots and now I can pick which snapshot I want. Now, if you look over here it says 12. We can go back here and we can see they're numbered 1 2 3 to 13 is the latest one.

[00:09:47]12 was the one before we made the changes. So, we can go into 12. Okay, and we can go into snapshot, go into Gary, go into documents and now in here you can see that original snapshot text file and text file two. And I can copy those out of there or do what it is that I need to do with them. So, there you go. Snapshots on SuSE. Okay, so here I am on my TerraMaster NAS and as you can see I've got two volumes and they're both using Btrfs, Btrfs and Btrfs. Now, there is a tool that you can use. I've got it installed. It's called snapshots.

[00:10:22]So, there it is there. Snapshot. You can install it using the normal app center installation. So, if we open up snapshots, okay, like this. Now, I'm also going to open up a normal file manager and you can see here in one of my volumes I've got a folder called Gary 2 and I've got this is an important file in it there like that. Okay, so what I'm going to do now is I'm going to say I want to create a snapshot in Gary 2. Okay, and we're just going to say go ahead and create that. Is it something you want to do on a schedule? No, I want to do it now. And of course you can set this up to do this every day, every week, whatever like that. Okay, so it's going to go ahead and it's created that snapshot. Now, as before, I'm going to Oh no, I've accidentally deleted my important file. Oh dear, what am I what am I going to do now? Well, I can go over to the restore function here and I can say I want to make a new restore.

[00:11:13]What do you want to which snapshot do you want to do it from? A local one?

[00:11:18]Yes, which one is it? Okay, and I can pick the snapshot that I want to restore.

[00:11:24]In this case, it's the one I've just made here, the manually taken one. Okay, and I want to restore it now. Next.

[00:11:31]Okay, you sure you want to do that? Yes, go ahead. It's running the restore.

[00:11:36]Okay, that has completed and if we go back over here now to our file browser, we can see there this is an important file. It is back, so it's been restored.

[00:11:45]Now, when you have snapshots, you can also make them visible, so allow snapshots to be visible. And this is quite useful because it means you can kind of browse in there. So, now if I look in here, there is this folder snapshot and I can go in there and here I can see all the snapshots that I've done. And if I go into this latest one here and there you can see here's all my files. This is an important file. So again, you can browse just like we did in openSUSE using the command line. You can browse in there and I can copy them out of here and so on. So, again, snapshots are absolutely brilliant this time on a TerraMaster NAS using Btrfs.

[00:12:17]Now, one thing to talk about, one of the disadvantages, is this freeing up of disk space because a snapshot keeps copies to blocks that were previously used in the past, if you were then to delete some big files now, okay, you might not actually get the disk space back because they're still in the snapshot. So, if you had a a big video file for example, and you put it in a snapshot and then you say, "Oh, I want to free up some disk space." and you delete that file, your disk space usage won't go down because actually it's still in the snapshot. So, that's something to be aware of that when reclaiming disk space, you have to take it into consideration what files are being pointed to in the snapshot as well. Now, the combination of RAID and snapshot, so for example, I showed you the TerraMaster NAS there, so I've got physically I've got two disks and they're running, you know, with mirroring, so if one disk fails, my data's still safe cuz it's still on the other disk. I can put in a new disk, it can rebuild it all and that's absolutely great. And then you combine that with file versioning really with the snapshot, so I can say take a snapshot of all my data today, do it next week, do it the week after that, and I've kind of got this historical uh backup which can also uh survive, you know, file deletes. Let's say by mistake I delete a file that I want. I go, "Oh, no, I deleted that file." Well, don't worry, it's in my snapshot. I can go back into the snapshot and bring it out of there. And also, a disk can fail. And I can say, "Don't worry, I'll rebuild the the disk by putting in a new disk and the RAID handles it." So, the combination of RAID and snapshots is really, really good for kind of managing historical backups that can protect you against accidental deletions, unwanted changes, uh and so on, and also physical drive failure.

[00:14:04]But, RAID with snapshots does provide excellent local protection, but there is still something it can't help you with, and that is if you get a burst pipe or something like that, fire, flooding, electrical damage, whatever. So, if you get a burst water pipe and it starts pouring water over your NAS unit or over your PC, then no matter how many disks you've got in there, no matter how many snapshots you've got in there, if that device gets physically damaged, the water damaged, that's it, you've lost all your data. So, that's why for a full backup scenario, you need another copy of it somewhere else, preferably on a another copy completely separate uh in a different place, physical location. So, if you do have fire, flood damage, theft damage, whatever, you can still recover your data.

[00:14:50]Uh and then for maybe for us home users in the cloud, or maybe it is on a on a separate device, in a different room, or you know, somewhere safer that you can if, you know, if you get a unlucky and something electricity problem and it causes a power supply to burn and it burns out something, you've got a separate device that wasn't affected by that. But, the more redundancy you have, the further distance physically, electrically from your two backups, that's the perfect backup solution.

[00:15:20]Okay, so there you have it, Btrfs and snapshots. Is it something you use already? Something you're now thinking about using? Love to hear your thoughts in the comments below. Okay, that's it.

[00:15:29]I'll see you in the next one.

[00:15:32]>> [music]