Data Refresh for Long-Term Storage

Added: 2026-06-04

[00:00:03][music] >> Welcome to another video from explainingcomputers.com.

[00:00:19]This time, I'm going to discuss different methods for refreshing the data on hard drives or SSDs that are left on powered for long periods.

[00:00:29]This video is a follow-up to my recent episode on long-term data storage in which I recommended rewriting the data on a long-term storage hard drive every 3 years and reading the data on SSDs every year.

[00:00:45]Many people asked how this may be achieved and therefore I'm making this video.

[00:00:51]And if you're wondering why we only have to read the data on an SSD to refresh it, it's because reading data forces the controller to assess its integrity and rewrite if required.

[00:01:05]Before we begin, it's really important to stress two things.

[00:01:10]Firstly, this video is focused entirely on hard drives and SSDs that are left on powered for long periods.

[00:01:18]In contrast, if a modern drive is connected to a computer and in regular use, then its controller will be regularly performing housekeeping activities that will refresh the data.

[00:01:30]For example, SSDs will constantly perform wear leveling which moves data around to prevent some blocks of flash memory cells wearing out before others.

[00:01:41]However, if you have got a hard drive or an SSD in storage for years on end without being connected to a computer, without being powered up, then the controller cannot perform such housekeeping activities and this is what this video is all about.

[00:01:57]Secondly, my recommendations for when to undertake data refresh are based on minimum data retention expectations, not the life expectancy of drive media, let alone the average or maximum time that data may be retained.

[00:02:13]To be clear, it's perfectly possible that you could save data on a hard drive or on a new TLC SSD and come back to it in 5 or even 10 years and it would still be okay.

[00:02:26]But, if every bit of data, and I stress every bit of data, was still intact after 10 years, particularly on an SSD, then you would be beating the odds. And hence, my recommendation is to refresh data after the minimum period it's likely to be retained, as this is the safest option.

[00:02:49]With these critical points noted, let's move on to how to refresh data.

[00:02:54]And here, three options are available, which are to take advantage of cold storage practicalities, to use specialist applications, or to use operating system utilities.

[00:03:15]Right. Here we have two of my five LaCie d2 backup hard drives. And for a couple of reasons on these, I find that I seldom need to use special software or operating system utilities to refresh their data.

[00:03:31]And this is because of the way these drives are managed over time, and so I thought it was worth mentioning.

[00:03:38]Firstly, as hard drive capacities increase and as the quantity of data I have backed up and archive grows, so my external drives get upgraded.

[00:03:49]For example, when When first purchased my LaCie D2s, they had capacities of between 160 and 500 GB.

[00:03:57]But, over time, they've all received new internal drives, with this one now containing a 16 TB Toshiba mechanism.

[00:04:06]And indeed, over the years, I've made videos about several of my external hard drive upgrades.

[00:04:14]A practical implication of upgrading to a larger internal mechanism is that data gets copied from one drive to another.

[00:04:22]So, if you upgrade backup hard drives and migrate that data every 3 years or so, you don't have to worry about data refresh.

[00:04:31]Secondly, if data refresh is due and a backup or archive drive does not need upgrading, I often find it has enough free capacity to make an on-drive copy, and this is how I often achieve a data refresh.

[00:04:46]So, for example, if an 8 TB drive only has 3 TB stored on it, I just duplicate that data on the same drive, making sure that it's verified, as I'll come to in a second. And I then delete the first copy, safe in the knowledge that all data has been rewritten.

[00:05:05]And even if a drive is over half full, on-drive duplication can be achieved in stages.

[00:05:12]So, for example, if an 8 TB drive contained 5 TB of data, 2.5 TB could be duplicated and verified, and the original deleted, and then the process repeated.

[00:05:25]It should be noted that this is likely to lead to data on a hard drive becoming fragmented.

[00:05:30]But, on a backup or archive drive, this may not be a concern, and certainly, it doesn't concern me.

[00:05:37]And you could always defrag the drive if you wished.

[00:05:42]If you rely on either migration to a new drive or an duplication to achieve data refresh, it's wise to make sure that all copies are verified.

[00:05:53]In Windows or on a Mac, you can do this using an application called TeraCopy, which is free for personal use and includes copy verification in both its free and pro versions.

[00:06:06]So, for example here in Windows, we have got TeraCopy running and we've also got my 16 TB LaCie d2 quadra connected to the computer.

[00:06:16]As we can see, it's only got about 5.4 TB of data stored on it and it's also only 22 months since I fitted the drive.

[00:06:25]So, a 3-year data refresh is not due.

[00:06:29]But, if it were, there would clearly be enough space for an on drive copy. And so, what I do is to open it up and to create a folder into which I copy everything else on the drive. And I do that using TeraCopy. Here it is and I've already set up a test data using just one folder and copying to the new folder we just looked at.

[00:06:51]I then go to options and click on verify files after transfer and then click on copy.

[00:06:59]As this progresses, let's use the magic of filmmaking to move forward in time until our copy and verification are complete.

[00:07:08]And there we go. We can see we've got a green here against verify. Everything is okay and we can also look in the log, which again shows verifying complete with no errors. And so, there we are.

[00:07:18]That's how easy it is to do a verified copy in TeraCopy, which as we noted is available for Windows and Mac.

[00:07:26]And if you want an alternative that's available for Linux and has a graphical interface, I'd recommend FreeFileSync, which can perform a file copy with verification if appropriately configured.

[00:07:45]>> Next, let's look at some specialist applications. And if you search for Drive Refresh Utility, the one that comes up again and again is Disk Fresh from Piriform Software.

[00:07:58]This is free to download, works in any version of Windows from XP onwards, and is dedicated to either reading and rewriting each sector of a hard drive or reading each sector of an SSD. And that's all there is to it.

[00:08:15]If we scroll down, we can see that the program hasn't been updated since July 2013. And this is not necessarily an issue as if a piece of software does its job, it doesn't need constant updates.

[00:08:29]And personally, I've been using Disk Fresh with no issues for many years.

[00:08:34]But inevitably, I would caution that you allow a 13-year-old piece of free software to rewrite or read all of the data on a drive at your own risk.

[00:08:44]And indeed, in general, note that you should never run any kind of data refresh utility on a drive that contains the only copy of any data.

[00:08:54]These points noted, here we have Disk Fresh up and running, and its interface is pretty self-explanatory.

[00:09:01]There's a very good help system over here, which includes a nice article all about why data refresh is necessary, but otherwise, it's simply a matter of selecting a drive or partition thereof.

[00:09:13]And right now on this particular system, we've got two SSDs physically connected here and here. The second one is formatted for Linux, so Windows can't see it, but the first one has got two NTFS partitions which appear up here.

[00:09:27]And then the third physical drive on this system is a test hard drive I've connected for, well, a test. And this again has got two partitions, which as we can see are formatted FAT32. And so, if we wanted to, we could use this fresh either on just this partition, or that partition, or indeed both partitions, or we could select the entire physical hard drive. And I think here, that's what we will do.

[00:09:49]By default, DiskFresh will read and then rewrite every sector, which is what is required to refresh a hard disk.

[00:09:57]But, there's also this checkbox over here for read-only mode, if you want to refresh an SSD or other flash media.

[00:10:05]Although here, we'll leave this unticked, as we're going to refresh a hard drive.

[00:10:10]It's then just a matter of which extent of refresh we want to select, where we've got two options. One is refresh range, which you can specify in sectors, which is useful if you want to conduct a refresh in stages. And indeed, as we can see, we can specify sectors as a percentage.

[00:10:27]But here, we will cancel, and we'll take the first option of just refresh to do the whole drive. Are we sure we want to continue? Yes, we are. And there we are, the process has started, which inevitably will take a little while. So, let's jump forward in time.

[00:10:46]And again.

[00:10:47]And again.

[00:10:49]Almost finished.

[00:10:51]And here we are, the refresh is complete, and the data on our hard drive should be safe from bit rot for at least 3 years.

[00:11:00]As this hopefully demonstrates, DiskFresh is very straightforward, and is therefore the application I personally use for hard drive data refresh when I don't rely on data migration or on drive copy.

[00:11:13]This said, I must caution that DiskFresh and similar applications, like those we're about to look at, are not ideal for refreshing SMR hard drives, as I'll discuss later in the video.

[00:11:26]Moving on, let's return to TeraCopy, as this can perform a file read test, which could be used to refresh the files on an SSD.

[00:11:37]And indeed, TeraCopy is my preferred SSD refresh application.

[00:11:43]All we need to do is to select our source. I'll go to add folders, and I've got an external SSD now connected to the system, which we could select as our folder. We could do the whole drive, although here, as a quick test, I'm just going to select one folder like that, and select folder. And then, I'll go to options because what TeraCopy is going to do is to read every file in order to create a checksum, and I don't want to save these checksums on the drive, so I'm just checking there are no options ticked to save the checksums. There aren't, so I'll go back to the file list, and click on test.

[00:12:19]And as usual, we'll now use the magic of filmmaking to speed on through.

[00:12:24]As we do so, I'd note that I find TeraCopy particularly useful if you just want to refresh certain files on an SSD, which is something you cannot do in Disk Fresh.

[00:12:35]And remember, the reason we only have to read data to refresh it on an SSD is that this forces the drive's controller to check data integrity and rewrite if required.

[00:12:47]Anyway, our test has now completed, and the data we selected on our SSD should now be refreshed.

[00:12:57]Moving on again, another option is a program called SpinRite from Gibson Research Corporation.

[00:13:04]This costs $89, and has a massive functionality for improving drive performance as well as data refresh.

[00:13:12]It's not an application I've used myself, but in recent discussions, two Explaining Computers channel members have told me that they've been using SpinRite successfully for over 20 years to avoid bit rot on hard drives and SSDs.

[00:13:28]SpinRite runs from a bootable USB drive and so can be used on most computers regardless of their installed operating system.

[00:13:37]A final program I've also not tried myself is Hard Disk Sentinel.

[00:13:43]This has a free trial as well as being available in standard, pro, and enterprise versions.

[00:13:50]However, as we can see over in this table, the ability to refresh a drive is only available in the professional version. This costs $33 and, like SpinRite, Hard Disk Sentinel offers a massive functionality far beyond data refresh.

[00:14:08]As I've said, Hard Disk Sentinel has a good reputation and I did consider getting the pro version to have a look at in this video.

[00:14:16]However, as I have no way of knowing if it refreshes data better than DiskFresh or TeraCopy, I thought I'd not muddy the waters and maintain my recommendations of using DiskFresh to refresh hard drives and TeraCopy to refresh SSDs.

[00:14:39]Rather than using a graphical application, it's possible to refresh drives using the Windows command prompt or the Linux terminal.

[00:14:48]However, this needs to be done with great care.

[00:14:52]And, as I've already outlined a range of options for drive refresh in Windows, to be safe, I've decided not to cover any Windows command line alternatives.

[00:15:03]For Linux users, SpinRite is a potential option if, with its $89 price tag.

[00:15:10]But, as none of the other applications I've talked about run in Linux, let's see how to refresh a hard drive or flash media in the Linux terminal.

[00:15:20]Starting with hard drives, we can read and rewrite all data using badblocks with syntax like this.

[00:15:29]Here, sudo executes the command with administrator rights, whilst the F flag forces badblocks to run even if the drive is mounted.

[00:15:39]B 4096 then sets a reasonable default block size, whilst P1 limits the number of passes to one, so the program does not repeat.

[00:15:50]Finally, NSV is critical, with the N parameter making sure that badblocks conducts a non-destructive scan, which does not destroy all data on the drive.

[00:16:02]S then enables progress tracking, with V displaying it on the screen.

[00:16:08]To try this out, I've got an external hard drive connected to the computer. We can see it over here. And if we do an lsblk list block devices in the terminal, we can see its two partitions are down here, and we can use badblocks either on a whole drive or a partition.

[00:16:23]And I think here, we'll choose the former. It'll take slightly less time to run. So, let's clear the screen to make things look tidy. There we go, and then enter our command.

[00:16:34]There we go, and always check you've got the N flag to ensure you're doing a non-destructive test. And having checked, let's now press enter.

[00:16:44]Where, as usual in Linux when entering a sudo command, we have to enter our password.

[00:16:50]And there we are, badblocks is now running a non-destructive read-write test, which, as previously, will take a lot of time. And so, once again, we'll now call on the magic of filmmaking to move forwards in time.

[00:17:06]And here we are, over 5 hours later, and we're almost there.

[00:17:11]And if we wait a second, yes, a refresh of our hard drive partition is complete.

[00:17:20]Next, let's refresh an SSD with a command that will read everything on the drive.

[00:17:27]There are many possible ways to achieve this, with a common approach being to copy everything on a drive to null.

[00:17:34]However, this can be tricky unless a drive has a totally flat file structure.

[00:17:39]And so here, we're going to use this command.

[00:17:43]And what this does is to use grep to recursively search every file in a location, which is a nifty way of forcing a read of every bit of data.

[00:17:53]For test, I've got a USB SSD connected.

[00:17:57]Here it is. And if we go back to the terminal and do an lsblk to list block devices to see where it is. It's actually got two partitions, but the main one is down here. We will refresh the files in this location, so let's copy that mount point like that. And we will uh clear the terminal just to be tidy, and then type the grep minus R to be recursive, our search term. I'm going to use Stanley, but you can use any term you like. It's purely a means to an end.

[00:18:24]And then on the end of that, we will put our location. Paste it in like that.

[00:18:29]And it's worth noting that unlike bad blocks, grep is happy to be given a mount point rather than a raw drive or partition. And indeed, this could be any folder which contains files you want to refresh on a flash media.

[00:18:44]Also note that the only output we're going to see will be any search matches.

[00:18:49]And to show this, on the SSD in a docs, and I think it's down in that folder there. Yes, it is. I've already created a test file which contains the word Stanley.

[00:18:59]So, with all this noted, let's run this command.

[00:19:03]There it goes. And almost immediately, it reports a test file match. But after that, there'll be no indication that anything is happening other than the flashing LED on the drive.

[00:19:16]So, let's speed on through until the end.

[00:19:20]Here we are, and we've now read every file on the drive, and in doing so, refreshed its data.

[00:19:34]Today, there are two methods that hard drives use to lay out their data tracks.

[00:19:39]The first is conventional magnetic recording or CMR, where the track pitch is determined by the size of the right head.

[00:19:48]And as this has to be larger than the read head, in CMR, there is wasted space between the tracks.

[00:19:56]To store more data in the same area, in shingled magnetic recording or SMR, the track pitch is determined by the read head. So, when tracks are laid down by the write head, they are allowed to overlap like shingles.

[00:20:12]This still allows data to be read because the read head does not overlap other tracks.

[00:20:18]But, data cannot be overwritten without corrupting overlapping tracks.

[00:20:24]Because of this, SMR drives organize data into append-only zones to which data can only be added. And when data needs to be changed, it's written to a new location and the previous one marked as obsolete.

[00:20:39]SMR drives also have a CMR cache, and I've explained them in more detail in my video on new hard drive technologies.

[00:20:49]But, the implication here is that SMR drives are far more difficult to refresh than CMR.

[00:20:56]You can use applications like DiskFresh to refresh SMR drives, but they will be very slow indeed and will place excessive wear on the drive mechanism.

[00:21:08]On drive copies are therefore more suitable for SMR refresh as they move data around rather than trying to rewrite each sector in turn, which an SMR drive cannot directly do.

[00:21:22]And ideally SMR data refresh is best achieved by doing a low-level format and copying data back to the drive from a backup. There really is no ideal refresh method for in situ SMR data.

[00:21:45]If you're serious about long-term data storage on unpowered hard drives or SSDs, then you should consider a data refresh strategy.

[00:21:55]However, given that everything we've looked at in this video is about either reading or rewriting all of the data on a drive, then I want to stress that everything we've covered is undertaken at your own risk.

[00:22:09]In particular, I want to stress yet again that you should never apply a mass data read or rewrite technique to a drive that you don't have a copy of as even a mass read has a small risk of a calamity.

[00:22:25]But now that's it for another video. If you've enjoyed what you've seen here, please press that like button. If you haven't subscribed, please subscribe and I hope to talk to you again very soon.

[00:22:43]>> [music]