Creating a Linux Data Backup Strategy on a Budget

Creating a Linux Data Backup Strategy on a Budget

The Joys of a Homelab and NAS Server

This year, I decided to build my first ever home storage server. It’s a 32 TB system that stores my personal and business data using open-source software. The server itself cost $531, and I bought four disks for $732, bringing the total cost to $1,263. It’s similar in price to off-the-shelf storage servers, but it offers more power and customizability. In this post, I’ll walk through how I chose the parts, what mistakes I made, and my recommendations for anyone interested in building their own.

Before and after of my 2022 homelab TrueNAS server build

If you’d prefer a video explanation, I recorded one on YouTube. NAS stands for network-attached storage. A NAS server’s primary job is storing data and making it available to other computers on your network. So, why have a whole dedicated server for data? After all, every computer stores data. I find it helpful to decouple data storage from my other systems. I upgrade my main workstation and laptop every two to three years, and migrating my data between computers was always a pain. A dedicated storage server eliminates most data migrations and facilitates sharing files between my systems.

I also have a lot of data. I’m a data hoarder, so I keep every digital photo I’ve ever taken, every email I’ve sent or received in the last 20 years, and source code for all of my personal projects. The total is currently 8.5 TB. The biggest data source is my DVD and Blu-Ray collection. I don’t like relying on streaming services to keep my favorite content available, so I still buy physical copies of movies and TV shows. As soon as I get a new disc, I rip the raw image and make a streamable video file. Between the raw ISO copy and the streamable MP4s, a single disc can occupy 60 GB of disk space. I still buy physical DVDs or Blu-Rays for anything I might watch a second time.

Homelab and NAS Beginnings

“Homelab” is a colloquial term that’s grown in popularity in the last few years. A homelab is a place in your home where you can experiment with IT hardware or software that you’d typically find in an office or data center. It can serve as a practice environment for new professional skills, or it can just be a place to play with interesting technology. If you’re new to the homelab world or have no experience building PCs, I recommend that you don’t build your own NAS. There are off-the-shelf solutions that offer similar functionality with a gentler learning curve.

Before building my own homelab NAS, I used a 4-disk Synology DS412+ for seven years. Honestly, I loved my Synology. It was one of the best purchases I ever made. It was a gentle introduction to the world of NAS servers, and it’s where I’d recommend you start if you’re not sure about the whole NAS thing.

My 10 TB Synology DS412+ has served me well for seven years. A few months ago, my Synology failed to boot and started making a clicking noise. A chill ran up my spine as I realized how dependent I’d become on this single device. Synology servers are not user-repairable, so if a part breaks after warranty, you have to replace the whole server. And if you’re dumb like me, and you’ve used a Synology-proprietary storage format, you can’t access your data without another Synology system. (Edit: A commenter on Hacker News showed me that you can recover a Synology Hybrid RAID volume from a non-Synology system.)

Fortunately, my old Synology recovered after I cleaned it out and reseated the disks, but it was an important wake-up call. I decided to switch to TrueNAS, as it offers an open-source implementation of an open storage format. TrueNAS (formerly known as FreeNAS) is one of the most popular operating systems for storage servers. It’s open-source, and it’s been around for almost 20 years, so it seemed like a reliable choice.

Understanding ZFS

TrueNAS uses ZFS, a filesystem designed specifically for storage servers. Traditional filesystems like NTFS or ext4 run on top of a data volume that manages low-level disk I/O. ZFS manages everything in the stack from the file-level logic down to disk I/O. ZFS’ comprehensive control gives it more power and performance than other filesystems. Some neat features of ZFS include:

  • Checksumming: ZFS verifies the integrity of data and automatically corrects silent data corruption.
  • Snapshots: ZFS can take instantaneous snapshots of a volume, allowing you to roll back changes or restore files.
  • Copy-on-write: ZFS avoids overwriting data, reducing write amplification and improving disk longevity.
  • Deduplication: ZFS can automatically identify and eliminate duplicate data blocks, saving storage space.

Before building this system, I had zero experience with ZFS, so I was excited to try it out. When I bought my Synology NAS, I initially installed three 4 TB drives and left the fourth slot empty. That gave me a total of 7 TB of usable space with Synology Hybrid Raid. Three years later, I was running out of space, so I added a fourth drive, bringing my total usable space to 10 TB. I decided to apply the same strategy for my new build. I wanted a system that met my current needs with room to grow. My rough target was to start with 20 TB of usable storage and extra headroom for up to 30 TB if I add disks later.

ZFS doesn’t let you add a new drive to an existing pool, but that feature is under active development. Hopefully, by the time I need to expand storage, the feature will be available in TrueNAS.

Planning the ZFS Disk Configuration

ZFS is designed to survive disk failures, so it stores each block of data redundantly. This feature complicates capacity planning because your total usable storage is not just the sum of each disk’s capacity. ZFS creates filesystems out of “pools” of disks. The more disks in the pool, the more efficiently ZFS can use their storage capacity. For example, if you give ZFS two 10 TB drives, you can only use half of your total disk capacity. If you instead use five 4 TB drives, ZFS gives you 14 TB of usable storage. Even though your total disk space is the same in either scenario, the five smaller drives give you 40% more usable space.

When you’re building a NAS server, you need to decide whether to use a smaller quantity of large disks or a larger quantity of small disks. Smaller drives are usually cheaper in terms of $/TB, but they’re more expensive to operate. Two 4 TB drives require twice the electricity of a single 8 TB drive. I wanted to minimize my server’s physical footprint, so I opted for fewer, larger drives.

ZFS offers different options for redundancy: raidz1, raidz2, and raidz3. The main difference is in robustness. raidz1 can survive one disk failure without losing data. raidz2 can survive two simultaneous disk failures, and raidz3 can survive three. What you gain in robustness, you pay for in usable storage. Given five 4 TB hard drives, here’s how much usable storage you’d get from each ZFS mode:

ZFS Mode Usable Storage
raidz1 12 TB
raidz2 8 TB
raidz3 4 TB

I chose raidz1. With only a handful of disks, the odds of two drives failing simultaneously is fairly low. Keep in mind that ZFS is not a backup strategy. ZFS can protect you against disk failure, but there are many threats to your data that ZFS won’t mitigate, such as accidental deletion, malware, or physical theft. I use restic to replicate everything important to encrypted cloud backups. The value of ZFS is that I don’t have to resort to my cloud backups if one drive dies, but I’ll have to recover from backups if two drives fail. That would be a pain, but it’s not worth giving up 20% of my server’s usable storage for raidz2.

Minimizing the Risk of Disk Failures

The more physical drives you have, the more defensive you should be about disk failure. If I had a pool of 20 disks, I’d probably use raidz2 or raidz3. Naively, the probability of two disks failing at once seems vanishingly small. Based on Backblaze’s stats, high-quality disk drives fail at 0.5-4% per year. A 4% risk per year is a 0.08% chance in any given week. Two simultaneous failures would happen once every 30,000 years, so I should be fine, right?

The problem is that disks aren’t statistically independent. If one disk fails, its neighbor has a substantially higher risk of dying. This is especially true if the disks are the same model, from the same manufacturing batch, and processed the same workloads. Further, rebuilding a ZFS pool puts an unusual amount of strain on all of the surviving disks. A disk that would have lasted a few more months under normal usage might die under the additional load of a pool rebuild.

Given these risks, I did what I could to reduce the risk of concurrent disk failures. I chose two different models of disk from two different manufacturers. To reduce the chances of getting disks from the same manufacturing batch, I bought them from different vendors. I can’t say how much this matters, but it didn’t increase costs significantly, so why not?

I purchased the same model of disk from two different vendors to decrease the chances of getting two disks from the same manufacturing batch.

Choosing the Hardware

The first decision was motherboard size. I’ve always appreciated my Synology DS412+’s compact form factor. I’ve never built a computer with a mini-ITX motherboard before, and this seemed like a good opportunity. I chose the ASUS Prime A320I-K for a few reasons:

  • The mini-ITX form factor keeps the case small.
  • It has onboard graphics, so I don’t need a dedicated GPU.
  • It’s one of the more affordable mini-ITX boards.

The ASUS Prime A320I-K supports onboard graphics in a mini-ITX form factor. I also looked at the B450, which was very similar but almost twice the price. The main advantage seemed to be better overclocking support, which I didn’t need.

From what I had read, ZFS is not very CPU-intensive. I ran a basic test by installing TrueNAS on a cheap Dell OptiPlex 7040 mini PC. It barely used the CPU, so it seemed safe to go with a low-powered option. My main criteria in a CPU was support for Radeon graphics so that I could use the A320 motherboard’s onboard HDMI output. The AMD Athlon 3000G is inexpensive and has native graphics support.

I settled on the AMD Athlon 3000G. At only $105, it’s a good value, it supports Radeon graphics, and it has decent CPU benchmarks.

When I built my last VM server, I used a Fractal Design case. It’s my favorite computer case ever, so I returned to Fractal Design on this build. I went with the Fractal Design Node 304 Black, a compact mini-ITX case. I liked the design because it’s closer to a cube than a tower. It has six drive bays, which allows me to start with enough drives and still have room to grow in the future.

The Fractal Design Node 304 Black is a mini-ITX case with space for six disks.

With six drive bays available in the case, I decided to start with four 8 TB disks, which translates to 22.5 TB of usable storage under raidz1. When I need to expand in the future, a fifth disk will bring me to 30.9 TB, and a sixth would get me 37 TB.

In the 8 TB range, there aren’t many drives below 7200 RPM, but you can go up to 10k RPM. For my NAS, speeds above 7200 RPM wouldn’t make a difference because the bottleneck is the network. A 10k RPM drive would be louder and consume more power but offer no practical gain in performance.

I initially tried checking Backblaze’s hard drive stats to avoid failure-prone disks, but they use drives on the pricier side. At one point, I was considering $400 drives for their impressively low 0.5% failure rate, but I realized it’s irrational to spend twice as much to reduce the failure rate by a few percent.

The last pitfall to avoid is shingled magnetic recording (SMR) technology. ZFS performs poorly on SMR drives, so if you’re building a NAS, avoid known SMR drives. If the drive is labeled as CMR, that’s conventional magnetic recording, which is fine for ZFS.

I chose the Toshiba N300 and the Seagate IronWolf. I saw positive reviews of both on the TrueNAS forums and reddit. Both models sold for $180-190, which was a good value for the storage space.

Toshiba N300 (left) and Seagate IronWolf (right)

TrueNAS needs a dedicated OS disk, but from what I’d read, it doesn’t demand much of it. The OS needs at least 2 GB of space, but TrueNAS infrequently reads or writes to the OS disk. The Kingston A400 is a fantastic value as a 120 GB M.2 SSD for only $32.

I went with the Kingston A400 because it was incredibly inexpensive — $32 for a 120 GB M.2 disk. I love M.2 disks! They don’t require any cabling. They just tuck away into the motherboard, take up nearly zero space, and you never have to touch them again.

In my research, I frequently found references to the “rule” that ZFS requires 1 GB of RAM for every TB of disk space in the system. According to ZFS developer Richard Yao, that rule is a myth. There are some RAM-hungry ZFS features like data deduplication, but ZFS runs fine with constrained memory.

I find memory extremely boring to shop for. I wish I had a more rigorous process for choosing RAM, but I couldn’t find trustworthy benchmarks or user reports for RAM. My process was:

  1. Find RAM modules compatible with the ASUS A320I-K motherboard.
  2. Choose the highest capacity modules I could afford.
  3. Pick a reputable brand.

That process led me to the CORSAIR Vengeance LPX 32GB CMK32GX4M2A2400C14 (2 x 16GB) for $128.

The CORSAIR Vengeance LPX 32GB CMK32GX4M2A2400C14 (2 x 16GB) is compatible with the A320I-K motherboard and is a decent price for 32 GB.

In terms of power capacity, basically any consumer PSU would have been sufficient. According to PCPartPicker, my system only requires 218 W. I would have picked a PSU in the 300-400 W range, but there weren’t semi-modular options with lower wattage. I went with the 500 W EVGA 110-BQ-0500-K1.

The EVGA 110-BQ-0500-K1 is a semi-modular PSU. At 500 W, it offers more than enough power for my build.

One item I’ve never purchased before was a 90-degree SATA cable. I didn’t realize I needed them until I saw that there wasn’t enough space between my motherboard and PSU to plug in a standard SATA cable. These slim 90-degree cables solved the problem.

It was such a tight squeeze between my PSU and motherboard that I needed 90-degree slim SATA cables.

Intentional Exclusions

There are a few components that I intentionally excluded from my build due to price, complexity, or physical space.

With scarce physical space and motherboard ports, I didn’t want a dedicated graphics card. I chose a motherboard and CPU combination that supports graphics rendering without an external card.

Many NAS builds include a host bus adaptor (HBA). An HBA is a chip that goes into the PCI slot of a motherboard and increases the number of disks the motherboard can support. ZFS requires you to reflash the HBA’s firmware in a process that sounds tedious and confusing. I decided to punt on the HBA until I need more storage. The ASUS A320I-K has four SATA ports, which is enough for my initial needs. I made sure to leave a PCI slot empty for a future HBA.

In researching different TrueNAS builds, I saw several posts claiming that ECC RAM (error correction code RAM) is a must-have to prevent data corruption. I ultimately decided against ECC RAM and just used standard, consumer-grade RAM. While I obviously don’t want my server to corrupt my data in RAM, I’ve also been using computers for the past 30 years without ECC RAM, and I’ve never noticed data corruption. If I were building a server for heavy load from multiple users all day, I’d spring for a build with ECC RAM. For home needs, I think simple consumer-grade RAM should be

Facebook
Pinterest
Twitter
LinkedIn