I’m planning on setting up a nas/home server (primarily storage with some jellyfin and nextcloud and such mixed in) and since it is primarily for data storage I’d like to follow the data preservation rules of 3-2-1 backups. 3 copies on 2 mediums with 1 offsite - well actually I’m more trying to go for a 2-1 with 2 copies and one offsite, but that’s besides the point. Now I’m wondering how to do the offsite backup properly.
My main goal would be to have an automatic system that does full system backups at a reasonable rate (I assume daily would be a bit much considering it’s gonna be a few TB worth of HDDs which aren’t exactly fast, but maybe weekly?) and then have 2-3 of those backups offsite at once as a sort of version control, if possible.
This has two components, the local upload system and the offsite storage provider. First the local system:
What is good software to encrypt the data before/while it’s uploaded?
While I’d preferably upload the data to a provider I trust, accidents happen, and since they don’t need to access the data, I’d prefer them not being able to, maliciously or not, so what is a good way to encrypt the data before it leaves my system?
What is a good way to upload the data?
After it has been encrypted, it needs to be sent. Is there any good software that can upload backups automatically on regular intervals? Maybe something that also handles the encryption part on the way?
Then there’s the offsite storage provider. Personally I’d appreciate as many suggestions as possible, as there is of course no one size fits all, so if you’ve got good experiences with any, please do send their names. I’m basically just looking for network attached drives. I send my data to them, I leave it there and trust it stays there, and in case too many drives in my system fail for RAID-Z to handle, so 2, I’d like to be able to get the data off there after I’ve replaced my drives. That’s all I really need from them.
For reference, this is gonna be my first NAS/Server/Anything of this sort. I realize it’s mostly a regular computer and am familiar enough with Linux, so I can handle that basic stuff, but for the things you wouldn’t do with a normal computer I am quite unfamiliar, so if any questions here seem dumb, I apologize. Thank you in advance for any information!
Amazon AWS Glacier
.
If you use ZFS this becomes easy, because you can do incremental backups at the block level.
I have my home lab server and do snapshots and sends to a server at my fathers house. Then I also have an external drive that I snapshot to as well.
If you are gonna go for TrueNAS, try Storj with TrueNAS Cloud task. TrueNAS made a partnership with Storj and the price is very good. https://www.truenas.com/truecloud-backup/
TlDr; The data is encrypted with restic and sent to Storj S3 storage that is further fragmenting it (and encrypting it too - so double encryption) into multiple pieces (with redundancy) and storing on other peoples TrueNASes (you can also provide your unused space btw and gain some small money back).
I am in process of setting this up (already run a working test backup) and I didn’t find anything that’s better than this integrated solution. Very cool!
Hetzner Storagebox
Just recently moved from an S3 cloud provider to a storagebox. Prices are ok and sub accounts help clean things up.
I use borg backup. It, and another tool called restic, are meant for creating encrypted backups. Further, it can create backups regularly and only backup differences. This means you could take a daily backup without making new copies of your entire library. They also allow you to, as part of compressing and encrypting, make a backup to a remote machine over ssh. I think you should start with either of those.
One provider thats built for being a cloud backup is borgbase. It can be a location you backup a borg (or restic I think) repository. There are others that are made to be easily accessed with these backup tools.
Lastly, I’ll mention that borg handles making a backup, but doesn’t handle the scheduling. Borgmatic is another tool that, given a yml configuration file, will perform the borgbackup commands on a schedule with the defined arguments. You could also use something like systemd/cron to run a schedule.
Personally, I use borgbackup configured in NixOS (which makes the systemd units for making daily backups) and I back up to a different computer in my house and to borgbase. I have 3 copies, 1 cloud and 2 in my home.
I’m just skipping that. How am I going to backup 48TB on an off-site backup?!
You ought to only be 3-2-1ing you irreplaceable/essential files like personal photos, videos, and documents. Unless you’re a huge photography guy i can believe that takes up 48TB
Get a tiny ITX box with a couple 20TB refurbished HDDs, stick it at a friend’s house
In theory. But I already spent my pension for those 64TB drives (raidz2) xD. Getting off-site backup for all of that feels like such a waste of money (until you regret it). I know it isn’t a backup, but I’m praying the Raidz2 will be enough protection.
Do you have to back up everything off site?
Maybe there are just a few critical files you need a disaster recovery plan for, and the rest is just covered by your raidz
The cost of storage is always more than double the sticker price. The hidden fee is that you need a second and maybe a third one and a system to put it all in. Most our operational lab cost is backups. I can’t replace the data if it’s lost.
Just a friendly reminder that RAID is not a backup…
Just consider if something accidentally overwrites some / all your files. This is a perfectly legit action and the checksums will happily match that new data, but your file(s) are gone…
Understanding the risks is half the battle, but we can only do what we can do.
Only back up the essentials like photos and documents or rare media.
Don’t care about stuff like Avengers 4K that can easily be reaquiredI don’t have Avengers 4K. It’s all just Linux ISOs.
This is what I’m currently doing, I use backblaze b2 for basically everything that’s not movies/shows/music/roms, along with backing up my docker stacks etc to the same external drive my media’s currently on.
I’m looking at a few good steps to upgrade this but nothing excessive:
- NAS for media and storing local backups
- Regular backups of everything but media to a small USB drive
- Get a big ass external HDD that I’ll update once a month with everything and keep in my storage unit and ditch backblaze
Not the cleanest setup but it’ll do the job. The media backup is definitely gonna be more of a 2-1-Pray system LMAO but at least the important things will be regularly taken care of
a “poor mans” backup can be useful for things like this, movie/tv/music collections, and will only be a few MB instead of TB.
if things go south at least you can rebuild your collection in time. obviously if there are some rare files that were hard to get then you can backup those ones up, but even at that it will probably still be a small backup
Yup. My “essential” backups are well under 1TB, the rest can be reacquired.
I got my parents to get a NAS box, stuck it in their basement. They need to back up their stuff anyway. I put in 2 18 TB drives (mirrored BTRFS raid1) from server part deals (peeps have said that site has jacked their prices, look for alts). They only need like 4 TB at most. I made a backup samba share for myself. It’s the cheapest symbology box possible, their software to make a samba share with a quota.
I then set up a wireguard connection on an RPi, taped that to the NAS, and wireguard to the local network with a batch script. Mount the samba share and then use restic to back up my data. It works great. Restic is encrypted, I don’t have to pay for storage monthly, their electricity is cheap af, they have backups, I keep tabs on it, everyone wins.
Next step is to go the opposite way for them, but no rush on that goal, I don’t think their basement would get totaled in a fire and I don’t think their house (other than the basement) would get totaled in a flood.
If you don’t have a friend or relative to do a box-at-their-house (peeps might be enticed with reciprocal backups), restic still fits the bill. Destination is encrypted, has simple commands to check data for validity.
Rclone crypt is not good enough. Too many issues (path length limits, password “obscured” but otherwise there, file structure preserved even if names are encrypted). On a VPS I use rclone to be a pass-through for restic to backup a small amount of data to a goog drive. Works great. Just don’t fuck with the rclone crypt for major stuff.
Lastly I do use rclone crypt to upload a copy of the restic binary to the destination, as the crypt means the binary can’t be fucked with and the binary there means that is all you need to recover the data (in addition to the restic password you stored safely!).
I just rsync it once in a while to a home server running my dad’s house. I want it done manually in a “pull” direction rather than a “push” in case I ever get hit with ransomware.
I rent a cheap 4.5 TB, 60€/year VPS. Encryption synchronization is handled by Proxmox as I have Proxmox Backup Server installed on it. I am planning to change it and just buy a miniPC with a cheap HDD that I’ll place at my family’s place to save money
Idrive has built in local encryption you can enable.
Rsync to a Hetzner storage box. I dont do ALL my data, just the nextcloud data. The rest is…linux ISOs… so I can redownload at my convenience.
I have an external storage unit a couple kilometers away and two 8TB hard drives with luks+btrfs. One of them is always in the box and after taking backups, when I feel like it, I detach the drive and bike to the box to switch. I’m currently researching btrbk for updating the backup drive on my pc automatically, it’s pretty manual atm. For most scenarios the automatic btrfs snapshots on my main disks are going to be enough anyway.
Put brand new drive into system, begin clone
When clone is done, pull drive out and place in a cardboard box
Take that box to my off-site storage (neighbors house) and bury it
(In truth I couldn’t afford to get to the 1 off-site in time and have potentially tragically lost almost 4TB of data that, while replacable, will take time because I don’t fucking remember what I even had lol. Gonna take the drives to a specialist tho cuz I think the plates are fine and it’s the actual reading mechanism that’s busted)
For this I use a python script run via cron to output an html directory file that lists all the folder contents and pushes it to my cloud storage. This way if I ever have a critical failure of replaceable media, I can just refer to my latest directory file.
I used to say restic and b2; lately, the b2 part has become more iffy, because of scuttlebutt, but for now it’s still my offsite and will remain so until and unless the situation resolves unfavorably.
Restic is the core. It supports multiple cloud providers, making configuration and use trivial. It encrypts before sending, so the destination never has access to unencrypted blobs. It does incremental backups, and supports FUSE vfs mounting of backups, making accessing historical versions of individual files extremely easy. It’s OSS, and a single binary executable; IMHO it’s at the top of its class, commercial or OSS.
B2 has been very good to me, and is a clear winner for this is case: writes and space are pennies a month, and it only gets more expensive if you’re doing a lot of reads. The UI is straightforward and easy to use, the API is good; if it weren’t for their recent legal and financial drama, I’d still unreservedly recommend them. As it is, you’d have you evaluate it yourself.
I’m running the same setup, restic -> b2. Offsite I have a daily rclone job to pull (the diffs) from b2. Works perfectly, cost is < 1€ per month.
