My personal Linux backup strategy

General Discussions
,
1

Hello guys, I want to share with you my personal backups strategy, sometimes we were in this conversation but I never went so deep.

My goal is backing up a Linux Os, which is installed on a 500gb ssd sata hard drive, which having 500mb of EFI boot partition and rest of the disk is occupied by an XFS partition, the main OS partition, with files, home folder, data, virtual machines folders and various installed software.
Before this 500gb hard drive I had Linux installed in a 240gb hard drive, but I prefer to switch to a 500gb one, which is less worn-out and more robust. The new disk is a Western Digital blue ssd SA510. Highly reccomended Western Digital disks.
More than this main hard drive my needing are backing up some files such as projects and other important stuffs which are located in a secondary hard drive, and backing up some other Windows machines.

My organization with backup is using a single external 4tb USB hard drive, I don’t own a NAS infrastructure, because my setup is an homelab, and having such complicated backup system is overweight.
I decided to save all my backups in a single NTFS partition, saving both Linux and Windows machines. I know I might use some native Linux tools such as Timeshift or Borg backups software solutions, but these tools requires to have a dedicated Linux file system, and the point is I don’t want to split hard drive in different partitions.

So, for backing up the main 500gb hard drive I decided to boot up the system in a Gparted Live environment and launch the command dd.
So far I discovered the dd tool can do disk images and skipping empty hard drive blocks, with the cli parameter: conv=sparse , so in the end the final .img will result smaller then the original disk size and also the entire process requires less time then doing a byte per byte disk image.
Beside the main disk backup I keep a separate EFI partition image (in case of extreme disaster)
The final command for the main os partition it will be:
dd if=/dev/sdx2 of=diskbackup.img bs=2048M conv=sparse status=progress
And the final command for EFI partition it will be:
dd if=/dev/sdx1 of=EFIdiskbackup.img bs=128M status=progress
Notice I’m using different block size because EFI partition is just 500mb and the imaging process will be quick enough.

For backing up the separate drive I’m still saving its backup on the same NTFS external hard drive: the hard drive where I have all the important stuffs is formatted in NTFS, and mounted in Linux with an fstab entry, so I take specific folder backup sync with the software AOMEI Backupper (I bought the legal license) which is booted in a live Windows PE environment.
For the other Windows PC it’s not a real problem, I just do incrementals backup with AOMEI Backuppers.

In the end I have all my backups in a single USB drive, which is daily disconnected from computers, moreover this hard drive is located in a box somewhere in my house and physically keylocked.
I have scheduled my manual backups once each two weeks, generally on sunday morning.
I know this might be considered a bit paranoid behaviour, but you guys just think I daily work alot with computer, and I can’t lose all my projects.

Might this post be useful to some of you which are struggling a little on backups, trying to find their way..
I just want to say I tried different backups solutions, mixing Linux and Windows tool and filesystems, but now I think this is my nearly final stage of developing the workflow.

3 Likes
2

My backup/restore solution is complicated by dmcrypt/LUKS full volume encryption.

After I build each new system, I create a disk image using “dd” piped to “gpg” (adding compression and encryption). The resulting “gpg” file then has parity archives created using “par2”. A few media errors should not ruin otherwise good files, and “par2” can regenerate missing or corrupt parts of the file(s).

Monthly system backups are handled by the following:

  • “dd” for the partitions that do not contain LVM (Logical Volume Manager) volumes
  • LVM snapshot of the root file system
  • “duplicity” backup (adding compression and encryption) of the LVM snapshot of the root file system
  • more “par2” parity archives

All image and backup content is sent to the NAS using “sshfs” (basically SFTP). From the NAS, it is then copied to various online and offline media (hard disk, SSD, USB flash drive, MicroSDXC, and BD-RE).

Restores have been tested, and I think I have my bases covered.

3 Likes
3

+1 for duplicity! My backup situation is not as complex as those outlined here, but I really like duplicity for what it does.

I run an old Dell PowerEdge R510 that hosts a Nextcloud server. My desktop/laptops run the Nextcloud client which “syncs” (read backs up) all of the important user files. Then on all of my “servers” I have my own bash script that I wrote to compress/encrypt and send those files to Nextcloud (yes I could use duplicity here too, but, I like to learn by doing so I wrote the script myself).

I then use duplicity (with encryption enabled) to backup the entirety of Nextcloud to an external USB drive, which I then (every couple of weeks) give to my dad to backup on an offsite NAS we share.

I’ve only tested minor parts of the restore and it worked for what I needed, not sure how a full disaster recovery would go though.

Reading through all of these setups has been educational for me!

3 Likes
4

@ricky89 solid plan, especially pulling EFI separately and keeping the external drive offline.

Made this a new thread as it’s really useful base that you have shared.

@MarshallJFlinkman your point about automation makes sense. Even a weekly cron with rsync or restic to a second drive (or cloud) could strike a nice balance with Ricky’s current “manual full image” approach.

@benowe1717 I liked your note on testing restores; too many of us make backups without verifying them. I’d add that rotating between two external drives could also reduce the single-drive risk that Ricky mentioned.

3 Likes
5

“There is no cloud, it’s just someone else’s computer.” :grinning_face:

2 Likes
6

I would use Duplicity but the fact in my usecase is I’m using a single external USB hard drive for backing up multiple Linux and Windows systems. Since I don’t want to split destination disk partitions I decided to keep all the disks formatted in NTFS, so actually I can’t use some sorta Linux native backup.

Duplicity, Borg, Timeshift, Restic etc etc are great tools for native Linux btw.

2 Likes
7

I use rsync to save changes to Nextcloud and a 20TB external hard drive. For my Windows there’s the OneCloud & Windows back up to the 20 TB external drive. So Mine’s fairly simple.

2 Likes
8

For uploading your backups on cloud you must have very fast upload connection speed :smiley:

2 Likes
9

Updates:

I decided to format my internal 1tb disk in Ext4: for sure performances are better in native Linux. Mostly for many tiny files, such as in node_modules folder, since I have all my projects stored on this drive.

For backing this drive I decided to use Duplicity just for some critical folders, storing them in a separate ext4 hard drive. (I found a 320gb usb spare disk somewhere : P )

3 Likes
10

I used to use either tar or dd to do backups; lately I use a couple of antiX tools; one is ISO Snapshot to create the backup; Live USB Maker allows me to create either a dd backup if it’s not antiX or a read/write backup if antiX or MX Linux (which share the tool).

1 Like
11

Is there a wiki page or KB article that you can share with us for the antiX ISO Snapshot tool?

12

The FAQ page has some mention of the tools, though they are not in depth. Actually logging in and accessing it is probably the best way to figure it out; it’s not difficult, but it’s not well documented either. Here is the FAQ page: antiX-FAQ antiX-FAQ and here is the Wiki page: antiX Linux fan. I’ll try to edit the page in the near future, since there is a need for better documentation.

1 Like
13

You guys are making me think I need to do better with my backup strategy.

It’s pretty simple, but it does work, and it works well.

All systems rsync to at least one local NAS: primary NAS or backup NAS. By “all systems,” I mean “all systems.” Every VM, every physical system. I use simple rsync for this. My NAS is running Synology DSM, and every volume is snapshotted. Every filesystem on the primary NAS is sent two places: local backup NAS, and offsite backup NAS.

I’m not using any encryption in my setup, except for the remote offsite NAS. (Well, okay, all laptops are running FileVault.) I guess I figure I have bigger problems if my local NAS is stolen or confiscated.

I personally think “I’ve got bigger problems if …” is a great method of problem solving. :smiley:

2 Likes
14

I often find myself taking this approach as well! It helps me prioritize which of the many possibilities is best for my situation.

I really like your approach to doing backups as well, very similar to mine I feel. I’ve not bothered with taking full machine snapshots in… years. It feels too easy to just re-install an OS, so as long as I’ve got all “important” files backed up somewhere, I feel confident I can get back up and running “in no time”.

2 Likes
15

Yeah, I’ve tried full system snapshots, and I find that it works better for me if I back up the data with plans to reinstall the OS separate from the restore.

I know that sync is not backup, but I consider my solution to be backup since all filesystems on the NAS are snapshotted, and snapshots are kept for hours/days/weeks/months.

I also try to automate system rebuilds – my macOS rebuild is pretty well scripted at this point, and some of my Linux systems have automated builds with Ansible. I would like to get more of that automated, but it will take a long time, and I’m straying far from the OP’s topic.

4 Likes
16

@unixdude nothing wrong with using NAS as a way to get common access to a backup storage area. Do you usually have the NAS mounted on each system or do you mount it only for copying or backups? In an enterprise environment there are frequently NAS in operation, but good shops have quite the “fortress” around their networks to protect them. For me in personal situations, I don’t keep a collection of systems mounted; I only turn on what I’m actually using; keeps my network security a bit simpler, especially when I’m switching systems and/or hardware at a moment’s notice.

2 Likes
17

I use the NFS automounter on my OSes, so when I issue a command like cd /auto/documents, the documents directory is mounted. And, there’s no fuss about the filer becoming unreachable – it just auto-unmounts.

At work we have NFS automount maps with tens of thousands of entries, and everything is auto-mounted at work too.

1 Like
18

I do something like this, nothing special:

Unbenanntes Diagramm
Unbenanntes Diagramm713Ă—324 49 KB

1 Like
19

I would like to buy a pair of tape drives (one for a spare), but every time I price them, I find them ridiculously overpriced for home use.

Instead of tape, I settled on sixteen 1 TB Western Digital Blue (CMR) hard drives. There are eleven monthly backups and five yearly backups. I bought two portable disk cases that hold eight disks each to store them. Each month I “rsync” the NAS folder containing the backups to the appropriate disk, and then return the disk to the case.

This in itself was an investment, but nothing compared to tape drive costs. The disks were approximately $40 each.

1 Like
20

@MarshallJFlinkman That’s a reasonable investment and one way to have a backup. Do you have any additional backups to these? My only concern would be what happens if your system happens to be turned on and you get a sudden electrical storm that does major damage?

I once had a power supply go out, despite, as far as I can remember, having the system turned off but plugged in. There was enough local damage that the local electronics store was also affected; since I was a regular customer and they couldn’t run their electronic cash register they GAVE me a FREE power supply. My point, though is do you also have an external backup, or do you physically disconnect these drives when you’re not performing backups?

Otherwise, in an electrical storm, even if rare, you are at risk.

2 Likes