Rsync Command in Linux with Examples

blog articles
1

Originally published at: rsync Command in Linux with Examples

The rsync command is one of the most popular and powerful tools in Linux for synchronizing files and directories between different locations. It is widely used for backups, remote file transfers, and directory syncing, offering robust features like incremental file transfers, compression, and file permission preservation. In this article, we’ll explore the basics of rsync, along with…

2

Thanks for the post @hydn

I’m curious if you’ve ever used Rsync or Rclone to migrate data from an old server to a new one. If you have, what commands did you use?

I’m asking because I’ve noticed that Rclone or Rsync sometimes don’t set the file permissions properly. Additionally, I believe that we shouldn’t sync all files between the servers, such as /proc or /dev.

What are your thoughts on this?

1 Like
3

Yes thats true and i think the best would be to use the dd for the backup and that works very well. You can easily back enormous amount of data and you can have no broken links.

Let me know if that works,
Gaurav

1 Like
4

I’ve used “dd” before, but I’ve encountered some issues. What commands do you use for server migration between two different servers? Do you migrate all files from the old server, or do you exclude some?

Thanks in advance.

5

I mainly use rsync, timeshift, clone for the server migration. If you have to migrate all the files from the one server to the another. Basically, i dont do that.

What i do and what is suggested is to 1. create a root node 2. After creating a root node and then mount the directive to the server. In this way the root node is only having a minimal install and for that a 2.5 HDD disk should be done as the faster booting time.

In case of the mount drives, RAID5-SATA should be done. This will allow the read and drive on the mount drives slow as compared to the booting time but that will not affect the root node. So building a distributed computing as compared to the monolithic system.

Following that connect another drive and after that set a time shift from one drive to another as a cron job. This will allow the complete system back up and at the same time, mounted drives will be safe incase the crash goes on.

Those who keep the data on the root node need to be worrying about the moving of the files and i dont recommend that.

Let me know if you need more information,
Gaurav

6

Thanks for the reply!

I totally agree with you. However, sometimes, some servers don’t have that separation and they keep their data on the root node. Additionally, the servers must be migrated from one provider to another. In that case, moving the hard drive is not possible either. I was wondering if there is a way to do migration in such cases.

7

Thanks for agreeing. In that case, rclone, sync, dd and such utilities. You can put them in the cron jon and following that you can put in the cron job another schedule,

for i in $(find . -iname "*"); do md5sum $i; done  >> md5sum.txt 
for i in $(find . -iname "*"); do md5sum $i; done >> copiedplace.md5sum.txt 

diff md5sum.txt copiedplace.md5sum.txt

done and it will tell you when the data has been migrated and if there is any loss you will be able to know which files broke.

Let me know, i can write one new automation tool in golang for the same.

Gaurav

1 Like
8

@serhattsnmz I have started writing the code for a new utility with in built cron job and time integration in Golang for the system backup and it integrates the system information also. I will finish it and will post it here and also send you the link.

The logic that i am implementing is use dd, rsync, rclone and then time backup and then compare using the

func main() {
time := time.Now()
time = time.String()
timeDate := time.Date()
# implementing a cron job 

currenttime := time.Now()
timeafterweek = time.Sub(time.Now())
if timeafterweek >= weektime {
exec.Command()
}
}

Let me know if you know additional ones or have any other ideas. This will be applicable and useful to the other people here and nothing else. If they use well and if not then also well.

Let me know,
Gaurav

1 Like
9

Thank you @gauravearn, I’ll try them.

10

@hydn @serhattsnmz Here is the same. GitHub - codecreatede/go-system-back: golang system backend application. no need to schedule end to end system backup based on time structs I started writing the code after finishing one application and i implemented the pure data structure as there is no such available command.

implementing a interface and then releasing a binary release. Will capture the time, make the structs and will compare the tiers and will move also the time based to the disk capture.

I have finished another part of the same and let me know if you can help check the same. I dont have a server so would appreciate if you can help check the same and ask for some additional implementation if needed.

Thank you,
Gaurav

1 Like