Introduction

Rsync, remote synchronize is known as a remote synchronization function of the software, it synchronizes files at the same time, can maintain the original file permissions, time, hard and soft links and other additional information. it is a “sync algorithm” that provides a quick way to synchronize files between clients and remote file servers, and can also be transferred via SSH, which is also very confidential, supports breakpoint continuation, data can be pushed or pulled, and is free of chargeS.

Rsync advantages：

• Ability to update entire directories and trees and file systems;
• Selectively maintain symbolic links, hard links, file ownership, permissions, devices, and time for folders and files;
• For installation, there are no special permission requirements;
• For multiple files, the internal pipeline reduces the file wait delay;
• Can use RSH, SSH or direct port as the transfer port;
• Support anonymous Rsync synchronization files are the ideal mirror tool;

Installation

Installing rsync is pretty easy, it doesn’t rely on any special libraries at all, so you can install it directly with yum.

yum install -y rsync 

The command rsync will appear once the installation is complete

Rsync works

Rsync works in three ways：
1.synchronization on the local file system.
2.the local host USES the remote shell to communicate with the remote host.
3.localhost connects rsync daemon on a remote host via network socket.

Here is the syntax works for each rsync:

Local:  
      rsync [OPTION...] SRC... [DEST]   
Access via remote shell:   
      Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]   
      Push: rsync [OPTION...] SRC... [USER@]HOST:DEST   
Access via rsync daemon:   
      Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]         
            rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]   
      Push: 
            rsync [OPTION...] SRC... [USER@]HOST::DEST         
            rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST 

Option

man rsync

Options
 -v, --verbose               increase verbosity
 -q, --quiet                 suppress non-error messages
     --no-motd               suppress daemon-mode MOTD (see manpage caveat)
 -c, --checksum              skip based on checksum, not mod-time & size
 -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
     --no-OPTION             turn off an implied OPTION (e.g. --no-D)
 -r, --recursive             recurse into directories
 -R, --relative              use relative path names
     --no-implied-dirs       don't send implied dirs with --relative
 -b, --backup                make backups (see --suffix & --backup-dir)
     --backup-dir=DIR        make backups into hierarchy based in DIR
     --suffix=SUFFIX         set backup suffix (default ~ w/o --backup-dir)
 -u, --update                skip files that are newer on the receiver
     --inplace               update destination files in-place (SEE MAN PAGE)
     --append                append data onto shorter files
     --append-verify         like --append, but with old data in file checksum
 -d, --dirs                  transfer directories without recursing
 -l, --links                 copy symlinks as symlinks
 -L, --copy-links            transform symlink into referent file/dir
     --copy-unsafe-links     only "unsafe" symlinks are transformed
     --safe-links            ignore symlinks that point outside the source tree
 -k, --copy-dirlinks         transform symlink to a dir into referent dir
 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
 -H, --hard-links            preserve hard links
 -p, --perms                 preserve permissions
 -E, --executability         preserve the file's executability
     --chmod=CHMOD           affect file and/or directory permissions
 -A, --acls                  preserve ACLs (implies --perms)
 -X, --xattrs                preserve extended attributes
 -o, --owner                 preserve owner (super-user only)
 -g, --group                 preserve group
     --devices               preserve device files (super-user only)
     --copy-devices          copy device contents as regular file
     --specials              preserve special files
 -D                          same as --devices --specials
 -t, --times                 preserve modification times
 -O, --omit-dir-times        omit directories from --times
     --super                 receiver attempts super-user activities
     --fake-super            store/recover privileged attrs using xattrs
 -S, --sparse                handle sparse files efficiently
 -n, --dry-run               perform a trial run with no changes made
 -W, --whole-file            copy files whole (without delta-xfer algorithm)
 -x, --one-file-system       don't cross filesystem boundaries
 -B, --block-size=SIZE       force a fixed checksum block-size
 -e, --rsh=COMMAND           specify the remote shell to use
     --rsync-path=PROGRAM    specify the rsync to run on the remote machine
     --existing              skip creating new files on receiver
     --ignore-existing       skip updating files that already exist on receiver
     --remove-source-files   sender removes synchronized files (non-dirs)
     --del                   an alias for --delete-during
     --delete                delete extraneous files from destination dirs
     --delete-before         receiver deletes before transfer, not during
     --delete-during         receiver deletes during the transfer
     --delete-delay          find deletions during, delete after
     --delete-after          receiver deletes after transfer, not during
     --delete-excluded       also delete excluded files from destination dirs
     --ignore-errors         delete even if there are I/O errors
     --force                 force deletion of directories even if not empty
     --max-delete=NUM        don't delete more than NUM files
     --max-size=SIZE         don't transfer any file larger than SIZE
     --min-size=SIZE         don't transfer any file smaller than SIZE
     --partial               keep partially transferred files
     --partial-dir=DIR       put a partially transferred file into DIR
     --delay-updates         put all updated files into place at transfer's end
 -m, --prune-empty-dirs      prune empty directory chains from the file-list
     --numeric-ids           don't map uid/gid values by user/group name
     --timeout=SECONDS       set I/O timeout in seconds
     --contimeout=SECONDS    set daemon connection timeout in seconds
 -I, --ignore-times          don't skip files that match in size and mod-time
     --size-only             skip files that match in size
     --modify-window=NUM     compare mod-times with reduced accuracy
 -T, --temp-dir=DIR          create temporary files in directory DIR
 -y, --fuzzy                 find similar file for basis if no dest file
     --compare-dest=DIR      also compare destination files relative to DIR
     --copy-dest=DIR         … and include copies of unchanged files
     --link-dest=DIR         hardlink to files in DIR when unchanged
 -z, --compress              compress file data during the transfer
     --compress-level=NUM    explicitly set compression level
     --skip-compress=LIST    skip compressing files with a suffix in LIST
 -C, --cvs-exclude           auto-ignore files the same way CVS does
 -f, --filter=RULE           add a file-filtering RULE
 -F                          same as --filter='dir-merge /.rsync-filter'
                             repeated: --filter='- .rsync-filter'
     --exclude=PATTERN       exclude files matching PATTERN
     --exclude-from=FILE     read exclude patterns from FILE
     --include=PATTERN       don't exclude files matching PATTERN
     --include-from=FILE     read include patterns from FILE
     --files-from=FILE       read list of source-file names from FILE
 -0, --from0                 all *-from/filter files are delimited by 0s
 -s, --protect-args          no space-splitting; only wildcard special-chars
     --address=ADDRESS       bind address for outgoing socket to daemon
     --port=PORT             specify double-colon alternate port number
     --sockopts=OPTIONS      specify custom TCP options
     --blocking-io           use blocking I/O for the remote shell
     --stats                 give some file-transfer stats
 -8, --8-bit-output          leave high-bit chars unescaped in output
 -h, --human-readable        output numbers in a human-readable format
     --progress              show progress during transfer
 -P                          same as --partial --progress
 -i, --itemize-changes       output a change-summary for all updates
     --out-format=FORMAT     output updates using the specified FORMAT
     --log-file=FILE         log what we're doing to the specified FILE
     --log-file-format=FMT   log updates using the specified FMT
     --password-file=FILE    read daemon-access password from FILE
     --list-only             list the files instead of copying them
     --bwlimit=KBPS          limit I/O bandwidth; KBytes per second
     --write-batch=FILE      write a batched update to FILE
     --only-write-batch=FILE like --write-batch but w/o updating destination
     --read-batch=FILE       read a batched update from FILE
     --protocol=NUM          force an older protocol version to be used
     --iconv=CONVERT_SPEC    request charset conversion of filenames
     --checksum-seed=NUM     set block/file checksum seed (advanced)
 -4, --ipv4                  prefer IPv4
 -6, --ipv6                  prefer IPv6
     --version               print version number
(-h) --help                  show this help (-h is --help only if used alone)

Although there are many options, the most common combination of options is “avz”, which compresses and displays some information and transmits it in archive mode.

Use of rsync

Synchronize data on the local disk

The following commands synchronize individual files on your local computer from one location to another.

rsync -zvh backup.tar /tmp/backup/

We can use it ‘–progress ‘option. It shows the file that was transferred and the time remaining.

rsync -zvh backup.tar /tmp/backup/ --progress

The commands will transfer or synchronize all files in the same directory to different directories on the same computer.

rsync -avzh /root/tomcat/ /tmp/backup/tomcat/ --progress

Use the –include and –exclude options

These two options allow us to include exclude files and folders by specifying parameters with which these options can help us specify which files and folders to include and exclude during synchronization. You do not want to be moved to these files or directories.
Here, in this example, the rsync command will include only those files and directories that end with a “log” and exclude end with a “war” files and directories.

rsync -avzh  --include '*log' --exclude '*war' /root/tomcat/ /tmp/backup/tomcat/ --progress

Use the –delete

If the file or directory does not exist at the source but already exists at the destination, you may need to delete the existing file/directory at the destination during synchronization.

We can use the ‘–delete’ option to delete files that do not exist in the source directory. Source and target are synchronized. Now, create a new target file, test.txt.

touch /tmp/backup/tomcat/123.txt
ls /tmp/backup/tomcat/          
rsync -avzh --delete  /root/tomcat/ /tmp/backup/tomcat/ --progress
ls /tmp/backup/tomcat/                                            

The target has the name test.txt, and when synchronizing the new file with the source of the ‘-delete’ option, it removes the file test.txt.

Sets the maximum size of the file to transfer

You can do this with the — max-size option. Here, in this example, the maximum file size is 200K, so the command will only pass smaller files equal to or greater than 200K.

rsync -avzh --max-size='200k'  /root/tomcat/ /tmp/backup/tomcat/ --progress

Set bandwidth limits and transfer files

You can transfer data from one machine to another at the same time with the help of the ‘–bwlimit’ option. This option helps limit I/O bandwidth.

rsync -avzh --bwlimit=10  /root/tomcat/ /tmp/backup/tomcat/ --progress
rsync -avzh --bwlimit=50  /root/tomcat/ /tmp/backup/tomcat/ --progress

Remotely synchronize data using ssh-based rsync

Through the introduction of synchronizing data on a local disk, we learned about the most common options. The options for remote transport are the same as for local transport. So we’ll cover only the basic format of remote transmission. If you need to use other options, you can choose according to the above description

Use SSH to copy files from the local server to the remote server

To specify the protocol to use rsync, you need to give the protocol you want to use the “-e” option. Here, in this example, we will use “SSH” with the “-e” option “-p” to specify the transport port and perform the data transfer.

rsync -zvh -e "ssh -p12598" backup.tar root@172.16.6.250:/tmp/backup/ --progress

Copy files from remote server to local server via SSH

rsync -zvh -e "ssh -p12598"  root@172.16.6.250:/tmp/backup/*  /tmp/  --progress

conclusion

The above article introduced two working modes of rsync: local transmission and remote transmission via SSH protocol. The third working mode is transmission via rsync Daemon. Rsync Daemon commands are basically the same as using SSH. The only change is that the destination address is transferred according to the alias defined in Rsync Daemon instead of IP.

If you want to know more about rsync, please leave me a message To learn more about Linux, please follow my link. Thank you

The post Synchronous data Using rsync Command in Linux appeared first on Linux Windows and android Tutorials.