Introduction
Rsync, remote synchronize is known as a remote synchronization function of the software, it synchronizes files at the same time, can maintain the original file permissions, time, hard and soft links and other additional information. it is a “sync algorithm” that provides a quick way to synchronize files between clients and remote file servers, and can also be transferred via SSH, which is also very confidential, supports breakpoint continuation, data can be pushed or pulled, and is free of chargeS.
Rsync advantages：
- Ability to update entire directories and trees and file systems;
- Selectively maintain symbolic links, hard links, file ownership, permissions, devices, and time for folders and files;
- For installation, there are no special permission requirements;
- For multiple files, the internal pipeline reduces the file wait delay;
- Can use RSH, SSH or direct port as the transfer port;
- Support anonymous Rsync synchronization files are the ideal mirror tool;
Installation
Installing rsync is pretty easy, it doesn’t rely on any special libraries at all, so you can install it directly with yum.
yum install -y rsync
The command rsync will appear once the installation is complete
Rsync works
Rsync works in three ways：
1.synchronization on the local file system.
2.the local host USES the remote shell to communicate with the remote host.
3.localhost connects rsync daemon on a remote host via network socket.
Here is the syntax works for each rsync:
Local:
rsync [OPTION...] SRC... [DEST]
Access via remote shell:
Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
Push: rsync [OPTION...] SRC... [USER@]HOST:DEST
Access via rsync daemon:
Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
Push:
rsync [OPTION...] SRC... [USER@]HOST::DEST
rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST
Option
man rsync
Options
-v, --verbose increase verbosity
-q, --quiet suppress non-error messages
--no-motd suppress daemon-mode MOTD (see manpage caveat)
-c, --checksum skip based on checksum, not mod-time & size
-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
--no-OPTION turn off an implied OPTION (e.g. --no-D)
-r, --recursive recurse into directories
-R, --relative use relative path names
--no-implied-dirs don't send implied dirs with --relative
-b, --backup make backups (see --suffix & --backup-dir)
--backup-dir=DIR make backups into hierarchy based in DIR
--suffix=SUFFIX set backup suffix (default ~ w/o --backup-dir)
-u, --update skip files that are newer on the receiver
--inplace update destination files in-place (SEE MAN PAGE)
--append append data onto shorter files
--append-verify like --append, but with old data in file checksum
-d, --dirs transfer directories without recursing
-l, --links copy symlinks as symlinks
-L, --copy-links transform symlink into referent file/dir
--copy-unsafe-links only "unsafe" symlinks are transformed
--safe-links ignore symlinks that point outside the source tree
-k, --copy-dirlinks transform symlink to a dir into referent dir
-K, --keep-dirlinks treat symlinked dir on receiver as dir
-H, --hard-links preserve hard links
-p, --perms preserve permissions
-E, --executability preserve the file's executability
--chmod=CHMOD affect file and/or directory permissions
-A, --acls preserve ACLs (implies --perms)
-X, --xattrs preserve extended attributes
-o, --owner preserve owner (super-user only)
-g, --group preserve group
--devices preserve device files (super-user only)
--copy-devices copy device contents as regular file
--specials preserve special files
-D same as --devices --specials
-t, --times preserve modification times
-O, --omit-dir-times omit directories from --times
--super receiver attempts super-user activities
--fake-super store/recover privileged attrs using xattrs
-S, --sparse handle sparse files efficiently
-n, --dry-run perform a trial run with no changes made
-W, --whole-file copy files whole (without delta-xfer algorithm)
-x, --one-file-system don't cross filesystem boundaries
-B, --block-size=SIZE force a fixed checksum block-size
-e, --rsh=COMMAND specify the remote shell to use
--rsync-path=PROGRAM specify the rsync to run on the remote machine
--existing skip creating new files on receiver
--ignore-existing skip updating files that already exist on receiver
--remove-source-files sender removes synchronized files (non-dirs)
--del an alias for --delete-during
--delete delete extraneous files from destination dirs
--delete-before receiver deletes before transfer, not during
--delete-during receiver deletes during the transfer
--delete-delay find deletions during, delete after
--delete-after receiver deletes after transfer, not during
--delete-excluded also delete excluded files from destination dirs
--ignore-errors delete even if there are I/O errors
--force force deletion of directories even if not empty
--max-delete=NUM don't delete more than NUM files
--max-size=SIZE don't transfer any file larger than SIZE
--min-size=SIZE don't transfer any file smaller than SIZE
--partial keep partially transferred files
--partial-dir=DIR put a partially transferred file into DIR
--delay-updates put all updated files into place at transfer's end
-m, --prune-empty-dirs prune empty directory chains from the file-list
--numeric-ids don't map uid/gid values by user/group name
--timeout=SECONDS set I/O timeout in seconds
--contimeout=SECONDS set daemon connection timeout in seconds
-I, --ignore-times don't skip files that match in size and mod-time
--size-only skip files that match in size
--modify-window=NUM compare mod-times with reduced accuracy
-T, --temp-dir=DIR create temporary files in directory DIR
-y, --fuzzy find similar file for basis if no dest file
--compare-dest=DIR also compare destination files relative to DIR
--copy-dest=DIR … and include copies of unchanged files
--link-dest=DIR hardlink to files in DIR when unchanged
-z, --compress compress file data during the transfer
--compress-level=NUM explicitly set compression level
--skip-compress=LIST skip compressing files with a suffix in LIST
-C, --cvs-exclude auto-ignore files the same way CVS does
-f, --filter=RULE add a file-filtering RULE
-F same as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter'
--exclude=PATTERN exclude files matching PATTERN
--exclude-from=FILE read exclude patterns from FILE
--include=PATTERN don't exclude files matching PATTERN
--include-from=FILE read include patterns from FILE
--files-from=FILE read list of source-file names from FILE
-0, --from0 all *-from/filter files are delimited by 0s
-s, --protect-args no space-splitting; only wildcard special-chars
--address=ADDRESS bind address for outgoing socket to daemon
--port=PORT specify double-colon alternate port number
--sockopts=OPTIONS specify custom TCP options
--blocking-io use blocking I/O for the remote shell
--stats give some file-transfer stats
-8, --8-bit-output leave high-bit chars unescaped in output
-h, --human-readable output numbers in a human-readable format
--progress show progress during transfer
-P same as --partial --progress
-i, --itemize-changes output a change-summary for all updates
--out-format=FORMAT output updates using the specified FORMAT
--log-file=FILE log what we're doing to the specified FILE
--log-file-format=FMT log updates using the specified FMT
--password-file=FILE read daemon-access password from FILE
--list-only list the files instead of copying them
--bwlimit=KBPS limit I/O bandwidth; KBytes per second
--write-batch=FILE write a batched update to FILE
--only-write-batch=FILE like --write-batch but w/o updating destination
--read-batch=FILE read a batched update from FILE
--protocol=NUM force an older protocol version to be used
--iconv=CONVERT_SPEC request charset conversion of filenames
--checksum-seed=NUM set block/file checksum seed (advanced)
-4, --ipv4 prefer IPv4
-6, --ipv6 prefer IPv6
--version print version number
(-h) --help show this help (-h is --help only if used alone)
Although there are many options, the most common combination of options is “avz”, which compresses and displays some information and transmits it in archive mode.
Use of rsync
Synchronize data on the local disk
The following commands synchronize individual files on your local computer from one location to another.
rsync -zvh backup.tar /tmp/backup/
We can use it ‘–progress ‘option. It shows the file that was transferred and the time remaining.
rsync -zvh backup.tar /tmp/backup/ --progress
The commands will transfer or synchronize all files in the same directory to different directories on the same computer.
rsync -avzh /root/tomcat/ /tmp/backup/tomcat/ --progress
Use the –include and –exclude options
These two options allow us to include exclude files and folders by specifying parameters with which these options can help us specify which files and folders to include and exclude during synchronization. You do not want to be moved to these files or directories.
Here, in this example, the rsync command will include only those files and directories that end with a “log” and exclude end with a “war” files and directories.
rsync -avzh --include '*log' --exclude '*war' /root/tomcat/ /tmp/backup/tomcat/ --progress
Use the –delete
If the file or directory does not exist at the source but already exists at the destination, you may need to delete the existing file/directory at the destination during synchronization.
We can use the ‘–delete’ option to delete files that do not exist in the source directory. Source and target are synchronized. Now, create a new target file, test.txt.
touch /tmp/backup/tomcat/123.txt ls /tmp/backup/tomcat/ rsync -avzh --delete /root/tomcat/ /tmp/backup/tomcat/ --progress ls /tmp/backup/tomcat/
The target has the name test.txt, and when synchronizing the new file with the source of the ‘-delete’ option, it removes the file test.txt.
Sets the maximum size of the file to transfer
You can do this with the — max-size option. Here, in this example, the maximum file size is 200K, so the command will only pass smaller files equal to or greater than 200K.
rsync -avzh --max-size='200k' /root/tomcat/ /tmp/backup/tomcat/ --progress
Set bandwidth limits and transfer files
You can transfer data from one machine to another at the same time with the help of the ‘–bwlimit’ option. This option helps limit I/O bandwidth.
rsync -avzh --bwlimit=10 /root/tomcat/ /tmp/backup/tomcat/ --progress rsync -avzh --bwlimit=50 /root/tomcat/ /tmp/backup/tomcat/ --progress
Remotely synchronize data using ssh-based rsync
Through the introduction of synchronizing data on a local disk, we learned about the most common options. The options for remote transport are the same as for local transport. So we’ll cover only the basic format of remote transmission. If you need to use other options, you can choose according to the above description
Use SSH to copy files from the local server to the remote server
To specify the protocol to use rsync, you need to give the protocol you want to use the “-e” option. Here, in this example, we will use “SSH” with the “-e” option “-p” to specify the transport port and perform the data transfer.
rsync -zvh -e "ssh -p12598" backup.tar [email protected]:/tmp/backup/ --progress
Copy files from remote server to local server via SSH
rsync -zvh -e "ssh -p12598" [email protected]:/tmp/backup/* /tmp/ --progress
conclusion
The above article introduced two working modes of rsync: local transmission and remote transmission via SSH protocol. The third working mode is transmission via rsync Daemon. Rsync Daemon commands are basically the same as using SSH. The only change is that the destination address is transferred according to the alias defined in Rsync Daemon instead of IP.
If you want to know more about rsync, please leave me a message To learn more about Linux, please follow my link. Thank you
