This comprehensive Linux guide expects that you run the following commands as root user but if you decide to run the commands as a different user then ensure that the user has sudo access and that you precede each of the privileged commands with sudo

rsync command is used to sync (or copy) files and directories locally and remotely. One of the important feature of rsync is that it works on “delta transfer algorithm”, means it will only sync or copy the changes from source to destination instead of copying the whole file which ultimately reduce amount of data sent over network. With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.

In this tutorial, we are using commands with examples. So, replace the “path” with the file path, “filename” with the name of the file and “.ext” with the extension of the file.

Some Important Features:

  • Speed : First time, rsync replicates the whole content between the source and destination directories. Next time, rsync transfers only the changed blocks or bytes to the destination location, which makes the transfer really fast.
  • Security : rsync allows encryption of data using ssh protocol during transfer.
  • Low Bandwidth: rsync uses compression and decompression of data block by block at the sending and receiving end respectively. So the bandwidth used by rsync will be always less compared to other file transfer protocols like FTP.
  • Privileges : No special privileges are required to install and execute rsync.

Installing rsync On Linux

Red-Hat Based Systems

On a red-hat based system, you can install rsync by running the following command

[email protected]:~$ yum install rsync

Debian Based Systems

On a debian based system, you can install rsync by running the following command

[email protected]:~$ apt-get install rsync

General Syntax

The general syntax of the rsync command is given below:

[email protected]:~$ rsync options source destination

Options

-v–verboseincrease verbosity
-q–quietsuppress non-error messages
–no-motdsuppress daemon-mode MOTD (see caveat)
-c–checksumskip based on checksum, not mod-time & size
-a–archivearchive mode; equals -rlptgoD (no -H,-A,-X)
–no-OPTIONturn off an implied OPTION (e.g., –no-D)
-r–recursiverecurse into directories
-R–relativeuse relative path names
–no-implied-dirsdon’t send implied dirs with –relative
-b, –backupmake backups (see –suffix & –backup-dir)
–backup-dir=DIRmake backups into hierarchy based in DIR
–suffix=SUFFIXbackup suffix (default ~ w/o –backup-dir)
-u–updateskip files that are newer on the receiver
–inplaceupdate destination files in-place
–appendappend data onto shorter files
–append-verify–append w/old data in file checksum
-d–dirstransfer directories without recursing
-l–linkscopy symlinks as symlinks
-L–copy-linkstransform symlink into referent file/dir
–copy-unsafe-linksonly “unsafe” symlinks are transformed
–safe-linksignore symlinks that point outside the tree
-k–copy-dirlinkstransform symlink to dir into referent dir
-K–keep-dirlinkstreat symlinked dir on receiver as dir
-H–hard-linkspreserve hard links
-p–permspreserve permissions
-E–executabilitypreserve executability
–chmod=CHMODaffect file and/or directory permissions
-A–aclspreserve ACLs (implies -p)
-X–xattrspreserve extended attributes
-o–ownerpreserve owner (super-user only)
-g–grouppreserve group
–devicespreserve device files (super-user only)
–specialspreserve special files
-Dsame as –devices –specials
-t–timespreserve modification times
-O–omit-dir-timesomit directories from –times
–superreceiver attempts super-user activities
–fake-superstore/recover privileged attrs using xattrs
-S, –sparsehandle sparse files efficiently
-n–dry-runperform a trial run with no changes made
-W–whole-filecopy files whole (w/o delta-xfer algorithm)
-x–one-file-systemdon’t cross filesystem boundaries
-B–block-size=SIZEforce a fixed checksum block-size
-e–rsh=COMMANDspecify the remote shell to use
–rsync-path=PROGRAMspecify the rsync to run on remote machine
–existingskip creating new files on receiver
–ignore-existingskip updating files that exist on receiver
–remove-source-filessender removes synchronized files (non-dir)
–delan alias for –delete-during
–deletedelete extraneous files from dest dirs
–delete-beforereceiver deletes before transfer, not during
–delete-duringreceiver deletes during the transfer
–delete-delayfind deletions during, delete after
–delete-afterreceiver deletes after transfer, not during
–delete-excludedalso delete excluded files from dest dirs
–ignore-errorsdelete even if there are I/O errors
–forceforce deletion of dirs even if not empty
–max-delete=NUMdon’t delete more than NUM files
–max-size=SIZEdon’t transfer any file larger than SIZE
–min-size=SIZEdon’t transfer any file smaller than SIZE
–partialkeep partially transferred files
–partial-dir=DIRput a partially transferred file into DIR
–delay-updatesput all updated files into place at end
-m–prune-empty-dirsprune empty directory chains from file-list
–numeric-idsdon’t map uid/gid values by user/group name
–timeout=SECONDSset I/O timeout in seconds
–contimeout=SECONDSset daemon connection timeout in seconds
-I–ignore-timesdon’t skip files that match size and time
–size-onlyskip files that match in size
–modify-window=NUMcompare mod-times with reduced accuracy
-T–temp-dir=DIRcreate temporary files in directory DIR
-y–fuzzyfind similar file for basis if no dest file
–compare-dest=DIRalso compare received files relative to DIR
–copy-dest=DIR… and include copies of unchanged files
–link-dest=DIRhardlink to files in DIR when unchanged
-z–compresscompress file data during the transfer
–compress-level=NUMexplicitly set compression level
–skip-compress=LISTskip compressing files with suffix in LIST
-C–cvs-excludeauto-ignore files in the same way CVS does
-f–filter=RULEadd a file-filtering RULE
-Fsame as –filter=’dir-merge /.rsync-filter’

repeated: –filter=’- .rsync-filter’
–exclude=PATTERNexclude files matching PATTERN
–exclude-from=FILEread exclude patterns from FILE
–include=PATTERNdon’t exclude files matching PATTERN
–include-from=FILEread include patterns from FILE
–files-from=FILEread list of source-file names from FILE
-0–from0all *from/filter files are delimited by 0s
-s–protect-argsno space-splitting; wildcard chars only
–address=ADDRESSbind address for outgoing socket to daemon
–port=PORTspecify double-colon alternate port number
–sockopts=OPTIONSspecify custom TCP options
–blocking-iouse blocking I/O for the remote shell
–statsgive some file-transfer stats
-8–8-bit-outputleave high-bit chars unescaped in output
-h–human-readableoutput numbers in a human-readable format
–progressshow progress during transfer
-Psame as –partial –progress
-i, –itemize-changesoutput a change-summary for all updates
–out-format=FORMAToutput updates using the specified FORMAT
–log-file=FILElog what we’re doing to the specified FILE
–log-file-format=FMTlog updates using the specified FMT
–password-file=FILEread daemon-access password from FILE
–list-onlylist the files instead of copying them
–bwlimit=KBPSlimit I/O bandwidth; KBytes per second
–write-batch=FILEwrite a batched update to FILE
–only-write-batch=FILElike –write-batch but w/o updating dest
–read-batch=FILEread a batched update from FILE
–protocol=NUMforce an older protocol version to be used
–iconv=CONVERT_SPECrequest charset conversion of file names
–checksum-seed=NUMset block/file checksum seed (advanced)
-4–ipv4prefer IPv4
-6–ipv6prefer IPv6
–versionprint version number
-h–helpshow help

rsync can also be run as a daemon, in which case the following options are accepted:

–daemonrun as an rsync daemon
–address=ADDRESSbind to the specified address
–bwlimit=KBPSlimit I/O bandwidth; KBytes per second
–config=FILEspecify alternate rsyncd.conf file
–no-detachdo not detach from the parent
–port=PORTlisten on alternate port number
–log-file=FILEoverride the “log file” setting
–log-file-format=FMToverride the “log format” setting
–sockopts=OPTIONSspecify custom TCP options
-v–verboseincrease verbosity
-4–ipv4prefer IPv4
-6–ipv6prefer IPv6
-h–helpshow this help (if used after –daemon)

Copy/Sync Files Or Directories From/To Local Computer

If you want to sync a single file on a local machine from one location to another location, you can use the following command:

[email protected]:~$ rsync -zvh filename.ext /path/to/target/directory

After running this command, the file “filename.ext” will be copied to the target directory.

[email protected]:~$ rsync -zvh /path/to/source/directory /path/to/target/directory

After running this command, the source directory will be copied to the target directory.

Copy/Sync Files Or Directories From A Remote Server

If you want to copy or sync a file or directory from a remote server to your local computer, you can run the following command

[email protected]:~$ rsync -avzh [email protected]:/home/darkcoder /tmp

[email protected]'s password:

receiving incremental file list

created directory /tmp

darkcoder/

darkcoder/file1.txt

darkcoder/file2.txt

sent 87 bytes  received 2.1M bytes  523.19K bytes/sec

total size is 2.1M  speedup is 1.00

This command will copy the directory “darkcoder” to “temp”. But to perform that, it would ask for the password for the remote server.

Copy/Sync Files Or Directories To A Remote Server

If you want to copy or sync a file or directory to a remote server from your local computer, you can run the following command

[email protected]:~$ rsync -avzh /tmp/darkcoder [email protected]:/home

After running this command, you will be asked to provide the password to the remote server. And after the connection is established, the directory “darkcoder” will be copied to the “home” directory of the remote server.

Copy Files Or Directories From A Remote Server Over SSH

If you want to copy or sync a file or directory from a remote server to your local computer over SSH protocol, you can use the -e option, run the following command

[email protected]:~$ rsync -avzhe ssh [email protected]:/home/darkcoder /tmp

Copy Files Or Directories To A Remote Server Over SSH

If you want to copy or sync a file or directory to a remote server from your local computer over SSH protocol, you can use the -e option, run the following command

[email protected]:~$ rsync -avzhe ssh /tmp/darkcoder [email protected]:/home

Display Progress While Transferring Data With rsync

You can use –progress option with rsync. This option tells rsync to print information showing the progress of the transfer. This gives a bored user something to watch. Implies –verbose if it wasn’t already specified.

[email protected]:~$ rsync -avzhe ssh --progress [email protected]:/home/darkcoder /tmp

[email protected]'s password:

receiving incremental file list

created directory /tmp

darkcoder/

darkcoder/file1.txt
         1.02M 100%        2.72MB/s        0:00:00 (xfer#1, to-check=3/5)

Copy Files And Directories With include/exclude Option

These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.

[email protected]:~$ rsync -avzhe ssh --include '*.txt" --exclude '*.*' /tmp/darkcoder [email protected]:/home

Using delete Option

You can also use --delete option with rsync command. This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g., “dir” or “dir/“) without using a wildcard for the directory’s contents (e.g., “dir/*“) since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files’ parent directory. Files that are excluded from the transfer are also excluded from being deleted unless you use the –delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section). Prior to rsync 2.6.7, this option would have no effect unless –recursive was enabled. Beginning with 2.6.7, deletions will also occur when –dirs (-d) is enabled, but only for directories whose contents are being copied.

This option can be dangerous if used incorrectly. It is a very good idea to first try a run using the –dry-run option (-n) to see what files are going to be deleted.

[email protected]:~$ rsync -avzh --delete [email protected]:/home/darkcoder.txt /tmp

Password:
receiving file list ... done
deleting darkcoder.txt
/tmp

Set The Maximum Size Of Files

You can use the --max-size option to set the maximum size of the files to be transferred. This tells rsync to avoid transferring any file that is larger than the specified SIZE. The SIZE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g., “–max-size=1.5m“).

[email protected]:~$ rsync -avzhe ssh --max-size='500k' [email protected]:/home/darkcoder /tmp

Moving Files Instead Of Copying

If you want to move the files i.e. to delete the source files after the transfer is complete, you can use the –remove-source-files option.

[email protected]:~$ rsync --remove-source-files -zvh filename.ext /path/to/target/directory

Transfer Files With A Bandwidth Limit

You can set the bandwidth limit while transferring data from one machine to another machine with the the help of --bwlimit option. This option allows you to specify a maximum transfer rate in kilobytes per second. This option is most effective when using rsync with large files (several megabytes and up).

[email protected]:~$ rsync --bwlimit=200 -avzh [email protected]:/home/darkcoder /tmp

Transfer Files With A Dry Run

This makes rsync perform a trial run that doesn’t make any changes (and produces mostly the same output as a real run). It is most commonly used in combination with the -v, --verbose and/or -i, --itemize-changes options to see what an rsync command is going to do before one actually runs it.

[email protected]:~$ rsync --dry-run --remove-source-files -zvh filename.ext /path/to/target/directory