This comprehensive Linux guide expects that you run the following commands as root user but if you decide to run the commands as a different user then ensure that the user has
sudoaccess and that you precede each of the privileged commands withsudo
rsync command is used to sync (or copy) files and directories locally and remotely. One of the important feature of rsync is that it works on “delta transfer algorithm”, means it will only sync or copy the changes from source to destination instead of copying the whole file which ultimately reduce amount of data sent over network. With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.
In this tutorial, we are using commands with examples. So, replace the “path” with the file path, “filename” with the name of the file and “.ext” with the extension of the file.
Some Important Features:
- Speed : First time, rsync replicates the whole content between the source and destination directories. Next time, rsync transfers only the changed blocks or bytes to the destination location, which makes the transfer really fast.
- Security : rsync allows encryption of data using ssh protocol during transfer.
- Low Bandwidth: rsync uses compression and decompression of data block by block at the sending and receiving end respectively. So the bandwidth used by rsync will be always less compared to other file transfer protocols like FTP.
- Privileges : No special privileges are required to install and execute rsync.
Installing rsync On Linux
Red-Hat Based Systems
On a red-hat based system, you can install rsync by running the following command
root@codesposts:~$ yum install rsyncDebian Based Systems
On a debian based system, you can install rsync by running the following command
root@codesposts:~$ apt-get install rsyncGeneral Syntax
The general syntax of the rsync command is given below:
root@codesposts:~$ rsync options source destinationOptions
| -v, –verbose | increase verbosity |
| -q, –quiet | suppress non-error messages |
| –no-motd | suppress daemon-mode MOTD (see caveat) |
| -c, –checksum | skip based on checksum, not mod-time & size |
| -a, –archive | archive mode; equals -rlptgoD (no -H,-A,-X) |
| –no-OPTION | turn off an implied OPTION (e.g., –no-D) |
| -r, –recursive | recurse into directories |
| -R, –relative | use relative path names |
| –no-implied-dirs | don’t send implied dirs with –relative |
| -b, –backup | make backups (see –suffix & –backup-dir) |
| –backup-dir=DIR | make backups into hierarchy based in DIR |
| –suffix=SUFFIX | backup suffix (default ~ w/o –backup-dir) |
| -u, –update | skip files that are newer on the receiver |
| –inplace | update destination files in-place |
| –append | append data onto shorter files |
| –append-verify | –append w/old data in file checksum |
| -d, –dirs | transfer directories without recursing |
| -l, –links | copy symlinks as symlinks |
| -L, –copy-links | transform symlink into referent file/dir |
| –copy-unsafe-links | only “unsafe” symlinks are transformed |
| –safe-links | ignore symlinks that point outside the tree |
| -k, –copy-dirlinks | transform symlink to dir into referent dir |
| -K, –keep-dirlinks | treat symlinked dir on receiver as dir |
| -H, –hard-links | preserve hard links |
| -p, –perms | preserve permissions |
| -E, –executability | preserve executability |
| –chmod=CHMOD | affect file and/or directory permissions |
| -A, –acls | preserve ACLs (implies -p) |
| -X, –xattrs | preserve extended attributes |
| -o, –owner | preserve owner (super-user only) |
| -g, –group | preserve group |
| –devices | preserve device files (super-user only) |
| –specials | preserve special files |
| -D | same as –devices –specials |
| -t, –times | preserve modification times |
| -O, –omit-dir-times | omit directories from –times |
| –super | receiver attempts super-user activities |
| –fake-super | store/recover privileged attrs using xattrs |
| -S, –sparse | handle sparse files efficiently |
| -n, –dry-run | perform a trial run with no changes made |
| -W, –whole-file | copy files whole (w/o delta-xfer algorithm) |
| -x, –one-file-system | don’t cross filesystem boundaries |
| -B, –block-size=SIZE | force a fixed checksum block-size |
| -e, –rsh=COMMAND | specify the remote shell to use |
| –rsync-path=PROGRAM | specify the rsync to run on remote machine |
| –existing | skip creating new files on receiver |
| –ignore-existing | skip updating files that exist on receiver |
| –remove-source-files | sender removes synchronized files (non-dir) |
| –del | an alias for –delete-during |
| –delete | delete extraneous files from dest dirs |
| –delete-before | receiver deletes before transfer, not during |
| –delete-during | receiver deletes during the transfer |
| –delete-delay | find deletions during, delete after |
| –delete-after | receiver deletes after transfer, not during |
| –delete-excluded | also delete excluded files from dest dirs |
| –ignore-errors | delete even if there are I/O errors |
| –force | force deletion of dirs even if not empty |
| –max-delete=NUM | don’t delete more than NUM files |
| –max-size=SIZE | don’t transfer any file larger than SIZE |
| –min-size=SIZE | don’t transfer any file smaller than SIZE |
| –partial | keep partially transferred files |
| –partial-dir=DIR | put a partially transferred file into DIR |
| –delay-updates | put all updated files into place at end |
| -m, –prune-empty-dirs | prune empty directory chains from file-list |
| –numeric-ids | don’t map uid/gid values by user/group name |
| –timeout=SECONDS | set I/O timeout in seconds |
| –contimeout=SECONDS | set daemon connection timeout in seconds |
| -I, –ignore-times | don’t skip files that match size and time |
| –size-only | skip files that match in size |
| –modify-window=NUM | compare mod-times with reduced accuracy |
| -T, –temp-dir=DIR | create temporary files in directory DIR |
| -y, –fuzzy | find similar file for basis if no dest file |
| –compare-dest=DIR | also compare received files relative to DIR |
| –copy-dest=DIR | … and include copies of unchanged files |
| –link-dest=DIR | hardlink to files in DIR when unchanged |
| -z, –compress | compress file data during the transfer |
| –compress-level=NUM | explicitly set compression level |
| –skip-compress=LIST | skip compressing files with suffix in LIST |
| -C, –cvs-exclude | auto-ignore files in the same way CVS does |
| -f, –filter=RULE | add a file-filtering RULE |
| -F | same as –filter=’dir-merge /.rsync-filter’ repeated: –filter=’- .rsync-filter’ |
| –exclude=PATTERN | exclude files matching PATTERN |
| –exclude-from=FILE | read exclude patterns from FILE |
| –include=PATTERN | don’t exclude files matching PATTERN |
| –include-from=FILE | read include patterns from FILE |
| –files-from=FILE | read list of source-file names from FILE |
| -0, –from0 | all *from/filter files are delimited by 0s |
| -s, –protect-args | no space-splitting; wildcard chars only |
| –address=ADDRESS | bind address for outgoing socket to daemon |
| –port=PORT | specify double-colon alternate port number |
| –sockopts=OPTIONS | specify custom TCP options |
| –blocking-io | use blocking I/O for the remote shell |
| –stats | give some file-transfer stats |
| -8, –8-bit-output | leave high-bit chars unescaped in output |
| -h, –human-readable | output numbers in a human-readable format |
| –progress | show progress during transfer |
| -P | same as –partial –progress |
| -i, –itemize-changes | output a change-summary for all updates |
| –out-format=FORMAT | output updates using the specified FORMAT |
| –log-file=FILE | log what we’re doing to the specified FILE |
| –log-file-format=FMT | log updates using the specified FMT |
| –password-file=FILE | read daemon-access password from FILE |
| –list-only | list the files instead of copying them |
| –bwlimit=KBPS | limit I/O bandwidth; KBytes per second |
| –write-batch=FILE | write a batched update to FILE |
| –only-write-batch=FILE | like –write-batch but w/o updating dest |
| –read-batch=FILE | read a batched update from FILE |
| –protocol=NUM | force an older protocol version to be used |
| –iconv=CONVERT_SPEC | request charset conversion of file names |
| –checksum-seed=NUM | set block/file checksum seed (advanced) |
| -4, –ipv4 | prefer IPv4 |
| -6, –ipv6 | prefer IPv6 |
| –version | print version number |
| -h, –help | show help |
rsync can also be run as a daemon, in which case the following options are accepted:
| –daemon | run as an rsync daemon |
| –address=ADDRESS | bind to the specified address |
| –bwlimit=KBPS | limit I/O bandwidth; KBytes per second |
| –config=FILE | specify alternate rsyncd.conf file |
| –no-detach | do not detach from the parent |
| –port=PORT | listen on alternate port number |
| –log-file=FILE | override the “log file” setting |
| –log-file-format=FMT | override the “log format” setting |
| –sockopts=OPTIONS | specify custom TCP options |
| -v, –verbose | increase verbosity |
| -4, –ipv4 | prefer IPv4 |
| -6, –ipv6 | prefer IPv6 |
| -h, –help | show this help (if used after –daemon) |
Copy/Sync Files Or Directories From/To Local Computer
If you want to sync a single file on a local machine from one location to another location, you can use the following command:
root@codesposts:~$ rsync -zvh filename.ext /path/to/target/directoryAfter running this command, the file “filename.ext” will be copied to the target directory.
root@codesposts:~$ rsync -zvh /path/to/source/directory /path/to/target/directoryAfter running this command, the source directory will be copied to the target directory.
Copy/Sync Files Or Directories From A Remote Server
If you want to copy or sync a file or directory from a remote server to your local computer, you can run the following command
root@codesposts:~$ rsync -avzh [email protected]:/home/darkcoder /tmp
[email protected]'s password:
receiving incremental file list
created directory /tmp
darkcoder/
darkcoder/file1.txt
darkcoder/file2.txt
sent 87 bytes received 2.1M bytes 523.19K bytes/sec
total size is 2.1M speedup is 1.00This command will copy the directory “darkcoder” to “temp”. But to perform that, it would ask for the password for the remote server.
Copy/Sync Files Or Directories To A Remote Server
If you want to copy or sync a file or directory to a remote server from your local computer, you can run the following command
root@codesposts:~$ rsync -avzh /tmp/darkcoder [email protected]:/homeAfter running this command, you will be asked to provide the password to the remote server. And after the connection is established, the directory “darkcoder” will be copied to the “home” directory of the remote server.
Copy Files Or Directories From A Remote Server Over SSH
If you want to copy or sync a file or directory from a remote server to your local computer over SSH protocol, you can use the -e option, run the following command
root@codesposts:~$ rsync -avzhe ssh [email protected]:/home/darkcoder /tmpCopy Files Or Directories To A Remote Server Over SSH
If you want to copy or sync a file or directory to a remote server from your local computer over SSH protocol, you can use the -e option, run the following command
root@codesposts:~$ rsync -avzhe ssh /tmp/darkcoder [email protected]:/homeDisplay Progress While Transferring Data With rsync
You can use –progress option with rsync. This option tells rsync to print information showing the progress of the transfer. This gives a bored user something to watch. Implies –verbose if it wasn’t already specified.
root@codesposts:~$ rsync -avzhe ssh --progress [email protected]:/home/darkcoder /tmp
[email protected]'s password:
receiving incremental file list
created directory /tmp
darkcoder/
darkcoder/file1.txt
1.02M 100% 2.72MB/s 0:00:00 (xfer#1, to-check=3/5)
Copy Files And Directories With include/exclude Option
These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.
root@codesposts:~$ rsync -avzhe ssh --include '*.txt" --exclude '*.*' /tmp/darkcoder [email protected]:/homeUsing delete Option
You can also use --delete option with rsync command. This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g., “dir” or “dir/“) without using a wildcard for the directory’s contents (e.g., “dir/*“) since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files’ parent directory. Files that are excluded from the transfer are also excluded from being deleted unless you use the –delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section). Prior to rsync 2.6.7, this option would have no effect unless –recursive was enabled. Beginning with 2.6.7, deletions will also occur when –dirs (-d) is enabled, but only for directories whose contents are being copied.
This option can be dangerous if used incorrectly. It is a very good idea to first try a run using the –dry-run option (-n) to see what files are going to be deleted.
root@codesposts:~$ rsync -avzh --delete [email protected]:/home/darkcoder.txt /tmp
Password:
receiving file list ... done
deleting darkcoder.txt
/tmpSet The Maximum Size Of Files
You can use the --max-size option to set the maximum size of the files to be transferred. This tells rsync to avoid transferring any file that is larger than the specified SIZE. The SIZE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g., “–max-size=1.5m“).
root@codesposts:~$ rsync -avzhe ssh --max-size='500k' [email protected]:/home/darkcoder /tmpMoving Files Instead Of Copying
If you want to move the files i.e. to delete the source files after the transfer is complete, you can use the –remove-source-files option.
root@codesposts:~$ rsync --remove-source-files -zvh filename.ext /path/to/target/directoryTransfer Files With A Bandwidth Limit
You can set the bandwidth limit while transferring data from one machine to another machine with the the help of --bwlimit option. This option allows you to specify a maximum transfer rate in kilobytes per second. This option is most effective when using rsync with large files (several megabytes and up).
root@codesposts:~$ rsync --bwlimit=200 -avzh [email protected]:/home/darkcoder /tmpTransfer Files With A Dry Run
This makes rsync perform a trial run that doesn’t make any changes (and produces mostly the same output as a real run). It is most commonly used in combination with the -v, --verbose and/or -i, --itemize-changes options to see what an rsync command is going to do before one actually runs it.
root@codesposts:~$ rsync --dry-run --remove-source-files -zvh filename.ext /path/to/target/directory
