Knowledgebase
emscom > emscom Help Desk > Knowledgebase

Search help:


Linux rsync common usage

Solution

rsync is a command line file synchronisation application, which is available for

  • Natively for almost all Linux distributions
  • As a statically linked binary for ESXi server
  • Most versions of Windows by way of a cut down cygwin environment

The rsync man page does not list the multitude of options in alphabetical order, which can make finding particular options long winded.  Some options are mutually exclusive and other options cancel other options.

Some common options and combinations are as follows.

Basic options suitable for local copy

options: -a archive, -v verbose, -h human readable, - P progress report

rsync -avhP /path/source/ /destination/

Copies the content of /path/source/. to /destination/.  and leaves intact any pre-existing files in /destination/ which are not in /path/source/

rsync -avhP /path/source /destination/

Creates a parent folder /destination/source/ and copies the contents of /path/source/. to it.

rsync -avhP --delete /path/source /destination/

Create a mirror of /path/source in /destination/ Delete any pre existing files in /destination/. which are not in source/.

 

Rsync MacOS Time Machine Backup Archive

MacOS includes rsync V2.69 accessible from the command line.

Rsync can be used to copy the Time Machine Backups.backupdb folder to and from volumes configured for use with time machine.  Hardlinks in the Backup.backupdb sub tree must be preserved using the -H parameter

rsync -avhHp /Volumes/TimeMachine/Backups.backupdb /Volumes/New/.

 

Optimising Throughput

Typical rsync operations carried out over fast LANs, often become bottlenecked at well below 10MBps (100Mbps).  Careful selection of the command options can improve throughput significantly.  Testing on a GBE LAN realised an improvement, from ~5MBps (50Mbps) to ~30MBps (300Mbps) for a 2GB file. 

The primary bottlenecks are caused by crc and compression algorithms within rsync, and the encryption algorithm within SSH.  Compression is disabled by default but often enabled without thought.  CRC calculations are not necessary during the initial replication, as there are no files in the destination to check against.  SSH encryption can be speeded up by selecting a less secure algorithm, such as blowfish, by passing switches to SSH through the rsync -e environment option.

  • rsync compression is enabled and disabled with the -z switch.
  • rsync crc checking may be disabled during initial replication with the [upper case] -W (--whole-file) switch.
  • ssh blowfish encryption is selected with -e '-c blowfish'

rsync -e 'ssh -c blowfish' -avhPW user@host.local:~/work ~/backup/lanhost/


Options Suitable for Remote copy

Option: -z compress in transit

rsync -avhPz user@10.20.30.40:~/source /destination/

Connect to the host 10.20.30.40 using SSH as user and copy the content of /home/user/source to the local folder /destination/source/. Compress files before transmission and decompress before writing.

Option -e environment

rsync -e "-p 9022" -avhPz /path/source user@fqdn.net:/backup/

Pass the parameters -p 9022 to SSH, to connect to the custom SSH port 9022 on a host with the domain name fqdn.net,  as user.  Copy the contents from the local folder /path/source/. to the folder /backup/ on he remote host.

 

Incremental Backup Options

Option -b backup, --backup= use archive folder

rsync - avhPzb --delete --backup-dir=/backup/archive/ /path/source /backup/today/

Mirrors the /path/source folder to /backup/today/.  Pre-existing files in /backup/today/ which do not appear in /path/source/. are moved to the /backup/archive/ folder.  Using the -b and --backup-dir options in a script, combined with date commands, allows an incremental backup to be constructed.

 

Windows Usage

rsync use under windows is essentially the same as linux.  Path references may not use the colon ':' character however.  Drives are referenced relative to the /cygdrive/ virual path.

rsync -avhP /cygdrive/c/documents\ and\ settings/ /cygdrive/r/backuppath/

 

Transfer Speeds

Data transfer between remote drives mapped with samba and wfs are much faster then using rsync's native ssh.  i.e. 10x to 100x

 

 
Was this article helpful? yes / no
Related articles robocopy
Time Machine backup rsync
ESXi OVF Tool
QNAP Backup disk not available
Yun user setup
Article details
Article ID: 71
Category: Linux
Date added: 30-03-2015 14:19:00
Views: 478
Rating (Votes): Article rated 3.0/5.0 (2)

 
« Go back

 
Powered by Help Desk Software HESK, in partnership with SysAid Technologies