Use 'on the fly' compression when using scp -c <cipher>


Two ideas: 1. Experiment with a different cipher 2. Use 'on the fly' compression

1. Experiment with a different cipher:

One idea is to test the various ciphers which are available through scp and try to determine which is the fastest for you. (The idea behind this is that it is the encryption of your data that is taking up the time.) Find the available ciphers by running the following:

andrew@athens:~$ ssh -Q cipher
3des-cbc
blowfish-cbc
cast128-cbc
arcfour
arcfour128
arcfour256
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc@lysator.liu.se
aes128-ctraes
192-ctraes
256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
chacha20-poly1305@openssh.com
andrew@athens:~$

This shows available ciphers on a default Xenial Xerus installation and can be added in using the syntax you have already suggested:

scp -c cipher [FILE] [USER]@[ADDRESS]:[PATH]

This option is passed directly to ssh hence testing ssh itself for the available ciphers. Hopefully you can find a cipher that will allow an increase in your transfer speed, when you have found this cipher you can place it in ~/.ssh/config rather than typing it in each time...

2. Use 'on the fly' compression:

scp is able to compress your transfers 'on the fly' meaning that the compressions is only used during transfer and will not be seen at either the host or remote ends. Add this in using the -C option:

scp -c cipher -C [FILE] [USER]@[ADDRESS]:[PATH]

Here is an example on my own system:

debug1: compress outgoing: raw data 365865, compressed 150118, factor 0.41
debug1: compress incoming: raw data 150118, compressed 365865, factor 2.44

The compression level is the same as the one used for gzip and if you really wish to experiment you can use the 'CompressionLevel' option which varies from 1-9, 1 being the fastest compression, 9 being the slowest with best compression and 6 being the default. This can be set in ~/.ssh/configas follows:

Host *Compression yesCompressionLevel 9

Bear in mind that this sets high compression for all scp / ssh transfers, it can be set for individual hosts if you wish...

Use the -v option to investigate the effects of any changes you make. You will note with experimentation that different data will have different compression ratios.