work - scp vs rsync speed
How does `scp` differ from `rsync`? (5)
An article about setting up Ghost blogging says to use
scp to copy from my local machine to a remote server:
scp -r ghost-0.3 [email protected]*your-server-ip*:~/
However, Railscast 339: Chef Solo Basics uses
scp to copy in the opposite direction (from the remote server to the local machine):
scp -r [email protected]:/var/chef .
In the same Railscast, when the author wants to copy files to the remote server (same direction as the first example), he uses
rsync -r . [email protected]:/var/chef
Why use the
rsync command if
scp will copy in both directions? How does
scp differ from
scp is best for one file.
OR a combination of
tar & compression for smaller data sets
like source code trees with small resources (ie: images, sqlite etc).
Yet, when you begin dealing with larger volumes say:
- media folders (40 GB)
- database backups (28 GB)
- mp3 libraries (100 GB)
It becomes impractical to build a zip/tar.gz file to transfer with scp at this point do to the physical limits of the hosted server.
As an exercise, you can do some gymnastics like piping
ssh and redirecting the results into a remote file. (saving the need to build
a swap or temporary clone aka zip or tar.gz)
rsync simplify's this process and allows you to transfer data without consuming any additional disc space.
Continuous (cron?) updates use minimal changes vs full cloned copies speed up large data migrations over time.
scp == small scale (with room to build compressed files on the same drive)
rsync == large scale (with the necessity to backup large data and no room left)
Difference b/w scp and rsync on different parameter
1. Performance over latency
scp : scp is relatively less optimise and speed
rsync : rsync is comparatively more optimise and speed
2. Interruption handling :
scp : scp command line tool cannot resume aborted downloads from lost network connections
rsync : If the above rsync session itself gets interrupted, you can resume it as many time as you want by typing the same command. rsync will automatically restart the transfer where it left off.
3. Command Example
$ scp source_file_path destination_file_path
$ cd /path/to/directory/of/partially_downloaded_file $ rsync -P --rsh=ssh [email protected]:bigdata.tgz ./bigdata.tgz
The "-P" option is the same as "--partial --progress", allowing rsync to work with partially downloaded files. The "--rsh=ssh" option tells rsync to use ssh as a remote shell.
4. Security :
scp is more secure, but when use rsync --rsh=ssh, then rsync would be as secure
man document to know more :
The major difference between these tools is how they copy files.
scp basically reads the source file and writes it to the destination. It performs a plain linear copy, locally, or over a network.
rsync also copies files locally or over a network. But it employs a special delta transfer algorithm and a few optimizations to make the operation a lot faster. Consider the call.
rsync A host:B
rsyncwill check files sizes and modification timestamps of both A and B, and skip any further processing if they match.
If the destination file B already exists, the delta transfer algorithm will make sure only differences between A and B are sent over the wire.
rsyncwill write data to a temporary file T, and then replace the destination file B with T to make the update look "atomic" to processes that might be using B.
Another difference between them concerns invocation.
rsync has a plethora of command line options, allowing the user to fine tune its behavior. It supports complex filter rules, runs in batch mode, daemon mode, etc.
scp has only a few switches.
In summary, use
scp for your day to day tasks. Commands that you type once in a while on your interactive shell. It's simpler to use, and in those cases
rsync optimizations won't help much.
For recurring tasks, like
cron jobs, use
rsync. As mentioned, on multiple invocations it will take advantage of data already transferred, performing very quickly and saving on resources. It is an excellent tool to keep two directories synchronized over a network.
Also, when dealing with large files, use
rsync with the
-P option. If the transfer is interrupted, you can resume it where it stopped by reissuing the command. See Sid Kshatriya's answer.
There's a distinction to me that
scp is always encrypted with ssh (secure shell), while
rsync isn't necessarily encrypted. More specifically,
rsync doesn't perform any encryption by itself; it's still capable of using other mechanisms (ssh for example) to perform encryption.
In addition to security, encryption also has a major impact on your transfer speed, as well as the CPU overhead. (My experience is that
rsync can be significantly faster than
Check out this post for when
rsync has encryption on.
rysnc can be useful to run on slow and unreliable connections. So if your download aborts in the middle of a large file rysnc will be able to continue from where it left off when invoked again.
rsync -vP [email protected]:/path/to/file .
The -P option preserves partially downloaded files and also shows progress.
As usual check