FOSS Unleashed

scp considered harmful

For whatever reason scp does not have a means to copy a symlink as a symlink. This makes scp absolutely terrible if you want to backup or archive a directory structure that contains a symlink cycle. If the directory you are wanting to copy contains a .wine directory, it very likely contains a symlink cycle.

As I have been burned by this in the past, I often argue against the usage of scp altogether, instead suggesting alternate means of transfering files. Either use sftp (via sftp, lftp, or filezilla), or just use tar | tar. I find tar piped into another instance of tar to be quite useful, and quite expressive once you get used to it.

# Copy directories from HOST
ssh HOST tar c dir1 dir2 dir3 | tar x

# Copy directories to HOST
tar c dir1 dir2 dir3 | ssh HOST tar x

# Copy directories to a specific directory on HOST
tar c dir1 dir2 dir3 | ssh HOST tar x -C ~/target/directory/

You can also put pv inbetween the tar transfers to get a progress bar.

The main thing to note with this technique is that tar c (note the lack of -f) produces a tarball to stdout, and tar x consumes a tarball from stdin. On occasion I find it helpful to copy files like this without ssh, generally I want to preserve a specific directory structure in these instances. Or I simply want the benefit of pv to get a sense of the progress of a larger file transfer.

Now, some of you might ask “why not rsync?” Which is a fair question, if you can rely on rsync being present on both sides (as it is requires this), rsync is in itself a very expressive tool. But that’s not something I can rely on. If there’s ssh on a host, there is also very likely to be tar.