Uploading a whole directory to a remote server with LFTP

As any user of a restricted VPS or PaaS product (like WPEngine) might know, there are sometimes restrictions that stop you from using ssh or scp.

In my most recent case I really needed to move about 30GB of tiny thumbnail files to WPEngine. I considered downloading them all to my machine with Cyberduck, then uploading them over, but that would have been way too slow. Cyberduck goes into a 'preparing...' cycle that seems to never end, and caps my CPU at 100%.

And in any case, SFTP doesn't support recursive put. What?!

Thankfully, there's LFTP. LFTP is available in just about any sane package manger and works in this fashion:

lftp sftp://user@mysub.domain.com

It uses the same syntax as sftp, plus a few awesome additions.
 
In my case, I wanted to move an entire wp-content/uploads folder over. This is really easy. Simply cd into your existing uploads directory, fire up lftp, and...

lftp user@site.wpengine.com:/wp-content/uploads/2012/10> mirror -R --parallel=20

This is the money shot. mirror -R  means "mirror in reverse", that is, move all local files to remote. And the parallel directive is extremely useful when moving tons of small files. I was seeing incredible (>6MB/sec) transfer rates of these tiny files between sites. And when uploading on my small comcast connection, I was able to saturate the pipe with parallelism this high.

Simply put, there exists no GUI tool that can pull something off this elegantly and quickly. Use it.