Home / server / Questions / 452978
Accepted
Zizzencs
Asked: 2012-11-29 04:48:15 +0800 CST

Copying files between linux machines with strong authentication without encryption

  • 772

I'm looking for a suitable program to copy files from one linux machine to another one. The program should be able to do authentication but it should not do encryption. The reason behind the latter is the lack of CPU power to do the encryption.

I copy backups from ~70 machines to a single backup server simultaneously. The single server is an HP Proliant DL360 G7, with 10 Gbps ethernet connection and an FC storage backend that can do 4 Gbps. Through FTP I can write ~400MB/sec to the storage (that's about what I want) but through ssh with arcfour I can only do ~100MB/sec while having 100% CPU usage. That's why I want file transfers not to be encrypted.

The alternatives that I found not really suitable:

  • rcp: no authentication, forget it
  • FTP: making the authentication "secure" (at least preventing plain-text password exchange) is possible but not really easy and I haven't found a method to force any FTP daemon to encrypt the control channel (for the authentication) and not to encrypt the data channel (for data transfers)
  • SCP/SFTP: in farely recent ssh(d) implementations you can't turn off encryption. The best you can do is to use the arcfour cypher for the encryption but it sill uses too much CPU power for my needs.
  • rsync over ssh: same problems as with SCP/SFTP.
  • plain rsync: from the documentation of rsyncd: "The authentication protocol used in rsync is a 128 bit MD4 based challenge response system. This is fairly weak protection, though (with at least one brute-force hash-finding algorithm publicly available), so if you want really top-quality security, then I recommend that you run rsync over ssh." It's a no-go.

Is there a protocol/program that can do exactly what I want?

(A big plus would be if it could work on windows as well and/or if it would support rsync-stlye copying/synchronization (e.g. copy only the differences).)

linux

3 Answers

  1. Best Answer

    You tried doing what I normally do, using a lower-weight encryption algorithm (like arcfour). When that becomes the bottleneck, the next approach I use is disabling ssh encryption entirely.

    One approach for this is to use hpn-ssh as your transport. My normal application for this is SSH copies across high-bandwidth, long-distance links (e.g. a 10GbE link between Chicago and New York with 16ms latency). hpn-ssh allows tuning of TCP windows there, but also has the option of disabling encryption entirely. That may help in your case.

    Also see: Why is my rsync so slow?

    • 9

  2. A couple of quick-fix options come to mind:

    • Stagger your backups so that they aren't all running at the same time.
    • Replace ssh with hpn-ssh (on the server and all 70 machines) and use its NONE cipher, which turns encryption off after authentication is complete.
    • 7

  3. WebDAV + authentication on Apache2 side sounds like an option :-)

    • 0