Home / server / Questions / 446933
Accepted
Speeddymon
Asked: 2012-11-09 15:44:11 +0800 CST

rsync unique situation

  • 772

Reference:
How do I do Multihop SCP transfers?
https://stackoverflow.com/questions/732039/rsyncing-files-between-two-remote-servers-get-errors-stating-rsync-command-not
http://ubuntuforums.org/showthread.php?t=748431

I have a situation that I really hope there is a solution for, although I've found nothing yet over the last couple of hours.

I have two servers that cannot talk to each other that I need to transfer data between. That's a fairly common situation that is easily overcome with some scp trickery. Here is the gotcha.

One server is accessed over a VPN, vpnserver the other server targetserver is accessed via a jump host jumphostserver.

I have ProxyCommand setup to allow me to proxy connections through jumphostserver to targetserver and I'm using the SSH Mux stuff (ControlMaster/ControlPersist/ControlPath) to allow connection sharing for all connections that are open.

My local machine can properly scp things between the two servers using the following command:

scp -3 vpnserver:/path/to/file targetserver:/path/to/destination

I can also rsync things directly from vpnserver to my local machine and from my local machine to targetserver (using the ssh proxy that goes through jumphostserver)

What I need to do is make my machine act as an intermediary the way that scp -3 allows it to, but do so using rsync, so that permissions, ownership, and (more importantly) the ACLs are properly copied to targetserver.

I had thought about trying to NFS export the filesystem of vpnserver and targetserver (re-exported through jumphostserver), but I don't have control over jumphostserver, only over vpnserver and targetserver.

rsync

2 Answers

  1. Best Answer

    If one of these two hosts could log into the other if the network connectivity were present, either with an SSH ID key on one recognized in authorized_keys on the other, or a password can be used, AND if port forwarding is not disabled, AND if running this traffic through your local machine is OK, then this can almost always be done via port forwarding.

    Let's call your local machine host A, and that host B has the credentials to log into host C with SSH if only it could reach host C.

    1. Log in to host B using the option to do REVERSE port forwarding listening to port 2222 on machine B and connecting to port 3333 on localhost (which will happen on machine A). On Linux this option is "-R 2222:localhost:3333".

    2. Log in to host C using normal port forwarding listening to port 3333 on machine A and connecting tp port 22 on localhost (which will happen on machine C). On Linux this option is "-L 3333:localhost:22".

    3. On host B start a test SSH session to host C to see if everything is set up right. The command to do is "ssh -p2222 usernameonhostc@localhost". If all is set up right, it will connect to port 2222 on B, be forwarded to port 3333 on A, and then be forwarded to port 22 on C. If your keys are right, you can just login. Otherwise you should ge a password prompt. Enter the password to be sure.

    If you can connect OK, close that test connection, and proceed with either #4 or #5 below.

    1. To send files from B to C, do this on B: rsync -e "ssh -p2222" -av /where/to/get/files/. usernameonhostc@localhost:/where/to/put/files

    2. To send files from C to B, do this on B: rsync -e "ssh -p2222" -av usernameonhostc@localhost:/where/to/get/files/. /where/to/put/files

    If you need to transfer very large amounts of data between the machines over the internet because your local access has limited bandwidth, then it will be more complicated, require knowing how the networks are restricted on B and C, and may not even be possible.

    Substitute the string "useronhostc" with your username that you use on host C. Leave "localhost" as is or change it only to "127.0.0.1". Provide the correct paths for the examples shown. Remove the "/." at the end of the source path if it is just a single file.

    Remember you must login to both B and C (from A) at the same time. Separate xterminal windows is suggested. These ssh logins will both be doing port forwarding. But you can also do commands through them. You could choose different port numbers than 2222 or 333 shown by example. Port 22 is probably require unless host C listens on a different port for some reason (I do that with my servers ... I'm not telling what port). If you change ports 2222 and/or 3333 you probably have to use numbers 1024 or above. Technically, you can even use the same number for both since the listens are on different machines.

    • 2

  2. If both systems have access to the internet, you could install the n2n package from ntop to create a host-to-host VPN link between the systems.

    • 0