protocol version mismatch -- is your shell clean?

One of your login scripts (.bashrc/.cshrc/etc.) is probably outputting data to the terminal (when it shouldn't be). This is causing ssh to error when it is connecting and getting ready to copy as it starts receiving extra data it doesn't expect. Remove output that is generated in the startup scripts.

You can check if your terminal is interactive and only output text by using the following code in a bashrc. Something equivalent exists for other shells as well:

if shopt -q login_shell; then
    [any code that outputs text here]
fi

or alternatively, like this, since the special parameter - contains i when the shell is interactive:

if echo "$-" | grep i > /dev/null; then
    [any code that outputs text here]
fi

For more information see: rsync via ssh from linux to windows sbs 2003 protocol mismatch

To diagnose this, make sure that the following is the output you get when you ssh in to the host:

USER@HOSTNAME's password: 
Last login: Mon Nov  7 22:54:30 2011 from YOURIP
[USER@HOSTNAME ~]$ 

If you get any newlines or other data you know that extra output is being sent. You could rename your .bashrc/.cshrc/.profile/etc. files to something else so that they won't output extra output. Of course there is still system files that could cause this. In that case, check with your sysadmin that the system files don't output data.

Troubleshooting this problem:

There is a simple way to test if your shell is clean, for an ssh connection: run a command from the ssh connection, rather than starting an interactive shell. The false command will immediately terminate without producing any output, so it is a good test:

bash$ ssh remotehost false
Enter passphrase for key '/home/user/.ssh/my_private_key': 
bash$

If that command line produces any output, one of your startup scripts is to blame:

bash$ ssh remotehost false
Enter passphrase for key '/home/user/.ssh/my_private_key': 
Welcome to RemoteHost!

This system is company property and is provided for authorized use only, 
as set forth in applicable written policies. Unauthorized use is prohibited 
and may be subject to discipline, civil suit and criminal prosecution.
 
Welcome back - You last logged in 16 days ago...
bash$

Another thing to check if you are getting this error is whether rsync is installed and locatable by ssh:

bash$ ssh remotehost "rsync --version"
Enter passphrase for key '/home/user/.ssh/my_private_key': 
rsync  version 3.0.9  protocol version 30
Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
bash$

If rsync is not in the path, you will instead see something like:

bash$ ssh remotehost "rsync --version"
Enter passphrase for key '/home/user/.ssh/my_private_key': 
bash: rsync: command not found
bash$

You can fix this by either installing rsync, or if it is installed but in an unusual location, passing the location to the rsync command line:

rsync -avvvz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" \
    --rsync-path="/usr/local/bin/rsync" \
    remoteuser@remotehost:/remote/dir /this/dir/

This is commonly caused by your shell's login stuff outputting stuff on a non-interactive shell. You can test if this is the case by doing:

ssh username@host "/bin/true" > testfile
ls -l testfile

If testfile is NOT 0 bytes, then the trouble is that your shell is outputting something. Check /etc/profile, .profile, .bashrc, .cshrc, etc. If it is, you can change it to check if your terminal is interactive and only output text by using the following code in a bashrc. Something equivalent exists for other shells as well:

if shopt -q login_shell; then
    [any code that outputs text here]
fi

or alternatively, like this, since the special parameter - contains i when the shell is interactive:

if echo "$-" | grep i > /dev/null; then
    [any code that outputs text here]
fi

However, if the test file is in fact 0 bytes, then your shell is behaving, but it is possible that you just have a very old version of rsync. You can tell the client end (assuming it is the newer end) to not advertise such a high version that the old rysnc server version doesn't recognize it. You can do this using the --protocol= option. In my case, using --protocol=30 did the trick.

If you are still having trouble, try ssh in as the user rsysnc is connecting with and try running rsync --version to see if the shell can find rsync. If you get something that says command not found, then rsync might not be installed on the machine you are connecting to or it might not be in the path. Rsync does have options for specifying the path of the remote end, read the man page(s).

This is a special case from the other replies, but is not very different from then.

To execute a rsync via ssh, you need shell access in ssh to execute the remote rsync command. If your ssh account only allows scp/sftp, you will not be able to start the remove rsync and fails giving this error.

This can be tested with the same command as above

ssh remotehost false

This one should fail and this one should be success

sftp remotehost

This proves that you have a sftp only access.

If you want and have permissions to do so, you can disable the sftp only access for that user, by editing the /etc/ssh/sshd_config and check for match and forcecommand entries.

You can also check this post

I got protocol version mismatch -- is your shell clean? simply because I hadn't installed rsync on the other end yet. sudo yum install rsync solved the problem.

I also observe this error when pulling files from an instance with rsync version 2.5.7 protocol version 26 to version 3.1.1:

protocol version mismatch - is your shell clean?
(see the rsync man page for an explanation)
rsync error: protocol incompatibility (code 2) at compat.c(62)
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.1]

But the remote shell had no login banner. Instead, the solution was to specify an older protocol (ref):

rsync --protocol=29 ...

The prompt will not be shown at all when directly executing a command, and non-interactively so. A simple google turns up first result: http://marc.info/?l=rsync&m=100263876212594&w=2 And since the shell can potentially be invoked, it must not display anything in non-interactive mode — like, when typing just "bash" into an existing prompt, nothing but the new prompt should appear.

This can be caused by an login message on the remote host such as "Your password is going to expire in 6 days" which RSYNC does not expect

In my case, the solution was as easy as removing the line cat /etc/motd line in ~/.bashrc.

Thank you Azendale, upon running your test I found that the file was not 0 bytes. So I went into all of my startup scripts specifically .bashrc and commented out...

#echo -e "Welcome to $LLVER ${USER}"
#echo " "
#date "+%A %d %B %Y, %T"
#free -m | awk 'NR==2{printf "Memory Usage: %s/%sMB (%.2f%%)\n", $3,$2,$3*100/$2 }'
#df -h | awk '$NF=="/"{printf "Disk Usage: %d/%dGB (%s)\n", $3,$2,$5}'
#echo "Support - https://www.linuxliteos.com/forums/ (Right click, Open Link)"
#echo "

I also deleted (well I copied them for backup) all the files in /etc/update-motd before I did this but I cannot definitely say if this had an impact on the solution. All I know is that scp finally works like it is supposed to and I'm happy.