Access CentOS 5 server over SSH with RSA public/private key from OSX

I'm trying to configure SSH access to my CentOS server using an RSA key rather than with a password but it refuses to accept the key. Here's what I've done so far:

On client machine (OSX):

ssh-keygen -b 4096 -C "[email protected]" -t rsa
// no passphrase

On Server (CentOS 5):

cat id_rsa.pub >> ~/.ssh/authorized_keys2
chmod 700 ~/.ssh
chmod 600 ~/.ssh/*
chmod go-w ~/

// Added to /etc/ssh/sshd_config:
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile  .ssh/authorized_keys2

// and finally restart sshd
/sbin/service sshd restart

Back on the client trying to connect:

ssh [email protected] -v

debug1: Found key in /Users/jwood/.ssh/known_hosts:16
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/jwood/.ssh/identity
debug1: Offering public key: /Users/jwood/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Offering public key: /Users/jwood/.ssh/id_dsa
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: password
[email protected]'s password:

As you can see, it tries to use the public key and fails falling back to password. What could be wrong with the config to cause this?

Thanks, J