It's unclear which known_hosts are troubling you - on the server or on the client. I used the instructions at http://blog.neutrino.es/2011/unlocking-a-luks-encrypted-root-partition-remotely-via-ssh/ on Debian. I am going to paraphrase that page and add some tweak I found useful. Contemporary Debian dropbox packages create all the needed keys on the server at installation time, but the referenced tutorial is so old, that it mentions cases when the keys need to be manually created. YMMV.

A. On the server. As you correctly mentioned, dropbear and OpenSSH keys differ, but are supposedly inter-convertible. The tutorial says that you could create on the server the host keys with:

dropbearkey -t rsa -f /etc/initramfs-tools/etc/dropbear/dropbear_rsa_host_key

In fact, it is a bit harder than that. For me on Debian Jessie, the steps were:

1. Create a OpenSSH key.

   ssh-keygen -t rsa -b 4096 -m PEM -f bootkey_rsa

2. Copy it on the server, that is, the computer where you are more likely to have dropbearconvert. There, convert it to dropbear format:

   /usr/lib/dropbear/dropbearconvert openssh dropbear bootkey_rsa bootkey_dropbear_rsa

3. Stil on the server, login to root, and extract from Dropbear key the public part to the file where boot dropbear looks for it:

   dropbearkey -y -f bootkey_dropbear_rsa | grep "^ssh-rsa" > /etc/dropbear-initramfs/authorized_keys

4. Update initramfs:

   update-initramfs -u -k all

5. Cleanup: remove bootkey_dropbear_rsa and bootkey_rsa from server.

Note that the location where the known_hosts is expected by update-initramfs, given above as /etc/dropbear-initramfs/authorized_keys , is flexible, and probably changes from distro to distro. To be sure you put it in the right place, read the source file from /usr/share/initramfs-tools/hooks/dropbear.

The other needed key is created with:

dropbearkey -t rsa -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear

B. On the client you need both kinds of keys, so there is another known_hosts. In the referenced tutorial, the command to connect is:

ssh -o "UserKnownHostsFile=~/.ssh/known_hosts.initramfs" \
-i "~/id_rsa.initramfs" [email protected] \
"echo -ne \"MyS3cr3tK3y\" >/lib/cryptsetup/passfifo"

One of the keys, id_rsa.initramfs, is the file /etc/initramfs-tools/etc/dropbear/dropbear_rsa_host_key or bootkey_rsa from the server section.

On the client known_hosts, there may be a conflict between the key you already have for the OpenSSH server and the dropbear server you just installed. I temporarily removed from known_hosts the keys for the regular OpenSSH service, connected using the command above (leaving out the -o parameter), got prompted if I trust the host-key, said yes, and got it appended to my known hosts. From there on, you need to move that last line to its own known_hosts (~/.ssh/known_hosts.initramfs in the example above).

Here's how you convert dropbear's key. The startup scripts are a little wonky for me too...

Boot your SERVER by connecting and unlocking the encrypted partition via your CLIENT

Install mandatory packages (on SERVER)

apt-get install dropbear initramfs-tools busybox

Append your desired public keys into the SERVER's authorized_keys file

Just copy and paste your public key(s) into /etc/dropbear-initramfs/authorized_keys on SERVER

Create the unlock script

Create following script in /etc/initramfs-tools/hooks/crypt_unlock.sh

#!/bin/sh

PREREQ="dropbear"

prereqs() {
  echo "$PREREQ"
}

case "$1" in
  prereqs)
    prereqs
    exit 0
  ;;
esac

. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions

if [ "${DROPBEAR}" != "n" ] && [ -r "/etc/crypttab" ] ; then
cat > "${DESTDIR}/bin/unlock" << EOF
#!/bin/sh
if PATH=/lib/unlock:/bin:/sbin /scripts/local-top/cryptroot; then
kill \`ps | grep cryptroot | grep -v "grep" | awk '{print \$1}'\`
# following line kill the remote shell right after the passphrase has
# been entered.
kill -9 \`ps | grep "\-sh" | grep -v "grep" | awk '{print \$1}'\`
exit 0
fi
exit 1
EOF

  chmod 755 "${DESTDIR}/bin/unlock"

  mkdir -p "${DESTDIR}/lib/unlock"
cat > "${DESTDIR}/lib/unlock/plymouth" << EOF
#!/bin/sh
[ "\$1" == "--ping" ] && exit 1
/bin/plymouth "\$@"
EOF

  chmod 755 "${DESTDIR}/lib/unlock/plymouth"

  echo To unlock root-partition run "unlock" >> ${DESTDIR}/etc/motd

fi

Make it executable:

chmod +x /etc/initramfs-tools/hooks/crypt_unlock.sh

Create a static IP (or skip this step to use DHCP)

Edit /etc/initramfs-tools/initramfs.conf to add (or change) the line:

IP=192.168.1.254::192.168.1.1:255.255.255.0::eth0:off

format [host ip]::[gateway ip]:[netmask]:[hostname]:[device]:[autoconf]

([hostname] can be omitted)

Update initialramfs

update-initramfs -u

Disable the dropbear service on boot so openssh is used after partition is decrypted

sudo update-rc.d dropbear disable

Testing

1. Reboot your server
2. Connect to your server via ssh [email protected] [-i ~/.ssh/id_rsa]

Alright everyone, so just in case you didn't find a solution and for anyone else who's pulling their hair out following online tutorials to get this up and running...

All the tutorials I found online say/imply that the file

authorized_keys

goes in

/etc/initramfs-tools/root/.ssh/

when in fact it's supposed to go in

/etc/dropbear-initramfs/

+1 @ceremcem for also showing this.