How bad is it to have multiple devices with the same SSH server keys?

Rather than storing host-specific data such as ssh host keys on the SD card or other read-only media, you can store this in NVRAM, which is what it's for on an embedded system. You'll need to do some custom scripting to store and retrieve the keys at boot time, but the scripts will be exactly the same for every device.

The impact of shipping the same key pair with all your devices is directly related to the security of the clients connecting to them, as it means that there is no way (from an SSH client) to uniquely identify the device it may be connecting to. Should your key pair be leaked, it could be used for MITM attacks.

On the other hand, regenerating the keys on each boot, will also trigger an alert on the clients.

For reference, from man ssh(1):

ssh automatically maintains and checks a database containing identification for all hosts it has ever been used with. Host keys are stored in ~/.ssh/known_hosts in the user's home directory. Additionally, the file /etc/ssh/ssh_known_hosts is automatically checked for known hosts. Any new hosts are automatically added to the user's file. If a host's identification ever changes, ssh warns about this and disables password authentication to prevent server spoofing or man-in-the-middle attacks, which could otherwise be used to circumvent the encryption. The StrictHostKeyChecking option can be used to control logins to machines whose host key is not known or has changed.

It sounds like in the first option, the SSH keys would be available on the SD card. So any user could take the card and read them out. So basically your private keys have become (mostly) public.

This will allow man-in-the-middle attacks, like the following:

1. A user sets up a SSH server with the private keys obtained from your device and gives that IP address to your technician.
2. Your technician inputs the root password over the SSH connection.
3. The user now knows the root password that is valid for all your devices.

However, you shouldn't be using root passwords in the first place, use ssh keys for authentication instead. Then the impact of shared server keys is pretty small if you only log on from a LAN.

SSH also provides forward secrecy, so an attacker has to be able to setup a false server to benefit from the keys; passively sniffing the traffic will not allow decrypting it.

I read this in horror! I who have done multiple machines in the same cluster with the same ssh host key would never dare do this. Do not under any circumstances allow machines with different sets of administrators to share ssh host keys. That way lies madness and screaming horror when you get posted for your lack of security.

Behold I tell you the truth, he who compromises one device compromises all of them. Once obtained one, expect bad people to jump from one to another at will and the security rolled back as though it were thin paper.

Since you mention that the SSH access is not used by the end user/customer you may want to turn off SSH access by default and only enable it temporarily when the device is put into "debug" mode.

You can then either ship all the devices with the same key, assuming that you have protected the "debug" mode so it can't be triggered remotely by someone trying to hack the device.

Or you have a new key generated when the device goes into "debug" mode - so you don't have to waste boot-time generating the keys each time the device is powered up.

Here's a sample attack scenario based on the constraints you have:

If your devices are, say a Raspberry Pi's for example. If I walk up and yank the SD card from one, I can stick the SD card in my own computer, find the sshd key and copy that to everywhere I want. Maybe I grab my own raspberry pi and a USB ethernet card. Now I can stick that in between a target device and wherever they're going and monitor for ssh connections. When I see that the target device is trying to make an ssh connection, I do this:

(target) <---> (my evil sshd <--> decrypted traffic <--> ssh) <---> (real server)
                                       |
                                       V
                                    log file

Oh, what's that? Your password is "I like cats"? Boy, this is an interesting email you sent to your wife. I bet it would be even more interesting if she read this email that you sent to your next door neighbors wife.

The possibilities are endless. And the target would never know, because the sshd key is identical to the one found on the real server. Depending on the physical security of the facilities that are receiving your device, this could be incredibly trivial. Don't do this.

Instead, do what you already propose but fix it. Before you write your image run something like this:

ssh-keygen -f some-new-server
cp some-new-server /path/to/the/mounted/image/sshd/key
cp some-new-server.pub /path/to/the/mounted/image/sshd/key.pub

And now every server has a new key. Because you really, really don't want to be distributing copies of a key. I mean honestly it's at least as bad as snapping a picture of your house keys and uploading them to the Internet with your home address.