I know how to set up passwordless / passphraseless ssh. But my understanding is that setting it up that way means that if someone get ahold of the id_dsa file, they can then log in from any machine.
Is it possible to set up, on server X, that Y is an "authorized key" only when it's coming from client Z?
You may restrict access by IP for every authorized_key. Just add the following line into
authorized_keys
on X something like:from="Z_IP" Y_id_rsa.pub
In this case server X will be accessible via SSH using Y's key only when accessing from Z's IP-address.
Also, you may add additional parameters. Here is examples: https://debian-administration.org/article/685/Restricting_SSH_logins_to_particular_IP_addresses