I'm encountering a weird problem where a Fedora Linux VPS server reports 100% CPU, and effectively becomes unusable, but I don't know why because the high load prevents me from SSHing into it to see what's wrong.
How do I prioritize or configure SSH so that I'm still able to connect even if some process is consuming all other CPU?
If this is for real a CPU / Load related problem you can (re)nice the sshd process on your server to give it a higher priority. To make that persistent you can add the nice inside the sshd init script.
Create a script which logs processes:
Setup a cron job which runs the script every minute:
Next time your system goes unresponsive, you'll have logging of what was doing it.
I was running the code that was hogging all the resources through Docker, so I just limited the amount of CPU that could be consumed by the container by starting it with
Because I have 8 cores, this means I should always have half a core for processing SSH sessions into the actual server running the container (unless some other process takes up those resources too).
For good measure I also limited the container's memory usage with
https://docs.docker.com/config/containers/resource_constraints/#cpu
If you don't mind my saying, I think you're asking the wrong question. You really want to find out what is using so may resources to the point of your not being able to log in. I have come across this problem numerous times and the solution is to track what is going on the server constantly. I came up with this solution which is as light as possible, reducing its chances of being stopped by a resource hog:
http://linuxtech.ie/wordpress/2012/09/05/finding-a-severe-resource-hog-on-your-server/
I hope that helps