After nearly a decade (yikes) of hosting my personal websites and applications with a decent shared web hosting provider (the one in pittsburgh) I've decided its time to venture out on my own and try a virtual private server.
The one thing that's been holding me back is I know my shared web host has a number of custom built security/instruction-detection systems setup to deal with problems common to hosting clients who want to run PHP applications. I'm loath to give up that peace of mind, but the extend of my admin skills are doing cause/effect style exploration with configurations, and parsing through logs to see "oh, that's how they f---ed us".
That context set, if I was going to setup an account with a linux based VPS for Apache web hosting
What steps would I take to secure the machine?
What additional monitoring packages could/should I install?
Step 1: Update your software as patches come out. If patches don't come out regularly for the projects you use, it's time to find alternatives that do.
Step 2: Offsite logging. Attackers may compromise a machine with PHP, but if you export the logs over the network to a stronger server, it's that much harder to cover tracks.
Step 3: Secure SSH logins. Run SSH on a nonstandard port. Require strong passwords, or even restrict logins to SSH keys. Install fail2ban or some other brute force detection tool to prevent being overwhelmed. Disable root access if your distro is silly enough to allow it.
Step 4: Update your software. It bears repeating. PHP is notoriously bad, and shared hosting can make it a massive pita to update, ingraining bad behavior. Projects like Debian package stuff you might find in PEAR and update. Subscribe to mailing lists and schedule time daily or weekly to address patching.
Step 5: Backups. When you do get hacked, it's safest to restore from a known good system. Incremental backups can help you with this.
There's tons of Intrusion Detection packages, like snort, aide, and acidbase. There's also pentesting tools like nessus/openvas.
I also like directing a simple uptime tool at VPS's, to document outages in case a refund is in order.
This is what I typically do:
These are in addition to items that others have suggested. Many more advanced tools exist, such as snort - I recommend that you look into them. However, this checklist should be good to get you going on a VPS.
jldugger offers many great ideas - I'd add on to that that you'll probably want to look into apache's mod_security as well as suexec. Mod_security comes with a bunch of pre-made filters that inspect http calls to your server and reject them if it sees something fishy. Suexec lets you run php/perl/etc scripts as the user that owns them as opposed to them all being run under the www-data user.