As a webdeveloper I have, like many, my own VPS. However, I am a webdeveloper, not a system manager. For me it's quite challenging to install a VPS and to keep it up to date and secure. It takes me days, if not weeks, to get a server up and running. I also hardly dare to update my server since I'm afraid I screw things up.
So my question to you is what resources can you recommend me that won't only tell me how to install a LAMP server, but also explains how to keep everything up to date and secure?
edit: Installing everything isn't the main problem, the main problem is configuring everything such that is is secure and that it works. Most tutorials just tell me how to install everything, but usually that's just not enough.
In my specific case I have a CentOS VPS for which I need LAMP, ftp and email. I also host multiple domains on one VPS.
There are two key parts to a server environment: Building it and Maintaining it.
The building part is "easy" -- Howtos will get you there (though the results might not be pretty).
The maintaining part is the hard part: Applying patches (even using modern package management tools) is a bit of a black art: You need to be ready and able to troubleshoot when things break, and you need the skills and knowledge to fix the brokenness (either by actually fixing it or by rolling back your change until you can figure out why it broke).
The best advice I can give you: Get a sysadmin (even a part-time one) to do quarterly assessments and patches. If you want to take this role over yourself then work with them until you feel comfortable.
The second best advice I can give you is build a test environment (any sysadmin you hire should insist on this) and do it yourself:
Note that your test environment need not be extravagant: A VirtualBox VM can often do the job.
A few other bits of free advice:
Have a test environment.
If you break a test environment you can always restore from a backup and try again.
If you break the production environment the users get angry.
(this leads directly to #2...)
Backups. Get you some (if you don't already have 'em).
Perform regular backups. Perform regular restore tests (to be sure your backups are usable).
Be confident that even if you make a mistake and destroy the production environment you can fix it.
Do not fear patching: Patch early, and Patch often.
Problems tend to arise when environments are set up and left alone without regular updates and testing: The chance of something breaking is roughly proportional to the number of changes (updated packages) involved, and regular updates work to keep that number small.
I practice (and strongly recomment) quarterly maintenance, with emergency fixes between windows as needed.
Your best bet is to roll with a proven distro that has a great package manager. Fedora, Cent, Redhat, OpenSuSe and Debian/Ubuntu all have these. All of these I have listed contain the packages for deploying a standard LAMP setup, and once you have the whole thing setup, you can start a system job, AKA cron job, to check and apply updates on a specified interval.