I'm running a traffic intense site plenty of dynamic content, mostly user-generated.
The server is a dedicated one and has a total of 4 Intel(R) Xeon(R) CPU X3210 @ 2.13GHz proccesors. I need to know to optimal values for ServerLimit and MaxClients apache's directives, considering that the server has 4GB of RAM and the MySQL database runs on a separate server. The panel is DirectAdmin with CentOS.
Below are my current directives, but during peak hours with over 5k users, an important lag is noticed - and it's not entirey MySQL's fault, because pages seem to be generated fast (I implemented a page generation time counter), but there is a long connection delay until the page starts responding and is sent to the browser.
<IfModule prefork.c>
StartServers 800
MinSpareServers 20
MaxSpareServers 60
ServerLimit 900
MaxClients 900
MaxRequestsPerChild 2000
</IfModule>
Timeout 90
KeepAlive On
KeepAliveTimeout 5
I should mention that monitoring the server using the top command, CPU usage never goes beyond 20% ~ 30% on peak hour. The MySQL server also has a 30~50% usage at that time, and I'm constantly working on fixing slow queries, but that's a different issue. I know it's not a DB bottleneck because static pages also take long to load on peak hours.
Any tips to optimize these values will be greatly appreciated, thanks.
Your MaxClients is WAY WAY WAY too high. What is the current size of your apache process? Multiply that x 900. Is that greater than 4GB? If so, the machine is likely going into swap. I usually start with MaxClients = 2x vCPUs in the box (grep -c processor /proc/cpuinfo). Which in this case would be about 8. Then make sure that MaxClients x apache process size isn't over 4GB.
You can up your MaxClients from there, depending on the type of connection that your clients have. (Dial-up users need to be spoonfed, etc.) But make sure you never get yourself into a swapping situation.
Then set your Min, Max, and Start servers to MaxClients. There's no real need to have them differ in a dedicated server environment.
Then do some testing with ab (as goose notes.)
I recommend playing around with apache's benchmark (ab) tool. You can play around with the values to match them to your traffic flow and see what type of responses you're getting as far as average load time & the like. At that point you can play around with the settings that you're talking about to try and optimize them. You should be able with ab to get a handle on the optimial performance for each performance tweak.
It wouldn't really be prudent for me to speak about your settings, however you also need to take your RAM into account as well because it sounds like you're eating up a lot of RAM with those settings. Although that's just speculation without any data. htop gives you a good visual read of your resources.
Also your load average could say a lot. I doubt that your usage is much higher than your total amount of cores from 20-30% cpu, but it's another indicator of how hard your server is actually working.
You need to obtain the average size of your apache process. With this number and the total size of your RAM you can calculate the MaxClients directive. Remember this: "A webserver should never ever have to swap" (Apache Performance Tuning)
Monitoring with top or htop is ok but you need a better view of all stats of your servers (cpu, ram, disk i/o, apache requests, mysql slow queries, etc ...) with some monitoring tool like ganglia or munin to find possible bottlenecks.