How to reduce memory usage on a Unix webserver

Thanks everyone for your answers! Following your suggestions I've been able to reduce my memory usage to 195M SWAP and 108M RSS, without touching my code (I'll definitely optimize it soon, but this was supposed to be a solution to get me out of trouble fast).

Here's the list of things I did:

Got rid of the wildcard used in VirtualHost entries. Instead of *:80 and *:443, I used the real IP of my server.

Changed Apache's prefork MPM. These are the values I ended up using:

StartServers           1
MinSpareServers        1 
MaxSpareServers        5 
ServerLimit           16
MaxClients            16
MaxRequestsPerChild    0
ListenBacklog        100

These are by no means magical numbers. I've spent some time trying different values and combination, and then testing them against the real usage of my server and everyone should do the same in their enviroment. For the record, my server receives close to 2M pvs/month, serving both dynamic pages and assets at a regular rate - no digg effect. The intention, again, was to reduce the memory footprint, not to improve performance or HA.

Reference:

• http://httpd.apache.org/docs/2.0/misc/perf-tuning.html
• http://httpd.apache.org/docs/2.2/mod/mpm_common.html

Tunned down Apache's KeepAlive. By setting KeepAliveTimeout to a lower value (2 in my case) I can expect less server processes just waiting on connections with idle clients that may not request any more content.

Reference: http://httpd.apache.org/docs/2.0/mod/core.html#keepalivetimeout

Removed MySQL's unused module. I added skip-innodb to MySQL's my.cnf. Massive memory consumption reduction.

There are also some remarkable good suggestions that I couldn't personally do:

• Remove PHP modules you do not need. The PHP on my server has most mods already compiled, I'll probably try my own minimal PHP on other VPS.
• Switch to nginx with php-fastcgi. That's another good advice that I'll be trying soon, but right now I can't risk the downtime.

I found this article on low-memory configurations for Apache and MySQL

To be very useful in laying out the configuration changes needed for low memory configurations. I tweaked them for my own situation but they should give you the tools needed to find the best fit for your environment

You're going to need to limit how many apache server processes are running, and being as close to the limit as you are, you're not going to be able to handle very much peaky traffic. Having a web server that's maxed out under normal usage is generally a Bad Idea (tm), as web traffic is nice and low for the most part until you get slashdotted or digged or fireballed or whatever.

The main issues are the number of apache processes that are running at any one point -- assuming prefork here, since I've only deployed PHP applications and PHP is not threadsafe. I don't have experience dimensioning the worker MPM. There are some items that are in shared memory, and some items that are in each process's memory.

You can reduce the total memory footprint by leaving out shared modules that you don't need. Basically, Apache comes configured from most hosts to do just about everything under the sun. If you're not using mod_userdir, then comment it out of your apache config. Just be careful how much you remove, because some of the things you may need or their dependencies are not intuitive! All of the modules should be documented on the apache.org website. The per process footprint is harder to get smaller; most apache configurations these days only come with the four essential modules compiled in. Beyond those four modules, most of the memory usage comes from either leaks or application RAM that's not garbage collected effectively, which is why you might want to set the number of requests handled by each process low.

You really want to keep your memory usage in RAM itself and not go into swap. Swap means I/O. I/O is slow and will drive your CPU usage through the roof as processes block while waiting for something to get shuffled out of swap.

For apache, remove the modules you do not use, as they just use additional memory. For MySQL, remove innodb/bbdb if you do not use them, and remove PHP modules you do not need.

Next you should configure apache MaxClients based on the size of one process and the amount of memory you want to give apache. Same goes for max connections on MySQL (I recommend the excellent MySQL Tuning Primer Script.

If you have control over your PHP app, make sure it doesn't use too much memory (e.g. in variables, especially static ones).

If you want to go further, you can replace apache+mod_php with nginx+fcgi setup, which will probably result in further memory reduction.

One last thing - you really don't want to swap on a web server. Just a little, to remove the unneeded stuff, but swapping regularily on a web server will result in a non-responsive web site.

Since you already met your target, here's few extra:

Since you removed all unnecessery php modules, you could to the same for apache. By default (depending on the installation thou) apache loads quite a bunch of extra modules and most of them are not really required for normal day to day usage. For example, there are bunch of authentication modules that are always loaded. deflate is typically not required unless you are trying to limit your bandwidth usage. Autoindex & status goes are also questionable.

And another one is that you can limit the amount of memory available for php in php.ini: memory_limit=xxxM

You could of course limit the number of processes apache can fork, however this would only work as a sudo-hard limit on your memory usage. From a lower level point of view you can use plimit to restrict the resources available to a process. Apply this to the parent and child processes inherit I believe.

However from a web server configuration point of view it can come down to how your code runs really! But bear in mind small things like using .htaccess files use more resources than using central apache configuration files (as they are read every time a request comes in, leading to greater overhead), something which is signification in large web sites.

One thing that might help memory growth over time is setting the httpd keepalive lower, but I'd test that carefully in case your application needs longer lived processes.

The server is running Solaris on a shared CPU, and I'm using a Apache + MySQL + PHP stack.

I don't have experience with Solaris, but the best thing you can do is to not use Apache/mod_php.

• Switch to nginx with php-fastcgi.
• Recompile php to use the minimal amount of plugins.
• Get rid of unnecessary processes like ntpd (use ntpdate), ftp (use scp) etc...