lima's questions

As far as I can tell, the load average on my server (Ubuntu Linux 8.04.1) is way too high, and in practice I see it slows down or stops serving during peak hours.

It's a fairly stock LAMP powering a single site (image hosting) that obviously servers a lot of content (images) from disk, but they need to go through PHP to be served. Aside from general advice to use a cache/proxy approach for this, I'm lost at why it's apparently using less than half of the available resources (4GB RAM, it's a Linode 4096).

I'm quite a noob at Linux, so please ask for whatever might be useful. This is a portion of htop (MySQL shows 98.9% CPU usage but that was marginal, it uses 0.*% almost all the time):

  1  [|||||||||||||||||||||||||||||||||||         69.0%]     Tasks: 355 total, 6 running

  2  [|||||||||||||||||||||||                     44.8%]     Load average: 18.32 15.02 11.58 
  3  [||||||||||||||||||||||||||||||||||||        71.9%]     Uptime: 01:10:22
  4  [|||||||||||||||||||||||||||||               57.9%]
  Mem[||||||||||||||||||||||||||||||||||||||2190/4096MB]
  Swp[|                                         0/127MB]

  PID USER     PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command                                                  
 2345 mysql     18   0  177M 72640  5140 S 98.9  1.7  7:47.58 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql 
 9350 www-data  16   0 48940 24304  4376 R 13.7  0.6  0:01.05 /usr/sbin/apache2 -k start
 9301 mysql     15   0  177M 72640  5140 S 10.0  1.7  0:00.17 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql 
 9186 mysql     17   0  177M 72640  5140 S 10.0  1.7  0:00.22 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql 
 9150 www-data  15   0 58400 33900  4476 S  8.1  0.8  0:02.03 /usr/sbin/apache2 -k start
 9077 mysql     15   0  177M 72640  5140 S  8.1  1.7  0:00.39 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql 
 9270 mysql     15   0  177M 72640  5140 S  7.5  1.7  0:00.12 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql 
 9037 mysql     16   0  177M 72640  5140 S  7.5  1.7  0:00.45 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql 
 9333 www-data  15   0 35724 11260  4560 S  6.2  0.3  0:03.88 /usr/sbin/apache2 -k start

This is the current apache2.conf, though I've tried lots of combinations and asked here in the past:

Timeout 90
KeepAlive On
MaxKeepAliveRequests 150
KeepAliveTimeout 3
<IfModule mpm_prefork_module>
    StartServers          1
    MinSpareServers       1
    MaxSpareServers      5
    MaxClients          275
    ServerLimit          275
    MaxRequestsPerChild   1250
</IfModule>

UPDATE: As asked, this is a portion of top:

top - 15:07:31 up  1:46,  2 users,  load average: 12.83, 10.64, 10.14
Tasks: 223 total,  17 running, 206 sleeping,   0 stopped,   0 zombie
Cpu(s): 84.3%us,  8.8%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  1.0%si,  5.9%st
Mem:   4194528k total,  3555696k used,   638832k free,    27748k buffers
Swap:   131064k total,      588k used,   130476k free,  1458672k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                   
 2345 mysql     17   0  180m  76m 5140 S   55  1.9  13:09.79 mysqld                                                    
12479 www-data  18   0 73224  47m 4552 S   48  1.2   0:03.74 apache2                                                   
12294 www-data  17   0 71788  46m 4472 R   39  1.1   0:05.78 apache2                                                   
12382 www-data  17   0 73744  48m 4460 R   33  1.2   0:03.19 apache2                                                   

UPDATE: As suggested (by Christopher Karel, thanks), here are the active processes (output from ps -efl | cut -c3- | egrep -v "^S"). On average, it shows 1-5 apache2 processes. Does this make sense given my current apache2.conf and load average?

T root     12519 12508  0  75   0 -   612 finish 15:07 pts/1    00:00:00 top
R www-data 18677  2774  1  76   0 - 17130 -      16:23 ?        00:00:04 /usr/sbin/apache2 -k start
R www-data 18965  2774  2  76   0 - 13397 -      16:26 ?        00:00:04 /usr/sbin/apache2 -k start
R www-data 19047  2774  2  76   0 - 11613 -      16:28 ?        00:00:00 /usr/sbin/apache2 -k start
R www-data 19088  2774 55  76   0 - 10482 -      16:29 ?        00:00:00 /usr/sbin/apache2 -k start
R www-data 19091  2774  0  81   0 -  8579 -      16:29 ?        00:00:00 /usr/sbin/apache2 -k start
R www-data 19092  2774  0  81   0 -  8355 -      16:29 ?        00:00:00 /usr/sbin/apache2 -k start
R www-data 19093  2774  0  82   0 -  8322 -      16:29 ?        00:00:00 /usr/sbin/apache2 -k start
R root     19094 18557  0  77   0 -   593 -      16:29 pts/2    00:00:00 ps -efl
R root     19095 18557  0  78   0 -   729 -      16:29 pts/2    00:00:00 -bash
R root     19096 18557  0  78   0 -   729 -      16:29 pts/2    00:00:00 -bash

I'm trying to migrate all of my domains from DNS Made Easy to Linodes DNS Manager. The easiest way I figured was the "Import a Zone" option on Linode, which works with AXFR. I then did the following (guied by this instructions):

On dnsmadeeasy.com:

1. Activated "Zone Transfer Setting (AXFR ACL)" in the "Advanced DNS Settings"
2. Went to "Zone Transfer Configuration" and added a "Zone Transfer Set (ACL)" named "Linode" and in "IPs to allow transfer" I added the ones required by Linode (based on this message: "Your nameserver must allow zone transfers (AXFR) from 69.164.199.240, 69.164.199.241, and 69.164.199.242"), each on its own line.
3. I went to my domains list and applied the corresponding AXFR ACL.
4. Waited for them to change from "Updating" status to "Active".

On manager.linode.com:

5. Tried to import any domain zone, using axfr1.dnsmadeeasy.com and axfr2.dnsmadeeasy.com as the "Remote Nameserver", depending on the name server my domain was delegated to.
6. Then it happens: Linode gives me a "AXFR Failed or the zone was empty!" error every time, only once I managed to get an "already listed domain" kind of error, because I was trying to import, well, and already (manually) imported domain. However, I could't even get the same error again.

What is that I'm missing?

I want to broadcast live video from a playlist created by me, but the actual video files (.flv's) are stored in some external file hosting which I can't admin, but it's currently allowing the use of flash players for playing its contents (progressively).

Which is the best solution for this? Can I use my current web server (Debian w/Apache) or should I use some specific server software? I've read some names such as Red5 and Wowza but I can't tell if those are solutions to my actual problem or how should I use them. Also, costs should be as low as possible for this project.

I'm currently using a Joyent Accelerator to host my webapps, and it's working fine, however I need to reduce costs so I'm downgrading my current plan and that imposes some new memory limits (256M rss, 512M swap). I wasn't too far over them yesterday, but after restarting Apache several times today, I'm now 411M rss, 721M swap (prstat -Z -s cpu).

Searching in Server Fault only gives me lots of ways and specific tools to monitor the server, but no advice on how to reduce/optimize it's memory usage. I've also seen this question, but I don't think it's good for this particular (or may I say generic?) situation.

The server is running Solaris on a shared CPU, and I'm using a Apache + MySQL + PHP stack.

I'm interested in knowing the steps one can take to troubleshot this and solve the issues. However, I'm also running out of time to lower my memory foot print and downgrade the plan before the current ends, so anything that can make magic and save the day is welcome as well :)