I'm seeing a weird issue on our web servers where the Apache processes seem to be remaining in limbo with 1 or 2 threads. This is disconcerting for me since I was just thinking I wanted to impose a stricter ServerLimit so that I can more safely up my php memory_limit.
Would anyone know of a nice way to troubleshoot this? Figure out what the hell those processes are doing? What state they are in?
I'm not entirely sure, but I would assume that this is caused by the fact that we never restart the Apache service, and do execute 2-3 reloads a day for various reasons.
As an example, one of the Web Servers is sitting around bored while the other is fairly taxed :
PID PPID RES THREADS
*15608 1 29 MB 1
8880 1 2 MB 3
14989 1 0 MB 1
25029 1 0 MB 1
2115 15608 3 MB 2
13058 15608 361 MB 2
14556 15608 157 MB 27
19962 15608 23 MB 1
21219 15608 387 MB 3
21600 15608 3 MB 3
21828 15608 3 MB 2
30152 15608 3 MB 2
In this example, there's 3 processes which I'm assuming were once the control process (pre reload), which are still running for whatever reason. And of the processes attached to the new control process, a bunch are still up with 2-3 threads each.
From 11 hours later, a lot of those same PIDs are still around with 1 thread. You'd think if MaxRequests had been reached the darned thing would have quit.
memory usage report for apache2
pid count : 11
thread count : 46
shared mem usage : 670 MB
total resident size : 691 MB
total cpu usage : 79 %
total memory usage : 7 %
average mem / pid : 62 MB
process breakdown
PID PPID RES THREADS
*15608 1 29 MB 1
8880 1 2 MB 3
14989 1 0 MB 1
25029 1 0 MB 1
2115 15608 3 MB 2
19962 15608 23 MB 1
21219 15608 387 MB 3
21600 15608 3 MB 3
21828 15608 3 MB 2
29239 15608 238 MB 27
30152 15608 3 MB 2
process tree
|-apache2-+-3*[apache2---{apache2}]
| |-apache2
| |-2*[apache2---2*[{apache2}]]
| `-apache2---26*[{apache2}]
|-apache2---2*[{apache2}]
|-2*[apache2]
First thing to check is whether these have spawned any additional processes (have you got pstree available?). Are the php exec functions enabled?
Other things to look at at whether you've got keepalives enabled and whether maxRequests is set to a sensible value. Are the threads idle or processing stuff?
Is the PHP talking to a database backend? Once the thread of execution goes into the DB client lib, PHP constraints regarding memory usage and time limits are no longer applied until the thread goes back to PHP. Are you seeing slow queries on your database?
Other things worth checking are whether the MTA is configured correctly - e.g. if you are using sendmail with a smart relay and the address you've configured is wrong, then php's mail() function can block for long periods.
Rob Olmos's reply regarding Rasmus Lerdorf's comment was made 6 years ago while there may still be some truth in the statement, it's not likely to apply to the mainstream extensions.
You didn't say what OS you're using - IMHO the benefits of threading on Linux compared to pre-forking are barely detectable.
Certainly if you've exhausted the other avenues and you don't have a compelling reason to stick with the threaded apache, then its worth trying the pre-fork - but do have a look elsewhere first.
Is PHP running via mod_php? The PHP install docs give this:
http://www.php.net/manual/en/install.unix.apache2.php
So I suspect that's where the issue is.