I am testing an app on Tomcat with one hour of crazy traffic from 200 users.
Everything is fine during the test.
But immediately AFTER the test (no more traffic), the number of threads rises inexplicably.
After a short but dangerous peak, it starts to decrease.
I computed an histogram to show the problem:
The yellow line (number of socketRead threads in Tomcat's thread pool) looks stable at first, but then jumps at the end of traffic (black vertical line).
QUESTION: Have you already seen this behavior, any idea what could be causing it?
Sometimes in production this phenomenon makes Tomcat reach maxThreads
, causing all sorts of other problems.
Cluster of 5 nodes, Tomcat 6.0.26 / JK / Alfresco
Details about the other kinds of threads, if needed:
- Red: socket read threads outside thread pool
- Blue: threads waiting in thread pool
- Green:
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill
- Purple: Application-side DAO threads (the app is Alfresco, used as a WebDAV server)
PS: If you have any interest in generating such graphs, I can make my tool open source, let me know.
This might be related to thread pooling.
http://webspherehelp.blogspot.co.uk/2010/01/tomcat-thread-pool-dont-shrink-as-you.html
might help.
YMMV