I am trying to identify if behaviour that I am observing is correct or if WildFly is leaking file handle descriptors.
During our standard performance testing after upgrading from WildFly 11 to 14 we ran into an issue regarding too many open files. After digging into things a bit more it looks like it is actually the number of pipes that WildFly has open that is increasing.
To help reproduce the problem I have created a simple JSF 2.2 application that contains a large image (100mb to simplify testing). I am retrieving the image using the standard JSF resource URL:
/contextroot/javax.faces.resource/css/images/big-image.png.xhtml
And have also tried adding omnifaces and using the unmapped resource handler URL:
/contextroot/javax.faces.resource/css/images/big-image.png
Adding Omnifaces did not change the behaviour I am seeing, and I have only included it as we first thought that it might have been a contributing factor.
Behaviour I am seeing:
WildFly starts and jstack reports that it has two threads matching default task-* which is the default value for task-core-threads
If I send in 5 concurrent requests for my large image, 3 new default task-* threads are spawned to serve the requests. 3 new Linux pipes will also be created.
If I stop my requests and wait for 2 minutes (the default value for task-keepalive) 3 of the threads will be removed. The pipes remain open.
Periodically - I believe about every 4.5 minutes some kind of clean up occurs and the pipes that were left over from the step above are removed.
However... If one of the original 2 worker threads is removed, e.g. task-1, task-3 and task-4 are removed, leaving task-2 and task-5, the pipe associated with task-1 is never cleaned up.
Over time these pipes add up and as far as I can tell they are never removed. Is this a leak somewhere, and if so where? JSF? WildFly? Undertow?
Things I have tried:
WildFly 14, 17 and 18
With and without Omnifaces (2.7 and 3.3)
Changing the min and max threads to be the same - this prevents handles building up, but I'd rather not go down this route