I have a third party application running on Windows Server 2003 SP2 that is getting an "Insufficient Server Resources" error. It just started exhibiting this behavior for no apparent reason. I ran ProcMon to see where it is happening. In each case, it happens to be reading a file on another file server. ProcMon shows the name of the file and the error "Insufficient Server Resources" at the same time as the application pops up it's own error message box that says essentially the same thing.
The ProcMon stack trace seems to indicate that the error emanates from fltmgr.sys (the Microsoft File System Filter Manager). The stack trace also contains calls into rdbss.sys and ntkrnlpa.exe. Neither the system event log nor the application event log show any errors.
I did find this hotfix -- but it seems to address a blue-screen crash which we're not seeing. Any ideas?
Additional Info: The other server (the file server) is also Windows 2003 SP2. The third party application is Concordance -- which I'm automating via COM. One critical point that I left out is that the application runs fine accessing this and other files on this server for a period and then generates this error. If I dismiss the dialog box, the driving application can continue to invoke Concordance (which in turn reads these files) for a while and the pattern repeats. It runs for a while and bombs out with the insufficient server resource error. But once I dismiss the dialog, it is as if the server no longer has a problem until, of course, it recurs. I've monitored memory and handles with Process Explorer during the period before the error pops up and nothing obvious is being depleted.
The only thing that springs to mind that might be a generic issue rather than one specific to ThirdPartyApplication(tm) is that if the machine it is reading the file from does not have sufficient client access licenses. With server editions it depends how you have the licensing configured and what licenses you have (for XP there is a fixed limit of 10 concurrent connections if I remember rightly). I have seen connections get "hung" in some cases on XP, such that no new file sharing connections can be opened until XP is restarted, though I have never seen this on a server edition, so the old faithful solution of restarting Windows may be a quick-fix work-around until you can find a more permanent solution.
If it isn't a licensing issue then you should probably contact the creators of that third party application as your first resort: they may be aware of a possible issue and already know how to fix or work around it.
ps. for questions such as this, it is helpful to specify the exact Windows version+variant that is installed on both machines, and what the third party application in question actually is - then if by chance someone else has hit the exact same problem previously you might get a quicker (more accurate) answer.
Is there an error code (0x0000000) logged in the event viewer or provided in the error box? I've ran across this problem a few times before, first was caused by an older version of Symantec enterprise (like 9.0) and the second time it was due to bad ram.
I've also seen some CRM applications that run off the network as well as quickbooks throw this error when the following is true;
Security permission issues (app unable to access the file)
Server 2003 getting in the way and blocking files from an untrusted server, usually occurs with remote executed exe files though
Antivirus autoprotect scanning the file while the app is trying to open it causing an i/o lock
simply bad error handling in the application
Can you copy/edit/view the file on the other server without issue outside of the application?