I am having issues with a Tomcat service that crashes (stops responding to requests) and needs to be sigkill'd and restarted every few days. My graphing shows that HEAP memory usage gets to about 3gb during the peaks and this is the most obvious symptom of the issue. From what I have read, adding in Xmx max memory limit for jvm is what dictates how much HEAP can be allocated. Xmx is set to over 10gb.
I have also played around with the argument MaxPermSize to no avail. I think this setting defines memory allocation for some application memory usage, but afaik it is different to HEAP memory itself.
Now I am wondering if there is an alternate setting I can pass in that defines the /actual/ HEAP memory limit? Otherwise, any tips on investigating the problem would be good, as I can't positively say it is memory limits being hit and the error logs do not show much.
I would first recommend that you do a crash course in Java and Tomcat. I can give you all sort of tips but in the end, you need to understand what you are doing to correctly administer a server.
That said, you can't tell the JVM that it can use more memory than your machine has available. If your Xmx is already above 10Gb, maybe you can increase it (if your server has more RAM available). But chances are, it's probably already set to it's maximum possibility by someone who set that up before you.
As to what is happening, your logs will tell you what is running out of memory - HEAP or PermGen. You need to understand what each of those are and you can read up on this here: Java HotSpot VM Options.
But frankly, looks like you have a memory leak going on in the application - unless you are running at the default PermGen value and you are running out of PermGen at which point it may not be a dramatic problem and just increasing PermGen will be fine.
PermGen is where code that is generated at runtime reside for example. With today's frameworks, it is typical that this runs out (Hibernate, Spring, AOP, etc). So a typical setting is 512Mb or 1Gb. Rarely need more than that in my experience, but it may vary depending on the applications.