Home / server / Questions / 1008523
Accepted
Amos
Asked: 2020-03-27 06:02:16 +0800 CST

ubuntu 18.04: clamav running, tomcat dying

  • 772

Ubuntu 18.04. 2Gb RAM + 512mb swap.

When running clamav, it consumes 800+mb of memory because it loads all the signatures into the memory. Because of that, I set it to run every day at 3am instead of ongoing.

So far, tomcat and clamav got along very well. Last night, at 3am, tomcat service was shut down when clamav started working.

[4643256.375812] OOM killed process 8145 (clamscan) total-vm:1149268kB, anon-rss:969476kB, file-rss:4kB
[7667218.452649] OOM killed process 8865 (java) total-vm:4568248kB, anon-rss:1067312kB, file-rss:0kB

Mar 26 03:00:31 user systemd[1]: tomcat.service: Main process exited, code=killed, status=9/KILL
Mar 26 03:00:31 user systemd[1]: tomcat.service: Failed with result 'signal'.
Mar 26 03:17:08 user systemd[1]: Reloading The Apache HTTP Server.
Mar 26 03:17:08 user systemd[1]: Reloaded The Apache HTTP Server.

I know upgrading is an immediate answer but until then, my questions are:

  1. Is there a way to run clamav without it consuming 800+mb?

  2. Is there a way to automatically restart tomcat if something like that happens again?

  3. Did Java really took 4,568,248kB = 4.5gb or am I missing something?

Edit inside tomcat.service file I have the following:

Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC"
Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom -Djava.awt.headless=true"
service clamav ubuntu-18.04 tomcat9

1 Answers

  1. Best Answer

    Java did not use 4.5 GiB of physical memory, that figure refers to virtual memory. You have about 128 TiB of virtual memory address space (cf. this answer) and it is used to access physical memory, but also files as if they were in memory. Most of the allocated virtual space does not use any physical memory and contains only 0s or points to data present on hard drives.

    What you are interested in is the anon-rss number: it is physical memory used by the process, mostly for private use, and is not just a cache/copy of a file (unlike file-rss). The kernel can not just delete this memory and you get an OOM.

    The answer by Janne to your previous question points you to a solution of your problem:

    1. You should tune Tomcat's memory settings, which can be done by adding options to the variable JAVA_OPTS in /etc/default/tomcat9. The most commonly used are -Xms (minimum heap size) and -Xmx (maximum heap size). E.g. you can use:

      JAVA_OPTS="-Djava.awt.headless=true -Xms256m -Xmx1024m"

      The JVM will use between 256 MiB and 1024 MiB of heap space (plus other kinds of Java memory) and might give some of it back, when it is not used.

    2. Your amount of swap is quite small, especially if you don't have a lot of RAM. Extend it to around 4 GiB (cf. this article) so that the kernel can swap some memory instead of killing Tomcat.

    3. To configure an automatic restart of Tomcat after failure, edit the tomcat.service file:

      systemctl edit --full tomcat.service

      and add to the [Service] section, e.g.:

      RestartSec=5s
Restart=on-failure

      You can find the possible values for Restart in the SystemD documentation.

    • 1