Home / server / Questions / 577425
In Process
rinogo
Asked: 2014-02-22 10:42:14 +0800 CST

How to determine what memory was freed

  • 772

We have a somewhat complex LAMP stack running a variety of websites using Litespeed instead of Apache on Centos 5.

Once everything is up and running, the memory usage of the system is fairly consistent - most of the memory is taken up by MySQL's caches, with the remainder of it for Litespeed (webserver and lsphp5) and OS-related operations.

Every once in a while, we experience a huge reduction in memory consumed by processes (around 3:55 am and 9:25 am in the screenshot below).

enter image description here

Everything continues working just fine - in fact, performance increases. I've checked the OOM killer and can't see any processes being killed, and MySQL's caches appear to remain intact (e.g. no deviation in MyISAM or InnoDB hit rate).

So, my question is, "How can we determine what memory is being freed (e.g. which processes it belonged to)?"

Note that it's ideal if we can do this analysis retroactively (e.g. on historical data). However, I'd also be happy to find a solution we can put in place to track future memory usage.

linux

1 Answers

  1. You could try using smem. It should be fairly simple to compile and use if not prepackaged for your OS.

    You can collect per-process statistics with the smemcap utility and process them elsewhere, or directly report to your monitoring solution by parsing the data.

    There is also plain old SNMP. The HOST-RESOURCES-MIB has a table that could be helpful.

    hrSWRunPerfMem OBJECT-TYPE
   SYNTAX KBytes
   ACCESS read-only
   STATUS mandatory
   DESCRIPTION
           "The total amount of real system memory allocated
           to this process."
   ::= { hrSWRunPerfEntry 2 }

    The hard thing to do with the data supplied would be correlation, though.

    Check this example using zabbix.

    • 1