This is a canonical question about how Unix operating systems report memory usage.
Similar Questions:
I have production server that is running Debian 6.0.6 Squeeze
#uname -a
Linux debsrv 2.6.32-5-xen-amd64 #1 SMP Sun Sep 23 13:49:30 UTC 2012 x86_64 GNU/Linux
Every day cron executes backup script as root:
#crontab -e
0 5 * * * /root/sites_backup.sh > /dev/null 2>&1
#nano /root/sites_backup.sh
#!/bin/bash
str=`date +%Y-%m-%d-%H-%M-%S`
tar pzcf /home/backups/sites/mysite-$str.tar.gz /var/sites/mysite/public_html/www
mysqldump -u mysite -pmypass mysite | gzip -9 > /home/backups/sites/mysite-$str.sql.gz
cd /home/backups/sites/
sha512sum mysite-$str* > /home/backups/sites/mysite-$str.tar.gz.DIGESTS
cd ~
Everything works perfectly, but I notice that Munin's memory graph shows increase of cache and buffers after backup.
Then I just download backup files and delete them. After deletion Munin's memory graph returns cache and buffers to the state that was before backup.
Here's Munin graph:
Externally hosted image was a dead link.
You are experiencing the Linux Ate My Ram issue.
Don't Panic.
This is NOT a problem.
Your system is Working As Designed.
The problem is not your OS -- the problem is your understanding of what "free" memory is.
Unix systems use memory for more than just running programs. Memory might be used for:
What follows is a brief (and largely incomplete) tour of how modern Unix systems report RAM usage.
What is Free memory (the OS definition)?
When a Unix system is reporting RAM as Free it means "I am not using this RAM for anything".
Free RAM is effectively worthless - It is not making your system faster, it's just sitting there being "free" in case something needs it. That something could be any of the three other items I mentioned above.
What are Cache and Buffer memory?
Cache and Buffer memory are RAM the operating system is using to make your system faster.
This memory is not needed for running programs right now, so your OS is using it to hold data that it needs frequently -- for example the C library (needed by pretty much every program you run) is almost always held in
cachememory, so the system doesn't have to go to the disk to find the instructions it needs to print "Hello World" on the screen.It's actually a lot more complicated than that -- there's shared memory, wired memory, etc. -- but for our purposes this simple explanation is adequate.
What is Active memory?
Active memory is part of we understand as "used" memory -- RAM that applications are using for whatever it is they do -- sorting spreadsheets, serving web pages, editing graphics, etc.
"Active" memory has been "active" recently -- the program claiming it has made use of its contents (reading or writing), and it's not considered a good candidate for swapping out.
What is Inactive memory?
Like Active memory, Inactive memory is RAM that applications are using for whatever it is they do. The difference is this memory hasn't been accessed in a while, so if push comes to shove the OS thinks it can be swapped out to disk and (with a little luck) the program claiming it won't ask for it again so it will never notice.
What is "Used" memory (the HUMAN definition)
What you and I think of as "Used" memory is, essentially, the sum of Active and Inactive memory. All the RAM currently claimed by applications for their use.
As long as you have more installed RAM than the sum of Active and Inactive memory (plus a nice safety margin of say 512-1024MB on top) you're in an OK place: Your OS probably won't be hitting swap and killing performance.
What is "Free" memory (the HUMAN definition)?
What you and I think of as "free" memory is the memory available to run programs.
This is slightly more complicated than just the "Free" figure your OS reports. When a program asks for RAM the operating system will try to get that RAM in the least disruptive way it can:
(This is about where performance usually goes to the dogs: Every time a program gets its turn on the CPU its swapped-out bits need to be brought back into RAM, which means some other program's Active memory has to be swapped out -- the high turnover in swap is called thrashing)
malloc()will fail. This is the POSIX-conforming behavior - the operating system will tell the program asking for RAM that it can't satisfy the request.The program can either ask for less RAM, or if it can't make do with a smaller chunk of memory it can clean up and exit. (If the program is badly written it will simply crash.)
In case you can't tell by my description here and my answer on the linked question, I think this is a terrible way to deal with the problem.
Why does Free RAM go up when you delete files?
In the example from the question here you noticed that it's possible to "Free" RAM by deleting the backup file -- the explanation for that is pretty simple: Since nothing is using that file (no open file handles) and it's no longer accessible from the filesystem (unlinked) the OS knows nobody will ever access that data again, and it purges the data from the filesystem cache.
This makes the OS report more Free memory, but has no impact on system performance.
This is the same "problem" as from Server refuses to use swap partition and a few other similar questions on this site. ( High Memory Usage on Linux Server, Memory Usage in LINUX, Web Server Running Low in Memory, etc.)
Pay attention to the fact that the memory consumption is from cache. This means it's keeping a file in memory. Cached memory is "free" memory. Instead of leaving the block of memory empty, your OS is keeping recently read files in that space. If an application does need that memory, it will be used by the application. Until then, it stands a chance to save your from having to read a file from the disk again if it is frequently referenced.
According to this graph, your effective memory consumption hasn't changed at all for the entire duration of the graph.
Something else to check if the above fails:
From: https://stackoverflow.com/a/5467207