Home / server / Questions / 477886
In Process
Blair Zajac
Asked: 2013-02-12 23:46:42 +0800 CST

Why does host swap out VMs when there are 16GB of buffer cache and swappiness = 0? [duplicate]

  • 772

I have a desktop running Ubuntu Quantal using OpenStack Folsom on an Intel i5 with 32 GB RAM and 2 GB swap. I'm running 7 VMs each sized like a EC2 m1.small, so 1.7 GB RAM each. I'm using KVM.

As I get up to running 5 or 6 concurrently, the host starts to swap them out:

top - 23:45:42 up 3 days,  1:51, 10 users,  load average: 0.37, 0.75, 1.15
Tasks: 418 total,   2 running, 413 sleeping,   3 stopped,   0 zombie
%Cpu(s):  8.8 us,  2.1 sy,  0.0 ni, 88.8 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
KiB Mem:  32864580 total, 32586956 used,   277624 free,   574236 buffers
KiB Swap:  1998844 total,  1113352 used,   885492 free, 16498252 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND           
24652 libvirt-  20   0 4169m 1.7g 7756 S   3.6  5.4   4:49.37 kvm               
25233 libvirt-  20   0 4450m 1.6g 7756 S   1.2  5.2   4:35.12 kvm               
25589 libvirt-  20   0 4163m 1.6g 7756 S   2.4  5.1   4:40.31 kvm               
 6562 root      39  19 2935m 658m 7460 S   0.0  2.1 100:05.62 java              
28393 libvirt-  20   0 4149m 624m 7756 S   0.0  1.9   2:25.01 kvm               
28106 libvirt-  20   0 4170m 617m 7756 S   0.0  1.9   2:18.17 kvm               
26519 libvirt-  20   0 4167m 590m 7756 S   0.0  1.8   2:22.16 kvm               
29399 libvirt-  20   0 4159m 589m 7756 S   0.0  1.8   2:19.94 kvm               


$ free -m
             total       used       free     shared    buffers     cached
Mem:         32094      31868        225          0        559      16175
-/+ buffers/cache:      15134      16959
Swap:         1951       1087        864


# /tmp/swap-used.sh |grep kvm
PID=944 - Swap used: 0 - (kvm-irqfd-clean )
PID=24652 - Swap used: 102468 - (kvm )
PID=25233 - Swap used: 108644 - (kvm )
PID=25589 - Swap used: 155768 - (kvm )
PID=26519 - Swap used: 192216 - (kvm )
PID=28106 - Swap used: 150796 - (kvm )
PID=28393 - Swap used: 208488 - (kvm )
PID=29399 - Swap used: 187388 - (kvm )

I've already tried setting swapiness to 20, then 10 and finally to 0, none of which has made a difference:

# cat /proc/sys/vm/swappiness 
0

I haven't rebooted the host since changing it from 60 to 0 (do I need to reboot)? I've also turned off swap completely with /sbin/swapoff -a; /bin/swapon -a. Immediately after re-enabling swap, I see this:

$ vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0  11384 247572 652736 16539748   11   10   291   228   13   12  7  3 87  3
 0  0  12968 234360 652756 16554432    0  317  8576  1240 3508 5470 17  2 75  5
 1  0  17068 243512 652768 16559508    0  820  9448  1216 3687 4845 20  2 77  2
 1  0  20040 233300 652772 16576152    0  594 12262   677 4436 5063 29  2 68  1
 1  0  22764 219156 652792 16594448    6  546 11962   727 3870 4559 28  1 68  2
 3  0  40832 229384 652776 16602440    0 3614 58404  4176 2051 6231 21  2 66 10
 1  0  52420 232236 652784 16613320    0 2318 42174  2512 1819 4026 15  2 77  6

I've got 15 GB of free memory that can be used without having the processes swapped out.

kvm-virtualization

1 Answers

  1. I've got 15 GB of free memory that can be used without having the processes swapped out.

    No, you don't.

               total       used       free     shared    buffers     cached
Mem:       32094      31868        225          0        559      16175

    You have 225MB of memory free.

    KiB Swap: 1998844 total, 1113352 used, 885492 free, 16498252 cached

    Look at all that wonderful cache the system has made, in part by getting unused junk out of precious RAM. Why are you trying to make it waste precious RAM?

    If you have 1GB of data in RAM that hasn't been used in hours, which makes more sense:

    1. You swap that out to disk when you're not busy and have 1GB more cache to use.

    2. You keep it in memory. You have 1GB less cache to use, and if that memory is ever needed for something else, you'll have to swap it out while you are busy.

    But if you think you know more about memory management than the people who wrote your operating system's memory management logic, go ahead and keep trying to find that magic "do everything faster and better" switch.

    • 8