I am not able to understand why would kernel issue this oom killer when I see enough memory is available:
I say enough memory is available after looking at
Normal
DMA
Normal free lines
This is an embedded nand flash based device with 256 MB RAM
Kernel : 2.6.31
myshellscript invoked oom-killer: gfp_mask=0xd0, order=2, oomkilladj=0
Backtrace:
[<c0106494>] (dump_backtrace+0x0/0x110) from [<c03641a0>] (dump_stack+0x18/0x1c)
r6:000000d0 r5:c9040c60 r4:00000002 r3:c0448690
[<c0364188>] (dump_stack+0x0/0x1c) from [<c015a314>] (oom_kill_process.clone.11+0x60/0x1b4)
[<c015a2b4>] (oom_kill_process.clone.11+0x0/0x1b4) from [<c015a738>] (__out_of_memory+0x154/0x178)
r8:c21e86e0 r7:001fb000 r6:00000002 r5:000000d0 r4:c9b6e000
[<c015a5e4>] (__out_of_memory+0x0/0x178) from [<c015a980>] (out_of_memory+0x68/0xa0)
[<c015a918>] (out_of_memory+0x0/0xa0) from [<c015d230>] (__alloc_pages_nodemask+0x42c/0x520)
r5:00000002 r4:000000d0
[<c015ce04>] (__alloc_pages_nodemask+0x0/0x520) from [<c015d388>] (__get_free_pages+0x18/0x44)
[<c015d370>] (__get_free_pages+0x0/0x44) from [<c0109418>] (get_pgd_slow+0x1c/0xe0)
[<c01093fc>] (get_pgd_slow+0x0/0xe0) from [<c0129ab0>] (mm_init.clone.43+0xb0/0xf0)
r7:c90858c0 r6:00000000 r5:c90858c0 r4:ce1a6680
[<c0129a00>] (mm_init.clone.43+0x0/0xf0) from [<c0129c40>] (mm_alloc+0x34/0x44)
r6:0009230c r5:c90858c0 r4:ce1a6680 r3:00000000
[<c0129c0c>] (mm_alloc+0x0/0x44) from [<c0180f70>] (bprm_mm_init+0x14/0x148)
r4:c5154000 r3:cd472564
[<c0180f5c>] (bprm_mm_init+0x0/0x148) from [<c01812d0>] (do_execve+0xa8/0x254)
[<c0181228>] (do_execve+0x0/0x254) from [<c0106000>] (sys_execve+0x3c/0x5c)
[<c0105fc4>] (sys_execve+0x0/0x5c) from [<c0102e80>] (ret_fast_syscall+0x0/0x2c)
r7:0000000b r6:0009230c r5:0009237c r4:000922fc
Mem-info:
DMA per-cpu:
CPU 0: hi: 18, btch: 3 usd: 0
Normal per-cpu:
CPU 0: hi: 42, btch: 7 usd: 0
Active_anon:28162 active_file:16 inactive_anon:18037
inactive_file:13 unevictable:0 dirty:0 writeback:0 unstable:0
free:9998 slab:2447 mapped:164 pagetables:701 bounce:0
DMA free:17128kB min:1560kB low:1948kB high:2340kB active_anon:51068kB inactive_anon:10320kB active_file:24kB inactive_file:0kB unevictable:0kB present:97536kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 158 158
Normal free:22864kB min:2600kB low:3248kB high:3900kB active_anon:61580kB inactive_anon:61828kB active_file:40kB inactive_file:52kB unevictable:0kB present:162560kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 2358*4kB 912*8kB 25*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 17128kB
Normal: 4266*4kB 657*8kB 32*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 22864kB
26591 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 0kB
Total swap = 0kB
65536 pages of RAM
10471 free pages
3967 reserved pages
2447 slab pages
892 shared page count
389 shared pages
620 mapped shared page count
177 mapped shared pages
0 pages swap cached
2481 dma reserved pages
19892 total user pages
20512 RSS sum by tasks
20512 RSS sum by page stats
164 user cache pages
26427 kernel cache pages
OK, so lets go through each bit.
Active memory is regions of memory that get thrown to the top of the LRU stack (basically get called a lot).
Inactive memory is stuff thats not being used a lot and is a swap nomination should memory need to be swapped.
Free is genuinely free memory About 40Mb. What gives?
The clue is in these lines:
This stuff denotes memory fragmentation. This basically shows you how much memory is contiguously available. And here is your problem.
Of free normal memory, you have no more than 32kb of available contiguous memory. Your memory is horribly fragmented. This means if any application needs to allocate more than 32k of space, there is no memory -- so oom killer arrives to kick something out to give memory to do this.
So, what can you do.
The clue to that is this:
Oh dear! No swap! So, memory thats committed -- just stays there. Newer kernels these days actually 'defragment' memory to make region of memory contiguous, older ones dont do it.
You had 70Mb of memory that could has been swapped! Plus this wouldn't have occurred all in one go but gradually so would not have been a hit for you. But no swap, so no luck. You also have little memory for pagecache which is also bad and slow for your system. This potentially could of given a lot more free contiguous space too which would of been nice for you.
My advice to you. Get yourself 768Mb of swap. Honestly, you really do your kernel a disservice by not enabling it.
Swap is really important for releasing unused memory (a quarter of it in your case) and also would have avoided the nasty fragmentation problems you've experienced as memory could have been swapped out and released more contiguous space. And even if it did get swapped back in, it could have been put back into a region of memory which give you larger contiguous gaps.