While debugging a problem with slow hg cloning from Windows clients, I found that hg clone performance could be significantly improved (10x faster) on EC2 2012 virtual instances by reducing the default os page size from 8gb to 1gb to force utilisation of (abundant) physical memory.
For my tests, I run this clone command:
hg clone -U --traceback --verbose --profile --time https://hg.mozilla.org/mozilla-central
On Windows Server 2012:
with default settings (OS manages the page size):
time: real 653.507 secs (user 115.891+0.000 sys 486.922+0.000)with manual settings (page file max size 1024mb):
time: real 179.646 secs (user 56.594+0.000 sys 58.234+0.000)
On Windows Server 2008:
with manual settings (page file max size 1024mb):
time: real 1273.845 secs (user 98.905+0.000 sys 592.585+0.000)
I have not been able to replicate the performance increase from 2012 on 2008 where the same command is always slow regardless of the OS page size.
I'm pretty confident that we can rule out things like software, network and hardware considerations because I've done my best to replicate the environments as accurately as possible (virtual hardware is identical: c3.2xlarge, 16gb ram, 80gb ssd, same network subnets, etc). I think (because of a comment from Matt Mackall on the hg bug I raised, on the significance of the user/sys timing ratios), that we're really looking at an OS level io or memory configuration problem.
Any ideas what else I should look at, besides the page file size?
I can't think of a reason that the page file should make a difference other than simply a size difference (and if you are doing non-wim based cloning of course you will have perf issues with the page file). Eliminating or reducing the page file size doesn't influence how much ram is used. See https://serverfault.com/a/75027/3528
All that being said for default windows setups could EC2 be doing some sort of memory balloning?