Home / server / Questions / 157130
In Process
Khushil
Asked: 2010-07-03 12:06:20 +0800 CST

24 Core Server Performance

  • 772

I have a quad hexacore system based on the Opteron 8345 chips - that's 24 cores in all with 48GB of ram. This box was meant to replace the current six Intel core box which my current LAMP stack runs on. I am however getting even worse performance than from the six core. I'm running:

  • Centos 5
  • MySQL 5.0
  • PHP 5.2.13
  • Apache 2.2.9

It's almost as if the box is too powerful? Has anyone had any experience with large SMP boxes like this? 

[root@gb-sanguine ~]# uname -a
Linux gb-sanguine.bountyonline.local 2.6.18-194.3.1.el5 #1 SMP Thu May 13 13:08:30 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

[root@gb-sanguine ~]# php -v
PHP 5.2.13 (cli) (built: Jun  2 2010 16:29:01)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies

[root@gb-sanguine ~]# httpd -l
Compiled in modules:
  core.c
  prefork.c
  http_core.c
  mod_so.c

[root@gb-sanguine ~]# httpd -v
Server version: Apache/2.2.15 (Unix)
Server built:   Jun  2 2010 11:15:28

[root@gb-sanguine ~]# mysql -V
mysql  Ver 14.12 Distrib 5.0.77, for redhat-linux-gnu (x86_64) using readline 5.1

Please see http://pastebin.com/raw.php?i=KSWxqmJe for dmesg output.

Many thanks in advance for your help folks.

linux centos lamp smp

6 Answers

  1. cat /proc/cpuinfo

    are all CPUs actually recognized? I don't know about CentOS, but, some kernels turn on SMP but limit the number of cores to 8 as a default.

    Does top show processes on each cpu? shift-O, j

    Is your application not multithreaded?

    -- It does recognize all 24 cores, so, that theory is effectively negated. The only other simple thing I can see is that your cpu powersave governor MIGHT be running the cpus at 800mhz rather than the 2.6 ghz.

    What does your application do? typical webapp? or is there a part that is not able to be multithreaded? Is mysql configured to use more cores (thread_concurrency)? When you watch the machine running, what is slower? network connection time? CPU throughput? disk access? sysctl.conf settings migrated over from the other box? any tweaks/modifications to mysql/apache/php configs that didn't get moved over?

    can you paste: cat /proc/mtrr

    • 4

  2. Cache memory issues. You need to be careful about how those cores are socketed and how the tasks spawn across them. I had a server go from fast 6-core to slow 8-core in a 12-core (2-socket) system. So, just having lots of cores and RAM does not mean that you should just blindly make use of them all.

    I would suggest using your 24-core system to run a virtualisation host like KVM. Then, it may be easier to partition your workload across a number of smaller VMs. You can pin specific cores to specific VMs. So, if your 24-core system is a 4-socketed 6-core or a 6-socketed 4-core, you can have fine-grained control over the performance. There will be a difference in performance between the two.

    Cheers.

    • 2

  3. Put a newer MySQL on it - 5.1 at least, or better, Percona's XtraDB. Make sure you're using InnoDB. MySQL 5.0 scales negatively (i.e. performance decreases) with more than 8 cores in my experiences.

    • 1

  4. Depending on the workload and kernel version, increasing the number of CPU may very well decrease performance (semaphore implementation of the Big Kernel Lock with heavy POSIX file locking usage comes to mind, this is something that has been observed in RHEL/CentOS 5).

    Have you checked whether you get better performance by reducing the number of CPUs? (maxcpus=)

    Otherwise, you should profile your application/system to track down the bottleneck. If this is CPU-bound, OProfile should help. SystemTap also comes to mind.

    • 1

  5. Did you see CPU performance bottlenecks in the past? You be seeing a throughput issue now, instead of a CPU bottleneck.

    With 6 sockets, the data needs to get from RAM to each CPU, and depending on how the cache is managed, you may be doing a lot more Cache misses (L1, L2, L3 cache in this context, that is usually on chip, or at least on-die in multi core instances) and be doing memory reads instead of L2 cache reads.

    Sometimes throwing resources at a problem does not help, if the problem was not a lack of resources.

    However +1 for kick ass hardware! Wow! 24 cores!

    • 0

  6. Actually, as a simple LAMP stack server, it is a bit too powerful.

    You can look into virtualize the box and it will replace not only what it is intended to replace and maybe some more servers in your farm.

    • 0