Home / server / Questions / 408915
Accepted
Wayne Conrad
Asked: 2012-07-19 09:14:17 +0800 CST

Linux Opteron system appears to be UMA but should be NUMA

  • 772

According to numactl, this dual CPU Opteron box is UMA rather than the expected NUMA:

$ numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
node 0 size: 65534 MB
node 0 free: 381 MB
node distances:
node   0 
  0:  10

I think it should be NUMA because there are four 4-core CPUs*. As I understand it, each CPU has its own memory channel; if a CPU needs to access memory in a non-local node, it must use hypertransport, which costs more time than accessing memory in the local node. AMD explains it here.

The motherboard has 16 RAM slots, 8 for each CPU. All 16 are populated with 4GB each for a total of 64GB. Some more particulars:

Why is numactl reporting that this box is UMA?

*There are two CPU dies per package, so the motherboard only has two CPU sockets.

linux opteron numa

1 Answers

  1. Best Answer

    The BIOS is hiding the NUMA reality behind the node interleaving setting. Setting that to Disabled will give you a true NUMA system as far as the OS is concerned. Not many systems really use NUMA effectively, which is why motherboard manufacturers default to make everything equally slow rather than, let the OS figure out what needs fast and slow access.

    • 1