Home / server / Questions / 125473
Accepted
Philippe
Asked: 2010-03-24 10:37:47 +0800 CST

How to manage processes-to-CPU cores affinities?

  • 772

I use a distributed user-space filesystem (GlusterFS) and I would like to be sure GlusterFS processes will always have the computing power they need.

Each execution node of my grid have 2 CPU, with 4 cores per CPU and 2 threads per core (16 "processors" are seen by Linux).

My goal is to guarantee that GlusterFS processes have enough processing power to be reliable, responsive and fast. (There is no marketing here, just the dreams of a sysadmin ;-)

I consider two main points :

  • GlusterFS processes
  • I/O for data access (on local disks, or remote disks)

I thought about binding GlusterFS instances on a specific "processor".

I would like to be sure that :

  • No grid job will impact the kernel and the GlusterFS instances
  • Researchers jobs won't be affected by system processes (I'd like to reserve a pool of cores to job execution and be sure that no system process will use these CPUs)

But what about I/O ? As we handle a huge amount of data (several terabytes), we'll have a lot of interuptions.

How can I distribute these operations on my processors ? What are the "best practices" ?

Thanks for your comments!

central-processing-unit linux glusterfs affinity grid

2 Answers

  1. Best Answer

    Pinning a userspace to a specific processor sometimes can make it perform more consistently; but not the kernel. (check taskset)

    if it were possible, you would be turning the userspace/kernel switchtme (already a measurable factor in an OS performance) into an inter-processor communication / synchronization issue. Many orders of magnitude worse.

    edit: now that you've removed the 'pin the kernel' idea, it's a lot more reasonable. Yes, you can use taskset to start all grid processes and leave one or two CPUs free for GlusterFS. For an analogous example, in Xen systems is considered a 'best practice' to reserve one CPU for the Dom0, which handles all I/O.

    • 1

  2. That's why I don't want to use FUSE filesystems in production... I'm using PVFS2 instead.

    • 1