Home / server / Questions / 351352
Accepted
sequenceGeek
Asked: 2012-01-19 16:34:37 +0800 CST

Limit Hard Drive Write Speed by User in Linux

  • 772

Background:

I use a cluster with my coworkers and if anyone writes too many big files at the same time I can't even move the cursor in vim (the horror!). I'm not sure if it is more related to the number of files open or the "bandwith" of hard drive being used but it definitely only happens when a big write operation is taking place (including downloading big files). I know you can limit hard drive space by quotas per user, but I've never heard of being able to limit data transfer speeds.

Question

Is it possible to limit the amount of data written per second by a user in linux?

performance linux drive write

3 Answers

  1. Best Answer

    Use cgroups together with CFQ I/O scheduler (the default for many distributions). CFQ is aware of cgroups and can give any user, group or process only x% of disk I/O time.

    So, if you have one cgroup called sequenceGeek, having 90% of maximum resources, you can then have another cgroup called coworkers, having 10% of resources. Or something similarly fair.

    man cgrules.conf, man cgset and man cgconfig.conf should get you started. This at least in Fedora 16, did not have time to check other distributions right now if they have similar config file names. Probably they do.

    EDIT: Oh, just noticed from your comments you are using NFS. Then your network might be saturated and you need to use QoS with tc and iptables. Google for Advances Linux Routing How-To, it has ready-made examples of throttling the traffic and creating priority classes. Or if possible, do the throttling at your firewall/router/load balancer.

    Could also be an old kernel version having issues with NFS or bad NFS mount options.

    • 3

  2. If you cannot move your cursor, my guess is that your CPU is waiting on IO for a very high amount of time. You can check this by running top, and looking at the amount listed at %wa near the top.

    The blkio controller of the aforementioned cgroups is a method of controlling the amount of IO someone or some group can do. I'm in a bit of a hurry right now, but if you read the kernel docs (/usr/share/doc/kernel-<version>/Documentation/cgroups), I'm sure you'll manage. You will need a fairly recent kernel though, but I think anything above 2.6.32 (the EL6 kernel) will probably do.

    You need to configure the blkio.throttle.write_bps_device file in your cgroups filesystem. With that, together with the /etc/cgconfig.conf and /etc/cgrules.conf, you can limit any user or group in the amount of bandwidth on your storage.

    • 1

  3. I'm not entirely sure, but after reading a bit from man limits.conf, and assuming you're root, I'd suggest to add hard limits for other people's accounts on data (maximum data size), priority and nice. Something like setting their limits lower than yours, and then setting whichever app you're running's priority to higher than they can. Of course, you should research a bit more and test before making any changes, as it might seriously interfere with their user experience.

    Good luck.

    • 0