Home / server / Questions / 296686
Accepted
kvz
Asked: 2011-08-03 06:34:58 +0800 CST

Can I throttle tar without ionice?

  • 772

My tar command 

tar --remove-files -cvif 2011-08-02_14-05-09.tar 2011-08-02_14-05-09

Writes 80MB/s. That's beyond the capacity we can spare because MySQL starts queuing up writes and eventually we get monitoring alerts cause systems won't wait forever on MySQL and time out.

Q: Can I throttle tar to only do 20MB/s?

Didn't find anything in the manpage and we already tried with the lowest ionice class: /usr/bin/ionice -c3 tar <...> but that doesn't impact the MBs written/s and MySQL still freezes up.

Maybe ionice is ignored because we're on a /dev/md0 software RAID device?

amazon-ec2 rate-limiting software-raid ionice tar

2 Answers

  1. Best Answer

    You can use ratepipe.

    I downloaded and tried it out and it seems to DTRT. It should work for you:

    tar --remove-files -cvif - 2011-08-02_14-05-09 | ratepipe -r 20 > 2011-08-02_14-05-09.tar
    • 5

  2. You could probably write the tar to stdout (using -f -) and pipe it into something that throttles. I'll see if I can find something quickly.

    Since tar will not buffer much, it should wait on the reads.

    Edit: MikeB beat me to it: ratepipe will do.

    • 1