Home / server / Questions / 342827
In Process
larsks
Asked: 2011-12-21 09:30:36 +0800 CST

Benchmarking disk performance on large-memory systems

  • 772

I need to produce some comparative filesystem performance metrics for our virtual hosting environment. Specifically, I need to look at the difference between NFS and iSCSI and at the performance impact of changing the network MTU from 1500 to 9000. This is my first foray into filesystem performance metrics...there's a tremendous amount of information out there (and so many tools!) and I'm not sure exactly where to start.

Complicating things are the fact that these systems each have 48 cores and 256 GB of memory, and I'm not sure exactly how this might impact the setup of my performance measurement tools. For example, the iozone documentation recommends setting the maximum file size to be larger than the local memory of the system...but this suggests a maximum file size of at least 512GB.

I'm hoping someone can provide some suggestions (or a pointer to some documentation) for (a) setting up some reasonable tests, and (b) interpreting the results.

Thanks!

performance filesystems iozone

2 Answers

  1. You need to consider the common usage scenario you are benchmarking for. You seem to be doing benchmarks not to decide whether one drive is better than the other, but rather deciding, what technology and what setup you should use. And hat I mean is, when you want your results to help you choose what's best for your use-case, you should replicate the production enviroment as closely as possible.

    So, are you gonna have a lot of that memory available for caches and buffers? If so, make the expected amount of free memory free in the benchmarks as well. If not, try to make it as close as possible.

    You can limit amount of memory available to Linux using mem= kernel parameter - so eg. passing mem=2G will make linux use just 2GB of memory.

    Another way (a bit better for getting real-world results) is to write a simple app, that will allocate the memory and fill it with something.

    And the best way is to setup the box as if it was in production, make it use the memory it will use normaly and best yet, do the things you want it to do (if you can do that without causing IO).

    With server set up like that, you can run iozone, fio, or other benchmarks to get some numbers.

    • 2

  2. The blog below discusses disk benchmarking numerous times. Search its archives. The author is my friend and trust what he writes.

    http://scalability.org/

    • 0