Home / server / Questions / 517775
Accepted
HTF
Asked: 2013-06-23 05:10:34 +0800 CST

GlusterFS direct I/O mode

  • 772

Could you please help me to understand 'direct-io-mode' in GlusterFS.

I'm getting better read results with direct I/O mode disabled - is it mean that in disabled mode it's writing/reading from cache - is it a system cache or GlusterFS cache? What's the recommended method? I'm using 4 nodes with distributed-replicated volume for web serving:

Write: dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
echo 3 > /proc/sys/vm/drop_caches
Read: dd if=tempfile of=/dev/null bs=1M count=1024
Read-Read: dd if=tempfile of=/dev/null bs=1M count=1024

Write           Read            Re-Read
67.5 MB/s       94.9 MB/s       94.6 MB/s   direct-io-mode=enabled
65.8 MB/s       230 MB/s        226 MB/s    direct-io-mode=disabled
glusterfs

2 Answers

  1. Best Answer

    after a long search i've found this article. it's not glusterfs specific, but it may help to understand better: in short, direct i/o mode disable the file buffer implemented by the os for that mount point.

    after reading it i've come to this conclusion:

    Enabling or disabling may give you better transfer rate depending on which application is using your glusterfs volume: if you are using applications or databases that handle by themselves caches you are going to have some boost enabling it ( because of the missing overhead of a second buffer ). if you are just doing file accesses probably disabling it will increase your performances because of less network access due to local file cache.

    • 6

  2. You enable direct-io-mode by"mount -t glusterfs XXX:/testvol -o direct-io-mode=enable mountpoint"??If so, the file was not cached in GlusterFS client kernel,in other words,the cache is system cache.But,the file was cached in GlusterFS server kernel.

    • -1