Home / server / Questions / 234783
In Process
kagali-san
Asked: 2011-02-13 08:57:01 +0800 CST

Linux's best filesystem to work with 10000's of files without overloading the system I/O

  • 772

It is known that certain AMD64 Linuxes are subject of being unresponsive under heavy disk I/O (see Gentoo forums: AMD64 system slow/unresponsive during disk access (Part 2)), unfortunately have such one.

I want to put /var/tmp/portage and /usr/portage trees to a separate partition, but what FS to choose for it?

Requirements:

* for journaling, performance is preffered over safe data read/write operations
* optimized to read/write 10000 of small files

Candidates:

* ext2 without any journaling
* BtrFS

In Phoronix tests, BtrFS had demonstrated a good random access performance (fat better than XFS thereby it may be less CPU-aggressive). However, unpacking operation seems to be faster with XFS there, but it was tested that unpacking kernel tree to XFS makes my system to react slower for 51% disregard of any renice'd processes and/or schedulers.

Why no ReiserFS? Google'd this (q: reiserfs ext2 cpu): 1 Apr 2006 ... Surprisingly, the ReiserFS and the XFS used significantly more CPU to remove file tree (86% and 65%) when other FS used about 15% (Ext3 and ... Is it same now?

performance linux filesystems performance-tuning gentoo

3 Answers

  1. Ext4, Btrfs, Reiser4, Reiser3 (with notail option). XFS sucks on metadata intensive operations (like rm'ing lots of files).

    • 2

  2. You don't really need journalling for those directories (since they can easily be restored), so no matter which filesystem you choose, you can disable journalling.

    Back when i used Gentoo, i used ReiserFS 3 for /usr/portage. The reason is simple: it was pretty fast, and it saved a lot of space compared to other filesystems when you are dealing with a huge number of very small files. (using the "notail" option removes this benefit of course) I did notice that over time performance on that filesystem would decrease. This is due to the fact that there are very frequent writes/deletes on /usr/portage and no matter what filesystem you use, after a while fragmentation is gonna make it slower. So sometimes i just took a targz from /usr/portage, reformatted the partition, and unpacked the targz on it again... this gives you a new /usr/portage with no fragmentation.

    I don't know if i would still use ReiserFS now.. i would probably go for ext4 (with journalling disabled) or ext2.

    About /var/tmp/portage... if you have enough ram, you might want to map that on tmpfs. That way, it will use ram as much as possible, but go to swap when the data does not fit in ram.

    • 1

  3. Ext2 is rather slow compared with the other OS, It may also have problems with very large numbers of files. I'd strongly recommend NOT using it - ext4 with journalling off is a different kettle of fish altogether.

    Personally, I'd lean towards Reiserfs for a filesystem with lots of small files - it's specifically optimized for this kind of operation. But I mainly work with webservers - where the access is predominantly read/only (I use ext4 for db). The rate at which it uses CPU is not such a big deal - indeed, higher CPU would be a pretty good indicator that less actual file I/O is required to complete an operation!

    Looking up the original article - the comment you quoted specifically relates to the behaviour for large files - I'm no expert on Gentoo, but I believe this is irrelevant to portage.

    • 0