Home / server / Questions / 356507
Accepted
Grodriguez
Asked: 2012-02-04 03:50:19 +0800 CST

"safe" ext4 configuration for systems running unattended

  • 772

I have a system running linux that must run unattended for long periods of time. The system uses industrial CF card for storage. Most of the time there are no writes to flash, although every now and then some configuration data/settings can be modified. The system must be resistant to power failures.

I would like to use ext4 for this. What is the best way to configure ext4 for this kind of setup? Bearing in mind that:

  • Performance is not a problem at all (especially write performance)
  • Upon power loss, the system should always boot in a clean state, even if that means that data written in the last few seconds is lost
  • If it is possible to avoid fsck, then all the better.

(I am aware of this related question: Prevent data corruption on ext4/Linux drive on power loss)

ext4

4 Answers

  1. Best Answer

    I've worked in building a system for automation on boats, and there was a prerequisite: in every moment the power could go down and everything must boostrap again correctly.

    My solution was to build a Gentoo-based initramfs system, with only a rw folder for application and configurations (this is the approach used by every router/firewall vendors). This solution add an additional layer of complexity when dealing with system upgrades, but assure you that the system will ALWAYS boot.

    Regarding your specific question, you should keep EXT4 journal enabled for having faster fsck (of a few secods), use the data=journal mount option, lower the commit option or use sync option to keep buffers always empty.

    Refs: http://www.kernel.org/doc/Documentation/filesystems/ext4.txt

    • 13

  2. I will preface this by saying that as far as I'm concerned, EXT (in all of its incarnations) is a pretty awful filesystem -- I have seen more "interesting" cases of filesystem corruption in the relatively small number of Linux/EXT{2,3,4} systems I've administered than I have in the relatively large number of Not-EXT filesystems I've had occasion to use.
    If at all possible try to pick a more robust filesystem. You'll thank yourself when the inevitable happens.

    That being said and all my personal biases out in the open and pushed aside, EXT4 does have three features I can think of that might help you out:

    • Journaling
      EXT4 can be a Journaled filesystem, if you want it to be. Enable the journaling feature (and specifically set the data-journaling mode to journal via tune2fs or as a mount option).
      This incurs a performance hit as all data must be written out to the EXT journal before it gets "committed" to the filesystem (every write basically happens twice) but it ensures you can always recover as far as a journal replay will get you without any problems.

    • SYNChronous Mounts
      When safety is paramount mounting a filesystem with the sync option is always a good idea. This forces all writes to disk immediately - again this is a performance hit, but a good idea if you expect power failures or random strangers yanking the CF card out.

    • Limit writable filesystems as much as possible This one isn't EXT specific, but the all-too-common Linux philosophy of "just create one big root partition and dump everything into it" is, quite frankly, stupid. Create a proper filesystem structure (/, /var, /usr, /home, etc...), and mount as many of the filesystems read-only as possible.
      This used to be common advice for unix systems for the sake of security, but in your case it has an added benefit: You can't corrupt a filesystem if you can't write to it.

    • 13

  3. EXT4 doesn't sound like the best choice for your system; I would suggest looking at a log-structured filesystem. These work by treating data as a constant stream of write updates against a virtual stream, with a pointer that points the latest 'head'. Updates occur by writing data and metadata to the storage, then updating the pointer. In the case of a crash after the write but before the pointer update the latest data is lost but the filesystem is consistent.

    Two candidate filesystems are LogFS and NILFS. Both are available in the mainline Linux kernel.

    • 7

  4. I'm intrigued about the device your building. You're after the reliability of an embedded device while using a filesystem that isn't really suited.

    Ext4 (and family) is a fine general purpose filesystem with (I guess) many billions of hours of use on varied hardware and use cases. However, what your asking for doesn't really fit with ext4. The pointers from voretaq7 and Giovanni will help get the best out of using ext4 if you have to, but the real answer is to use something more suited to your requirements. Steve has given you a couple of options. If you keep pulling power from an ext4 FS you'll eventually get a mess.

    If this is a one off system that you're building you should make the choice to use something more suitable or accept that there will be problems at some point. It might only be 1 power outage out of 100 or 1 out of 1000. That could be good enough for you to take the risk and the device could likely run for a long time (years) without any manual intervention.

    If this is a product that you intend to widely deploy/bring to market you have the choice to use something more suitable. Or you take the business decision to support a percentage of the devices that will brick every year and either need replacing or manual intervention to recover them.

    • 1