Home / server / Questions / 80039
Accepted
Natalie Adams
Asked: 2009-10-31 14:05:22 +0800 CST

Parallel file systems

  • 772

I am looking for a "Parallel File System" that is easy to setup, maintain, and scalable.

My goal is to have a single mount point on a Linux machine that applications can read/write using standard coding practices (such as C++) that essentially is several mount points. I am not looking for speed, I am looking for storage and reliability; though speed would be a plus obviously.

So for example lets say:
/mnt1 -> smb://srvr1/share
/mnt2 -> smb://srvr2/share
/vfs -> mnt1 + mnt2

I looked at ZFS and although they support actual files as disks, they don't recommend it. Anyone have any suggestions on this?

linux

6 Answers

  1. Most of these type of file systems aren't very easy to setup and maintain. But GlusterFS may fit your needs and it's about as easy as these things get.

    Other options would be AFS, Lustre(very difficult to setup and maintain), and pNFS.

    There is a new interesting project called SheepDog but it's primary focus is as a backend for virtual machines.

    • 1

  2. While you say "parallel" filesystem what it sounds like you're actually describing is a "stack" of filesystem. That's what UnionFS does. While I'm not sure if it supports using SMB mounts, it's worth a look.

    There's also aufs (Another Union FS) which, as the name implies, is another implementation of the same concept.

    • 1

  3. Do you really need to combine mountpoints?

    You're probably better off using a mixture of RAID and LVM instead, assuming your storage requirements aren't into SAN territory. At least keep things down to a single filesystem rather than adding in an extra meta filesystem. (Which is seriously what you are talking about!)

    There are projects like UnionFS, already in another answer, which address this sort of problem. However, there are huge issues with that. It's great for "I want to start with this CD and make a few changes to it" but actually distributing writes across a bunch of equalish filesystems is a different story.

    Otherwise you're in "cluster filesystem" territory, and those generally aren't built out of CIFS shares...

    • 1
  4. Best Answer

    What you want is a Samba RAID.

    First, mount the samba shares (perhaps add them to the fstab), and create files to act as block devices on each samba share:

    dd if=/dev/zero of=/where/you/mounted/the/share/raid bs=10000M count=1

    (replace 10000M with the desired size)

    Follow the instructions here, but for block devices, use the files you have just created. This will probably be quite slow.

    • 1

  5. Maybe you can also have a look at mhddfs, similar to aufs:

    http://svn.uvw.ru/mhddfs/trunk/README

    There even is a Debian package, al least in Sid.

    • 1

  6. There are several parallel filesystems. However most are quite complex to set up, and optimized for supercomputing. See PVFS, Gluster, Lustre, CEPH...

    PVFS is by far the easiest to set up IMHO, it can be resized easily and re-exported as samba and NFS.

    Gluster is based on Fuse so it carries usual Fuse limitations (performance, no nfs-kernel-server support) but is generally very good.

    Lustre really requires quite a lot of nodes to be interesting, and is mostly useless out of the render farm/computing nodes applications.

    Let it be clear : if you're not an excellent linux administrator, forget it. All of these systems requires that you know how to compile and install kernels and kernels modules, edit tons of obscure configuration files and work entirely from the console. Certainly not for the Ubuntu noob crowd.

    • 0