I'm searching for the best way to set up filesystems/partitions for a storage server with the specialty that partial data loss is not a problem.
Lets say I got 4x3TB disks and I want to arrange them in a way that a) I got the maximum space available for the storage partition and b) the system partition(s) are mirrored and can take the failure of one disk. (partial data loss is not such a big deal in this scenario, lets say we store reproducible data, but it is very time consuming to recreate it)
My current approach is the following:
- RAID1->EXT4 on two ~100GB partitions on different disk for system files.
- LVM->DMCRYPT->EXT4 with the rest of partitions and disks. (storage should be encrypted)
The interesting thing here is: Would I be able to mount the dmcrypt device after lets say disk #3 (while disk #1 and disk #2 are the ones with the RAID partitions) failed (worst case scenario, all data is lost) and got replaced? What about the ext4 filesystem, would I be able to repair it? I'm certain I would loose all data thats stored on that specific disk, but what about the rest?
Maybe I'm missing some filesystem/program that can handle such situations well, but most of the times it's all about data recovery, which is not very helpful in this case. I'm open to any different approach that would fit.
EDIT: Of course I could do DMCRYPT->EXT4 on all storage partitions, but I want one mount point for it and of course not the hassle to open all dmcrypts on reboot (with the above setup I only have to do it once).
If you don't care about partial data loss and you don't want any redundancy - i would suggest you create separate file system on every physical disk and split it "program" way. I guess that you're using it for hashed data or something - in that case often is possible to split datas into directories - for example - based on md5 sum of files - and spliting data on different disks partialy. Mouting it - is not problem - you can have shared key for decrypt block devices placed on non encrypted storage.
If you want redundancy - simply use raid5 or something like it. Or - more "software" storage - gluster or ceph.
Standard filesystems aren't prepared for situation when part of space where they are will disappear.
Other than the underlying 100GB system partition there is no RAID involved at all. Everything else will depend upon how your LVM and it's underlying components are organised - information that you have not provided.