I have an HP DL360 G7 server I plan to install KVM and ZFS on. The purpose is going to be a lab in a box. I have the 8 drive bays in the front loaded with 4 drives and an SSD (for the ZFS ZIL cache). My goal was to keep the disk array away from the actual OS disk. What I am wonder is weather or not it is safe to boot a Linux server installation from a USB drive for "production" use. The server has an embedded USB / SD card reader on the motherboard for VMware and other embedded solutions. This raises a question to me because once VMware is loaded it stay in memory. On the other hand a Linux install does not (atleast not 100%). I am concerned if I load the OS on a USB drive (or SD card) I will burn the SD card out.
Can anyone please give me some insight on this? I am wondering what my options are. The way I see it my options currently are make Linux boot from the ZFS array or use a USB drive. The first option would be okay if I could make grub play nice with ZFS root booting. Which to my knowledge is broken or at least needs some nasty hack to pull it off.
Like I said any help you could provide would be great.
You can, but you probably shouldn't.
The DL360 G7 doesn't present disks in a JBOD fashion. If you're using the onboard HP Smart Array controller, this won't work the way you expect.
Depending on the OS you use, there are swap and other I/O activity considerations.
Why go through all of this? If you want ZFS, just use it on the server in a baremetal OS installation. ZFS root is not great... however, you have an HP controller that's capable of multiple logical drives and better reporting/monitoring than you'd get otherwise. The ZIL SSD is probably not even necessary.
See: ZFS best practices with hardware RAID
Can you give a better idea of why you wanted to keep the OS and data drives separate?
Yes, why not? I do it regularly. But having said that, it's not entirely trivial to do it well. If you just straight install Linux to a USB disk it's usually really slow and doesn't really work that great.
However, if you use a tool like
unetbootinit'll create a more optimal installation with which you can also install some configuration.FreeNAS actually is designed to run this way, so that you have all of your HDD's and SSD's available for storage and the OS is booted off flash onto a RAM disk. Although FreeNAS is not Linux based, I mention it here to tell you this is not an unusual solution and is used in production.
When/if you do get it to work, make sure you back up your USB stick every time you make changes to it's stored configuration.
Define "safe". As in:
I would not trust a file server to be booted from a single USB pen, unless it's a server just for testing/lab etc. We use USB drives (expensive ones) in our vSphere Cluster, and I've encountered failures many times.
You've indicated that you have replaced the built-in smartarray controller with a LSI HBA card. If I were you I'd dedicate two drives (2x200GB MLC SATA SSD or similar) for the operating system + swapfile, and create some sort of mirror on them. I have no idea if Linux can boot from ZFS in a stable manner, so you should consider using the operating systems native software raid with a common, trusted filesystem on top (ext3, ext4, xfs).