We are using RHEL5.3 with a Clarion SAN attached by FC.
Our situation is that we have a number of LUNs presented to Hosts and we want to dynamically present the LUNs to Xen Guests. We are not sure on what the best practice approach is to set this up. The Xen guests will form a cluster together and need the LUNs only for data partitions, i.e. when they are actively running services.
So one approach would be to always present all disks to all Xen guests, and then rely up on the cluster software, and mount itself, to not mount the disk twice in two locations. This sounds kinda risky and also is not very secure (one cracked guest can see/destroy all the data).
Another approach would be to dynamically add and remove the disks from the Xen guests at the dom0 level (using xm block-attach). This could work but sounds slightly complicated, I'm wondering whether Red Hat Cluster Suite supports this in some way or whether there are scripts to do this.
Yet another approach would be to have the LUNs endpointed at the Xen guests themselves - I'm not sure whether this is technically possible since the multipathing has to be done at the Host level.
Have you considered a clustered file system like GFS?
I had the same question. My answer was to present the LUNs to all DomUs and have the DomUs control the access.
A better approach might be to run the cluster-lvm on your DomUs. (/etc/lvm/lvm.conf locking_type=3 on RH should do the trick). After that present all LUNs to all DomUs, create PVs, VGs and LVs as needed from within a DomU.
I would not go for gfs or ocfs if you don`t need concurrent access on the same LUNs.