Home / server / Questions / 79891
Accepted
Karl Katzke
Asked: 2009-10-31 06:42:30 +0800 CST

Solaris Multipathed I/O - Physically Removing a Disk

  • 772

I am a Solaris newbie. I have a Solaris 10u8 machine that is running an attached J4400 and some internal drives. We're using multipathed SAS I/O (stmsboot; mpathadm), so the device mount points have been moved off from their normal c0t5d0 to long strings -- in the case of c0t5d0, it's now /dev/rdsk/c6t5000CCA00A274EDCd0. (I can see this with stmsboot -l0.

Normally, when replacing a disk on a Solaris system, I would run cfgadm -c unconfigure c0::dsk/c0t5d0. However, cfgadm -l does not list c6, nor does it list any disks. In fact, running cfgadm gets me the following: 

bash# cfgadm -l /dev/rdsk/c0t5d0
Ap_Id                          Type         Receptacle   Occupant     Condition
/dev/rdsk/c0t5d0: No matching library found
bash# cfgadm -l /dev/rdsk/c6t5000CCA00A274EDCd0
cfgadm: Attachment point not found
bash# cfgadm -l c6t5000CCA00A274EDCd0
Ap_Id                          Type         Receptacle   Occupant     Condition
c6t5000CCA00A274EDCd0: No matching library found

So -- how do I administer and remove physical devices that are in multipath-managed controllers on Solaris 10u8?

solaris

2 Answers

  1. Best Answer

    cfgadm -al only showed the controllers, not any of the disks.

    The answer, straight from a Sun employee on the zfs-discuss mailing list, is that cfgadm doesn't understand SAS multipathing yet -- and worse, it doesn't understand multipathing in the sense that you can't edit your device configuration! That means that you can't remove a disk and insert a new one (because they have different GUIDs) while the system's running without breaking at very least mpathadm. What good is multipathing if replacing a hot-swappable disk requires a reboot?

    So basically, while you can multipath I/O via SAS to an external array, god forbid you need to remove a disk!

    This isn't true for fishworks, aka the Sun Unified Storage devices. They've got tools that can handle it, but those tools haven't made it outside of those products yet. The cfgadm and mpathadm problems have also been fixed in Nevada / 126.

    • 2

  2. Maybe it's better to take the local disks out of the multipath setup. In /kernel/drv/fp.conf you can disable mpxio for a specific controller. Here is a example from a SunFire V880 with an internal fibre channel adapter.

    name="fp" parent="/pci@8,600000/SUNW,qlc@2" port=0 mpxio-disable="yes";

    But nethertheless what is cfgadm -al showing?

    • 1