Home / server / Questions / 506583
Accepted
sbrattla
Asked: 2013-05-10 21:34:08 +0800 CST

Do SAS expanders work transparently with SAS controllers?

  • 772

I'm new to SAS expanders and I'm trying to get things right. I understand that SAS expanders, when used with SAS controllers which supports expanders, allows you to utilise the full capacity of supported drives on a controller.

However, say I've got a SAS controller with 4 x SSF-8087 ports, which by itself would support 16 disks. I could then, as far as I understand, instead connect each of these ports to an expander to increase disk capacity.

However, will the SAS controller "see" all disks as if they were connected directly to the controller and not through an expander. Could I look at controllers as switches in a network? Would I be able to create a RAID from a disk on expander 1 and expander 3?

UPDATE:

I don't really have a specific project where I need to use SAS expanders. I'm just curious about how they work, and was thinking about how it would work if I used one or more SAS expanders with a LSI MegaRAID controller (9260-16i) which is running in one of our servers. However, I haven't really put any thought into the distinction between "local" disks and disks in another enclosure. The LSI MegaRAID controller has 4 SFF-8087 ports, which makes it capable of accomodating 16 drives.

sas

2 Answers

  1. Best Answer

    First, stop and think about your realistic goals...

    The short answer to your question is that SAS is a switching protocol. The drives connected via expanders are all accessible and can be addressed transparently by a single controller.

    However, the rest of your questions are going to be a function of your particular controller, your storage architecture and the backplanes in the storage enclosure(s) you end up using.

    It's not possible to give a specific answer without more information.

    • How many disks do you need?
    • What RAID controller or SAS HBA are you using?
    • Will these disks be internal or external?
    • What specific SAS expander card are you looking to use (it's not like there are many on the market)?
    • What server and enclosure hardware are you considering?
    • What is the application/reason for this setup? (ZFS? Storage Spaces?)

    Let's use your example of a controller with 4 x SAS SFF-8087 ports. These are meant for communication inside of a chassis. Each SFF-8087 port is a 4-lane connector that can accommodate 4 drives. 16 drives total, in your case... Servers typically only have 4, 8, 12, 16, or 24/25 internal drive bay capacities.

    • At the 4 and 8-drive level, the internal server drive backplane will have a 1:1 lane-to-drive ratio and typically have one or two SFF-8087 ports. No oversubscription!

    • For 12-port servers, the expander is usually built into the backplane, and the input will be a single SFF-8087 port. Oversubscription!

    • For servers with 16 internal bays, you may see four separate SFF-8087 ports intended to be served by a single or multiple SAS/RAID controllers. But at 16-ports, you may have an expander on the backplane instead.

    • Almost every 24 or 25-bay server I've encountered has expanders built onto the drive backplane and accommodate one or two SFF-8087 ports. The exception is something like the HP ProLiant DL370 G6, which could accept three 8-bay drive cages, each requiring 2 SFF-8087 ports.

    So when do people use PCIe card SAS expanders?

    Note: HP makes a great PCIe SAS expander. There's an entire community built around its use for storage projects.

    The most common case I see is a motherboard or PCIe controller with TWO SFF-8087 ports and a need to connect more than 8 drives split across multiple drive cages. I can't envision something like your scenario where you have 16 ports available on a controller and would want to multiply that via expanders to accommodate more internal drives. If you're working with a server with more than 16 disks, chances are that you'll be encountering a backplane expander somewhere. This would make the PCIe expander unnecessary.

    This is all different when you look at external storage, though. The transport there would be a SAS SFF-8088 external cable. Most external drive shelves/JBODs have expanders built-in.

    Example RAID configuration output from an HP Smart Array P410i controller connected to two drive cages inside of an HP ProLiant DL370 G6 server through an HP SAS expander card. Note the card is represented as an enclosure (Vendor ID HP, Model HP SAS EXP Card) and has its own SAS WWN.

    Smart Array P410i in Slot 0 (Embedded)    (sn: 5001438006503640)

   array A (Solid State SATA, Unused Space: 0  MB)

      logicaldrive 1 (223.5 GB, RAID 0, OK)

      physicaldrive 5C:2:8 (port 5C:box 2:bay 8, Solid State SATA, 240.0 GB, OK)

   array B (SAS, Unused Space: 0  MB)

      logicaldrive 2 (68.3 GB, RAID 1, OK)

      physicaldrive 7C:2:1 (port 7C:box 2:bay 1, SAS, 72 GB, OK)
      physicaldrive 7C:2:2 (port 7C:box 2:bay 2, SAS, 72 GB, OK)

   array C (SATA, Unused Space: 2651025  MB)

      logicaldrive 3 (1000.0 GB, RAID 1+0, OK)
      logicaldrive 4 (500.0 GB, RAID 1+0, OK)

      physicaldrive 2C:1:1 (port 2C:box 1:bay 1, SATA, 1 TB, OK)
      physicaldrive 2C:1:2 (port 2C:box 1:bay 2, SATA, 1 TB, OK)
      physicaldrive 2C:1:3 (port 2C:box 1:bay 3, SATA, 1 TB, OK)
      physicaldrive 2C:1:4 (port 2C:box 1:bay 4, SATA, 1 TB, OK)
      physicaldrive 3C:1:5 (port 3C:box 1:bay 5, SATA, 1 TB, OK)
      physicaldrive 3C:1:6 (port 3C:box 1:bay 6, SATA, 1 TB, OK)

   Expander 250 (WWID: 50014380133153E6, Box: 1)
   Enclosure SEP (Vendor ID HP, Model HP SAS EXP Card) 248 (WWID: 50014380133153E5, Box: 2)
   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 249 (WWID: 500143800650364F)

    enter image description here

    • 11

  2. I can agree with the thing that a expander enclosure should not put behind another expander enclosure, from my experiance it looks like to work but hsving lot of very weird behaviours like seeing the drives and initialising them.. now having both expander enclosures connected seperately to the SAS raid card from main server. it seems to work lot better but still have some issues but that can be related to the drives themself. The expanders are not the same type but the first one i run for years without issue on that raidcontroller, now the second expander is different, so have to find out the troubles i have, it can be related to anything software or hardware. here the specs i use at the moment;

    -Highpoint 2722 SAS raidcard with 2 external SAS 8088 ports - expander enclosure 1 with 36 port 24 bays with Chenbro expander(not on backplane) - expander enclosure 2 with 24 bay and Gooxi jbod expander backplane

    The raidcard sees both expanders , the Chenbro as a 24 4U enclosure and the Gooxi one even with the hardware monitoring like fans and temps.

    Till sofar i have some troubles installing drives on the Gooxi and that takes loads of time-)

    • 0