Home / server / Questions / 1077561
In Process
gertvdijk
Asked: 2021-09-15 08:53:32 +0800 CST

How to ensure SAS-3 wide port configuration is enabled?

  • 772

I have:

  • A commodity Linux server, in this I have installed:
  • 1x HBA storage adapter Broadcom (LSI) 9500-8e having 2x mini-SAS-HD connector, connected to:
  • 1x Storage enclosure with dual input mini-SAS-HD connector, SAS-3 capable, using:
  • 2x mini-SAS-HD cables running between the controller and the enclosure.
  • A bunch of SAS 3.0 hard disks in the enclosure.

The enclosure and disks work just fine with a single cable attached, but they can optionally be connected with two. Using two cables is recommended by the manual of the enclosure:

enter image description here

(The other two mini-SAS-HD connectors listed in the specs of the enclosure are for daisy-chaining more enclosures, but I have just one.)

I'd like to know:

  • Main question: How do I know the controller and enclosure actually have 8x 12Gbit/s lanes (as provided by 2x four lanes of mini-SAS-HD) bandwidth available? (a.k.a. wide port configuration enabled)
  • How to tell which cables are connected? For reasons of monitoring, I'd like to know about cable failure, similar to an ethernet link status. It would be great to be able to alert when the second link goes down.

Please note that my question here is unrelated to a high-availability scenario; the enclosure I have does not support high-availability (multi-path). My question is about port configuration / bandwidth.

What I do know:

The Broadcom StorCLI reference manual suggests looking at the "load balance" mode status/configuration.

set loadbalancemode =[on|off] Enables (on) or disables (off) automatic load balancing between SAS phys or ports in a wide port configuration.

However, StorCLI reports neither on or off, it reports "None" in the "Policies Table":

Load Balance Mode = None

Listing the drives in the enclosure lists two connectors, though, so that sounds okay to me:

Drive /c0/e18/s0 Device attributes :
==================================
Manufacturer Id = HGST    
[...]
Device Speed = 12.0Gb/s
Link Speed = 12.0Gb/s
[...]
Connector Name = C0   & C1

However, all of the disks including the enclosure are listed in kernel messages with just C0 connector, e.g.: scsi 5:0:39:0: enclosure level(0x0001), connector name( C0 ). ?

sas storage lsi broadcom storcli

1 Answers

  1. set loadbalancemode =[on|off] setting has meaning only for Broadcom RAID controllers. As far as I know you don't have to worry about SAS wide ports in this scenario, they are handled at port layer of SAS protocol, and modern Broadcom SAS HBAs with Broadcom SAS expanders automatically configure wide ports up to x16.

    You can use smp_discover from smp_utils package to view detailed information about SAS topology. E.g. list of PHYs connected to expander.

    • 0