Is it possible to connect all 24 disks to one raid card ? How do I connect these physically ? Do I need an expander ? Or is it-mode flushing sufficient ? I read it-mode flushing permits up to 1024 drives and software raid. (mdadm bcache megacli etc usage is not a problem ?
The Hardware I got :
- OS : Linux
- Backplane : BPN-SAS3-846EL-1 https://www.supermicro.com/manuals/other/BPN-SAS3-846EL.pdf
- Raid Cards : LSI 9260-8i , 2 x LSI 2108 pci-e x8
- Cable: SFF-8643 to SFF-8087
That and WAY More - 250 are not unheard of SAS is a storage network protocol.
Using a SAS cabinet that has the logic boards and physical slots for the discs.
Generally no. I mean, you need one, but the cabinet has one or more of them.
Ah, the backplane. You are aware that there is DOCUMENTATION you should READ? You link to it, but you seem to think we should read it for you.
If you got that backplane - as a backplane, i.e. as single part - you bascially did not get a car but bought part of a car engine. This backplane is a part sold separately for upgrades and replacement IN STORAGE CHASSIS. As you can clearly read (if you read it) on 2-5 of the manual it has a TON of SAS HDD connectors on the back side. These generally are on the back side of a chassis with hot swap trays, where the discs then connect into those SAS connectors. And yes, those are then for SAS discs. The backplane also supports chaining, do you can have one SAS connection go from one backplane to the next. That is, btw., also described IN THE MANUAL YOU LINK - Chapter 3 (Dual Port and Cascading Configurations).
It turns out board has built in sas expander , I was able to connect 24 disks into one lsi 2108 and without it mode flushing choosing raid1 (it creates raid 10 automatically) they work as expected. Benchmarks show 1.5 GB per second read and write. A little low than I expected since each disk is rated 250mb/sec read speed. Used linux disks utility , hdparm and dd for benchmark. Cache was enabled didnt see much difference adding sofwtware striped sdd caches (bcache) with 1MB cutoff in small files copy, other than slightly increased IO. Tested this copying random sized files created with dd if=/dev/urandom smaller than 1MB, timed with time command of bash. This was my first setup, I have seen cards and cables first time in my life. Answer is although documentation does not state clearly up to how many disks a cable or an adapter or a built in backplane expander supports and although many people insist that it-mode firmware flushing is required , they work out of the box.