I'm trying to get dmraid to properly detect an existing RAID 10 array, which is working fine within windows 7. In summary, the drive setup and bios partitioning is as following:
2 x SSD (120 GB), RAID 0.
Partitioned into:
180 GB (win7)and58 GB (Ubuntu).Both detected and mounting fine in Ubuntu.
4 x HDD (2 TB), RAID 1+0. Single 4TB partition.
Not working in ubuntu.
The Ubuntu installation itself is on a partitioned striped disk pair, and works fine. The two partitions (180 GB and 58 GB) correspond to the following LVM's (I've taken the liberty to format the data for readability):
#$ sudo dmsetup info /dev/dm-{1,2,3,4,5}
|----------+--------+---------------+-----------+----------+--------|
| dev/dm-0 | ubuntu | pdc_hjijcjji | container | | 58 GB |
| dev/dm-1 | ubuntu | pdc_hjijcjji1 | / | ext4 | 54 GB |
| dev/dm-2 | ubuntu | pdc_hjijcjji2 | | extended | 4.3 GB |
| dev/dm-3 | win 7 | pdc_fjhhdeeg | container | | 180 GB |
| dev/dm-4 | ubuntu | pdc_hjijcjji5 | | swap | 4.3 GB |
| dev/dm-5 | win 7 | pdc_fjhhdeeg1 | | ntfs | 180 GB |
|----------+--------+---------------+-----------+----------+--------|
The Raid 10 array (which is the one I need help to get it working within Ubuntu) consists of four 2TB disks, and give a resulting 4TB array. It seems as if dmraid is aware of this array, given the following output:
#$ sudo dmraid -r |----------+--------+------------------+--------+---------+----------------+---------| | Device | Format | Name | Type | Status? | Size (sectors) | ? | |----------+--------+------------------+--------+---------+----------------+---------| | /dev/sdf | pdc | pdc_fjhhdeeg | stripe | ok | 175781248 | data@ 0 | | /dev/sde | pdc | pdc_fjhhdeeg | stripe | ok | 175781248 | data@ 0 | | /dev/sdd | pdc | pdc_bjibibahah-1 | stripe | ok | 1758766336 | data@ 0 | | /dev/sdc | pdc | pdc_bjibibahah-1 | stripe | ok | 1758766336 | data@ 0 | | /dev/sda | pdc | pdc_bjibibahah-0 | stripe | ok | 1758766336 | data@ 0 | | /dev/sdb | pdc | pdc_bjibibahah-0 | stripe | ok | 1758766336 | data@ 0 | |----------+--------+------------------+--------+---------+----------------+---------|
Which throws me off a bit, since I'd expect the array, pdc_hjijcjji, to show up here as well. Perhaps, since it's a partition within a striped disk, it is included within pdc_fjhhdeeg. In any case, the striped array is running fine, so I'm not too worried about it.
pdc_hjijcjji shows up when running dmraid -s:
#$ sudo dmraid -s |-----------+----------------+--------------+--------------| | Name | pdc_bjibibahah | pdc_fjhhdeeg | pdc_hjijcjji | |-----------+----------------+--------------+--------------| | | Superset | Active Set | Active Set | | Size (-h) | 1.677 TB | 167.6 GB | 54.0 GB | | Size | 3517532672 | 351562496 | 113281024 | | Stride | 128 | 128 | 128 | | Type | raid10 | stripe | stripe | | Status | ok | ok | ok | | Subsets | 2 | 0 | 0 | | Devs | 4 | 2 | 2 | | Spares | 0 | 0 | 0 | |-----------+----------------+--------------+--------------| # Size is in blocks of 512 bytes.
Where pdc_bjibibahah seems to correspond to the Raid10 array, except that it has a weird size (1.677 TB, as apposed to approx 4 TB). Trying to activate this using dmraid -ay results in a mounted 1.677 TB device with is reported as unallocated data.
Running sudo dmraid -s -si on the bjibibahah array:
#$ sudo dmraid -s -si |-----------+----------------+------------------+------------------| | Name | pdc_bjibibahah | pdc_bjibibahah-0 | pdc_bjibibahah-1 | |-----------+----------------+------------------+------------------| | | Superset | Subset | Subset | | Size (-h) | 1.638 TB | 1.638 TB | 1.638 TB | | Size | 3517532672 | 3517532672 | 3517532672 | | Stride | 128 | 128 | 128 | | Type | raid10 | stripe | stripe | | Status | ok | ok | ok | | Subsets | 2 | 0 | 0 | | Devs | 4 | 2 | 2 | | Spares | 0 | 0 | 0 | |-----------+----------------+------------------+------------------|
I've tried configuring the arrays manually, but with no success, and the man page isn't proving all too helpful either. If anyone has a suggestion as to how to configure dmraid, or to convince it to change the configuration of the RAID10 array, I would be really grateful.
Here are a few outputs, should it be relevant:
$ sudo dmraid -V dmraid version: 1.0.0.rc16 (2009.09.16) shared dmraid library version: 1.0.0.rc16 (2009.09.16) device-mapper version: 4.20.0 $ sudo dmsetup --version Library version: 1.02.48 (2010-05-20) Driver version: 4.20.0 $ uname -srvm Linux 3.0.0-16-generic #28-Ubuntu SMP Fri Jan 27 17:44:39 UTC 2012 x86_64 $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 11.10 Release: 11.10 Codename: oneiric
Also, the BIOS raid setup of the RAID10 array match the following disks within ubuntu: Code:
|---------+------------+--------------| | Port:ID | Assignment | OS Disk Name | |---------+------------+--------------| | 01:01 | LD 1-1 | /dev/sda | | 02:01 | LD 1-2 | /dev/sdb | | 03:01 | LD 1-3 | /dev/sdc | | 04:01 | LD 1-4 | /dev/sdd | |---------+------------+--------------|
Update: I forgot to mention that I'm on a Crosshair V motherboard, using the built in AMD SB950 controller.
Update 2: When running sudo dmraid -s -ccs pdc I get the following error messages:
ERROR: pdc: wrong # of devices in RAID set "pdc_fjhhdeeg" [1/2] on /dev/sdf ERROR: pdc: wrong # of devices in RAID set "pdc_fjhhdeeg" [1/2] on /dev/sde ERROR: pdc: wrong # of devices in RAID set "pdc_bjibibahah-1" [1/2] on /dev/sdd ERROR: pdc: wrong # of devices in RAID set "pdc_bjibibahah-1" [1/2] on /dev/sdc ERROR: pdc: wrong # of devices in RAID set "pdc_bjibibahah-0" [1/2] on /dev/sdb ERROR: pdc: wrong # of devices in RAID set "pdc_bjibibahah-0" [1/2] on /dev/sda *ERR*
If I omit the specifier pdc, and just write sudo dmraid -s -ccs I get the even more cryptic:
*ERR* *ERR* *ERR*
Update 3: I've managed to remove the badly configured array using sudo dmraid -rE as suggested in an askubuntu QA here. I'm still getting the "ERROR: pdc: wrong # of devices in RAID set ..." errors, which seem to be impeding further progress. I'll keep searching for solutions to this problem.
Edit: Is there any information I can add that would help anyone help me?
OK :) So here's your goal:
4 x HDD (2 TB), RAID 1+0. Single 4TB partition.Seeing that it's only reporting around 1.6TB, that much missing storage suggests a meta-data problem to me.
I would suggest tearing down the entire array and ensure the metadata is wiped by using
dmraid -rE. Then, once those disks don't show up anymore, try making a simple RAID1 from the 2 2T disks, your container capacity should be closer to 1.8-1.9 then 1.6. Once that is verified, then create your stripe, which ought to just add the two together. If this doesn't work you might have a bug in either the firmware or the dmraid system itself.dmraid is ummm... interesting, given the choice I would prefer not to use it. You're basically taking all the complexity of md-raid, without any increase in performance, and then mixing in a slew of different metadata formats from different vendors, and then sharing that metadata with multiple operating systems, trusting that the OS you shared it with doesn't ruin the metadata.
You might want to test actually replacing failed members after you get this configuration stable, a RAID is just a big single point of failure if you can't service it. Good luck.
In response to your dmraid -rE issue At this point you're only option is dd if=/dev/zero ... and wipe the disks out.
I have the very same problem and I believe I finally (after several days of googling without a satisfactory answer) found the problem.
I believe this is a BUG - a 32bit unsigned int is used for the number of sectors somewhere (dmraid ?).
Note, the number of sectors on (my) 2TB disk: 3906898048, which multiplied by 512B/sector gives 2.0TB or 1.8TiB When striped, it should double, but instead it gives: 3518828800 sectors, multiplied by 512 B/sector results to 1.8TB or 1.6TiB.
Considering that Windows 10 reads the correct 4TB size on that RAID10 partition, the error is likely on Linux side.
Converted to binary:
3906898048 -> 0000 1110 1000 1101 1110 1000 1000 1000 0000 2x 3906898048 -> 0001 1101 0001 1011 1101 0001 0001 0000 0000 3518828800 -> 0000 1101 0001 1011 1101 0001 0001 0000 0000Can please somebody look into it and create a patch? Thank you.