Home / server / Questions / 472708
Accepted
Kevin
Asked: 2013-01-27 13:44:39 +0800 CST

Unexpected RAID space overhead

  • 772

I'm using an HP Smart Array P400 and seeing the controller take a rather large amount of overhead I would not expect and am wondering where it is going.

I have six SAS drives hooked up. They're all marked 146GB except one. (One happens to be 300GB but this won't matter for RAID.) I'm not sure if this means 146,000,000,000 bytes or 156,766,306,304 bytes or what.

In ACU, under Physical Drives it shows them as 146GB. When I create an array from them, unused space (before creating a logical disk or setting the redundancy level) shows as 820.2GB.

Since 146*6=876GB and not 820.2GB, at first I would have thought that the disk sizes were being quoted in decimal gigs (GB = 10^9) and the array size in binary gigs (GiB = 2^30).

However, if I assume this, the numbers still don't work out. 146GB in binary would be 135.973GiB, and six of them would be 815.839GiB.

815.8GiB is smaller than the 820.2GB that ACU is quoting as the array size, which logically means it must be quoting both the drive sizes and array size in the same units (be them either binary or decimal gigs).

But if this is the case, then 55.8GB, or a whopping 6.4% of the array has mysteriously vanished.

Now, I know the RAID controller probably places some metadata on the drives so I can't expect 100% of the space to be available. But I would expect this metadata should only be on the order of a few megabytes at most. What accounts for a loss of 55.8GB over six drives?

To clarify, we are not talking about losses due to redundancy. For instance, RAID1-0 makes 50% of the space available, for six drives RAID5 makes 83.3% of the space available, etc., but what I'm talking about here is space that is lost before redundancy is even chosen. This space would be lost even with RAID0, which should expose nearly 100% of the space.

raid

1 Answers

  1. Best Answer

    I wouldn't really worry about it. It is what it is. You can't change it. Plan accordingly.

    You've encountered the difference between Gigabyte and Gibibyte.

    (1024 x 1024 x 1024) = 1,073,741,824 bytes
(1000 x 1000 x 1000) = 1,000,000,000 bytes

To convert from GB to GiB, divide by 1.073741824.

    That accounts for the drive size difference.

    Here's a 6-disk RAID 1+0 array comprised of 300GB SAS drives on a Smart Array P410 controller. Instead of 900GB usable space, it's 838GB:

      Logical Drive: 1
     Size: 838.1 GB
     Fault Tolerance: RAID 1+0
     Array Accelerator: Enabled
     Disk Name: /dev/cciss/c0d0

     Mirror Group 0:
        physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 300 GB, OK)
        physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 300 GB, OK)
        physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 300 GB, OK)
     Mirror Group 1:
        physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 300 GB, OK)
        physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 300 GB, OK)
        physicaldrive 2I:1:6 (port 2I:box 1:bay 6, SAS, 300 GB, OK)

    However, the same disks, when run in a Nexenta/ZFS setup with LSI SAS controllers, show the following during format:

       7. c10t5000C5001A9EC14Fd0 <HP-EG0300FAWHV-HPDE-279.40GB>
      /scsi_vhci/disk@g5000c5001a9ec14f
   8. c10t5000C500339A0D17d0 <HP-EG0300FAWHV-HPDE-279.40GB>
      /scsi_vhci/disk@g5000c500339a0d17
   9. c10t5000C500339A1B7Fd0 <HP-EG0300FAWHV-HPDE-279.40GB>
      /scsi_vhci/disk@g5000c500339a1b7f

    So I'm really working with 279.4GB disks. (3 x 279.4) = 838.2GB, which is close to the 838.1GB available space provided in the Smart Array-based logical drive.

    Running the same check for a 146GB drive on one of my ZFS systems shows the disks registering as:

      1. c17t5000CCA00A1A15BDd0 <HP-DG0146FARVU-HPDD-136.73GB>
     /pci@0,0/pci8086,3410@9/pci1000,3020@0/iport@10/disk@w5000cca00a1a15bd,0

    So, 136.73GB. (6 x 136.73) = 820.38, versus the 820.2GB you see on your system.

    This means that your usable space is just a function of the drive's reported size and definitely not an issue with the HP Smart Array RAID controller overhead.

    • 2