Home / server / Questions / 182048
Accepted
JD Frias
Asked: 2010-09-17 15:15:21 +0800 CST

mdadm: Swapping out small hard drives for bigger ones in a RAID5, How to partition?

  • 772

So I got the gist of swapping out smaller hard drivers for bigger ones in an MDADM RAID5:

Fail and remove one drive.

mdadm -f /dev/md0 /dev/sdd1
mdadm -r /dev/md0 /dev/sdd1

Shutdown, swap the hard drive, partition and add back to array

mdadm --add /dev/md0 /dev/sdd1

Wait for resync and continue with all the remaining drives.

Finally grow the array

mdadm --grow /dev/md0 --size=max

My question is how do you partition the new bigger drives? I see different instructions around the net and I know most of them are outdated so I wanted to see if anyone has done it recently.

  1. Partition the new drives to be the same size as the old drives in the array

  2. Partition the new drives to be the maximum size of the drive.

I have a really simple setup, 4 drives RAID5 350GB each. I only have one partition on each drive and build a single array. This is a data archive disk so no need for multiple arrays or partitions.

The other approach I am thinking is backup to one drive, build a 3 drive array, add the backup drive to the array. That is:

  • Remove one drive (350GB)
  • Add a new drive (1TB)
  • Backup all my data into the new drive (my raid data will fit in a new drive)
  • Take out the remaining 3 (350GB) old drives
  • Put in the rest of the 3 (1TB) new drives
  • Build a 3 drive array
  • Copy the data to the array from the 1st new drive
  • Wipe the backup disk and add to the array to make it 4 drives.

Is this safer? Faster? Thoughts?

partition mdadm raid5 swapping

4 Answers

  1. Current versions of md tools support replacing a device while keeping the RAID (and so, the redundancy) working.

    You need ability to add at least one more device the the computer but after that, you won't need to keep the array in degraded state for hours (or days, in case of current multi-TB HDDs) while it rebuilds.

    First you need to add a disk as a spare to the array (assuming 4 drives in RAID):

    mdadm /dev/md0 --add /dev/sde1

    Then, you tell Linux to start moving the data to the new drive:

    mdadm /dev/md0 --replace /dev/sda1 --with /dev/sde1

    After replacement is finished, the device is marked as faulty, so you need to remove it from array:

    mdadm /dev/md0 --remove /dev/sda1

    Repeat for other drives in the array.

    If you have ability to connect multiple additional drives, you can do that even for all drives at the same time, all while keeping the array online and with full redundancy. So following is a valid set of commands:

    mdadm /dev/md0 --add /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1
mdadm /dev/md0 --replace /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 --with /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1

    Wait until finish, remove old drives:

    mdadm /dev/md0 --remove /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
    • 9
  2. Best Answer

    Eventually you will need to have the "raid partition" to have the maximum size of the disk; it doesn't matter if you do that upon rebuild or if you repartition once your array is renewed.

    I've recently done something similar, it's fairly fast. Note that you can set the raid rebuild speed (/proc/sys/dev/raid/speed_limit_min/max), by default it's limited for background rebuilds.

    Note there is one step missing in your plan: you will need to instruct the thing on top of your md-layer to grow as well (filesystem, or lvm pv, or ...)

    However, if you can do it (can keep the system down for longer, invest more time and have enough disk slots) it's probably safer to copy everything onto a backup disk. The data remains in plain sight at all times, which I find comforting.

    • 2

  3. The simplest/safest way to end up with an array made up of larger disks would be to

    • copy all the data to an external drive
    • remove all the small disks and replace them with large ones
    • create the new array on the larger disks
    • copy the data back

    This would also give you two copies of the data: one on the external drive, and one still on the collection of smaller drives you'd removed. (Keep careful track of which disk is which!)

    I don't think you can grow RAID-5 arrays by adding a disk into it. You would be creating a 3-disk RAID-5 array, and the best you could do after copying data to the new array would be to mark the fourth disk as a hot-spare... which isn't exactly the same thing as a 4-disk RAID-5 array.

    • 1

  4. I'm asking myself the same question, just with 4x 400 GB disks RAID5 (lets call it /dev/md1) replaced with much bigger disks (something between 2 and 4 TB). LVM on top of that.

    One not so elegant solution I figured out, is the following:

    Remove one disk (preferable to most error-prone one according to the disks' SMART values), replace it with a new one. Partition the new disk with one partition of the same size as the old ones and a second partition for the remainder. Add the same-sized partition to the RAID /dev/md1 and wait for resync to finish.

    Then do the same again with the next disk. Now you've got two additional still unused partitions. Start a new RAID 5 on them. Add the new RAID5 (e.g. /dev/md2) as physical volume (PV) to the volume group used on /dev/md1. Already after two disks being replaced you already have more redundant disk space available.

    Do the same for the third and forth disk, add them as additional disks to /dev/md2 and run pvresize on the device afterwards.

    I though suspect that I'll do the same as suggested twice already in this thread: Put all stuff (currently about 700G of the 1.2 TB the RAID has) on an external disk (possibly one of the new ones) and then start a new RAID5 array with three disks, copy the data back and then add the last disk which was used as temporary storage.

    • 1