RAID notes

From Smop.co.uk

Linux software RAID is fabulous. Unless you have a hardware RAID card with battery backed controller, why use anything else? It's cheaper, often faster, very robust, very flexible and doesn't use a PCI slot.

The definitive source of documentation is the manpage, however you really want a few notes - the examples are much clearer than the main documentation. So, here are some notes:

• Create a RAID-1 set across two disks, one of which isn't there yet:
  • mdadm -C /dev/md0 --level=1 --raid-devices=2 /dev/hda5 missing
• Add 2nd disk to previously defined array:
  • mdadm /dev/md0 -a /dev/hdd5

Whilst we wait for that to sync, let's look at /proc/mdstat which shows what is going on:

Personalities : [raid1] 
md0 : active raid1 hdd5[2] hda5[0]
      59954432 blocks [2/1] [U_]
      [=>...................]  recovery =  5.4% (3295232/59954432) finish=29.2min speed=32320K/sec

I didn't find the Software RAID HOWTO fabulously clear on what this means, so let's disect it with the help of this German explanation.

The first line is easy - it says the kernel knows about RAID-1 (mirroring) - where a number of disks have identical information on them.

The next line says that the array is active and has two disks in it - hdd5 and hda5, we'll come back to the numbers later.

The third line says how large the array is, it then says that there should be 2 disks in it, however only 1 is correct. One disk is marked "Up", the other is marked as down ("_"). However, "U_" does _not_ refer to hdd5 and hda5 respectively - that is the confusing bit!

We know that there should be 2 disks (since it says [2/..]), these will be numbered 0 and 1. If it said there are [7/..] disks (e.g. RAID-5) then the disks would be 0,1,2,3,4,5,6. Now we return to those numbers in that second line - these are the disk numbers in the array. hda5 is numbered disk 0 and thus one of the mirrors. hdd5 is numbered disk 2 - what is that? If it was disk 1 then it would be the other mirror, since it's disk 2 then it means that it is not currently in the array - i.e. it is not synced. It's either a hot-swap spare or it is being synced.

The third line says that it is in fact being synced.

Incidentally, the HOWTO says that if a disk is marked with an "F" - e.g. "hda5[0](F)" then it means that it has failed. I don't know if it would be automatically renumbered to e.g. "[2](F)" or whether it keeps the number it had before the failure.

Later on I added another array (md1) which has two mirrored disks, however as in the example right at the start, I've only initially created it with one member (i.e. it is degraded). You can also see that the second array has finally synced - the recovery line is now longer display, both disks are marked "Up", the array has 2/2 members and hdd5 is now disk 1 rather than disk 2.

Personalities : [raid1] 
md1 : active raid1 hdd1[1]
      96256 blocks [2/1] [_U]
      
md0 : active raid1 hdd5[1] hda5[0]
      59954432 blocks [2/2] [UU]
      
unused devices: <none>

recovering

Take this scenario where /dev/hdd5 has failed (pay attention tothe "F" not to the ordering of the "U_"):

md0 : active raid1 hdd5[2](F) hda5[0]
      59954432 blocks [2/1] [U_]

• this is just temporary so remove the duff disk and re-add it:
  • mdadm -r /dev/md0 /dev/hdd5
  • mdadm -a /dev/md0 /dev/hdd5
• linux has reset the disk by now so we need to re-run hdparm to make it quick again:
  • hdparm -c3 -m8 -u1 -d1 -W1 /dev/hdd