Recently I had a hard drive fail. It was part of a Linux software RAID 1 (mirrored drives), so we lost no data, and just needed to replace hardware. However, the raid does requires rebuilding. A hardware array would usually automatically rebuild upon drive replacement, but this needed some help.
When you look at a “normal” array, you see something like this:
That’s the normal state – what you want it to look like. When a drive has failed and been replaced, it looks like this:
Notice that it doesn’t list the failed drive parts, and that an underscore appears beside each U. This shows that only one drive is active in these arrays – we have no mirror.
Another command that will show us the state of the raid drives is “mdadm”
As this shows, we presently only have one drive in the array.
Although I already knew that /dev/hdb was the other part of the raid array, you can look at /etc/raidtab to see how the raid was defined:
To get the mirrored drives working properly again, we need to run fdisk to see what partitions are on the working drive:
Duplicate that on /dev/hdb. Use “n” to create the parttions, and “t” to change their type to “fd” to match. Once this is done, use “raidhotadd”:
The rebuilding can be seen in /proc/mdstat:
The md0, a small array, has already completed rebuilding (UU), while md1 has only begun. After it finishes, it will show:
I was a little surprised that this process wasn’t entirely automatic. There’s no reason it couldn’t be. This is an older Linux install; I don’t know if more modern versions will just automatically rebuild.
*Originally published at http://www.aplawrence.com
A.P. Lawrence provides SCO Unix and Linux consulting services http://www.pcunix.com
