With 4 easy steps replace broken HDD on Linux machine which is configured for software raid1.
Consider a Linux system with software RAID1 drive named /dev/sda and /dev/sdb, among these two drive one drive has failed.
1. Identify failed drive.
How to Identify which drive has failed?
To identify which RAID array has failed, look at the string containing [UU]. Each “U” represents a healthy partition in the RAID array. If you see [UU] the array is healthy. If you see a missing “U” like [U_] then the RAID array is degraded or faulty.
# cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sda1 102336 blocks super 1.0 [2/1] [U_] md2 : active raid1 sda3 233147200 blocks super 1.1 [2/1] [U_] bitmap: 2/2 pages [8KB], 65565KB chunk md1 : active (auto-read-only) raid1 sda2 1037000 blocks super 1.1 [2/1] [U_]
Above output shows, One “U” is missing in RAID arrays md0, md1, and md2 of drive “sdb” so we have found “disk sdb” is faulty.
2. Remove failed drive.
Before removing drive we have to mark drive partition failed and then remove. To do that run below command.
To mark the partitions “fail” from the RAID array:
# mdadm --manage /dev/md0 --fail /dev/sdb1 # mdadm --manage /dev/md1 --fail /dev/sdb2 # mdadm --manage /dev/md2 --fail /dev/sdb3
To remove the partitions from the RAID array:
# mdadm --manage /dev/md0 --remove /dev/sdb1 # mdadm --manage /dev/md1 --remove /dev/sdb2 # mdadm --manage /dev/md2 --remove /dev/sdb3
Now you can power off the system and physically replace the defective drive:
# shutdown -h now
3. Adding new drive.
After installing new hard drive, verify new drive detected by the system and has same device name ie sdb to verify run the command.
"fdisk -l /dev/sdb"
After confirming we have to add the new drive to raid array.
Also, we have to create same partition table structure which was on the old drive.
To do that we can refer existing partition tables structure from drive “sda”.
To copy partition tables structure from existing sda to sdb run below command
# sfdisk -d /dev/sda | sfdisk /dev/sdb
After partitions are configured on the newly installed hard drive, we can add the partitions to the RAID arrays.
Add for partition md0
# mdadm --manage /dev/md0 --add /dev/sdb1 mdadm: added /dev/sdb1
Repeat this command for each partition by changing /dev/md# and /dev/sdb#:
# mdadm --manage /dev/md1 --add /dev/sdb2 mdadm: added /dev/sdb2
# mdadm --manage /dev/md2 --add /dev/sdb3 mdadm: added /dev/sdb3
Now we can check that the partitions are being synchronized by issuing:
# cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sdb1 sda1 102336 blocks super 1.0 [2/2] [UU] md2 : active raid1 sdb3 sda3 233147200 blocks super 1.1 [2/1] [U_] [>....................] recovery = 0.3% (968342/233147200) finish=14.9 min speed=252264K/sec bitmap: 2/2 pages [8KB], 65565KB chunk md1 : active raid1 sdb2 sda2 1037000 blocks super 1.1 [2/2] [UU]
Once all drives have synchronized your RAID array will be back to normal.
4. Configure GRUB to the new hard drive MBR
We need to install GRUB (short for GNU GRand Unified Bootloader) on the new drive’s MBR (short for Master Boot Record). This is so that if the other drive fails the new drive will be able to boot into the operating system.
Enter the GRUB command line:
# grub Locate grub setup files: grub> find /grub/stage1 find /grub/stage1 (hd8,0) (hd9,0) Install grub on the MBR: grub> device (hd8) /dev/sdb grub> root (hd8,0) grub> setup (hd8) grub> quit