Replace failed drive in linux software raid 1

With 4 easy steps replace broken HDD on Linux machine which is configured for software raid1.

Consider a Linux system with software RAID1  drive named /dev/sda and /dev/sdb, among these two drive one drive has failed.

1. Identify failed drive.

How to Identify which drive has failed?

To identify which RAID array has failed, look at the string containing [UU]. Each “U” represents a healthy partition in the RAID array. If you see [UU] the array is healthy. If you see a missing “U” like [U_] then the RAID array is degraded or faulty.

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0]
102336 blocks super 1.0 [2/1] [U_]

md2 : active raid1 sda3[0]
233147200 blocks super 1.1 [2/1] [U_]
bitmap: 2/2 pages [8KB], 65565KB chunk

md1 : active (auto-read-only) raid1 sda2[0]
1037000 blocks super 1.1 [2/1] [U_]

Above output shows, One “U” is missing in RAID arrays md0, md1, and md2   of drive “sdb” so we have found “disk sdb” is faulty.

2. Remove failed drive.

Before removing drive we have to mark drive partition failed and then remove. To do that run below command.

To mark the partitions “fail” from the RAID array:

# mdadm --manage /dev/md0 --fail /dev/sdb1 
# mdadm --manage /dev/md1 --fail /dev/sdb2 
# mdadm --manage /dev/md2 --fail /dev/sdb3

To remove the partitions from the RAID array:

# mdadm --manage /dev/md0 --remove /dev/sdb1 
# mdadm --manage /dev/md1 --remove /dev/sdb2 
# mdadm --manage /dev/md2 --remove /dev/sdb3

Now you can power off the system and physically replace the defective drive:

# shutdown -h now

3. Adding new drive.

After installing new hard drive, verify new drive detected by the system and has same device name ie sdb to verify run the command.

"fdisk -l /dev/sdb"

After confirming we have to add the new drive to raid array.
Also, we have to create same partition table structure which was on the old drive.
To do that we can refer existing partition tables structure from drive “sda”.

To copy partition tables structure from existing sda to  sdb run below command

# sfdisk -d /dev/sda | sfdisk /dev/sdb

After partitions are configured on the newly installed hard drive, we can add the partitions to the RAID arrays.
Add for partition md0

# mdadm --manage /dev/md0 --add /dev/sdb1 
mdadm: added /dev/sdb1

Repeat this command for each partition by changing /dev/md# and /dev/sdb#:

# mdadm --manage /dev/md1 --add /dev/sdb2 
mdadm: added /dev/sdb2
# mdadm --manage /dev/md2 --add /dev/sdb3 
mdadm: added /dev/sdb3

Now we can check that the partitions are being synchronized by issuing:

# cat /proc/mdstat 
Personalities : [raid1] 
md0 : active raid1 sdb1[2] sda1[0] 
102336 blocks super 1.0 [2/2] [UU] 

md2 : active raid1 sdb3[2] sda3[0] 
233147200 blocks super 1.1 [2/1] [U_] 
[>....................] recovery = 0.3% (968342/233147200) finish=14.9 min speed=252264K/sec 
bitmap: 2/2 pages [8KB], 65565KB chunk

md1 : active raid1 sdb2[2] sda2[0] 
1037000 blocks super 1.1 [2/2] [UU]

Once all drives have synchronized your RAID array will be back to normal.

4. Configure GRUB to the new hard drive MBR

We need to install GRUB (short for GNU GRand Unified Bootloader) on the new drive’s MBR (short for Master Boot Record). This is so that if the other drive fails the new drive will be able to boot into the operating system.

Enter the GRUB command line:

# grub

Locate grub setup files:

grub> find /grub/stage1
find /grub/stage1
(hd8,0)
(hd9,0)

Install grub on the MBR:

grub> device (hd8) /dev/sdb
grub> root (hd8,0)
grub> setup (hd8)
grub> quit

Author: Sudo Admin

IT Admin

Leave a Reply

avatar
  Subscribe  
Notify of