If the drive should be replaced, then
1. start with determining which one is broken. See details of md device
# mdadm –detail /dev/mdX
# smartctl -a /dev/sdY
smartmontools required to be installed
2. replace required drive physically
3. See if the drive appeared
# lshw | grep /dev/sd[a-z]$ -A 3 -B 3
4. Careful! Transfer partition information from working disk to new installed disk
# sfdisk -d /dev/SOURCEDISK | sfdsik /dev/TARGETDISK
if source disk is sda for example, target sdb, then:
# sfdisk -d /dev/sda | sfdsik /dev/sdb
5. add partitions to md drives
# mdadm /dev/mdX –add /dev/sdY1
6. enjoy to watch progress of rebuild
$ watch -n 10 cat /proc/mdstat
7. If failed drive is not removed automatically from mdadm configuration, remove it by “–remove” command. It might require to set that drive failed, read man mdadm how to do it.
8. If mdX device is bootable device, then it is needed to cause system to boot from devices in mdX. In case grub is used, then:
# grub-install /dev/sdY