++++++++++++++++++++++++++++++++++++++
use raidtools to restore failed array
command: raidhotremove, raidhotadd
situation: one of array is failed (for example: /dev/sdb3)
+++++++++++++++++++++++++++++++++++++++
#cat /proc/mdstat
md1 : active raid1 sdb2[1] sda2[0]
20482752 blocks [2/2] [UU]
md2 : active raid1 sdb3[1][F] sda3[0]
20482752 blocks [2/2] [U_]
md3 : active raid1 sdb5[1] sda5[0]
10241280 blocks [2/2] [UU]
md4 : active raid1 sdb6[1] sda6[0]
10241280 blocks [2/2] [UU]
md5 : active raid1 sdb7[1] sda7[0]
5116544 blocks [2/2] [UU]
md0 : active raid1 sdb8[1] sda8[0]
2096384 blocks [2/2] [UU]
md6 : active raid1 sdb9[1] sda9[0]
2048192 blocks [2/2] [UU]
++++++++++++++++++++++++++++++++++++++++
The sdb3 is failed, before to restore it you might should use fsck or fsck.ext3(2) to fix partion like this:
++++++++++++++++++++++++++++++++++++++++
#fsck /dev/sdb3
+++++++++++++++++++++++++++++++++++++++
Then use command to restore array
+++++++++++++++++++++++++++++++++++++++
#raidhotremove /dev/md2 /dev/sdb3
#raidhotadd /dev/md2 /dev/sdb3
#watch -n1 cat /proc/mdstat (you will see)
md2 : active raid1 hdb3[2] hda3[1]
119684160 blocks [2/1] [_U]
[>………………..] recovery = 0.2% (250108/119684160)
finish=198.8min speed=10004K/sec after few minutes #cat /proc/mdstat
md1 : active raid1 sdb2[1] sda2[0]
20482752 blocks [2/2] [UU]
md2 : active raid1 sdb3[1] sda3[0]
20482752 blocks [2/2] [UU]
md3 : active raid1 sdb5[1] sda5[0]
10241280 blocks [2/2] [UU]
md4 : active raid1 sdb6[1] sda6[0]
10241280 blocks [2/2] [UU]
md5 : active raid1 sdb7[1] sda7[0]
5116544 blocks [2/2] [UU]
md0 : active raid1 sdb8[1] sda8[0]
2096384 blocks [2/2] [UU]
md6 : active raid1 sdb9[1] sda9[0]
2048192 blocks [2/2] [UU]
It’s success to restored