Results 1 to 17 of 17
  1. #1
    Join Date
    Aug 2010
    Posts
    231

    how to fix degraded Soft Raid?

    Hi,

    How can i fix this degraded raid? Someone can help me to figure out?

    ...but i am not sure i am adding the correct drive / partition. Possible for someone to confirm the command i need to use for this?

    Thank you,



    [[email protected] ~]# mdadm --detail /dev/md*
    mdadm: /dev/md does not appear to be an md device
    /dev/md2:
    Version : 0.90
    Creation Time : Sun Nov 3 01:42:24 2013
    Raid Level : raid1
    Array Size : 307198912 (292.97 GiB 314.57 GB)
    Used Dev Size : 307198912 (292.97 GiB 314.57 GB)
    Raid Devices : 2
    Total Devices : 1
    Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Wed Dec 18 12:31:45 2013
    State : clean, degraded
    Active Devices : 1
    Working Devices : 1
    Failed Devices : 0
    Spare Devices : 0

    UUID : 92707d2b:a91fF102:a4d2adc2:26fd5302
    Events : 0.601403

    Number Major Minor RaidDevice State
    0 8 2 0 active sync /dev/sda2
    1 0 0 1 removed
    /dev/md4:
    Version : 0.90
    Creation Time : Sun Nov 3 01:42:30 2013
    Raid Level : raid1
    Array Size : 1629926336 (1554.42 GiB 1669.04 GB)
    Used Dev Size : 1629926336 (1554.42 GiB 1669.04 GB)
    Raid Devices : 2
    Total Devices : 1
    Preferred Minor : 4
    Persistence : Superblock is persistent

    Update Time : Wed Dec 18 12:31:52 2013
    State : active, degraded
    Active Devices : 1
    Working Devices : 1
    Failed Devices : 0
    Spare Devices : 0

    UUID : 731d0588:afdc5dc4:a4d2add2:26fb5352
    Events : 0.3394981

    Number Major Minor RaidDevice State
    0 8 4 0 active sync /dev/sda4
    1 0 0 1 removed








    [[email protected] ~]# cat /proc/mdstat
    Personalities : [raid1]
    md4 : active raid1 sda4[0]
    1629926336 blocks [2/1] [U_]

    md2 : active raid1 sda2[0]
    307198912 blocks [2/1] [U_]

    unused devices: <none>
    Last edited by martin33; 12-18-2013 at 01:43 PM.

  2. #2
    Join Date
    Mar 2003
    Location
    WebHostingTalk
    Posts
    16,963
    Moved > Hosting Security and Technology.
    Specially 4 You
    .
    JoneSolutions.Com ( Jones.Solutions ) is on the net 24/7 providing stable and reliable web hosting solutions and services since 2001

  3. #3
    Join Date
    Aug 2010
    Posts
    231
    Hi,

    Sorry if i posted at the wrong place

    Someone can tell me what to do exactly to fix this?

    Thank you,

    Martin

  4. #4
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    First step would be to run some tests to see if the /dev/sdb disk is failing. You wouldn't want to reinsert a bad disk.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  5. Try to run a SMART check on the drive.
    If it really fails, you will have to remove the drive from the md array. Install a new drive, copy clean drive partitions to the new drive, and then reinsert the new drive to the md array.
    Hostabulous | cPanel (Linux) & Plesk (Windows) Hosting KVM VPS R1Soft backups | Proudly Canadian
    Cloudflare LiteSpeed Cloudlinux Remote backups Anti-Spam Web App Firewall Canada/US/Germany

  6. #6
    Join Date
    Aug 2010
    Posts
    231
    Hi,

    The disk was changed by the ovh support. I already ran a disk check. The 2 disks are fine.

    Now they said i have to re-add the disk to the raid array.

    But they don't want to provide any command for this - they said they don't maintain software.

    So i am stuck. I don't know a lot about raid.

    Here is also a parted -l in case if that may help :


    parted -l
    Model: ATA ST2000DM001-9YN1 (scsi)
    Disk /dev/sda: 2000GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: gpt

    Number Start End Size File system Name Flags
    1 20,5kB 1049kB 1029kB primary bios_grub
    2 2097kB 315GB 315GB ext4 primary raid
    3 315GB 331GB 16,8GB linux-swap(v1) primary raid
    4 331GB 2000GB 1669GB logical raid


    Error: /dev/sdb: unrecognised disk label

    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1011-real: 16,1GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 9665MB 9664MB primary ext4
    2 9665MB 10,7GB 1073MB primary linux-swap(v1)


    Error: /dev/mapper/vm-ntheskzz9a-cow: unrecognised disk label

    Model: Linux device-mapper (snapshot) (dm)
    Disk /dev/mapper/vm-ntheskzz9a: 16,1GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 9665MB 9664MB primary ext4
    2 9665MB 10,7GB 1073MB primary linux-swap(v1)


    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1012: 16,1GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 8591MB 8590MB primary ext4
    2 8591MB 10,7GB 2146MB primary linux-swap(v1)


    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1008: 16,1GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 8591MB 8590MB primary ext4
    2 8591MB 10,7GB 2146MB primary linux-swap(v1)


    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1010: 10,7GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 9665MB 9664MB primary ext4
    2 9665MB 10,7GB 1073MB primary


    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1002: 10,7GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 9665MB 9664MB primary ext4
    2 9665MB 10,7GB 1073MB primary linux-swap(v1)


    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1004: 21,5GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 21,5GB 21,5GB primary ext4


    Model: Linux device-mapper (snapshot-origin) (dm)
    Disk /dev/mapper/vm-vs1011: 16,1GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 9665MB 9664MB primary ext4
    2 9665MB 10,7GB 1073MB primary linux-swap(v1)


    Model: Linux device-mapper (linear) (dm)
    Disk /dev/mapper/vm-vs1006: 10,7GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1049kB 10,7GB 10,7GB primary ext4


    Model: Unknown (unknown)
    Disk /dev/md2: 315GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: loop

    Number Start End Size File system Flags
    1 0,00B 315GB 315GB ext4


    Error: /dev/md4: unrecognised disk label




    and again the other commands, just to clarify for the next post :

    [[email protected] ~]# mdadm --detail /dev/md*
    mdadm: /dev/md does not appear to be an md device
    /dev/md2:
    Version : 0.90
    Creation Time : Sun Nov 3 01:42:24 2013
    Raid Level : raid1
    Array Size : 307198912 (292.97 GiB 314.57 GB)
    Used Dev Size : 307198912 (292.97 GiB 314.57 GB)
    Raid Devices : 2
    Total Devices : 1
    Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Wed Dec 18 12:31:45 2013
    State : clean, degraded
    Active Devices : 1
    Working Devices : 1
    Failed Devices : 0
    Spare Devices : 0

    UUID : 92707d2b:a91fF102:a4d2adc2:26fd5302
    Events : 0.601403

    Number Major Minor RaidDevice State
    0 8 2 0 active sync /dev/sda2
    1 0 0 1 removed
    /dev/md4:
    Version : 0.90
    Creation Time : Sun Nov 3 01:42:30 2013
    Raid Level : raid1
    Array Size : 1629926336 (1554.42 GiB 1669.04 GB)
    Used Dev Size : 1629926336 (1554.42 GiB 1669.04 GB)
    Raid Devices : 2
    Total Devices : 1
    Preferred Minor : 4
    Persistence : Superblock is persistent

    Update Time : Wed Dec 18 12:31:52 2013
    State : active, degraded
    Active Devices : 1
    Working Devices : 1
    Failed Devices : 0
    Spare Devices : 0

    UUID : 731d0588:afdc5dc4:a4d2add2:26fb5352
    Events : 0.3394981

    Number Major Minor RaidDevice State
    0 8 4 0 active sync /dev/sda4
    1 0 0 1 removed







    [[email protected] ~]# cat /proc/mdstat
    Personalities : [raid1]
    md4 : active raid1 sda4[0]
    1629926336 blocks [2/1] [U_]

    md2 : active raid1 sda2[0]
    307198912 blocks [2/1] [U_]

    unused devices: <none>
    Last edited by martin33; 12-21-2013 at 11:21 PM.

  7. #7
    Join Date
    Aug 2010
    Posts
    231
    I know how to get the infos required to fix this. ...but i don't know how to fix Please help

    In other datacenters, it was always the datacenter support who was performing this kind of steps.

  8. #8
    Join Date
    Aug 2010
    Posts
    231
    Here is what i am going to use...

    mdadm /dev/md4 --manage --add /dev/sdb1

    Am i ok with this command or if i will whipe out the disk? :-P

    Just need to confirm.

  9. #9
    This is normal for OVH, the idea of OVH is, you need the hardware, the network, but not the hand holding of a managed server.

    They are great if you know what you are doing, if you don't, well.. you could have problems. (Such as this)

    I will try to help you learn how to rebuild an RAID array

    Disclaimer: Generally, this should work. If you encounter any errors, stop immediately and figure out the error and what is causing it. I am not responsible for any data loss etc etc...

    Anyhow.. this should get you up and running.

    Step 1: Partition Layouts

    You will need to ensure both drives have the same partition layout if you do not then you cannot add the drive to the array (correctly anyway).

    Code:
    $ sgdisk -R /dev/sdb /dev/sda
    Now ensure /dev/sdb and /dev/sda are the correct way around. You do not want this mixed up. Mix it up and the working drive will loose it's partition scheme (Not good).

    Syntax as follows:
    Code:
    sgdisk -R TARGET SOURCE
    Target being the new drive, and source being a working drive which is active in an array.

    Step 2: Partition GUIDs

    Now, you need to ensure your new drive and partitions have unique GUIDs. This is important because otherwise the array would see drive B as drive A and visa versa. It would probably just break.

    Code:
    sgdisk -G /dev/sdb
    This will generate new GUIDs on the selected drive. Do not do this on a drive already in an array.. ever.

    Step 3: Rebuilding Array
    Last thing you do, is add the drive back to the array.

    Now this you have to check with your current array. For example from your output:

    Code:
    [[email protected] ~]# cat /proc/mdstat
    Personalities : [raid1]
    md4 : active raid1 sda4[0]
    1629926336 blocks [2/1] [U_]
    
    md2 : active raid1 sda2[0]
    307198912 blocks [2/1] [U_]
    
    unused devices: <none>
    In your setup sda4 is part of md4 and sda2 is part of md2, this means you need to add sdb4 to md4 and sdb2 to md2.

    Simple really.

    So let's add them.

    Code:
    mdadm /dev/md2 -a /dev/sdb2
    mdadm /dev/md4 -a /dev/sdb4
    Step 4: Activate SWAP (Optional)
    If you want both drives to provide swap space (it was probably like this before) then you need to activate swap on /dev/sdb.

    This part is easy, from your output above I can see your swap partition is sda3 and sdb3.

    So simple is simple:
    Code:
    swapon /dev/sdb3
    Both drives should now have swap activated.

    Step 5: Check system
    Well, it's that time, just confirm everything worked and off you go.

    Now you should see your array rebuilding when running:

    Code:
    cat /proc/mdstat
    And running

    Code:
    free -m
    You should see the total swap between the two drives.

    If all checks out, you're good to go

    Hope this helps, enjoy!

  10. #10
    Join Date
    Aug 2010
    Posts
    231
    Ok!

    ...now i understand how it work

    I'm going to try.

    Thank you so much for this great tutorial Seems clear to me now!

  11. #11
    Join Date
    Aug 2010
    Posts
    231
    Hi,

    I followed the intructions but i got this for the swap :

    swapon /dev/sdb3
    swapon: /dev/sdb3: read swap header failed: Invalid argument

    All else is working real fine now. What can i do concerning the swap?

    This is not a big problem, since the server can run without, but...

    Thank you,

    Martin

  12. #12
    Join Date
    Aug 2010
    Posts
    231
    Edit :

    i posted a question, but i am fine now Found why, no need to answer, so i removed it

    Thank you!
    Last edited by martin33; 12-23-2013 at 11:28 PM.

  13. Probably because /dev/sdb3 is not your swap partition
    Hostabulous | cPanel (Linux) & Plesk (Windows) Hosting KVM VPS R1Soft backups | Proudly Canadian
    Cloudflare LiteSpeed Cloudlinux Remote backups Anti-Spam Web App Firewall Canada/US/Germany

  14. #14
    Join Date
    Mar 2005
    Location
    Ten1/0/2
    Posts
    2,509
    There is only 1 step that was actually missing that is critical and the very first step. BACKUP! Make sure you have a good backup that you can restore from should bad sh!t happen...
    CPanel Shared and Reseller Hosting, OpenVZ VPS Hosting. West Coast (LA) Servers and Nodes
    Running Linux since 1.0.8 Kernel!
    Providing Internet Services since 1995 and Hosting Since 2004

  15. #15
    Join Date
    Aug 2010
    Posts
    231
    Yes i do have backups, and yes **** is happening.

    What wrong exactly?

    It say this for sda :

    Model: ATA ST2000DM001-9YN1 (scsi)
    Disk /dev/sda: 2000GB
    Sector size (logical/physical): 512B/4096B
    Partition Table: gpt

    Number Start End Size File system Name Flags
    1 20,5kB 1049kB 1029kB primary bios_grub
    2 2097kB 315GB 315GB ext4 primary raid
    3 315GB 331GB 16,8GB linux-swap(v1) primary raid
    4 331GB 2000GB 1669GB logical raid

    ...so i also assumed sdb3 was going to be a swap partition.

    Now i can see i am loosing all data on the server.

    What can i do ?

  16. #16
    Join Date
    Aug 2010
    Posts
    231
    I think i know what's the only solution :

    to restore.
    Last edited by martin33; 12-24-2013 at 03:17 AM.

  17. #17
    Join Date
    Aug 2010
    Posts
    231
    Ok.
    Something else was missing in this tutorial :

    NEVER REBOOT WHILE YOU RUN MANUAL RE-SYNC!
    I didn't had to bare-metal I will still migrate this host just in case, cause it seems a little dirty to me now

    If ever you reboot during a sync then wait until the sync complete. It should start to sync again automatically. Wait even if your server is not 100% working (in our case the vm's were all offline, but the host was up). ...when the sync complete, reboot another time and it should be working fine.

    Also : the sync may take a very long time before it complete Even on the day after, you should have a check before you reboot, if you have large disks.

    All i had to do is restore the vm's. Fortunately, we do both local and remote backups frequently Going to migrate this to another host now ...just in case...

Similar Threads

  1. Replies: 19
    Last Post: 10-22-2013, 09:51 AM
  2. Degraded Raid - how can i fix this?
    By martin33 in forum Hosting Security and Technology
    Replies: 16
    Last Post: 09-16-2013, 03:37 AM
  3. Adaptec RAID 10 degraded but all HDD are reported as optimal
    By ServerZoo in forum Colocation and Data Centers
    Replies: 3
    Last Post: 10-24-2011, 01:46 AM
  4. Degraded soft raid 1
    By Lost Eagle in forum Hosting Security and Technology
    Replies: 14
    Last Post: 07-30-2011, 08:41 AM
  5. Degraded RAID Status - Third Time?
    By Technolojesus in forum Hosting Security and Technology
    Replies: 6
    Last Post: 10-08-2010, 10:03 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •