OT: Cannot setup raid1 for data under Debian

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

OT: Cannot setup raid1 for data under Debian

Robert Rozman
Hi,

I hope you won't get mad for posting OT. I'm banging my head against the
wall trying to setup raid1 with 2 identical
disks - it works when mounted manually, but it won't restore after boot...

I followed this procedure :

mdadm --zero-superblock /dev/sda1
mdadm --zero-superblock /dev/sdb1

mdadm --create
/dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
/dev/sdb1

mkfs.ext3 /dev/md0

echo "DEVICES /dev/sda1 /dev/sdb1" > /etc/mdadm/mdadm.conf
mdadm --brief --detail --verbose /dev/md0 >> /etc/mdadm/mdadm.conf

-----------------------
then added this entry into fstab :
/dev/md0       /raid1           ext3    defaults,user_xattr 0       2

then added two kernel modules to load : raid1, sd_mod

but after reboot - it stops in console on fsck saying :
fsck.ext3: Invalid argument while trying to open /dev/md0
/dev/md0:
The superblock could not be read or does not descibe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt. and you might try running e2fsck with an alternate
superblock:
 e2fsck -b 8193 <device>

Also when I try : mount /raid1 it gets same error.
But if I do this :
 mdadm --create
/dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
/dev/sdb1

then mount /raid1 works.... I just cannot get raid1 to start by it self...

any advice ?

Thanks in advance,

regards,

Rob.





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Matthew Williams-2
Did you make sure to include the raid-related kernel options inside the
kernel proper and not as modules?

Matt

Robert Rozman wrote:
> Hi,
>
> I hope you won't get mad for posting OT. I'm banging my head against the
> wall trying to setup raid1 with 2 identical
> disks - it works when mounted manually, but it won't restore after boot...

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Nigel Titley
In reply to this post by Robert Rozman
Robert Rozman wrote:

> Hi,
>
> I hope you won't get mad for posting OT. I'm banging my head against
> the wall trying to setup raid1 with 2 identical
> disks - it works when mounted manually, but it won't restore after
> boot...

I run my Misterhouse box with two identical RAID disks, just like you.
I'll make some comments below. First off I labelled the partitions as
RAID (I think the kernel needs this to actually detect that they are
RAID at bootup)

>
> I followed this procedure :
>
> mdadm --zero-superblock /dev/sda1
> mdadm --zero-superblock /dev/sdb1
>
Didn't use this at all

> mdadm --create
> /dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
> /dev/sdb1

mdadm --create /dev/md0 --level=1 /dev/sda1 /dev/sdb1

most of the defaults should be fine. I don't *think* you need the --auto
flag.

>
> mkfs.ext3 /dev/md0

fine

>
> echo "DEVICES /dev/sda1 /dev/sdb1" > /etc/mdadm/mdadm.conf
> mdadm --brief --detail --verbose /dev/md0 >> /etc/mdadm/mdadm.conf
>
mdadm doesn't need a config file these days so this is unnecessary

> -----------------------
> then added this entry into fstab :
> /dev/md0       /raid1           ext3    defaults,user_xattr 0       2

Ok

>
> then added two kernel modules to load : raid1, sd_mod

Shouldn't need this.

>
> but after reboot - it stops in console on fsck saying :
> fsck.ext3: Invalid argument while trying to open /dev/md0
> /dev/md0:
> The superblock could not be read or does not descibe a correct ext2
> filesystem. If the device is valid and it really contains an ext2
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt. and you might try running e2fsck with an alternate
> superblock:
> e2fsck -b 8193 <device>
>
> Also when I try : mount /raid1 it gets same error.
> But if I do this :
> mdadm --create
> /dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
> /dev/sdb1
>
> then mount /raid1 works.... I just cannot get raid1 to start by it
> self...
>
I'm pretty sure that you need to label the partitions as RAID. Basically
what is happening is that there is nothing to tell linux that the
partitions are RAID, so it doesn't bother to bind them together at boot
time. This is why you are having to do it by hand each time.

A minor comment, you are connecting both disks to the same IDE bus (one
as master and one as slave). This has performance and reliability
implications. Performance because every command to the slave has to go
through the master, and also because when reading from the disks the md
driver will read blocks alternately from each disk, which gives a
performance improvement over plain disks . Reliability for the same
reason, but also because if the IDE channel fails (or the bus shorts)
then you lose both disks together.

I have it set up as each disk running as master on the two built in IDE
channels. The CD runs as a slave on the first channel. I have both /boot
and / set up as type 1 RAID.

This is all on Debian Sarge, by the way.

Hope this helps

Nigel


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Nigel Titley
In reply to this post by Matthew Williams-2
Matthew Williams wrote:

>Did you make sure to include the raid-related kernel options inside the
>kernel proper and not as modules?
>  
>
I don't think he's booting off it, so it shouldn't make any difference.
Most modern distos have it built in anyway.

Nigel


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Robert Rozman
In reply to this post by Nigel Titley

----- Original Message -----
From: "Nigel Titley" <[hidden email]>
To: <[hidden email]>
Sent: Friday, February 10, 2006 4:58 PM
Subject: Re: [mh] OT: Cannot setup raid1 for data under Debian


> Robert Rozman wrote:
>
>> Hi,
>>
>> I hope you won't get mad for posting OT. I'm banging my head against the
>> wall trying to setup raid1 with 2 identical
>> disks - it works when mounted manually, but it won't restore after
>> boot...
>
> I run my Misterhouse box with two identical RAID disks, just like you.
> I'll make some comments below. First off I labelled the partitions as RAID
> (I think the kernel needs this to actually detect that they are RAID at
> bootup)
>
>>
>> I followed this procedure :
>>
>> mdadm --zero-superblock /dev/sda1
>> mdadm --zero-superblock /dev/sdb1
>>
> Didn't use this at all
>
>> mdadm --create
>> /dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
>> /dev/sdb1
>
> mdadm --create /dev/md0 --level=1 /dev/sda1 /dev/sdb1
>
> most of the defaults should be fine. I don't *think* you need the --auto
> flag.
>
>>
>> mkfs.ext3 /dev/md0
>
> fine
>
>>
>> echo "DEVICES /dev/sda1 /dev/sdb1" > /etc/mdadm/mdadm.conf
>> mdadm --brief --detail --verbose /dev/md0 >> /etc/mdadm/mdadm.conf
>>
> mdadm doesn't need a config file these days so this is unnecessary
>
>> -----------------------
>> then added this entry into fstab :
>> /dev/md0       /raid1           ext3    defaults,user_xattr 0       2
>
> Ok
>
>>
>> then added two kernel modules to load : raid1, sd_mod
>
> Shouldn't need this.
>

I was advised to. Remember I boot from other disk, I'd just like to have
this Raid1 for data on some mount point.

>>
>> but after reboot - it stops in console on fsck saying :
>> fsck.ext3: Invalid argument while trying to open /dev/md0
>> /dev/md0:
>> The superblock could not be read or does not descibe a correct ext2
>> filesystem. If the device is valid and it really contains an ext2
>> filesystem (and not swap or ufs or something else), then the superblock
>> is corrupt. and you might try running e2fsck with an alternate
>> superblock:
>> e2fsck -b 8193 <device>
>>
>> Also when I try : mount /raid1 it gets same error.
>> But if I do this :
>> mdadm --create
>> /dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
>> /dev/sdb1
>>
>> then mount /raid1 works.... I just cannot get raid1 to start by it
>> self...
>>
> I'm pretty sure that you need to label the partitions as RAID. Basically
> what is happening is that there is nothing to tell linux that the
> partitions are RAID, so it doesn't bother to bind them together at boot
> time. This is why you are having to do it by hand each time.
>
I have done it - if I get it right - partitions are setup to be $fd type
(raid autodetectable...)


> A minor comment, you are connecting both disks to the same IDE bus (one as
> master and one as slave). This has performance and reliability
> implications. Performance because every command to the slave has to go
> through the master, and also because when reading from the disks the md
> driver will read blocks alternately from each disk, which gives a
> performance improvement over plain disks . Reliability for the same
> reason, but also because if the IDE channel fails (or the bus shorts) then
> you lose both disks together.
>
> I have it set up as each disk running as master on the two built in IDE
> channels. The CD runs as a slave on the first channel. I have both /boot
> and / set up as type 1 RAID.
>
> This is all on Debian Sarge, by the way.

The problem lies only in fact that I have to manually do this :

mdadm --create
/dev/md0 --auto --verbose  --chunk=4 --level=1 --raid-devices=2 /dev/sda1
/dev/sdb1

and then can mount manually and it works - but I'd like to do it
automatically ....

Thanks in advance,

regards,

Rob.


>
> Hope this helps
>
> Nigel
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> ________________________________________________________
> To unsubscribe from this list, go to:
> http://sourceforge.net/mail/?group_id=1365
>



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Gaetan Lord
In reply to this post by Robert Rozman
Robert Rozman wrote:
 > Hi,
 >
 > I hope you won't get mad for posting OT. I'm banging my head against
the wall trying to setup raid1 with 2 identical
 > disks - it works when mounted manually, but it won't restore after
boot...
 >
 >

Rob, I don't use raid but I just want to provide another way to do
backup

Raid1 is nice, because it allow you to get an exact copy of a drive in
realtime, but ....

- This is software raid, so the system is actually doing the mirror, for
  every disk access.

- The mirroring occurs only when the system is live, the system is not
  constraint to backup schedule, so it could be shutdown, but useless with
  misterhouse

- Many people use raid1, to be safe if they have a disk crash, but there is
  no safety for user error. So, if you erase a file, then it is erased on
  both drive.

What I think is the most important with misterhouse, is the code we write,
and not the operating system. On the operating system side, the most
important
is the configuration.  This is a very small amount of data, compare to the
whole disk content. I forget to add, personal files like pictures and MP3
which are also important.

I prefer to use an approach with "rsync", which allow me the following

- The backup is done only once a day, when there is no activity on my
system.

- Rsync copy on the second drive, only what was change (the difference),
  and not the whole content of the drive. So, if you have a 20Mb log file,
  and you change a single bit, it will not copy the 20Mb on the second drive
  but only a single block of data containing the change. This increase the
  performance a lot.

- Because it is done once a day, I still have a copy from last night, if
I do
  any "user" mistake.

- Finally, because rsync could work in client-server mode. I do a secure
  rsync copy of the most important files, to a remote system. It's nice
  to have a second local copy of your drive, but if your system is burn
  or flood with water, then, none of them might be usable. I have a friend
  with ssh port open on his linux box. So, I rsync my important file once
  a night to his system. And again because it transfer only the difference,
  the backup usually doesn't take more than 2-3 minutes.

Just my 2C

I'm in vacation soon, and will put a lot of doc in the wiki page. I will
take time to document this, and also include my script to handle the rsync
procedure.


 



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Nigel Titley
Gaetan Lord wrote:

> Rob, I don't use raid but I just want to provide another way to do
> backup
>
Well, he never said he wanted RAID for backup. Maybe like me he wants
his Misterhouse system to keep running even if he has a disk crash.
Having said that I agree with everything you say about RAID not being a
substitute for backups, which is why all my home systems are backed up
to tape as well as running RAID on the vital systems (like my mailserver
and the Misterhouse box (if Misterhouse fails then my heating system
stops working, and my front door locks don't work)).

> Raid1 is nice, because it allow you to get an exact copy of a drive in
> realtime, but ....
>
> - This is software raid, so the system is actually doing the mirror, for
>  every disk access.

This actually speeds up read access as the md driver will read from the
two raid disks alternately.

>
> - The mirroring occurs only when the system is live, the system is not
>  constraint to backup schedule, so it could be shutdown, but useless with
>  misterhouse

That is why he wants his RAID system to come up at system start. RAID is
only useful if it runs all the time.

>
> - Many people use raid1, to be safe if they have a disk crash, but
> there is
>  no safety for user error. So, if you erase a file, then it is erased on
>  both drive.

Absolutely agreed.

>
> What I think is the most important with misterhouse, is the code we
> write,
> and not the operating system. On the operating system side, the most
> important
> is the configuration.  This is a very small amount of data, compare to
> the
> whole disk content. I forget to add, personal files like pictures and MP3
> which are also important.
>
> I prefer to use an approach with "rsync", which allow me the following
>
Different problem You are talking about backup, he is talking about
hardware failure (which is the only thing that RAID 1 is good for).

All the best

Nigel


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Nigel Titley
In reply to this post by Robert Rozman
Robert Rozman wrote:

>
> I was advised to. Remember I boot from other disk, I'd just like to
> have this Raid1 for data on some mount point

No problem, it does not harm.

> .
> The problem lies only in fact that I have to manually do this :
>
> mdadm --create /dev/md0 --auto --verbose  --chunk=4 --level=1
> --raid-devices=2 /dev/sda1 /dev/sdb1
>
> and then can mount manually and it works - but I'd like to do it
> automatically ....

I think your problem is that you are zeroing the persistent superblock.
What version of kernel, and mdadm are you using? You must not zero the
persistent superblock if you want the array to be auto-detected at startup.

Nigel


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365

Reply | Threaded
Open this post in threaded view
|

Re: OT: Cannot setup raid1 for data under Debian

Robert Rozman
In reply to this post by Gaetan Lord

----- Original Message -----
From: "Gaetan Lord" <[hidden email]>
To: <[hidden email]>
Sent: Friday, February 10, 2006 6:42 PM
Subject: Re: [mh] OT: Cannot setup raid1 for data under Debian


> Robert Rozman wrote:
> > Hi,
> >
> > I hope you won't get mad for posting OT. I'm banging my head against
> the wall trying to setup raid1 with 2 identical
> > disks - it works when mounted manually, but it won't restore after
> boot...
> >
> >
>
> Rob, I don't use raid but I just want to provide another way to do
> backup
>
> Raid1 is nice, because it allow you to get an exact copy of a drive in
> realtime, but ....
>
> - This is software raid, so the system is actually doing the mirror, for
>  every disk access.
>
> - The mirroring occurs only when the system is live, the system is not
>  constraint to backup schedule, so it could be shutdown, but useless with
>  misterhouse
>
> - Many people use raid1, to be safe if they have a disk crash, but there
> is
>  no safety for user error. So, if you erase a file, then it is erased on
>  both drive.
>
> What I think is the most important with misterhouse, is the code we write,
> and not the operating system. On the operating system side, the most
> important
> is the configuration.  This is a very small amount of data, compare to the
> whole disk content. I forget to add, personal files like pictures and MP3
> which are also important.
>
> I prefer to use an approach with "rsync", which allow me the following
>
> - The backup is done only once a day, when there is no activity on my
> system.
>
> - Rsync copy on the second drive, only what was change (the difference),
>  and not the whole content of the drive. So, if you have a 20Mb log file,
>  and you change a single bit, it will not copy the 20Mb on the second
> drive
>  but only a single block of data containing the change. This increase the
>  performance a lot.
>
> - Because it is done once a day, I still have a copy from last night, if I
> do
>  any "user" mistake.
>
> - Finally, because rsync could work in client-server mode. I do a secure
>  rsync copy of the most important files, to a remote system. It's nice
>  to have a second local copy of your drive, but if your system is burn
>  or flood with water, then, none of them might be usable. I have a friend
>  with ssh port open on his linux box. So, I rsync my important file once
>  a night to his system. And again because it transfer only the difference,
>  the backup usually doesn't take more than 2-3 minutes.
>
> Just my 2C
>
> I'm in vacation soon, and will put a lot of doc in the wiki page. I will
> take time to document this, and also include my script to handle the rsync
> procedure.
>
Hi,

the way you described is I think even better. Please keep us posted with
more info....

Regards,

Rob.

>
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> ________________________________________________________
> To unsubscribe from this list, go to:
> http://sourceforge.net/mail/?group_id=1365
>



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
________________________________________________________
To unsubscribe from this list, go to: http://sourceforge.net/mail/?group_id=1365