OK, this is more for self-education than anything else, though I do
have systems that I need to put software RAID on. 

Kernel is 2.4.25, mdadm (unknown version - I'm not near the machines).

I'm trying to get picture of how software RAID works and where it
stores its information.

With a normal disk the first track is not (I believe) part of any
filesystem and starts with the MBD+Partition Table (first block) and
with GRUB the next block is the GRUB stage1 and then following is the
stage1.5. After that I don't know. Presumably the Stage2 is loaded
from the RAID filesystem. I dunno!

With RAID1 there are two disks which presumably are identical from
track1 on, but I don't believe that track1 itself is actually copied
in a RAID. 

When I boot from a RAID, I presumably get the MBR PT and stage1 from
the boot disk. How does the "boot disk" get determined? I presume its
just the first disk with a bootable flag and is a physical disk and
not a RAID disk, since RAID is not up and running at this stage. At
some stage the RAID kicks in and the "persistent" superblock is
presumably accessed somewhere and used.

Does anyone know if this is documented anywhere? Does anyone know
where the RAID superblock lives? Anyone point out the errors in the
above story?

Cheers,

Cliff
-- 

The National Party manifesto can be viewed here:

http://www.labour.org.nz/policy/index.html

begin  Enkidu dedi ki:
  --8<--

> When I boot from a RAID, I presumably get the MBR PT and stage1 from
> the boot disk. How does the "boot disk" get determined? I presume its
> just the first disk with a bootable flag and is a physical disk and
> not a RAID disk, since RAID is not up and running at this stage. At
> some stage the RAID kicks in and the "persistent" superblock is
> presumably accessed somewhere and used.

Lilo only supports booting from RAID-1, and both disks are identical in
RAID-1 so no superblock need to be accessed to boot. Lilo puts MBR on
both disks so that whichever one you boot from, it can read in the
kernel and initrd image from the booted disk (not from the "array").
Only then the array is started.

This (identical disk images) doesn't hold true for RAID 0 or 5, so it is
not possible to boot directly from a RAID-5 array (i.e. /boot residing
in the array). For RAID-5, there should be a separate (single or
raid-1) /boot partition to boot the kernel from, but the root partition
can reside in the raid-5 array, an idea which I don't like due to
certain pitfalls (with their workarounds).

> Does anyone know if this is documented anywhere?

Linux RAID documentation is indeed sparse, but lilo docs should help on
this specific issue. There are also kernel and raid-tools docs and RAID
howto for general info.

-- 
Abdullah        | aramazan@ |
Ramazanoglu     | myrealbox |
________________| D.0.T c�m |__

On Tue, 28 Dec 2004 01:19:54 +0200, Abdullah Ramazanoglu
<abdullah@ramazanoglu.tr> wrote:

> Cliff asked:
>> Does anyone know if this is documented anywhere?
>
>Linux RAID documentation is indeed sparse, but lilo docs should help on
>this specific issue. There are also kernel and raid-tools docs and RAID
>howto for general info.
>
Hmm. Most of what I've been able to find is over 2 years old and
refers to raidtools. I'm using mdadm.

Cheers,

Cliff
-- 

The National Party manifesto can be viewed here:

http://www.labour.org.nz/policy/index.html

In article <1prus0lfcj0mcemm845sq9d2tng7jgkl2m@4ax.com>,
 Enkidu <enkidu@cliffp.com> wrote:

>When I boot from a RAID, I presumably get the MBR PT and stage1 from
>the boot disk. How does the "boot disk" get determined? I presume its
>just the first disk with a bootable flag and is a physical disk and
>not a RAID disk, since RAID is not up and running at this stage.

The boot order is determined by the BIOS--it simply scans all the 
devices it knows about in the specified order until it finds one with 
some marker that (presumably) says "I've got a valid boot record". Yes, 
it is possible to boot from RAID arrays in this way. I'm not sure how 
that works, but it's handled entirely within the BIOS--since the Linux 
kernel doesn't use the BIOS, it must have its own RAID driver module 
loaded, either in the kernel image or the initrd, otherwise it won't be 
able to access the drive after booting.

I have three different client machines that boot in this way, one from a 
RAID-5 array and the other two from RAID-1.

>At some stage the RAID kicks in and the "persistent" superblock is
>presumably accessed somewhere and used.

Once Linux gets control and has initialized the relevant drivers 
(whether built-in on or modules in the initrd), it can access the RAID 
array as it does any other disk.

>Does anyone know if this is documented anywhere? Does anyone know
>where the RAID superblock lives?

Presumably the RAID-dependent structures are internal to the RAID 
controller and driver--all the Linux filesystem code sees is the logical 
volume.

I found this document 
<http://www.uruk.org/orig-grub/PC_partitioning.txt> in a link from the 
original GRUB documentation. It describes partition tables, CHS 
addressing, the MBR, and all that gory stuff.

On Sat, 08 Jan 2005 15:52:59 +1300, Lawrence D'Oliveiro
<ldo@geek-central.gen.new_zealand> wrote:

>In article <1prus0lfcj0mcemm845sq9d2tng7jgkl2m@4ax.com>,
> Enkidu <enkidu@cliffp.com> wrote:
>
>>When I boot from a RAID, I presumably get the MBR PT and stage1 from
>>the boot disk. How does the "boot disk" get determined? I presume its
>>just the first disk with a bootable flag and is a physical disk and
>>not a RAID disk, since RAID is not up and running at this stage.
>
>The boot order is determined by the BIOS--it simply scans all the 
>devices it knows about in the specified order until it finds one with 
>some marker that (presumably) says "I've got a valid boot record". Yes, 
>it is possible to boot from RAID arrays in this way. I'm not sure how 
>that works, but it's handled entirely within the BIOS--since the Linux 
>kernel doesn't use the BIOS, it must have its own RAID driver module 
>loaded, either in the kernel image or the initrd, otherwise it won't be 
>able to access the drive after booting.
>
I'm talking about software RAID, not hardware RAID. As far as the BIOS
is concerned there is NO RAID controller, and just a lot of IDE (or
SCSI) disks The first disk must have an MBR and PT in the first block.

When the system boots, it uses the first disk's MBR and PT. Any
bootstrap program (lilo and grub) will reside in the subsequent
sectors of the HDD. To make the second disk bootable, it has to have
these sectors copied to it from the first HDD. 

The bootstrap program locates the kernel using a trimmed down file
system driver and this then loads the kernel. If the kernel has RAID
support built in, the RAID is autodetected or manually built at this
stage. If the root file system is on a RAIDed partition, the array is
built at this time and the root file system switched to it. 

This much I know. 

I also know that there are two ways that the information for building
the software array can be obtained. The first is a conf file and the
second is the RAID superblock.

Ordinary file system superblocks are found at the start of the
partition and at intervals over the whole disk (in the block groups
actually).

The RAID superblock must be elsewhere on the disk. This is referred to
as a "persistent superblock" and contains the information necessary to
rebuild the array among other things.

I've since been able to find out that the RAID superblock resides at
the end of a RAIDed partition, actually in the last complete 64kb set
of blocks.
>
>I have three different client machines that boot in this way, one from a 
>RAID-5 array and the other two from RAID-1.
>
Software RAID will only boot from a RAID-1 partition.
>
>>At some stage the RAID kicks in and the "persistent" superblock is
>>presumably accessed somewhere and used.
>
>Once Linux gets control and has initialized the relevant drivers 
>(whether built-in on or modules in the initrd), it can access the RAID 
>array as it does any other disk.
>
That's true of hardware RAID and to some extent of software RAID.
>
>>Does anyone know if this is documented anywhere? Does anyone know
>>where the RAID superblock lives?
>
>Presumably the RAID-dependent structures are internal to the RAID 
>controller and driver--all the Linux filesystem code sees is the logical 
>volume.
>
No, not for software RAID.
>
>I found this document 
><http://www.uruk.org/orig-grub/PC_partitioning.txt> in a link from the 
>original GRUB documentation. It describes partition tables, CHS 
>addressing, the MBR, and all that gory stuff.
>


Cheers,

Cliff
-- 

The National Party manifesto can be viewed here:

http://www.labour.org.nz/policy/index.html