There is an article on Linux.com that has a counter-intuitive result, it 
claims that RAID1 is faster than RAID0 and significantly faster than a 
single drive in a multitasking environment.

http://www.linux.com/feature/124256

There are a number of possible explanations for this, 1) the article is 
wrong. 2) Software RAID1 distributes read requests across both drives so 
that in a multitasking situation the disk subsystem can do twice as many 
read transactions. 3) Software RAID1 makes a simultaneous read request to 
both drives and then uses the first to return. 4) The software is able to 
to determine the head positions on both drives and then makes the read 
request to the drive that has it's head closest to the desired block.

Can anyone tell me what the RAID software is really doing?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

General Schvantzkopf wrote:
 > There is an article on Linux.com that has a counter-intuitive result,
 > it claims that RAID1 is faster than RAID0 and significantly faster
 > than a single drive in a multitasking environment.
 >
 > http://www.linux.com/feature/124256
 >
 > There are a number of possible explanations for this, 1) the article
 > is wrong. 2) Software RAID1 distributes read requests across both
 > drives so that in a multitasking situation the disk subsystem can do
 > twice as many read transactions. 3) Software RAID1 makes a
 > simultaneous read request to both drives and then uses the first to
 > return. 4) The software is able to to determine the head positions on
 > both drives and then makes the read request to the drive that has
 > it's head closest to the desired block.
 >
 > Can anyone tell me what the RAID software is really doing?

There are a number of faults with that study.

Firstly, unless the author used different hard disk controllers for the
then RAID1 should be at best equal to single drive performance.  In my
experience there is a very slight overhead that while is measurable is
not noticeable in most deployments.  I would suggest the source of this
authors unexplained results are more than likely down to 'sampling
error'.  If the author went through the trouble to be scientific and get
lots of samples then this type of error would be very unlikely to have
occurred.

Secondly, the author uses the the term 'significantly faster' very
flippantly.  Scientifically, something is 'significantly different' if a
manipulation by an experimenter reliably gives differences in samples
from the same population.  The size of the difference makes NO
contribution to whether something is 'significantly different'.
Furthermore, looking at the means between the samples that he presented
in the graphs (and he probably only did one sample per cohort!) although
there are probably 'significant' performance differences between the
means, that ACTUAL difference (the size of the difference) didn't look
like anything writing home about to me.  Ask yourself, if a single drive
takes 1 minute 43 seconds to do one task, is the fact that a RAID0 setup
would take 1 minute 35 seconds really a perceptible gain?  If that is a
perceptible gain, is it actually worth it given the cost and other
disadvantages?  (See my final point for disadvantages of RAID0.)

Thirdly while I could look up the fact that the hard disk he used were
7200rpm, he didn't test 10000rpm for an objective comparison.  There may
be further differences that are larger or possible smaller.

Fourthly real world difference, given the type of tasks he set and how
good modern hard disks are capable of deal with multiple-requests for
data, even if a desktop machine is laden with background tasks indexing
and prelinking for example, it is unlikely that most of the normal tasks
a desktop user does (document composing, checking email, browsing) would
be impacted.

Fifthly, proper RAID cards that are worth their salt probably start
around the price range of �75 and above; the one that came 'free' on
your motherboard is in a different class!  This is NOT the type of RAID
card we use out in the real world for proper setups!

Overall RAID0 for desktops is a really, really bad idea in the vast
majority of circumstances.  Minor performance gains are available to
people usually with double costs when you include a decent RAID card in
comparison to just buying a big and fast disk in the first place.  You
also draw lots of extra power (another hard disk and the RAID card needs
power too) and you DOUBLE the risk of losing your data to hard disk failure.

Don't be tempted by RAID0; buy a large fast hard disk and backup often
instead.  Read articles about fast hard disks or solid state drives
instead :)

- --
Regards,
Sheridan Hutchinson
Sheridan@Shezza.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHlL7GnBrliHqz8aARAi5AAJ48qv5i5CBERWHX148JV/Ikg1SBNgCePBQg
2ki6QwYDOEDXhOvhqG94jOM=
=j1x8
-----END PGP SIGNATURE-----

General Schvantzkopf <schvantzkopf@yahoo.com> writes:
>There are a number of possible explanations for this, 1) the article is 
>wrong. 2) Software RAID1 distributes read requests across both drives so 
>that in a multitasking situation the disk subsystem can do twice as many 
>read transactions. 3) Software RAID1 makes a simultaneous read request to 
>both drives and then uses the first to return. 4) The software is able to 
>to determine the head positions on both drives and then makes the read 
>request to the drive that has it's head closest to the desired block.

2 sounds most plausible to me.  4 and (to a lesser degree) 3 would be
cool, but I doubt that the md driver does this.  For 4 one would
probably not try to query the current head position; this is probably
not possible, and even if it is, is not a good idea, because the block
interface and (with command queuing) the disk interface is
asynchronous (it may serve other requests before serving the request
under consideration); instead, one might look at what requests have
been sent to the n block devices recently, and send the new request to
the device that has had the closest block request recently; however,
one has to balance this with load balancing.

I also note that one can configure the Barcelona-based AMD CPUs
(Phenoms and quad-core Opterons) to access the two memory channels in
an unganged fashion as an alternative to the traditional ganged setup.
In the unganged setup a cache line is read from only one channel,
which can have the following positive effects: the other channel is
free to serve a different request, and there are more different open
pages in the DIMMs (with the ganges setup each pair of DIMMs has the
same pages open).  From what I read, which setup is faster depends on
the workload, but for many multi-tasking loads the unganged setup is a
little faster.

In a similar vein, I note that the new ATI RV670 graphics chips
(Radeon 3850, 3870) reduces the bits in the memory interface compared
to the R600 (Radeon 2900) by keeping the same number (8) of memory
controllers/channels, but making each channel narrower (from 64 to 32
bits), and hardly loses performance from this.

- anton
-- 
M. Anton Ertl                    Some things have to be seen to be believed
anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
http://www.complang.tuwien.ac.at/anton/home.html

General Schvantzkopf <schvantzkopf@yahoo.com> writes:
>There is an article on Linux.com that has a counter-intuitive result, it 
>claims that RAID1 is faster than RAID0 and significantly faster than a 
>single drive in a multitasking environment.
>
>http://www.linux.com/feature/124256

Inspired by this, I did a little micro-benchmarking on a machine that
has both a software RAID0 and a software RAID1 on the same disks (but
of course, in different regions, so there may be differences coming
from that).  The disks are a WDC WD4000KD-55NSB2 and a ST3400620AS;
the kernel is the Debian 2.6.18-4-amd64 kernel.

First benchmark: Create 2 20GB files in parallel, with:

for i in 1 2; do time dd if=/dev/zero of=$i bs=1024k count=20480 & done

Results: the numbers produced by dd:

       process 1             process2
RAID1  869.537s (24.7 MB/s)  935.002s (23.0MB/s)
RAID0  583.512s (36.8 MB/s)  601.59s  (35.7MB/s)

No surprise here: RAID1 has to write to two disks, so it is slower.

Next benchmark: Read these files in parallel, with:

for i in 1 2; do time dd if=$i bs=1024k count=20480|wc -c & done

Results: the numbers produced by dd:

       process 1             process2
RAID1  696.336s (30.8MB/s)   703.792s (30.5MB/s)
RAID0  460.54s  (46.6MB/s)   466.837s (46.0MB/s)

So, even though it's theoretically possible to be as fast with RAID1
in this case as with RAID0, this does not happen.

However, things may be different for accesses that are seek-limited
instead of accesses like these that are bandwidth-limited.  It's a
little more involved to benchmark that, so I won't do that for now.

Just for comparison, when doing a similar test with a single large
file, I saw 47MB/s on the RAID1 and 89MB/s on the RAID0.

- anton
-- 
M. Anton Ertl                    Some things have to be seen to be believed
anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
http://www.complang.tuwien.ac.at/anton/home.html

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>However, things may be different for accesses that are seek-limited
>instead of accesses like these that are bandwidth-limited.  It's a
>little more involved to benchmark that, so I won't do that for now.

But I got around to it:  I created a 50GB file as follows:

dd if=/dev/zero of=x bs=1024k count=51200

Then I produced random accesses from multiple processes to this file,
e.g. as follows:

for i in 0 1 2 3 4 5 6 7 8; do /usr/bin/time -f %es gforth random.fs -e "9$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait

I.e., each process does 100 random accesses within the file.  I always
varied the seed to avoid accessing blocks that had already been
accessed.  Here's the raw data:

#proc   RAID1
1       1.98s
2       1.46s 1.53s
3       2.25s 2.41s 2.58s
4       2.79s 2.94s 2.96s 2.97s
5       3.23s 3.60s 3.76s 4.09s 4.15s
6       3.70s 4.21s 4.43s 4.67s 4.76s 4.77s
7       4.47s 5.13s 5.51s 5.62s 5.66s 5.74s 5.83s
8       5.65s 5.82s 5.84s 5.86s 5.98s 6.17s 6.19s 6.21s
9       6.67s 6.76s 6.84s 6.86s 6.97s 7.00s 7.20s 7.26s 7.43s

#proc   RAID0
1       1.60s
2       2.17s 2.21s
3       2.78s 3.28s 3.32s
4       3.58s 3.70s 4.03s 4.14s
5       3.92s 4.30s 4.36s 4.87s 4.92s
6       4.37s 4.62s 4.99s 5.14s 5.27s 5.34s
7       5.80s 5.97s 6.35s 6.37s 6.59s 6.79s 6.80s
8       5.64s 5.86s 6.06s 6.41s 6.63s 6.67s 6.85s 6.86s
9       6.60s 6.84s 6.98s 7.09s 7.21s 7.29s 7.31s 7.43s 7.48s

I used the same seeds for both file systems.

So, it looks like md RAID1 is faster than md RAID0 for seek-limited
workloads with a few (but more than 1) independent threads/processes.
It's not clear why this should be the case for such small accesses
(1K), 

Another relevant hardware detail on this setup: I think that native
command queuing is only available (and apparently enabled) on only one
of the two disk drives on the tested system.

- anton
-- 
M. Anton Ertl                    Some things have to be seen to be believed
anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
http://www.complang.tuwien.ac.at/anton/home.html

On Tue, 22 Jan 2008 20:27:54 +0000, Anton Ertl wrote:

> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>However, things may be different for accesses that are seek-limited
>>instead of accesses like these that are bandwidth-limited.  It's a
>>little more involved to benchmark that, so I won't do that for now.
> 
> But I got around to it:  I created a 50GB file as follows:
> 
> dd if=/dev/zero of=x bs=1024k count=51200
> 
> Then I produced random accesses from multiple processes to this file,
> e.g. as follows:
> 
> for i in 0 1 2 3 4 5 6 7 8; do /usr/bin/time -f %es gforth random.fs -e
> "9$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do
> 52428800 random block drop loop ; foo bye" & done; wait
> 
> I.e., each process does 100 random accesses within the file.  I always
> varied the seed to avoid accessing blocks that had already been
> accessed.  Here's the raw data:
> 
> #proc   RAID1
> 1       1.98s
> 2       1.46s 1.53s
> 3       2.25s 2.41s 2.58s
> 4       2.79s 2.94s 2.96s 2.97s
> 5       3.23s 3.60s 3.76s 4.09s 4.15s 6       3.70s 4.21s 4.43s 4.67s
> 4.76s 4.77s 7       4.47s 5.13s 5.51s 5.62s 5.66s 5.74s 5.83s 8      
> 5.65s 5.82s 5.84s 5.86s 5.98s 6.17s 6.19s 6.21s 9       6.67s 6.76s
> 6.84s 6.86s 6.97s 7.00s 7.20s 7.26s 7.43s
> 
> #proc   RAID0
> 1       1.60s
> 2       2.17s 2.21s
> 3       2.78s 3.28s 3.32s
> 4       3.58s 3.70s 4.03s 4.14s
> 5       3.92s 4.30s 4.36s 4.87s 4.92s 6       4.37s 4.62s 4.99s 5.14s
> 5.27s 5.34s 7       5.80s 5.97s 6.35s 6.37s 6.59s 6.79s 6.80s 8      
> 5.64s 5.86s 6.06s 6.41s 6.63s 6.67s 6.85s 6.86s 9       6.60s 6.84s
> 6.98s 7.09s 7.21s 7.29s 7.31s 7.43s 7.48s
> 
> I used the same seeds for both file systems.
> 
> So, it looks like md RAID1 is faster than md RAID0 for seek-limited
> workloads with a few (but more than 1) independent threads/processes.
> It's not clear why this should be the case for such small accesses (1K),
> 
> Another relevant hardware detail on this setup: I think that native
> command queuing is only available (and apparently enabled) on only one
> of the two disk drives on the tested system.
> 
> - anton

Interesting. Do you want to post your code somewhere so that other people 
can try it.

General Schvantzkopf <schvantzkopf@yahoo.com> writes:
>Interesting. Do you want to post your code somewhere so that other people 
>can try it.

The code is in my posting (you may have to "apt-get install gforth" or
similar).  In order to spare you coming up with the variations
yourself, here are the commands for all numbers of processes:

dd if=/dev/zero of=x bs=1024k count=51200

#now do something to flush the cache; umounting and remounting the
#file system may work.

for i in 3; do /usr/bin/time -f %es gforth random.fs -e "200$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 6 7; do /usr/bin/time -f %es gforth random.fs -e "200$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2; do /usr/bin/time -f %es gforth random.fs -e "3$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2 3; do /usr/bin/time -f %es gforth random.fs -e "4$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2 3 4; do /usr/bin/time -f %es gforth random.fs -e "5$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2 3 4 5; do /usr/bin/time -f %es gforth random.fs -e "6$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2 3 4 5 6; do /usr/bin/time -f %es gforth random.fs -e "7$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2 3 4 5 6 7; do /usr/bin/time -f %es gforth random.fs -e "8$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
for i in 0 1 2 3 4 5 6 7 8; do /usr/bin/time -f %es gforth random.fs -e "9$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait

- anton
-- 
M. Anton Ertl                    Some things have to be seen to be believed
anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
http://www.complang.tuwien.ac.at/anton/home.html