I'm trying to use dd to image the partitions on an old drive, using the
command:

dd if=/dev/hdd4 of=~/tmp/hdd4

The process has been running since about 1:30AM this morning.
According to gkrellm, /dev/hdd is getting (mostly) sustained transfer
rates of 131K/s, while the device on which the destination file
resides, /dev/hdb, is seeing continuous transfer rates ranging from
4.9MB/s to 6.3MB/s.

What would cause that?  From watching the image file grow, the 131K/s
number sounds a lot more accurate.

I ran an offline SMART test on /dev/hdd before beginning the transfer.
Here's the results of a subsequent "smartctl -a /dev/hdd" :

Some of you might remember my post about trying to get a LaCie internal
drive working.smartctl version 5.32 Copyright (C) 2002-4 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model:     ST310212A
Serial Number:    7EG09BDY
Firmware Version: 3.02
Device is:        Not in smartctl database [for details use: -P
showall]
ATA Version is:   5
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Sat Apr  2 13:47:14 2005 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection
activity
                                        was completed without error.
                                        Auto Offline Data Collection:
Enabled.
Self-test execution status:      (   0) The previous self-test routine
completed
                                        without error or no self-test
has ever
                                        been run.
Total time to complete Offline
data collection:                 (2400) seconds.
Offline data collection
capabilities:                    (0x1d) SMART execute Offline
immediate.
                                        No Auto Offline data collection
support.
                                        Abort Offline collection upon
new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test
supported.
                                        No Selective Self-test
supported.
SMART capabilities:            (0x0003) Saves SMART data before
entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        No General Purpose Logging
support.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (  14) minutes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE
UPDATED  WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate     0x0008   091   082   000    Old_age   Offline
     -       85094852
3 Spin_Up_Time            0x0006   098   098   000    Old_age   Always
     -       0
4 Start_Stop_Count        0x0013   100   100   020    Pre-fail  Always
     -       109
5 Reallocated_Sector_Ct   0x0013   100   100   036    Pre-fail  Always
     -       22
7 Seek_Error_Rate         0x0009   058   054   030    Pre-fail  Offline
     -       223372185858
9 Power_On_Hours          0x0012   095   095   000    Old_age   Always
     -       4918
10 Spin_Retry_Count        0x0013   100   100   090    Pre-fail  Always
      -       0
12 Power_Cycle_Count       0x0013   100   100   000    Pre-fail  Always
      -       109
197 Current_Pending_Sector  0x0030   100   100   000    Old_age
Offline      -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age
Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   200   200   000    Old_age
Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1

(/results)

Edited it a bit, to make it fit better in 80 columns. According to the
"Offline data collection status", the offline test ran without error.
There are 22 relocated sectors, but I can't a mere 22 seeks, over the
course of the entire disk, holding down the transfer rate like that.

According to hdparm, the drive is using DMA.  The master device on the
IDE channel is a 16X DVD-ROM drive, so I'd have to believe there's
decent DMA hardware servicing the drive.

What could be wrong?  Are there any other diagnostic utilities I can
run on it? (Without pulling out the Ultimate Boot CD, that is.)

Mike Mol wrote:
> I'm trying to use dd to image the partitions on an old drive, using the
> command:
> 
> dd if=/dev/hdd4 of=~/tmp/hdd4
> 
> The process has been running since about 1:30AM this morning.
> According to gkrellm, /dev/hdd is getting (mostly) sustained transfer
> rates of 131K/s, while the device on which the destination file
> resides, /dev/hdb, is seeing continuous transfer rates ranging from
> 4.9MB/s to 6.3MB/s.
> 
> What would cause that?  From watching the image file grow, the 131K/s
> number sounds a lot more accurate.

Have tried playing with the bs (blocksize) paramter to dd?  That can 
make a *huge* difference in overall throughput.

John-Paul Stewart wrote:
> Mike Mol wrote:
> > I'm trying to use dd to image the partitions on an old drive, using
the
> > command:
> >
> > dd if=/dev/hdd4 of=~/tmp/hdd4
> >
> > The process has been running since about 1:30AM this morning.
> > According to gkrellm, /dev/hdd is getting (mostly) sustained
transfer
> > rates of 131K/s, while the device on which the destination file
> > resides, /dev/hdb, is seeing continuous transfer rates ranging from
> > 4.9MB/s to 6.3MB/s.
> >
> > What would cause that?  From watching the image file grow, the
131K/s
> > number sounds a lot more accurate.
>
> Have tried playing with the bs (blocksize) paramter to dd?  That can
> make a *huge* difference in overall throughput.

Hm.  I'm now running a parallel dd operation with a bs=1M argument.
I'm seeing a sustained 7MB+ transfer rate on /dev/hdd.  Works like a
charm, despite having two (supposedly) demanding reads going to the
same device.

(and a few seconds later)

Canceled the first dd operation.  Second operation now hovers in the
9-11MB/s range. Thanks a bunch.

That still doesn't explain to me why the receiving drive was
transferring data at such a disproportional rate, though.

Mike Mol wrote:
> John-Paul Stewart wrote:
> > Mike Mol wrote:
> > > I'm trying to use dd to image the partitions on an old drive,
using
> the
> > > command:
> > >
> > > dd if=/dev/hdd4 of=~/tmp/hdd4
> > >
> > > The process has been running since about 1:30AM this morning.
> > > According to gkrellm, /dev/hdd is getting (mostly) sustained
> transfer
> > > rates of 131K/s, while the device on which the destination file
> > > resides, /dev/hdb, is seeing continuous transfer rates ranging
from
> > > 4.9MB/s to 6.3MB/s.
> > >
> > > What would cause that?  From watching the image file grow, the
> 131K/s
> > > number sounds a lot more accurate.
> >
> > Have tried playing with the bs (blocksize) paramter to dd?  That
can
> > make a *huge* difference in overall throughput.
>
> Hm.  I'm now running a parallel dd operation with a bs=1M argument.
> I'm seeing a sustained 7MB+ transfer rate on /dev/hdd.  Works like a
> charm, despite having two (supposedly) demanding reads going to the
> same device.
>
> (and a few seconds later)
>
> Canceled the first dd operation.  Second operation now hovers in the
> 9-11MB/s range. Thanks a bunch.
>
> That still doesn't explain to me why the receiving drive was
> transferring data at such a disproportional rate, though.

That's because doing a bunch of teenie, tiny reads with the default dd
block size has more impact on the read side of the operation.  The
writes are "written" to the write buffer from the I/O cache (memory),
then flushed (written) to disk in big, consolidated operations.

With so many I/O subsystems running around these days with the 2.6.X
kernels in addition to the 2.4.20+(?) kernels, even the degree of
impact would likely differ.  Not sure how the "throughput" is
measured/calculated with gkrellm or how the differences in kernels
would effect its output.

hth,
prg