Odd problem syncing files, I was using a script from the internet
to sync some files to and from a central location. I noted that the
files were sync-ing twice. I wrote the following test script to check
things out. It look as though rsync does not update timestamps
properly

This bug seems a really massive one to me in that every backup type
operation based on rsync has messed up dates.


=============start============
#! /bin/bash

remote_dir="/Volumes/webdav/Common"
file_to_mirror="test.txt"

if [ ! -e ~/$file_to_mirror ]
        then
        touch ~/$file_to_mirror
fi

if [ ~/$file_to_mirror -nt $remote_dir/$file_to_mirror ]
        then # our file is newer
        echo "Copying to remote dir"
        rsync -aEXvu --rsync-path="/usr/local/bin/rsync" \
            ~/$file_to_mirror $remote_dir/$file_to_mirror
elif [ ~/$file_to_mirror -ot $remote_dir/$file_to_mirror ]
        then # mirror fileis newer
        echo "Updating from remote dir"
        rsync -aEXvu --rsync-path="/usr/local/bin/rsync"  \
            $remote_dir/$file_to_mirror ~/$file_to_mirror
else
        echo "Both files same!"
fi

stat $remote_dir/$file_to_mirror ~/$file_to_mirror
=============end============

I started off 1) with the test file the same in both places. I guess
my fist issue is why is "stat" printing four times and what are the
four times? I then 2) touched the local file to "change" it and reran
the script; the remote copy was updated as expected. Good.


Now at 3) things begin to go bad. Rerunning the sync again I should
see that the local and remote copies are identical... seeing as that
is what step two did! However the files are deemed to differ and the
local copy is updated from the remote.

At stage 4) things seem OK again. But why the double transfer?

It appears that the version of rsync supplied with tiger, leopard and
snow leopard are all 2.6.9 which contains a bug which affects the
modification date on the copy if the -E flag is present. So I upgraded
both ends to use 3.0.6 but see the same effect. Followed the
instructions
from http://www.bombich.com/mactips/rsync.html when upgrading rsync.

In the following I wrapped the output from stat to stop my mailer from
doing so! Local is a MacBookPro runing snow leapard, remote is MacMini
running tiger. Using webdav to access the remote MacMini.


1)=======================================
fergus: ./test-ot.bash
Both files same!
788529167 56 -rwx------ 1 fergus fergus 0 0
  "May 26 10:44:25 2010"
  "May 26 10:44:25 2010"
  "May 26 10:44:25 2010"
  "May 26 10:44:25 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
234881026 6990277 -rwx------ 1 fergus fergus 0 0
  "May 26 10:44:26 2010"
  "May 26 10:44:25 2010"
  "May 26 10:44:26 2010"
  "May 26 10:44:25 2010" 4096 0 0 /Users/fergus/test.txt



2)=======================================
fergus: touch test.txt
fergus: stat /Volumes/webdav/Common/test.txt test.txt
788529167 56 -rwx------ 1 fergus fergus 0 0
  "May 26 10:44:25 2010"
  "May 26 10:44:25 2010"
  "May 26 10:44:25 2010"
  "May 26 10:44:25 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
234881026 6990277 -rwx------ 1 fergus fergus 0 0
  "May 26 10:44:48 2010"
  "May 26 10:44:48 2010"
  "May 26 10:44:48 2010"
  "May 26 10:44:25 2010" 4096 0 0 test.txt
fergus:
fergus:
fergus: ./test-ot.bash
Copying to remote dir
sending incremental file list
test.txt

sent 89 bytes  received 31 bytes  240.00 bytes/sec
total size is 0  speedup is 0.00
788529167 57 -rwx------ 1 fergus fergus 0 0
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
234881026 6990277 -rwx------ 1 fergus fergus 0 0
  "May 26 10:44:48 2010"
  "May 26 10:44:48 2010"
  "May 26 10:44:48 2010"
  "May 26 10:44:25 2010" 4096 0 0 /Users/fergus/test.txt



3)=======================================
fergus: ./test-ot.bash
Updating from remote dir
sending incremental file list
test.txt

sent 89 bytes  received 31 bytes  240.00 bytes/sec
total size is 0  speedup is 0.00
788529167 57 -rwx------ 1 fergus fergus 0 0
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
234881026 6990281 -rwx------ 1 fergus fergus 0 0
  "May 26 10:45:14 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:14 2010"
  "May 26 10:45:09 2010" 4096 0 0 /Users/fergus/test.txt



4)=======================================
fergus: ./test-ot.bash
Both files same!
788529167 57 -rwx------ 1 fergus fergus 0 0
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:09 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
234881026 6990281 -rwx------ 1 fergus fergus 0 0
  "May 26 10:45:14 2010"
  "May 26 10:45:09 2010"
  "May 26 10:45:14 2010"
  "May 26 10:45:09 2010" 4096 0 0 /Users/fergus/test.txt
fergus:

Fergus McMenemie <fergus@twig-me-uk.not.here> wrote:

> Odd problem syncing files, I was using a script from the internet
> to sync some files to and from a central location. I noted that the
> files were sync-ing twice. I wrote the following test script to check
> things out. It look as though rsync does not update timestamps
> properly

> This bug seems a really massive one to me in that every backup type
> operation based on rsync has messed up dates.


I dunno if it's the script but one thing with rsync and macs is, the 
volume you are writing the files to, need that "ignore ownership on this
volume" box checked on the drive info.

This of course applies to getting files from a remote host.

If left at default, it doesn't really matter what options you send to rsync
for preserving ownership/permissions/dates, the OSX will just ignore it.

If you are sending files from your mac to an rsync server, one thing to keep
in mind is, it's probably not running as root in the rsyncd.conf so again,
the host will place it's own ownership and permissions on the files, likely
screwing up the time stamps you want to preserve.

Other than that, the 3.0.6 works like a charm.

-bruce
bje@ripco.com

On 05-26-2010 15:07, Bruce Esquibel wrote:
> I dunno if it's the script but one thing with rsync and macs is, the
> volume you are writing the files to, need that "ignore ownership on this
> volume" box checked on the drive info.

Not so.  I've been using rsync for ages to keep all or part of
two or more Macs identical.  Both HFS+ and UFS volumes.  I have
never "ignored ownership."

-- 
Wes Groleau

   Hillary Insults Virgen de Guadalupe?
   http://Ideas.Lang-Learn.us/russell?itemid=1531

In article <1jj3lud.1kfkzxft0aauuN%fergus@twig-me-uk.not.here>,
 fergus@twig-me-uk.not.here (Fergus McMenemie) wrote:

> Odd problem syncing files, I was using a script from the internet
> to sync some files to and from a central location. I noted that the
> files were sync-ing twice. I wrote the following test script to check
> things out. It look as though rsync does not update timestamps
> properly
> 
> This bug seems a really massive one to me in that every backup type
> operation based on rsync has messed up dates.
> 
> 
> =============start============
> #! /bin/bash
> 
> remote_dir="/Volumes/webdav/Common"
> file_to_mirror="test.txt"
> 
> if [ ! -e ~/$file_to_mirror ]
>         then
>         touch ~/$file_to_mirror
> fi
> 
> if [ ~/$file_to_mirror -nt $remote_dir/$file_to_mirror ]
>         then # our file is newer
>         echo "Copying to remote dir"
>         rsync -aEXvu --rsync-path="/usr/local/bin/rsync" \
>             ~/$file_to_mirror $remote_dir/$file_to_mirror
> elif [ ~/$file_to_mirror -ot $remote_dir/$file_to_mirror ]
>         then # mirror fileis newer
>         echo "Updating from remote dir"
>         rsync -aEXvu --rsync-path="/usr/local/bin/rsync"  \
>             $remote_dir/$file_to_mirror ~/$file_to_mirror
> else
>         echo "Both files same!"
> fi
> 
> stat $remote_dir/$file_to_mirror ~/$file_to_mirror
> =============end============
> 
> I started off 1) with the test file the same in both places. I guess
> my fist issue is why is "stat" printing four times and what are the
> four times? I then 2) touched the local file to "change" it and reran
> the script; the remote copy was updated as expected. Good.
> 
> 
> Now at 3) things begin to go bad. Rerunning the sync again I should
> see that the local and remote copies are identical... seeing as that
> is what step two did! However the files are deemed to differ and the
> local copy is updated from the remote.
> 
> At stage 4) things seem OK again. But why the double transfer?
> 
> It appears that the version of rsync supplied with tiger, leopard and
> snow leopard are all 2.6.9 which contains a bug which affects the
> modification date on the copy if the -E flag is present. So I upgraded
> both ends to use 3.0.6 but see the same effect. Followed the
> instructions
> from http://www.bombich.com/mactips/rsync.html when upgrading rsync.
> 
> In the following I wrapped the output from stat to stop my mailer from
> doing so! Local is a MacBookPro runing snow leapard, remote is MacMini
> running tiger. Using webdav to access the remote MacMini.
> 
> 
> 1)=======================================
> fergus: ./test-ot.bash
> Both files same!
> 788529167 56 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:44:25 2010"
>   "May 26 10:44:25 2010"
>   "May 26 10:44:25 2010"
>   "May 26 10:44:25 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
> 234881026 6990277 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:44:26 2010"
>   "May 26 10:44:25 2010"
>   "May 26 10:44:26 2010"
>   "May 26 10:44:25 2010" 4096 0 0 /Users/fergus/test.txt
> 
> 
> 
> 2)=======================================
> fergus: touch test.txt
> fergus: stat /Volumes/webdav/Common/test.txt test.txt
> 788529167 56 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:44:25 2010"
>   "May 26 10:44:25 2010"
>   "May 26 10:44:25 2010"
>   "May 26 10:44:25 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
> 234881026 6990277 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:44:48 2010"
>   "May 26 10:44:48 2010"
>   "May 26 10:44:48 2010"
>   "May 26 10:44:25 2010" 4096 0 0 test.txt
> fergus:
> fergus:
> fergus: ./test-ot.bash
> Copying to remote dir
> sending incremental file list
> test.txt
> 
> sent 89 bytes  received 31 bytes  240.00 bytes/sec
> total size is 0  speedup is 0.00
> 788529167 57 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
> 234881026 6990277 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:44:48 2010"
>   "May 26 10:44:48 2010"
>   "May 26 10:44:48 2010"
>   "May 26 10:44:25 2010" 4096 0 0 /Users/fergus/test.txt
> 
> 
> 
> 3)=======================================
> fergus: ./test-ot.bash
> Updating from remote dir
> sending incremental file list
> test.txt
> 
> sent 89 bytes  received 31 bytes  240.00 bytes/sec
> total size is 0  speedup is 0.00
> 788529167 57 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
> 234881026 6990281 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:45:14 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:14 2010"
>   "May 26 10:45:09 2010" 4096 0 0 /Users/fergus/test.txt
> 
> 
> 
> 4)=======================================
> fergus: ./test-ot.bash
> Both files same!
> 788529167 57 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:09 2010" 4096 0 0 /Volumes/webdav/Common/test.txt
> 234881026 6990281 -rwx------ 1 fergus fergus 0 0
>   "May 26 10:45:14 2010"
>   "May 26 10:45:09 2010"
>   "May 26 10:45:14 2010"
>   "May 26 10:45:09 2010" 4096 0 0 /Users/fergus/test.txt
> fergus:

Is the destination volume FAT32?  FAT only maintains resolutions 
at 2 second intervals.

If the destination is a FAT file system, then use the 
--modify-window=1 option so rsync will be more tolerant of the FAT 
short comings.

Wes Groleau <Groleau+news@freeshell.org> wrote:

> Not so.  I've been using rsync for ages to keep all or part of
> two or more Macs identical.  Both HFS+ and UFS volumes.  I have
> never "ignored ownership."

Between two macs, probably.

Going to and from something else, like Solaris 10, sparc or x86, it
won't work right. The OP didn't say, or at least wasn't clear on this, the
kind of remote host.

Even if the owner was root, it'll all end up looking like this...

-rw-r--r--   1 _unknown  _unknown           2 Feb 21  2009 Desktop DF
-rw-r--r--   1 _unknown  _unknown    92022858 Feb  9  2009 databases-020209.tgz
-rw-r--r--   1 _unknown  _unknown    96934920 Feb  9  2009 databases-020909.tgz
-rw-r--r--   1 _unknown  _unknown   903741440 Feb 27  2009 databases-022709.tgz

The owner/group permissions definetly are shot, but the time stamps aren't.

Unless he clarifies the setup it's just second guessing. I never had to use 
anything more complex with rsync than -avzo and it always worked reliable
once the rsyncd.conf was configured properly and that ownership thing turned
off.

-bruce
bje@ripco.com

Bruce Esquibel <bje@ripco.com> wrote:

> > Not so.  I've been using rsync for ages to keep all or part of
> > two or more Macs identical.  Both HFS+ and UFS volumes.  I have
> > never "ignored ownership."
> 
> Between two macs, probably.
> 
> Going to and from something else, like Solaris 10, sparc or x86, it
> won't work right. The OP didn't say, or at least wasn't clear on this, the
> kind of remote host.


Sorry I thought I was clear. "Local is a MacBookPro runing snow leapard,
remote is MacMini running tiger. Using webdav to access the remote
MacMini." Reading though these helpful comments I looked into the
volume type I was syncing to (hfs+) and the way it was mounted (webdav).
I tired syncing again but using a regualr afp mount rather than webdav.

Worked fine!

Once you know what you are looking for, Google is the place to find
it, and "webdav timestamps" reveals all kind of articles indicating 
problmes along the lines I saw. I guess I can now adjust things to
get my script to work. 

Thanks all!

On 05-27-2010 17:21, Fergus McMenemie wrote:
> Sorry I thought I was clear. "Local is a MacBookPro runing snow leapard,
> remote is MacMini running tiger. Using webdav to access the remote
> MacMini." Reading though these helpful comments I looked into the
> volume type I was syncing to (hfs+) and the way it was mounted (webdav).
> I tired syncing again but using a regualr afp mount rather than webdav.
>
> Worked fine!

And mine worked over NFS.

-- 
Wes Groleau

   Third World Comes to the U.S.
   http://Ideas.Lang-Learn.us/russell?itemid=1505

In message <htln7l$olg$1@remote5bge0.ripco.com> 
  Bruce Esquibel <bje@ripco.com> wrote:
> Wes Groleau <Groleau+news@freeshell.org> wrote:

>> Not so.  I've been using rsync for ages to keep all or part of
>> two or more Macs identical.  Both HFS+ and UFS volumes.  I have
>> never "ignored ownership."

> Between two macs, probably.

> Going to and from something else, like Solaris 10, sparc or x86, it
> won't work right. The OP didn't say, or at least wasn't clear on this, the
> kind of remote host.

Works fine from my FreeBSD servers.

-- 
'There's Mr Dibbler.' 'What's he selling this time?' 'I don't think he's
trying to sell anything, Mr Poons.' 'It's that bad? Then we're probably
in lots of trouble.' --Reaper Man

In article <1jj67rl.17e86zi1m9ltogN%fergus@twig-me-uk.not.here>,
 fergus@twig-me-uk.not.here (Fergus McMenemie) wrote:

> Bruce Esquibel <bje@ripco.com> wrote:
> 
> > > Not so.  I've been using rsync for ages to keep all or part of
> > > two or more Macs identical.  Both HFS+ and UFS volumes.  I have
> > > never "ignored ownership."
> > 
> > Between two macs, probably.
> > 
> > Going to and from something else, like Solaris 10, sparc or x86, it
> > won't work right. The OP didn't say, or at least wasn't clear on this, the
> > kind of remote host.
> 
> 
> Sorry I thought I was clear. "Local is a MacBookPro runing snow leapard,
> remote is MacMini running tiger. Using webdav to access the remote
> MacMini." Reading though these helpful comments I looked into the
> volume type I was syncing to (hfs+) and the way it was mounted (webdav).
> I tired syncing again but using a regualr afp mount rather than webdav.
> 
> Worked fine!
> 
> Once you know what you are looking for, Google is the place to find
> it, and "webdav timestamps" reveals all kind of articles indicating 
> problmes along the lines I saw. I guess I can now adjust things to
> get my script to work. 
> 
> Thanks all!

Yeah. I usually default to AFP when networking Mac to Mac.

-- 
Send responses to the relevant news group rather than email to me.
E-mail sent to this address may be devoured by my very hungry SPAM
filter. Due to Google's refusal to prevent spammers from posting
messages through their servers, I often ignore posts from Google
Groups. Use a real news client if you want me to see your posts.

JR