f



Compare All Files to All Files in 2 Tree Directories?

How can we compare all files in one directory (including
subdirectories) against all files in another directory (including
subdirectories) to find out which files are anywhere under either or
both? I have backup files on different drives, but I have moved some
files to different directories, so the paths do not match for some
files.

I have been able to run comm on sorted lists of files under both
directories using find, sort, and comm, but I use the find format
'"%f","%T@","%s"\n', so I cannot tell where most of the files listed
from comm are exactly located.

I would like to use '"%f","%T@","%s","%H"\n', so the path would be
displayed, but then comm wouldn't comm what I want to comm.

Is there a way to comm only part of the lines? Partial line comm or
choose fields to comm?

Is there an easier or another way with other commands or scripts or
programs to compare the files without comparing the paths and comparing
not the directories?

* "%f","%T@","%s","%H" - filename, modified time, size, (partial) path.

0
jeolmeun (2)
2/10/2006 6:01:17 AM
comp.unix.shell 15484 articles. 3 followers. Post Follow

19 Replies
776 Views

Similar Articles

[PageSpeed] 32

I want to find out if the same file A is under /1/2/3/ on one drive and
under /4/5/6/ on another drive. I could do a find for file A, but I
want to find out for all files under /1/ and /4/.

0
jeolmeun
2/10/2006 6:09:07 AM
jeolmeun wrote:
> I want to find out if the same file A is under /1/2/3/ on one drive and
> under /4/5/6/ on another drive. I could do a find for file A, but I
> want to find out for all files under /1/ and /4/.

Let's see if this fits your need:

find /1 /4 -type f | perl -F/ -alne '$k=$F[-1];$h{$k}=$_ and next if
$F[1]eq"1";print"comm $_ $h{$k}" if exists $h{$k}'

When using the basename as hash keys, problems may occur when you have
duplicated filenames under /1/..be careful....

Xicheng

0
Xicheng
2/10/2006 6:54:26 AM
jeolmeun wrote:
> I want to find out if the same file A is under /1/2/3/ on one drive and
> under /4/5/6/ on another drive. I could do a find for file A, but I
> want to find out for all files under /1/ and /4/.

Run this Ruby program with
ruby common-files.rb /dir1 /dir2

##  Find files common to two directories.  Descend into subdirectories.
lists = [ Hash.new { [] }, Hash.new { [] } ]
2.times {|i|
  Dir[ ARGV[i] + "/**/*" ].each{|path|
    lists[i][ File.basename(path) ] += [ path ]  if File.file?( path )
  }
}
lists.first.each_key { |name|
  if lists.last.include?( name )
    puts "---  #{ name }"
    puts lists.first[ name ], lists.last[ name ]
  end
}

0
William
2/10/2006 10:21:00 AM
jeolmeun wrote:

> I want to find out if the same file A is under /1/2/3/ on one drive and
> under /4/5/6/ on another drive. I could do a find for file A, but I
> want to find out for all files under /1/ and /4/.
> 


So if a file named A existed under /1/2/3/ and also under /4/5/6/7/8/9, 
would that be a match or does it need to be at the same depth in the 
directory hierarchy? What if /1/2/3/A had different contents from 
/4/5/6/A? What if /1/2/3/A was a file and /4/5/6/A was a directory or a 
symbolic link or....

	Ed.
0
Ed
2/10/2006 2:29:21 PM
"jeolmeun" <jeolmeun@gmail.com> writes:

> How can we compare all files in one directory (including
> subdirectories) against all files in another directory (including
> subdirectories) to find out which files are anywhere under either or
> both? 

This should work

  diff -r dir1 dir2

but if you have more than the occasional difference it may generate pages
and pages of output.

I usually start with

  diff -rqb dir1 dir2

and then diff the files I'm really interested on, or use either of the
above, redirecting stdout and stderr to a file

  diff -r dir1 dir2 1>diffs 2>&1

-- HASM




0
HASM
2/10/2006 3:45:24 PM
On 9 Feb 2006 22:01:17 -0800, jeolmeun wrote:
[...]
> I would like to use '"%f","%T@","%s","%H"\n', so the path would be
> displayed, but then comm wouldn't comm what I want to comm.
>
> Is there a way to comm only part of the lines? Partial line comm or
> choose fields to comm?
[...]

Use join(1)?

-- 
Stephane
0
Stephane
2/10/2006 3:48:24 PM
> So if a file named A existed under /1/2/3/ and also under /4/5/6/7/8/9,
> would that be a match or does it need to be at the same depth in the
> directory hierarchy?

Yes. That would be a match. It does not need to be the same depth.

> What if /1/2/3/A had different contents from
> /4/5/6/A? What if /1/2/3/A was a file and /4/5/6/A was a directory or a
> symbolic link or....

I would like to check for different contents perfectly, but I think
comparing filename, modified date, and byte size should be sufficient.

If the filename, modified date, and byte size are the same that would
be a match. Althought, I wouldn't mind knowing how to compare their
hash or md5.

If /1/2/3/A was a file and /4/5/6/A was a directory or symbolic link, I
do not want them to be a match.

0
jeolmeun
2/10/2006 4:04:22 PM
In article <1139550929.187653.254950@g43g2000cwa.googlegroups.com>,
jeolmeun <jeolmeun@gmail.com> wrote:
>Is there a way to comm only part of the lines? Partial line comm or
>choose fields to comm?

You could use this:

ftp://ftp.armory.com/pub/scripts/ucomm

It will be much slower, though, as it's written in gawk.

	John
-- 
John DuBois  spcecdt@armory.com  KC6QKZ/AE  http://www.armory.com/~spcecdt/
0
spcecdt
2/10/2006 4:11:17 PM
Thanks. I think join does what I wanted.

I have two files: t1 and t4.

admin@ubuntu:/tmp$ cat t1
"A","1139587765","0"|"1/2/A"
"B","1164212520","0"|"1/2/B"
"B","1164212520","0"|"1/2/3/B"
"B","1164212520","0"|"1/2/4/B"
"B","1164212520","0"|"1/2/3/B"
admin@ubuntu:/tmp$ cat t4
"A","1139587770","0"|"4/A"
"A","1139587771","0"|"4/5/A"
"B","1164212520","0"|"4/B"
"B","1164212520","0"|"4/5/B"
"B","1164212520","0"|"4/5/6/B"

admin@ubuntu:/tmp$ join -t '|' -j 1 -v 1 t1 t4
"A","1139587765","0"|"1/2/A"
admin@ubuntu:/tmp$ join -t '|' -j 1 -v 2 t1 t4
"A","1139587770","0"|"4/A"
"A","1139587771","0"|"4/5/A"

admin@ubuntu:/tmp$ join -t '|' -j 1 t1 t4
"B","1164212520","0"|"1/2/B"|"4/B"
"B","1164212520","0"|"1/2/B"|"4/5/B"
"B","1164212520","0"|"1/2/B"|"4/5/6/B"
"B","1164212520","0"|"1/2/3/B"|"4/B"
"B","1164212520","0"|"1/2/3/B"|"4/5/B"
"B","1164212520","0"|"1/2/3/B"|"4/5/6/B"
"B","1164212520","0"|"1/2/4/B"|"4/B"
"B","1164212520","0"|"1/2/4/B"|"4/5/B"
"B","1164212520","0"|"1/2/4/B"|"4/5/6/B"
"B","1164212520","0"|"1/2/3/B"|"4/B"
"B","1164212520","0"|"1/2/3/B"|"4/5/B"
"B","1164212520","0"|"1/2/3/B"|"4/5/6/B"

Is there a way to combine the above to output:
"B","1164212520","0"|"1/2/B"|"1/2/3/B"|"1/2/4/B"|"4/B"|"4/5/B"|"4/5/6/B"
or an output format similar?


Stephane Chazelas wrote:
> On 9 Feb 2006 22:01:17 -0800, jeolmeun wrote:
> [...]
> > I would like to use '"%f","%T@","%s","%H"\n', so the path would be
> > displayed, but then comm wouldn't comm what I want to comm.
> >
> > Is there a way to comm only part of the lines? Partial line comm or
> > choose fields to comm?
> [...]
> 
> Use join(1)?
> 
> -- 
> Stephane

0
jeolmeun
2/10/2006 5:01:51 PM
jeolmeun wrote:
> > So if a file named A existed under /1/2/3/ and also under /4/5/6/7/8/9,
> > would that be a match or does it need to be at the same depth in the
> > directory hierarchy?
>
> Yes. That would be a match. It does not need to be the same depth.
>
> > What if /1/2/3/A had different contents from
> > /4/5/6/A? What if /1/2/3/A was a file and /4/5/6/A was a directory or a
> > symbolic link or....
>
> I would like to check for different contents perfectly, but I think
> comparing filename, modified date, and byte size should be sufficient.
>
> If the filename, modified date, and byte size are the same that would
> be a match. Althought, I wouldn't mind knowing how to compare their
> hash or md5.

use "find . -type f -ls", you can get filename(you can extract the
basename), file-size, and a modified-time(?), and then concatenate them
into a single string.. you can then use these strings as hash keys to
filter out duplicated files.

here is an example of the cmd "find -ls"
1095623    4 -rw-r--r--   1 xicheng  xicheng       727 Feb  9 00:35
../A/B/C/k9.txt

then use "perl -alne -------"  your hash key can be written as
following:

   $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];

# "727Feb900:35k9.txt"

Xicheng

> If /1/2/3/A was a file and /4/5/6/A was a directory or symbolic link, I
> do not want them to,be a match.

0
Xicheng
2/10/2006 5:08:29 PM
This is what I have now which seems to do what I wanted. Is an easier
or more simplified way?

admin@ubuntu:/tmp$ find 1 -type f -printf '"%f","%T@","%s"|"%h"\n' > h1
admin@ubuntu:/tmp$ find 4 -type f -printf '"%f","%T@","%s"|"%h"\n' > h4
admin@ubuntu:/tmp$ cat h1
"A","1139587765","0"|"1/2"
"B","1164212520","0"|"1/2"
"C","1139591213","0"|"1/2"
"D","1139591223","0"|"1/2"
"G","1139591226","0"|"1/2"
"K","1139591231","0"|"1"
admin@ubuntu:/tmp$ cat h4
"A","1139587770","0"|"4"
"B","1164212520","0"|"4"
"E","1139591241","0"|"4"
"M","1139591273","0"|"4/5"
"P","1139591275","0"|"4/5"
admin@ubuntu:/tmp$ sort -t '|' -k 1 h1 > hs1
admin@ubuntu:/tmp$ sort -t '|' -k 1 h4 > hs4
admin@ubuntu:/tmp$ cat hs1
"A","1139587765","0"|"1/2"
"B","1164212520","0"|"1/2"
"C","1139591213","0"|"1/2"
"D","1139591223","0"|"1/2"
"G","1139591226","0"|"1/2"
"K","1139591231","0"|"1"
admin@ubuntu:/tmp$ cat hs4
"A","1139587770","0"|"4"
"B","1164212520","0"|"4"
"E","1139591241","0"|"4"
"M","1139591273","0"|"4/5"
"P","1139591275","0"|"4/5"
admin@ubuntu:/tmp$ join -t '|' -j 1 h1 h4
"B","1164212520","0"|"1/2"|"4"
admin@ubuntu:/tmp$ join -t '|' -j 1 -v 1 h1 h4
"A","1139587765","0"|"1/2"
"C","1139591213","0"|"1/2"
"D","1139591223","0"|"1/2"
"G","1139591226","0"|"1/2"
"K","1139591231","0"|"1"
admin@ubuntu:/tmp$ join -t '|' -j 1 -v 2 h1 h4
"A","1139587770","0"|"4"
"E","1139591241","0"|"4"
"M","1139591273","0"|"4/5"
"P","1139591275","0"|"4/5"

0
jeolmeun
2/10/2006 5:18:40 PM
correction: Is there an easier
or more simplified way?

0
jeolmeun
2/10/2006 5:20:10 PM
correction:
join -t '|' -j 1 -v 2 h1 h4
should be
join -t '|' -j 1 -v 2 hs1 hs4

0
jeolmeun
2/10/2006 5:39:56 PM
jeolmeun wrote:
> This is what I have now which seems to do what I wanted. Is an easier
> or more simplified way?

you can do it in one command and no need to make two temporary files:

replace the $k in:

find /1 /4 -type f -ls | perl -F/ -alne '$k=$F[-1];$h{$k}=$_ and next
if$F[1]eq"1";print"comm $_ $h{$k}" if exists $h{$k}'

with
   $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];

BTW, if you use relative dirname like "4" instead of absolute dirname
like '/4', then all $F[1] should be replaced by $F[0]..

find dir1 dir2 -type f -ls | perl -F/ -alne '
    $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];
    $h{$k}=$_ and next if$F[0]eq"dir1";
    print"comm $_ $h{$k}" if exists $h{$k}'

this way you keep all directory and subdirectories information:
Best,
Xicheng

> admin@ubuntu:/tmp$ find 1 -type f -printf '"%f","%T@","%s"|"%h"\n' > h1
> admin@ubuntu:/tmp$ find 4 -type f -printf '"%f","%T@","%s"|"%h"\n' > h4
> admin@ubuntu:/tmp$ cat h1
> "A","1139587765","0"|"1/2"
> "B","1164212520","0"|"1/2"
> "C","1139591213","0"|"1/2"
> "D","1139591223","0"|"1/2"
> "G","1139591226","0"|"1/2"
> "K","1139591231","0"|"1"
> admin@ubuntu:/tmp$ cat h4
> "A","1139587770","0"|"4"
> "B","1164212520","0"|"4"
> "E","1139591241","0"|"4"
> "M","1139591273","0"|"4/5"
> "P","1139591275","0"|"4/5"
> admin@ubuntu:/tmp$ sort -t '|' -k 1 h1 > hs1
> admin@ubuntu:/tmp$ sort -t '|' -k 1 h4 > hs4
> admin@ubuntu:/tmp$ cat hs1
> "A","1139587765","0"|"1/2"
> "B","1164212520","0"|"1/2"
> "C","1139591213","0"|"1/2"
> "D","1139591223","0"|"1/2"
> "G","1139591226","0"|"1/2"
> "K","1139591231","0"|"1"
> admin@ubuntu:/tmp$ cat hs4
> "A","1139587770","0"|"4"
> "B","1164212520","0"|"4"
> "E","1139591241","0"|"4"
> "M","1139591273","0"|"4/5"
> "P","1139591275","0"|"4/5"
> admin@ubuntu:/tmp$ join -t '|' -j 1 h1 h4
> "B","1164212520","0"|"1/2"|"4"
> admin@ubuntu:/tmp$ join -t '|' -j 1 -v 1 h1 h4
> "A","1139587765","0"|"1/2"
> "C","1139591213","0"|"1/2"
> "D","1139591223","0"|"1/2"
> "G","1139591226","0"|"1/2"
> "K","1139591231","0"|"1"
> admin@ubuntu:/tmp$ join -t '|' -j 1 -v 2 h1 h4
> "A","1139587770","0"|"4"
> "E","1139591241","0"|"4"
> "M","1139591273","0"|"4/5"
> "P","1139591275","0"|"4/5"

0
Xicheng
2/10/2006 5:56:27 PM
Thank you all for your help.

0
jeolmeun
2/10/2006 6:02:23 PM
Xicheng wrote:
> jeolmeun wrote:
> > This is what I have now which seems to do what I wanted. Is an easier
> > or more simplified way?
>
> you can do it in one command and no need to make two temporary files:
>
> replace the $k in:
>
> find /1 /4 -type f -ls | perl -F/ -alne '$k=$F[-1];$h{$k}=$_ and next
> if$F[1]eq"1";print"comm $_ $h{$k}" if exists $h{$k}'
>
> with
>    $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];
>
> BTW, if you use relative dirname like "4" instead of absolute dirname
> like '/4', then all $F[1] should be replaced by $F[0]..
>
> find dir1 dir2 -type f -ls | perl -F/ -alne '
>     $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];
>     $h{$k}=$_ and next if$F[0]eq"dir1";
                                            ~~~
Just noticed this should also be changed when add "-ls" option to
"find", so you might change the following clause from:
     if $F[0] eq "dir1"
to:
     if (split/\//,$F[-1])[0] eq "dir1"

Best,
Xicheng

>     print"comm $_ $h{$k}" if exists $h{$k}'
>
> this way you keep all directory and subdirectories information:
> Best,
> Xicheng
>
> > admin@ubuntu:/tmp$ find 1 -type f -printf '"%f","%T@","%s"|"%h"\n' > h1
> > admin@ubuntu:/tmp$ find 4 -type f -printf '"%f","%T@","%s"|"%h"\n' > h4
> > admin@ubuntu:/tmp$ cat h1
> > "A","1139587765","0"|"1/2"
> > "B","1164212520","0"|"1/2"
> > "C","1139591213","0"|"1/2"
> > "D","1139591223","0"|"1/2"
> > "G","1139591226","0"|"1/2"
> > "K","1139591231","0"|"1"
> > admin@ubuntu:/tmp$ cat h4
> > "A","1139587770","0"|"4"
> > "B","1164212520","0"|"4"
> > "E","1139591241","0"|"4"
> > "M","1139591273","0"|"4/5"
> > "P","1139591275","0"|"4/5"
> > admin@ubuntu:/tmp$ sort -t '|' -k 1 h1 > hs1
> > admin@ubuntu:/tmp$ sort -t '|' -k 1 h4 > hs4
> > admin@ubuntu:/tmp$ cat hs1
> > "A","1139587765","0"|"1/2"
> > "B","1164212520","0"|"1/2"
> > "C","1139591213","0"|"1/2"
> > "D","1139591223","0"|"1/2"
> > "G","1139591226","0"|"1/2"
> > "K","1139591231","0"|"1"
> > admin@ubuntu:/tmp$ cat hs4
> > "A","1139587770","0"|"4"
> > "B","1164212520","0"|"4"
> > "E","1139591241","0"|"4"
> > "M","1139591273","0"|"4/5"
> > "P","1139591275","0"|"4/5"
> > admin@ubuntu:/tmp$ join -t '|' -j 1 h1 h4
> > "B","1164212520","0"|"1/2"|"4"
> > admin@ubuntu:/tmp$ join -t '|' -j 1 -v 1 h1 h4
> > "A","1139587765","0"|"1/2"
> > "C","1139591213","0"|"1/2"
> > "D","1139591223","0"|"1/2"
> > "G","1139591226","0"|"1/2"
> > "K","1139591231","0"|"1"
> > admin@ubuntu:/tmp$ join -t '|' -j 1 -v 2 h1 h4
> > "A","1139587770","0"|"4"
> > "E","1139591241","0"|"4"
> > "M","1139591273","0"|"4/5"
> > "P","1139591275","0"|"4/5"

0
Xicheng
2/10/2006 6:04:18 PM
jeolmeun wrote:
> > So if a file named A existed under /1/2/3/ and also under /4/5/6/7/8/9,
> > would that be a match or does it need to be at the same depth in the
> > directory hierarchy?
>
> Yes. That would be a match. It does not need to be the same depth.
>
> > What if /1/2/3/A had different contents from
> > /4/5/6/A? What if /1/2/3/A was a file and /4/5/6/A was a directory or a
> > symbolic link or....
>
> I would like to check for different contents perfectly, but I think
> comparing filename, modified date, and byte size should be sufficient.
>
> If the filename, modified date, and byte size are the same that would
> be a match. Althought, I wouldn't mind knowing how to compare their
> hash or md5.
>
> If /1/2/3/A was a file and /4/5/6/A was a directory or symbolic link, I
> do not want them to be a match.

This Ruby program will do it:

##  Find files common to two directories.  Descend into subdirectories.
lists = [ Hash.new { [] }, Hash.new { [] } ]
for i in 0 .. 1
  Dir[ ARGV[i] + "/**/*" ].each{|path|
    lists[i][
      [ File.basename(path), File.size(path), File.mtime(path) ]
    ] += [ path ]  if File.file?( path )
  }
end

lists.first.each_key { |key|
  if lists.last.include?( key )
    puts "---  #{ key[0,2].join('  ') } bytes"
    puts lists.first[ key ], lists.last[ key ]
  end 
}

0
William
2/10/2006 6:21:46 PM
Xicheng wrote:
> jeolmeun wrote:
> > This is what I have now which seems to do what I wanted. Is an easier
> > or more simplified way?
>
> you can do it in one command and no need to make two temporary files:
>
> replace the $k in:
>
> find /1 /4 -type f -ls | perl -F/ -alne '$k=$F[-1];$h{$k}=$_ and next
> if$F[1]eq"1";print"comm $_ $h{$k}" if exists $h{$k}'
>
> with
>    $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];
>
> BTW, if you use relative dirname like "4" instead of absolute dirname
> like '/4', then all $F[1] should be replaced by $F[0]..
>
> find dir1 dir2 -type f -ls | perl -F/ -alne '
>     $k=$F[6].$F[7].$F[8].$F[9].(split/\//,$F[-1])[-1];
>     $h{$k}=$_ and next if(split/\//,$F[-1])[0]eq"dir1";
>     print"comm $_ $h{$k}" if exists $h{$k}'
>

I may need to improve this code coz the previous one contains too much
redundent operations..

find dir1 dir2 -type f -ls | perl -F/ -alne '
    $k = "@F[6..9]".( split"/" )[-1];
    $h{$k} = $_ and next if m{[^/]*dir1/};
    print "comm $_ $h{$k}" if $h{$k}'

the keys now contains some spaces like "727 Feb 9 00:35k9.txt", can add
a BEGIN block in perl code to elimilate these spaces:  BEGIN{ $" = "" }

Best,
Xicheng

>
> > admin@ubuntu:/tmp$ find 1 -type f -printf '"%f","%T@","%s"|"%h"\n' > h1
> > admin@ubuntu:/tmp$ find 4 -type f -printf '"%f","%T@","%s"|"%h"\n' > h4
> > admin@ubuntu:/tmp$ cat h1
> > "A","1139587765","0"|"1/2"
> > "B","1164212520","0"|"1/2"
> > "C","1139591213","0"|"1/2"
> > "D","1139591223","0"|"1/2"
> > "G","1139591226","0"|"1/2"
> > "K","1139591231","0"|"1"
> > admin@ubuntu:/tmp$ cat h4
> > "A","1139587770","0"|"4"
> > "B","1164212520","0"|"4"
> > "E","1139591241","0"|"4"
> > "M","1139591273","0"|"4/5"
> > "P","1139591275","0"|"4/5"
> > admin@ubuntu:/tmp$ sort -t '|' -k 1 h1 > hs1
> > admin@ubuntu:/tmp$ sort -t '|' -k 1 h4 > hs4
> > admin@ubuntu:/tmp$ cat hs1
> > "A","1139587765","0"|"1/2"
> > "B","1164212520","0"|"1/2"
> > "C","1139591213","0"|"1/2"
> > "D","1139591223","0"|"1/2"
> > "G","1139591226","0"|"1/2"
> > "K","1139591231","0"|"1"
> > admin@ubuntu:/tmp$ cat hs4
> > "A","1139587770","0"|"4"
> > "B","1164212520","0"|"4"
> > "E","1139591241","0"|"4"
> > "M","1139591273","0"|"4/5"
> > "P","1139591275","0"|"4/5"
> > admin@ubuntu:/tmp$ join -t '|' -j 1 h1 h4
> > "B","1164212520","0"|"1/2"|"4"
> > admin@ubuntu:/tmp$ join -t '|' -j 1 -v 1 h1 h4
> > "A","1139587765","0"|"1/2"
> > "C","1139591213","0"|"1/2"
> > "D","1139591223","0"|"1/2"
> > "G","1139591226","0"|"1/2"
> > "K","1139591231","0"|"1"
> > admin@ubuntu:/tmp$ join -t '|' -j 1 -v 2 h1 h4
> > "A","1139587770","0"|"4"
> > "E","1139591241","0"|"4"
> > "M","1139591273","0"|"4/5"
> > "P","1139591275","0"|"4/5"

0
Xicheng
2/11/2006 2:52:27 AM
On 2006-02-10 07:01, jeolmeun wrote:
> How can we compare all files in one directory (including
> subdirectories) against all files in another directory (including
> subdirectories) to find out which files are anywhere under either or
> both? I have backup files on different drives, but I have moved some
> files to different directories, so the paths do not match for some
> files.
> 
> I have been able to run comm on sorted lists of files under both
> directories using find, sort, and comm, but I use the find format
> '"%f","%T@","%s"\n', so I cannot tell where most of the files listed
> from comm are exactly located.
> 
> I would like to use '"%f","%T@","%s","%H"\n', so the path would be
> displayed, but then comm wouldn't comm what I want to comm.
> 
> Is there a way to comm only part of the lines? Partial line comm or
> choose fields to comm?
> 
> Is there an easier or another way with other commands or scripts or
> programs to compare the files without comparing the paths and comparing
> not the directories?
> 
> * "%f","%T@","%s","%H" - filename, modified time, size, (partial) path.
> 


I should do:

rsync -avHn --delete /directory1/. /directory2/.

The important option here is "n" so rsync do _nothing_, just tell you 
which files it should update or remove if it was a real rsync.


/birre
0
birre
2/14/2006 11:49:20 AM
Reply:

Similar Artilces:

comparing file sizes from files in a directory #2
comparing file sizes from files in a directory Hello all! I need some help with this function I was trying to make but came up futile in. Basically I have this folder called 'original' and another one called 'new'. In these folders there are .NIF files. Both folders have the same subdirectory folder and file structure. So if there is an "original/black/dot.nif" then there will be a "new/black/dot.nif". The only difference between the two dot.nif files are that they may not be the same file size. I'm trying to create a function that will compare the dot.nif from the original folder with the dot.nif from the new folder and delete the larger dot.nif (thus keeping the smaller one wherever it may be). I would really appreciate some help. Basically I tried listing the files into an array for each of the two top folders (original and new) while trying to exclude subfolders. And then I was going to run it through a loop to determine which of the two files from the two arrays were larger/smaller. Alvin wrote: > comparing file sizes from files in a directory > > Hello all! > > I need some help with this function I was trying to make but came up > futile in. Basically I have this folder called 'original' and another > one called 'new'. In these folders there are .NIF files. Both folders > have the same subdirectory folder and file structure. So if there is > an "original/black/dot.nif" the...

files, directories, files, directories
Hi folks, I've been trying to make a decision and it's driving me crazy. Is a directory a file or is a directory NOT a file but a node? Should I have A) public interface IFile { IFileName FileName; IContent GetContent(); } public interface IDirectory extends IFile { } or B) public interface INode { INodeName NodeName; } public interface IFile extends INode { IContent GetContent(); } public interface IDirectory extends INode { } Method A is nice cause IFile becomes the base "Node" type and you can use names like "IFileName" which sounds nice compared to "INodeName" and fits with the "FileSystem" moniker. A has the disadvantage that IDirectory has a GetContent() method which isn't bad but can be considered a bit weird. It probably wouldn't be too hard to make users think of everything as a file. Method B is nice cause IDirectory doesn't have GetContent() -- that honour belongs only to IFile. B is at a disadvantage when it comes to naming: everything becomes a "Node". INodeName sounds rude compared to IFileName. This method feels more "pure" cause you're specialising features completely. Do you reckon I can get away with Method A? It doesn't seem as pure unless you spend a long time convincing yourself that (for all intents and purposes) a file *is* a node (it's just got a different name) and a directory *is* a file. A penny for anyone's thought...

comparing file sizes from files in a directory
Hello all! I need some help with this function I was trying to make but came up futile in. Basically I have this folder called 'original' and another one called 'new'. In these folders there are .NIF files. Both folders have the same subdirectory folder and file structure. So if there is an "original/black/dot.nif" then there will be a "new/black/dot.nif". The only difference between the two dot.nif files are that they may not be the same file size. I'm trying to create a function that will compare the dot.nif from the original folder with the dot.nif from the new folder and delete the larger dot.nif (thus keeping the smaller one wherever it may be). I would really appreciate some help. Basically I tried listing the files into an array for each of the two top folders (original and new) while trying to exclude subfolders. And then I was going to run it through a loop to determine which of the two files from the two arrays were larger/smaller. ...

Comparing 2 files to create a third file
Hi, I've got two files as follows: FILE 1 FILE 2 user5:12345 user3 user2:234239 user4 user3:34908 user5 user1:234342 user4:234993 and I want to use an awk script to generate a third file which contains the information contained in FILE 1 for only those users which are contained in FILE 2. So my FILE 3 in this sample would contain: FILE 3 user3:34908 user4:234993 user5:12345 Additional info: FILE 1 contains two fields separated with a ":" FILE 2 only has one field FILE 3 does not hav...

FAQ 5.2 How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? #2 554047
This is an excerpt from the latest version perlfaq5.pod, which comes with the standard Perl distribution. These postings aim to reduce the number of repeated questions as well as allow the community to review and update the answers. The latest version of the complete perlfaq is at http://faq.perl.org . -------------------------------------------------------------------- 5.2: How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? Use the Tie::File module, which is included in the standard distribution sinc...

FAQ 5.2 How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? #2 325998
This message is one of several periodic postings to comp.lang.perl.misc intended to make it easier for perl programmers to find answers to common questions. The core of this message represents an excerpt from the documentation provided with Perl. -------------------------------------------------------------------- 5.2: How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? Use the Tie::File module, which is included in the standard distribution since Perl 5.8.0. ------------------------------------------...

Comparing 2 files and writing diffirence to new file
Hi all I've got two files namely: FileOld FileNew I want to loop through FILEOLD and compare the records with the records in FILENEW. The records that is in FILEOLD and not in FILENEW I want to write to a new file called NEWFILE. My code looks like this but does not give me the correct results: OPEN(FILEOLD) OPEN(FILENEW) CLOSE(NEWFILE) EMPTY(NEWFILE) CREATE(NEWFILE) OPEN(NEWFILE) CLEAR(FILO:RECORD) CLEAR(FILN:RECORD) SET(FILO:SURNAME_KEY,FILO:SURNAME_KEY) SET(FILN:SURNAME_KEY,FILO:SURNAME_KEY) LOOP UNTIL EOF(FILEOLD) NEXT(FILEOLD) NEXT(FILENEW) TEMPSURNAME = FILO:SURNAME TEMPNAME = FILO:NAME GET(FILENEW,FILN:SURNAME_KEY) IF FILN:SURNAME <> TEMPSURNAME NEW:SURNAME= TEMPSURNAME NEW:NAME = TEMPNAME ADD(NEWFILE) TEMPSURNAME = FILO:SURNAME TEMPNAME = FILO:NAME . .. CLOSE(NEWFILE) CLOSE(FILEOLD) CLOSE(FILENEW) This is the only way I know how to do it, but the results is wrong. Please help Regards Nazla You have made a couple of errors in your routine. Here is what i would use. OPEN(FILEOLD) OPEN(FILENEW) IF EXISTS(NEWFILE) THEN REMOVE(NEWFILE) END CREATE(NEWFILE) OPEN(NEWFILE) CLEAR(FILO:RECORD) SET(FILO:SURNAME_KEY,FILO:SURNAME_KEY) LOOP NEXT(FILEOLD) IF ERRORCODE() THEN BREAK END FILN:SURNAME = FILO:SURNAME FILN:NAME = FILO:NAME ! i have no idea what your key looks like so i included this line GET(FILENEW, FILN:SURNAME_KEY) IF ERRORCODE() THEN NEW:SURNAME= FILN:SURNAME ...

file.lastmodified
Here's my problem... I have a file on many many users remote machines (with different OS's, etc) and I want to have my program test the date that the file that they are using, against the date of the most current version of the program avialable online. I want the program to be able to "see" if the program has changed without having a large overhead of downloading the new file. However File.lastModified gives an absolute time that is different by machine, and therein lies my problem. Thank you for any comments/suggestions! > I have a file on many ma...

libXm.so.2: cannot open shared object file: No such file or directory
I get the following: /tools/dfII/bin/32bit/libSelect.exe: error while loading shared libraries: libXm.so.2: cannot open shared object file: No such file or directory I therefore added /usr/local/lib to LD_LIBRARY_PATH as libXm.so.2 is there. I then get: Warning: XmeGetColorObjData(): not yet implemented! \o Incompatible Xm library detected! \o Label: 272 != 284 \o xmUseVersion: 2001 \o Motif version 2.1.-1717986918 must be installed. \o /tools/dfII/bin/32bit/libManager.exe is exiting with core file. \o Warning: XmeGetColorObjData(): not yet implemented! I think I am running openmotif 222. Any ideas? --- Erik Erik Wanta wrote: > I get the following: > /tools/dfII/bin/32bit/libSelect.exe: error while loading shared > libraries: libXm.so.2: cannot open shared object file: No such file or > directory > > I therefore added /usr/local/lib to LD_LIBRARY_PATH as libXm.so.2 is > there. I then get: > > Warning: XmeGetColorObjData(): not yet implemented! > \o Incompatible Xm library detected! > \o Label: 272 != 284 > \o xmUseVersion: 2001 > \o Motif version 2.1.-1717986918 must be installed. > \o /tools/dfII/bin/32bit/libManager.exe is exiting with core file. > \o Warning: XmeGetColorObjData(): not yet implemented! > > I think I am running openmotif 222. Any ideas? > --- > Erik Hi Erik, The chances are that another libXm is provided by cadence and it is this one that should be picked (because of whate...

comparing PID values of 2 Files in shell
Hi, There is a file having a list of running PIDs and another file having a list of registered PIDs. How can we check if each value of running PIDs are less or more than the registered PIDs i.e. comparing the value of each PID in both the files. Request you to pls give your inputs. Thanks a lot in advance. In article <a2d84811-4900-49be-ba59-264f887c7b15@b1g2000pra.googlegroups.com>, marconi <kirankumar.techy@gmail.com> wrote: > Hi, > > There is a file having a list of running PIDs and another file having > a list of registered PIDs. How can we check if each va...

portably encrypting a file system's partition, directory and/or file #2
Hi *, I have firewire and USB devices with partitions/logical drives, whole directories and/or files I would like to encrypt. The thing is that I need to be able to just plug in the thing on any x86 machine running a commercial OS that would just take it (and AFAIK the only filesystem that even a MAC would seamlessly 'mount' is vfat/FAT32) is there anyway to do that? I have read quite a bit about it and I still don't find exactly what I need. Also, why exactly does encryption belong in the kernel? I think once you make it a kernel-depending functionality 'portability...

tar file with . files and . directories
hi i want to create a tar file with all my . files and . directories in my home directory. i cannot create a tar file without having all my files in it is there a way to create such a tar file? thanx in advance cu hausi On Tue, 02 Mar 2004 23:00:47 +0000, Hannes wrote: > hi > > i want to create a tar file with all my . files and . directories in my > home directory. > > i cannot create a tar file without having all my files in it is there a > way to create such a tar file? > Not very clear what you want to do. If you don't want some files to be archived, use...

How to tar.gz a directory tree WITHOUT files greater than 30 MB file size?
How can I *.tar.gz a whole directory tree (say \home\myname\) but all files inside which are greater than 30 MB should not be included ? Matt Matt Stevens wrote: > How can I *.tar.gz a whole directory tree (say \home\myname\) but all files inside which are > greater than 30 MB should not be included ? Use "find -size" to generate a list of files less than a specific size. "man find" for more info. Then use "tar -T" to tar the files. "man tar" for more info. You might even be able to pipe them together (- for -T), although I've never tried....

FAQ 5.2: How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file?
This message is one of several periodic postings to comp.lang.perl.misc intended to make it easier for perl programmers to find answers to common questions. The core of this message represents an excerpt from the documentation provided with Perl. -------------------------------------------------------------------- 5.2: How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? Use the Tie::File module, which is included in the standard distribution since Perl 5.8.0. ------------------------------------------...

Import Multiple Excel Files into Multiple files or One Single File #2
Now, the code below seems ok to me, but when I run it, I get this error: Error in read_all_files_in_folder (line 27) xlswrite(fOut, numData, s); numData: empty 0x0 double %folders = {'C:\Users\Excel\Desktop\Excel_Files\'}; %# get input XLS files dName = uigetdir('.', 'Select folder containing Excel XLS files'); if dName==0, error('No folder selected'); end files = dir( fullfile(dName,'*.xls') ); files = strcat(dName, filesep, {files.name}'); %' %# prepare output XLS file [fName dName] = uiputfile({'*.xls' 'Excel (*.xls)'}, 'Output File', 'final.xls'); if dName==0, error('No file selected'); end fOut = fullfile(dName,fName); %# process NUM_SHEETS = 1; %# number of sheets per file for s=1:NUM_SHEETS %# extract contents of same sheet from all files numData = cell(numel(files),1); for f=1:numel(files) numData{f} = xlsread(files{f}, s); end %# rearrange data numData = cat(3,numData{:}); numData = reshape(permute(numData,[3 1 2]), [], size(numData,2)); %# write data to corresponding sheet of output XLS file xlswrite(fOut, numData, s); end How can I modify this code to import all files in the folder into separate variables (any number of Excel files become any number of Matlab variables)? Also, how can I make the code import everything into one single variable (all data in f...

Solution to synchronize 300,000 files over FTP without file by file comparing?
Need help to effectively synchronize 3 machines (XP pro, 2000 pro and 98SE) with 3 different FTP Servers (Titan, Raiden and Serv U) Each machines has about 15 Gigabytes in total of 300,000 15k to 900k TIFF files spanning 15 levels of 6000 folders. Servers are connected by broadband, upload transfer rate is 640K on 2 Servers and 4000K on 1 server. 100 to 200 new files are added in each server daily and not much files are modified (10 to 30 in total maybe) We are using WS_ftp synchronize utility performing Server A to B sync (3 hours) then A to C sync (3 hours) and A to B again (3 hours) to make the files identical. That is a full day job. And most of the time is used for scanning and comparing files. We can't schedule the job because the synchronization halt from time to time. We need to reboot the FTP server or the client and start the task again. I doubt it is caused by the large number of files with small size, we tried to synchronize 4 DVD movies between 3 servers and the whole process finished in half the time needed and never halt. Recently the synchronization is getting slower and those non technical staffs (those who created 6000 folders) are complaining. I have tried Availl replicate from Availl.com, but it doesn't seem to operate well for such a large number of files. tried Syncback from 2brightsparks, replicate locally is great, but across ftp doesn't get the job done. tried remote control each server and backup newly added files but the...

file extention from file file location
Hi, I am new to perl (only working on this one problem) but I program in other languages. We have an upload script written in cgi. Can anyone tell me how to get the file extension from the variable $File1 (actual file location on uploader's computer) then find out if it is equal to "txt" Thanks in advance for not telling me to read the manual. Ron -- PLEASE NOTE: comp.infosystems.www.authoring.cgi is a SELF-MODERATED newsgroup. aa.net and boutell.com are NOT the originators of the articles and are NOT responsible for their content. HOW TO POST to co...

FAQ 5.2 How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? #3 554638
This is an excerpt from the latest version perlfaq5.pod, which comes with the standard Perl distribution. These postings aim to reduce the number of repeated questions as well as allow the community to review and update the answers. The latest version of the complete perlfaq is at http://faq.perl.org . -------------------------------------------------------------------- 5.2: How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file? Use the Tie::File module, which is included in the standard distribution since Perl 5.8.0. -------------------------------------------------------------------- The perlfaq-workers, a group of volunteers, maintain the perlfaq. They are not necessarily experts in every domain where Perl might show up, so please include as much information as possible and relevant in any corrections. The perlfaq-workers also don't have access to every operating system or platform, so please include relevant details for corrections to examples that do not work on particular platforms. Working code is greatly appreciated. If you'd like to help maintain the perlfaq, see the details in perlfaq.pod. -- Posted via a free Usenet account from http://www.teranews.com ...

Lookup between 2 files ( korn shell ) #2
Hello Thanks guys for your suggestions; FILE1=/home/pavi/DS.txt FILE2=/home/pavi/chain.txt awk -F ":" '{print $1}' $FILE1 > f1.txt awk -F " " '{print $2}' $FILE2 > f2.txt DS.txt contains; ALBD:xxxxx FLN:yyyyyyy WLG:ttttttt chain.txt contain; ECK: ECK: 331 ECK: ECK: 336 FED :FED :427 FIE :FIE: 212 FLN :FLN: 140 FLN :FLN :141 FMF: FMF: 183 FR1 :FR1: 303 FTT :FTT :423 GAT :GAT: 185 GEN :GEN: 396 GFY :GFY: 268 GIA :GIA: 228 GMD :GMD: 264 MIN: MIN: 158 MRO :MRO: 354 MY0 :MY0: 315 NCS :NCS: 390 OSC :ALBD: 377 OSC :ALBD :378 OSC: ALBD: 379 OSC ALBD 38...

How to compare the file+folder names of two directory trees for upper/lower case changes?
The problem of many so called "synchronization" tools is that they simply backup a directory tree from one source to a target.Almost all of them do NOT detect changes in upper/lower case of file or folder names. So assume I have two absolute equal directory trees beginning with D:\myprojects\* and W:\backup\myprojects\v2009\* Now I change some chars in a folder name in the source from lower to upper case resp. vice versa e.g. D:\myprojects\l1b\versionAA12\* to D:\myprojects\lib\Versionaa12\* This change is NOT detected by standard sync/backup programs. The folde...

Iteration through File.file? misses entries for which File.file?(entry) == true
Hello everyone, I'm new to Ruby, so I'm most likely making an elementary mistake. However, searching Google didn't help me find my answer, and after mucking around with irb I still haven't figured it out. I wrote a method that was intended to take all of the files in a given directory and put them into an array. Here's the code I wrote: def getFiles(dir) pwdFiles = Array.new Dir.foreach(dir) do |entry| pwdFiles.push(entry) if File.file?(entry) == true end end This works fine when called on the working directory --- that is, getFile...

File Upload
I'm writing a script to upload images along with articles to a directory on the server. I'm developing it offline on my WinXP and Apache 1.3.x laptop and it's working great, but when I move the script to the server, I get write errors when the file attempts to upload. So I set up a test script (included below) to test the directory structures all the way up to the file, and every directory appears to be writable, yet I can't write to the test file. I keep seeing suggestions to set the open_basedir directive in the php.ini file to solve this, but it's hasn't worked yet. Can anyone suggest what I can try to get the uploads working? <?php $filename = 'images/articles/test.txt'; $somecontent = "Add this to the file\n"; if (! file_exists($filename)) { print "The file $filename does not exist."; } // Check path availability up to the file if (! is_writable('e:/webroot/www/images/articles')) { print "Directory e:/webroot/www/images/articles is not writable<br>"; } else { print "Directory e:/webroot/www/images/articles is writable<br>"; } if (! is_writable('e:/webroot/www/images')) { print "Directory e:/webroot/www/images is not writable<br>"; } else { print "Directory e:/webroot/www/images is writable<br>"; } if (! is_writable('e:/webroot/www')) { print "Directory e:/webroot/www is not writable<br>"; } el...

Comparing files in a zip to files on drive
I am attempting to incrementally back up files using python. How can I compare a file on a hard drive to a file in a python created zip file to tell if the file has changed? Or should I be using some other method to determine if a file has changed? kj7ny wrote: > compare a file on a hard drive to a file in a python created zip file > to tell if the file has changed? >>> fileondisk = r'C:\Documents and Settings\power\Desktop\ES3dc+Template.3f' >>> zipondisk = r'C:\Documents and Settings\power\Desktop\temps.zip' >>> import zipfile >>> z = zipfile.ZipFile(zipondisk) >>> z.namelist() ['ES1+Template.3f', 'ES3sc+Template.3f', 'ES3dc+Template.3f'] >>> info = z.getinfo('ES3dc+Template.3f') >>> info.CRC 1620938006 >>> import binascii >>> checksum = 0 >>> fp = open(fileondisk, 'rb') >>> while 1: .... data = fp.read(32*1024) .... if not data: break .... checksum = binascii.crc32(data, checksum) .... >>> checksum 1620938006 >>> kj7ny wrote: > I am attempting to incrementally back up files using python. How can I > compare a file on a hard drive to a file in a python created zip file > to tell if the file has changed? Or should I be using some other > method to determine if a file has changed? > You can easily determine size and date from the hard drive/filesystem and from zip file...

changing files to ro, if files are for 2 days "untouched" #2
Hi, we need to share some data-files(mostly ms.doc) for 5 Windows based PC`s/clients (our host will run Debian/Samba). In order to be ablte to "protect" those files from overwriting/ moving, I am thinking if it`s possible to change those files to RO(read-only), if files are for 2 days "untouched" .. How? Any tips are appreciated very much. Thank you! John linux55@bluemail.ch wrote: > In order to be ablte to "protect" those files from overwriting/ > moving, I am thinking if it`s possible to change those files > to RO(read-only), if files are for ...

Web resources about - Compare All Files to All Files in 2 Tree Directories? - comp.unix.shell

Avicenna Directories - Wikipedia, the free encyclopedia
Between 2000-2007, the WHO published the directory electronically, along with updates received after the 2000 print publication. In August 2007, ...

To Complement Its Search Engine, Facebook Launches People, Pages, and Places Directories
Facebook is now closer than ever to becoming the phone book for the internet. The site recently launched a Directory of people , Pages , and ...

Directories
The source for IT industry news, views and analysis across the channel, business and technology

Princess Diana leaked royal directories to Murdoch tabloid, court hears
The former royal editor of Rupert Murdoch's now defunct British tabloid the News of the World told a court on Thursday that the late Princess ...

Telstra slashes 800 jobs from directories arm Sensis
TELSTRA is slashing 800 jobs from its struggling directories arm Sensis just one month after the telco giant sold a 70 per cent stake in the ...

Government To Stop Publishing Contact Information In Blue Pages Of Phone Directories
... has decided to stop publishing contact information for all of its departments and agencies in the blue pages section of telephone directories.A ...

File System Programming Guide: Accessing Files and Directories
Explains how to create and manage files and directories.

Fixing directories
Web directories are one of the key points of differentiation between plans on the Exchanges. A single company can offer half a dozen networks ...

Welcome to SBJ Local Directories
... newsletters, events, print subscription, digital subscription or other special offers. Welcome to Sacramento Business Journal's local directories. ...

Bradenton and Lakewood Ranch Guides - Directories and guides
The Bradenton Herald is your source for Bradenton and Lakewood Ranch directories and guides.

Resources last updated: 3/29/2016 2:09:23 AM