Hello,

We have an NFS partition exported though Solaris 9 and I have a question 
about removing files via a cron script.

For a while I've been doing this:
find /scratch -mtime +14 | xargs rm
But I feel somehow limited by this, since this has problems with 
filenames with spaces in, and also can not remove directories.
I can add the -R option, but I was wondering if the
find /scatch -mtime +14 -exec rm \{\} \;
would be better and is able to handle files with spaces and such in 
since I would like to do this as well for a shared Samba directory for 
tempfiles.

Maybe running -exec rmdir \{\} \; to get rid of the dirs, but how do I 
take care of the other problem?

Thanks,

Wimmy

-- 
+----------+
|  PLEASE  |
|  DO NOT  |
| FEED THE |
|  TROLLS  |
+----------+
     |  |
     |  |
   .\|.||/..

Wim Cossement <wcosseme@nospam.bcol.be> wrote:
> Hello,

> We have an NFS partition exported though Solaris 9 and I have a question 
> about removing files via a cron script.

> For a while I've been doing this:
> find /scratch -mtime +14 | xargs rm
> But I feel somehow limited by this, since this has problems with 
> filenames with spaces in, and also can not remove directories.

One solution is to not use xargs...

find /scratch -mtime +14 -exec rm {} +

As mentioned, that won't remove a directory.  So perhaps...

find /scratch ! -type d -mtime +14 -exec rm {} +

would be better.

Worse, the removal itself may update a directory so that the mtime
changes. 

Depending on your environment, you might want to just follow the above
with

find /scratch -type d -mtime +14 -exec rmdir {} +

If the directory isn't empty, the rmdir will just fail.  (Might want to
redirect errors...)  Directories would be emptied after 14 days, but
could hang around empty for another couple of weeks.  In most cases, the
disk space for that is small enough to not worry about it.

Otherwise, you could have a policy of simply removing empty directories
more often.  Drop the mtime on it down.

> I can add the -R option, but I was wondering if the
> find /scatch -mtime +14 -exec rm \{\} \;
> would be better and is able to handle files with spaces and such in 
> since I would like to do this as well for a shared Samba directory for 
> tempfiles.

-r (not -R) would be different.  It would mean that any directory which
was old enough to match would be removed, along with any files or
subdirectories, even if they were new.

Yes, exec'ing the rm directly avoids various issues with
spaces/newlines/etc. in filenames.  Another solution would be to get the
gnu find and xargs programs and use the -0 flag on them.

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Darren Dunham wrote:
> Wim Cossement <wcosseme@nospam.bcol.be> wrote:

>>We have an NFS partition exported though Solaris 9 and I have a question 
>>about removing files via a cron script.

> One solution is to not use xargs...
> 
> find /scratch -mtime +14 -exec rm {} +
> 
> As mentioned, that won't remove a directory.  So perhaps...
> 
> find /scratch ! -type d -mtime +14 -exec rm {} +
> 
> would be better.

Last time I did something like that, I was running it through cron
and trying to clean out /tmp.  I ran my script, and it removed
someone's /tmp/.X11-unix/X0 because they had been logged in for
weeks and weeks, and the named socket's mtime doesn't get updated
except when the X session is started (or when the socket is created?).
As a result, the user's X11 session just stopped working (couldn't
accept connections from new clients?) one day.  Oops.
The same thing might apply for named pipes as well.

Therefore, I would think this is better:

	find /scratch -type f -mtime +14 -exec rm '{}' '+'

> Worse, the removal itself may update a directory so that the mtime
> changes. 

That's not necessarily a terrible thing.  The directory can be
removed another 14 days later.  This gets a little silly if you
have something like this:

	mkdir -p 0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9
	touch 0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9/foo

Then it will take over a year for the cron job to remove the whole thing,
since it will just remove one component every 2 weeks.

You could solve this problem by queueing up the entire list of directories
to remove before removing any of them, but that would require a fair bit
of extra work for no big gain...

   - Logan

Wim Cossement wrote:
> Hello,
> 
> We have an NFS partition exported though Solaris 9 and I have a question 
> about removing files via a cron script.
> 
> For a while I've been doing this:
> find /scratch -mtime +14 | xargs rm
> But I feel somehow limited by this, since this has problems with 
> filenames with spaces in, and also can not remove directories.
> I can add the -R option, but I was wondering if the
> find /scatch -mtime +14 -exec rm \{\} \;
> would be better and is able to handle files with spaces and such in 
> since I would like to do this as well for a shared Samba directory for 
> tempfiles.
> 
> Maybe running -exec rmdir \{\} \; to get rid of the dirs, but how do I 
> take care of the other problem?
> 
> Thanks,
> 
> Wimmy
> 

Thanks for the tips guys!

I've been using some GNU packages for a while (Samba most importantly) 
via the pkg-get tool so I forgot all about this possiblilty to install 
up to date GNU software.

I'll be using gfind from now on... :-)

Regards,

Wimmy

Logan Shaw <lshaw-usenet@austin.rr.com> writes:

> Darren Dunham wrote:
>> Wim Cossement <wcosseme@nospam.bcol.be> wrote:
>
>>> We have an NFS partition exported though Solaris 9 and I have a
>>> question about removing files via a cron script.
>
>> One solution is to not use xargs...
>> find /scratch -mtime +14 -exec rm {} +
>> As mentioned, that won't remove a directory.  So perhaps...
>> find /scratch ! -type d -mtime +14 -exec rm {} +
>> would be better.
>
>> Worse, the removal itself may update a directory so that the mtime
>> changes.
>
> That's not necessarily a terrible thing.  The directory can be
> removed another 14 days later.  This gets a little silly if you
> have something like this:
>
> 	mkdir -p 0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9
> 	touch 0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9/foo
>
> Then it will take over a year for the cron job to remove the whole thing,
> since it will just remove one component every 2 weeks.

You might also want to look at rmdir -p

This will for file with space

 find . -type f -print | perl -pne 's/(.*)/"$1"/' | xargs rm
-SR

Sun <sfgroups@gmail.com> wrote:
> This will for file with space
> 
> find . -type f -print | perl -pne 's/(.*)/"$1"/' | xargs rm

But what about other special characters like tab, newline, or '"'? In
UNIX a filename can contain any character besides NUL and '/'.

Imagine if someone did:
mkdir -p "x^M/etc"
touch "x^M/etc/passwd"

Then your find produces something like:
% find . -print
[...]
/etc/passwd

Now imagine running a "find | xargs rm" over this directory...

Conclusion: Never use "find | xargs" on untrusted directories.

You should have find directly call rm (with the right arguments and not
fiddled through a pipe). To reduce the number of exec required you could
use the '+' parameter to -exec (instead of ';'):

find . -type f -exec rm {} +

"Wim Cossement" <wcosseme@nospam.bcol.be> wrote:

> Maybe running -exec rmdir \{\} \; to get rid of the dirs,
> but how do I take care of the other problem?

Does this help?

http://packages.debian.org/stable/admin/tmpreaper.html
http://www.stanford.edu/services/pubsw/package/system/tmpreaper.html

Wim Cossement <wcosseme@nospam.bcol.be> wrote:
> Thanks for the tips guys!

> I've been using some GNU packages for a while (Samba most importantly) 
> via the pkg-get tool so I forgot all about this possiblilty to install 
> up to date GNU software.

> I'll be using gfind from now on... :-)

What does that get you over 'find ... -exec rm {} +' ?

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Darren Dunham <ddunham@redwood.taos.com> writes:

> Wim Cossement <wcosseme@nospam.bcol.be> wrote:
> > Thanks for the tips guys!
> 
> > I've been using some GNU packages for a while (Samba most importantly) 
> > via the pkg-get tool so I forgot all about this possiblilty to install 
> > up to date GNU software.
> 
> > I'll be using gfind from now on... :-)
> 
> What does that get you over 'find ... -exec rm {} +' ?

GNU find has the -print0 option, which prints the file/directory name
as a null terminated string. You can have just about any character you
like in file names with that option. 

The GNU version of xargs has the ability to read null terminated file
names (--null or -0), so it could be used in this context as well.

Joe
-- 
I never veer to starboard 'cuz I never sail at all
   - The Pirates Who Don't do Anything

joe@invalid.address wrote:
> Darren Dunham <ddunham@redwood.taos.com> writes:

>> Wim Cossement <wcosseme@nospam.bcol.be> wrote:
>> > Thanks for the tips guys!
>> 
>> > I've been using some GNU packages for a while (Samba most importantly) 
>> > via the pkg-get tool so I forgot all about this possiblilty to install 
>> > up to date GNU software.
>> 
>> > I'll be using gfind from now on... :-)
>> 
>> What does that get you over 'find ... -exec rm {} +' ?

> GNU find has the -print0 option, which prints the file/directory name
> as a null terminated string. You can have just about any character you
> like in file names with that option. 

> The GNU version of xargs has the ability to read null terminated file
> names (--null or -0), so it could be used in this context as well.

Umm... true.  But what does that get you over 'find ... -exec rm {} +'?
I mean if you've already got them, fine.  But on a Solaris box that
didn't already have it, why not just use find directly?  It wouldn't
have any issues with filenames (because you're not printing or using
xargs) and it should be just as fast.

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

> Umm... true.  But what does that get you over 'find ... -exec rm {} +'?
> I mean if you've already got them, fine.  But on a Solaris box that
> didn't already have it, why not just use find directly?  It wouldn't
> have any issues with filenames (because you're not printing or using
> xargs) and it should be just as fast.

I don't have a clue ;-)

Daniel Rock wrote:
> But what about other special characters like tab, newline, or '"'? In
> UNIX a filename can contain any character besides NUL and '/'.
> 
> Imagine if someone did:
> mkdir -p "x^M/etc"
> touch "x^M/etc/passwd"

Well, I trust the apps as such that they don't generate tempfiles that 
are too weird.

And the filenames with spaces and stuff are all created from WinDoze 
boxes so the spaces should be the only thing to worry about.

Interesting points though...

Wimmy

Martin Carpenter wrote:
  > Does this help?
> 
> http://packages.debian.org/stable/admin/tmpreaper.html
> http://www.stanford.edu/services/pubsw/package/system/tmpreaper.html

Looks good, and I'll also install that Solarpack system, it looks kinda 
cool...
Because I'm using pkg-get, as mentioned before...

Wimmy

Wim Cossement <wcosseme@nospam.bcol.be> wrote:
> And the filenames with spaces and stuff are all created from WinDoze 
> boxes so the spaces should be the only thing to worry about.

Never underestimate what someone unfamiliar with 'vi' can do while
trying to figure out how to exit it...  *shudder*

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

In article <m38xy1nemp.fsf@invalid.address>,  <joe@invalid.address> wrote:
>Darren Dunham <ddunham@redwood.taos.com> writes:
>
>> Wim Cossement <wcosseme@nospam.bcol.be> wrote:

>> > I'll be using gfind from now on... :-)
>> 
>> What does that get you over 'find ... -exec rm {} +' ?
>
>GNU find has the -print0 option, which prints the file/directory name
>as a null terminated string. You can have just about any character you
>like in file names with that option. 
>
>The GNU version of xargs has the ability to read null terminated file
>names (--null or -0), so it could be used in this context as well.

So you like it to use a nonstandard option on gfind that forces you
to use another GNU tool and another non-standard option on this tool
and prefer this before using the simple and standard compliant 

	'find ... -exec rm {} +' ? 



-- 
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) J�rg Schilling D-13353 Berlin
      js@cs.tu-berlin.de		(uni)  
      schilling@fokus.fraunhofer.de	(work) Blog: http://schily.blogspot.com/
URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily

In article <dg9d92$m18$1@snic.vub.ac.be>,
Wim Cossement  <wcosseme@nospam.bcol.be> wrote:
>Martin Carpenter wrote:
>  > Does this help?
>> 
>> http://packages.debian.org/stable/admin/tmpreaper.html
>> http://www.stanford.edu/services/pubsw/package/system/tmpreaper.html
>
>Looks good, and I'll also install that Solarpack system, it looks kinda 
>cool...
>Because I'm using pkg-get, as mentioned before...

Solarpack -- first I've heard of it.

How does it compare to Blastwave and SFW?

Much overlap?

Thanks,

David