I have a lot of files I need to copy to another server using scp
(secure copy).  For efficiency reasons I want to minimize the number
of scp invocations.  So, logically I would like to do this:

find ... [files]  |  xargs -i scp {} user@server:/somedir

Unfortunately, I have to use the replacement string {} since I need to
provide a directory as the last parameter to scp.  However, this
causes xargs to invoke scp for each individual file.

I can see doing this using a double xargs like this:

find ... [files]  |  xargs echo scp user@server:/somedir |
    sed 's/.*somedir//' |
    xargs -i scp {} user@server:/somedir

Here the first xargs is used to put as many files together on a single
line as possible and then the second xargs is used to feed each line
of files into an scp.

Is there a better way to do this?  Somehow it seems like there should
be a way to get xargs to do this with a single xargs but I haven't
seen a way.

Denis

On 2010-01-05, sysdrk <sysdrk@wowway.com> wrote:
>
> I can see doing this using a double xargs like this:
>
> find ... [files]  |  xargs echo scp user@server:/somedir |
>     sed 's/.*somedir//' |
>     xargs -i scp {} user@server:/somedir
>
> Here the first xargs is used to put as many files together on a single
> line as possible and then the second xargs is used to feed each line
> of files into an scp.
>
> Is there a better way to do this?  Somehow it seems like there should
> be a way to get xargs to do this with a single xargs but I haven't
> seen a way.
>
You could use xargs to call a script or function that takes the 
destination as the first argument and calls scp.

scp_to () {
  dest="$1"
  shift
  scp "$@" "$dest"
}

sysdrk <sysdrk@wowway.com> writes:

> I have a lot of files I need to copy to another server using scp
> (secure copy).  For efficiency reasons I want to minimize the number
> of scp invocations.  So, logically I would like to do this:
>
> find ... [files]  |  xargs -i scp {} user@server:/somedir
>
> Unfortunately, I have to use the replacement string {} since I need to
> provide a directory as the last parameter to scp.  However, this
> causes xargs to invoke scp for each individual file.

If your names are "space safe" (it seems so from your example) then

  find ... | xargs | xargs -i scp {} user@server:/somedir

may do what you want.

-- 
Ben.

On Tue, 05 Jan 2010 16:55:55 +0000, Ben Bacarisse wrote:

> sysdrk <sysdrk@wowway.com> writes:
> 
>> I have a lot of files I need to copy to another server using scp
>> (secure copy).  For efficiency reasons I want to minimize the number of
>> scp invocations.  So, logically I would like to do this:
>>
>> find ... [files]  |  xargs -i scp {} user@server:/somedir
>>
>> Unfortunately, I have to use the replacement string {} since I need to
>> provide a directory as the last parameter to scp.  However, this causes
>> xargs to invoke scp for each individual file.
> 
> If your names are "space safe" (it seems so from your example) then
> 
>   find ... | xargs | xargs -i scp {} user@server:/somedir
> 
> may do what you want.

Other options you might consider
1) Use something like cpio, pax or a "tar" that can read the list of 
files from standard input. Then use "ssh" rather than "scp" to create a 
pipeline like
	find ... | pax -w | ssh user@server 'cd /somedir; pax -r'

2) Write your own wrapper for scp that reorders the arguments, e.g. myscp
#!/bin/sh
dest=$1
shift
scp "$@" "$dest"

and then use
	find ... | xargs myscp user@server:/somedir

Of course you don't actually need to create your own script to do this, 
you can use
find ... | xargs sh -c 'scp "$@" user@somehost:/somedir' dummy_arg0
but I think another file is cleaner, and you don't need to worry if you 
need the dummy_arg0 or not.

sysdrk schrieb:

> I have a lot of files I need to copy to another server using scp
> (secure copy).  For efficiency reasons I want to minimize the number
> of scp invocations.  So, logically I would like to do this:

> find ... [files]  |  xargs -i scp {} user@server:/somedir

> Unfortunately, I have to use the replacement string {} since I need to
> provide a directory as the last parameter to scp.  However, this
> causes xargs to invoke scp for each individual file.

If the list of files is short, without spaces and such, try this:

scp $(find ...) user@server:/somedir

Otherwise try this:

find ... | tar -c | ssh user@host 'tar -x -C /targetdir'

Karsten

-- 

   ()    My homepage is http://www.tecneeq.de/ and your homepage sucks�!
  <\/>
  _/\_          �) Unless it has animated gifs from 1996, then it rocks!

> > I have a lot of files I need to copy to another server using scp
> > (secure copy). =A0For efficiency reasons I want to minimize the number
> > of scp invocations. =A0So, logically I would like to do this:
> > find ... [files] =A0| =A0xargs -i scp {} user@server:/somedir
> > Unfortunately, I have to use the replacement string {} since I need to
> > provide a directory as the last parameter to scp. =A0However, this
> > causes xargs to invoke scp for each individual file.

Thanks for all the replies.  It looks like the wrapper suggestion is
the easiest and cleanest solution so I'm going to go with that.

Denis