f



Redirecting program output with SBCL's RUN-PROGRAM

I'm trying to figure out how to use the output stream of one program I
start with RUN-PROGRAM so it can be used as the input of another
program started with RUN-PROGRAM (i.e., the moral and perhaps literal
equivalent of piping). I've tried using a number of combinations of
the :INPUT, :OUTPUT and :WAIT keyword arguments, but nothing I've hit
upon has been productive so far. Any tips would be helpful; for
example, how would I go about doing something like "ls | grep lisp"
from the shell?

One of my attempts is

(defun piping-test ()
  (let ((grep-process (run-program "/usr/bin/grep" '("lisp")
				   :input :stream
				   :output :stream)))
    (unwind-protect
	 (with-open-stream (s (process-input grep-process))
	   (let ((ls-process (run-program "/bin/ls" '()
					  :output s)))
	     (when ls-process
	       (unwind-protect
		    (with-open-stream (o (process-output grep-process))
		      (loop
			 :for line := (read-line o nil nil)
			 :while line
			 :collect line))
		 (process-close ls-process)))))
      (when grep-process (process-close grep-process)))))

Running this in a SLIME REPL causes everything to hang until I break
with C-c C-c, so it's pretty obviously not the right thing, but I'm
not sure how to change it so it is the right thing.

Thanks,
Pillsy
0
pillsbury (453)
2/28/2010 6:26:51 PM
comp.lang.lisp 16861 articles. 3 followers. Post Follow

7 Replies
2020 Views

Similar Articles

[PageSpeed] 52

Pillsy <pillsbury@gmail.com> writes:

> I'm trying to figure out how to use the output stream of one program I
> start with RUN-PROGRAM so it can be used as the input of another
> program started with RUN-PROGRAM (i.e., the moral and perhaps literal
> equivalent of piping). I've tried using a number of combinations of
> the :INPUT, :OUTPUT and :WAIT keyword arguments, but nothing I've hit
> upon has been productive so far. Any tips would be helpful; for
> example, how would I go about doing something like "ls | grep lisp"
> from the shell?
>
> One of my attempts is
>
> (defun piping-test ()
>   (let ((grep-process (run-program "/usr/bin/grep" '("lisp")
> 				   :input :stream
> 				   :output :stream)))
>     (unwind-protect
> 	 (with-open-stream (s (process-input grep-process))
> 	   (let ((ls-process (run-program "/bin/ls" '()
> 					  :output s)))
> 	     (when ls-process
> 	       (unwind-protect
> 		    (with-open-stream (o (process-output grep-process))
> 		      (loop
> 			 :for line := (read-line o nil nil)
> 			 :while line
> 			 :collect line))
> 		 (process-close ls-process)))))
>       (when grep-process (process-close grep-process)))))
>
> Running this in a SLIME REPL causes everything to hang until I break
> with C-c C-c, so it's pretty obviously not the right thing, but I'm
> not sure how to change it so it is the right thing.

(defun piping-test () ; should be written as:
    (pipe
       (command "ls")
       (command "grep" :arguments '("lisp"))))

; implementation of pipe and command (and other verbs such as input,
; output, and error redirections), left as an exercise to the reader.
; scsh may be taken for inspiration. 

-- 
__Pascal Bourguignon__
0
pjb
2/28/2010 7:01:22 PM
On Feb 28, 2:01=A0pm, p...@informatimago.com (Pascal J. Bourguignon)
wrote:
> Pillsy <pillsb...@gmail.com> writes:

> > Any tips would be helpful; for
> > example, how would I go about doing something like "ls | grep lisp"
> > from the shell?

> > One of my attempts is

> > (defun piping-test ()
> > =A0 (let ((grep-process (run-program "/usr/bin/grep" '("lisp")
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :input :str=
eam
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :output :st=
ream)))
> > =A0 =A0 (unwind-protect
> > =A0 =A0 (with-open-stream (s (process-input grep-process))
> > =A0 =A0 =A0 (let ((ls-process (run-program "/bin/ls" '()
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0:output s)))
> > =A0 =A0 =A0 =A0 (when ls-process
> > =A0 =A0 =A0 =A0 =A0 (unwind-protect
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(with-open-stream (o (process-output gre=
p-process))
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(loop
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :for line :=3D (read-line o nil=
 nil)
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :while line
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :collect line))
> > =A0 =A0 =A0 =A0 =A0 =A0 (process-close ls-process)))))
> > =A0 =A0 =A0 (when grep-process (process-close grep-process)))))

> > Running this in a SLIME REPL causes everything to hang until I break
> > with C-c C-c, so it's pretty obviously not the right thing, but I'm
> > not sure how to change it so it is the right thing.

> (defun piping-test () ; should be written as:
> =A0 =A0 (pipe
> =A0 =A0 =A0 =A0(command "ls")
> =A0 =A0 =A0 =A0(command "grep" :arguments '("lisp"))))
>
> ; implementation of pipe and command (and other verbs such as input,
> ; output, and error redirections), left as an exercise to the reader.
> ; scsh may be taken for inspiration.

The problem I'm having is exactly in your "exercise to the reader".
It's not entirely clear how the sources for another implementation of
an entirely different implementation are going to be useful in
figuring out the implementation-specific functionality of SBCL.

Cheers,
Pillsy
0
Pillsy
3/1/2010 12:36:29 PM
On 2/28/10 1:26 PM, Pillsy wrote:
> (defun piping-test ()
>   (let ((grep-process (run-program "/usr/bin/grep" '("lisp")
> 				   :input :stream
> 				   :output :stream)))
>     (unwind-protect
> 	 (with-open-stream (s (process-input grep-process))
> 	   (let ((ls-process (run-program "/bin/ls" '()
> 					  :output s)))
> 	     (when ls-process
> 	       (unwind-protect
> 		    (with-open-stream (o (process-output grep-process))
> 		      (loop
> 			 :for line := (read-line o nil nil)
> 			 :while line
> 			 :collect line))
> 		 (process-close ls-process)))))
>       (when grep-process (process-close grep-process)))))

Try this one.  It works for me with CMUCL; perhaps it works with SBCL.
It basically starts ls first before starting grep.

(defun piping-test2 ()
  (let ((ls-process (run-program "/bin/ls" '()
				   :wait nil
				   :output :stream)))
    (unwind-protect
	 (with-open-stream (s (process-output ls-process))
	   (let ((grep-process (run-program "/usr/bin/grep" '("lisp")
					  :input s
					  :output :stream)))
	     (when grep-process
	       (unwind-protect
		    (with-open-stream (o (process-output grep-process))
		      (loop
			 :for line := (read-line o nil nil)
			 :while line
			 :collect line))
		 (process-close grep-process)))))
      (when ls-process (process-close ls-process)))))

Ray

0
Raymond
3/1/2010 2:16:43 PM
On Mar 1, 9:16=A0am, Raymond Toy <toy.raym...@gmail.com> wrote:
> On 2/28/10 1:26 PM, Pillsy wrote:
>
>
>
> > (defun piping-test ()
> > =A0 (let ((grep-process (run-program "/usr/bin/grep" '("lisp")
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :input :str=
eam
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :output :st=
ream)))
> > =A0 =A0 (unwind-protect
> > =A0 =A0 (with-open-stream (s (process-input grep-process))
> > =A0 =A0 =A0 (let ((ls-process (run-program "/bin/ls" '()
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0:output s)))
> > =A0 =A0 =A0 =A0 (when ls-process
> > =A0 =A0 =A0 =A0 =A0 (unwind-protect
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(with-open-stream (o (process-output gre=
p-process))
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(loop
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :for line :=3D (read-line o nil=
 nil)
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :while line
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 :collect line))
> > =A0 =A0 =A0 =A0 =A0 =A0 (process-close ls-process)))))
> > =A0 =A0 =A0 (when grep-process (process-close grep-process)))))
>
> Try this one. =A0It works for me with CMUCL; perhaps it works with SBCL.
> It basically starts ls first before starting grep.
>
> (defun piping-test2 ()
> =A0 (let ((ls-process (run-program "/bin/ls" '()
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:w=
ait nil
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:o=
utput :stream)))
> =A0 =A0 (unwind-protect
> =A0 =A0 =A0 =A0 =A0(with-open-stream (s (process-output ls-process))
> =A0 =A0 =A0 =A0 =A0 =A0(let ((grep-process (run-program "/usr/bin/grep" '=
("lisp")
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 :input s
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 :output :stream)))
> =A0 =A0 =A0 =A0 =A0 =A0 =A0(when grep-process
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(unwind-protect
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (with-open-stream (o (process-out=
put grep-process))
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (loop
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:for line :=3D (read-l=
ine o nil nil)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:while line
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:collect line))
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(process-close grep-process)))))
> =A0 =A0 =A0 (when ls-process (process-close ls-process)))))
>
> Ray

Confirmed on on RHEL5 (slightly different paths) and SBCL (only added
sb-ext package qualifiers)

(let ((ls-process (sb-ext:run-program "/bin/ls" '()
				      :wait nil
				      :output :stream)))
  (unwind-protect
       (with-open-stream (s (sb-ext:process-output ls-process))
	 (let ((grep-process (sb-ext:run-program "/bin/grep" '("lisp")
						 :input s
						 :output :stream)))
	   (when grep-process
	     (unwind-protect
		  (with-open-stream (o (sb-ext:process-output grep-process))
		    (loop
		       :for line :=3D (read-line o nil nil)
		       :while line
		       :collect line))
	       (sb-ext:process-close grep-process)))))
    (when ls-process (sb-ext:process-close ls-process))))

Thanks,

Mirko
0
Mirko
3/1/2010 6:49:06 PM
On Mar 1, 9:16=A0am, Raymond Toy <toy.raym...@gmail.com> wrote:
[...]
> Try this one. =A0It works for me with CMUCL; perhaps it works with SBCL.
> It basically starts ls first before starting grep.

Thanks, this works! Not only does it work with SBCL, with minor
modifications it also works with CCL[1].

One interesting wrinkle is that this takes a really long time (one
whole second) on SBCL when run through SLIME. It's orders of magnitude
faster when run from an SBCL on the command line (or when used through
the CCL Cocoa listener). This is disappointing, since I was hoping to
put something together that would let me "live" in SLIME and SBCL
instead of having to divide time between it and a terminal window.

Cheers,
Pillsy

> (defun piping-test2 ()
> =A0 (let ((ls-process (run-program "/bin/ls" '()
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:w=
ait nil
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:o=
utput :stream)))
> =A0 =A0 (unwind-protect
> =A0 =A0 =A0 =A0 =A0(with-open-stream (s (process-output ls-process))
> =A0 =A0 =A0 =A0 =A0 =A0(let ((grep-process (run-program "/usr/bin/grep" '=
("lisp")
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 :input s
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 :output :stream)))
> =A0 =A0 =A0 =A0 =A0 =A0 =A0(when grep-process
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(unwind-protect
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (with-open-stream (o (process-out=
put grep-process))
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (loop
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:for line :=3D (read-l=
ine o nil nil)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:while line
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:collect line))
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(process-close grep-process)))))
> =A0 =A0 =A0 (when ls-process (process-close ls-process)))))

[1] CCL's RUN-PROGRAM is evidently a pretty close copy of the one
provided by SBCL and CMUCL.
0
Pillsy
3/2/2010 2:20:55 PM
On 3/2/10 9:20 AM, Pillsy wrote:
> On Mar 1, 9:16 am, Raymond Toy <toy.raym...@gmail.com> wrote:
> [...]
>> Try this one.  It works for me with CMUCL; perhaps it works with SBCL.
>> It basically starts ls first before starting grep.
> 
> Thanks, this works! Not only does it work with SBCL, with minor
> modifications it also works with CCL[1].

Cool.

But if you want to do pipes like this, why not just do something like
(untested):

(run-program "/bin/sh" '("-c" "ls | grep lisp") :output <output stream>)

Ray
0
Raymond
3/2/2010 3:21:25 PM
On Mar 2, 10:21=A0am, Raymond Toy <toy.raym...@gmail.com> wrote:
> On 3/2/10 9:20 AM, Pillsy wrote:
[...]
> > Thanks, this works! Not only does it work with SBCL, with minor
> > modifications it also works with CCL[1].

> Cool.

> But if you want to do pipes like this, why not just do something like
> (untested):

> (run-program "/bin/sh" '("-c" "ls | grep lisp") :output <output stream>)

That means giviing up on the possibility of sticking Lisp code into
the pipeline, which is (part of) my ultimate goal here. Perhaps that's
just completely unworkable though, or this is the wrong level of
abstraction to attack the goal at.

Cheers,
Pillsy
0
Pillsy
3/2/2010 3:27:07 PM
Reply: