f



Process.waitFor() returns, but threads reading output/error streams are blocked on read

java gurus,

i'm seeing some odd behavior running a process via Runtime.exec on redhat 
linux (seen it on RHEL AS 2.1, 3, 4) with 1.4 and 1.5 JVMs (the only ones 
i've tried).  if i start a specific process via Runtime.exec and set up 
threads to read in the out/err streams, i find that after Process.waitFor() 
returns (~5 seconds) the threads are still stuck in InputStream.read().  i 
would have expected them to return -1 (or fail with IOExceptions) almost 
immediately after process termination.  i've tried closing the streams from 
the main thread and destroying the process, but that was not enough to break 
the threads out of read.  i found that if i changed my reader threads to 
poll InputStream.available() they will find the streams closed after the 
main thread closes them, although available() will return 0 indefinitely if 
the main thread fails to close the streams manually.

i'll include the java source below, but the problem seems extremely specific 
to the program i'm running in Runtime.exec().  i've been unable to duplicate 
the problem with another program.  the program is a non-interactive command 
line tool (i'll call "cmd") that creates a socket connection to another 
process on the same machine and requests an action to be performed (in my 
case the action is "-start", passed in on the command line).  the program 
takes about 5 seconds to complete.  upon successful completion no data is 
written to standard out/err.  the exit code is 0 on success, non-zero on 
failure.  i can confirm in the process list that the program does terminate 
(which Process.waitFor() tells me anyway), and i can also confirm the 
program performed the action i expected.  i discovered that when my java 
process is hung if i terminate the process the command line tool connects to 
that will break the threads out of read() with a -1 response.  that leads me 
to believe that something was not cleaned up when the command line tool 
exited.  i've also found that if i invoke the command line tool 
differently - e.g. i run "cmd -start" after a previous run of "cmd -start" 
(close to a no-op) or run "cmd -status" - the java process terminates 
normally.  so something about the first call to "cmd -start" preventes 
InputStream.read() from returning.

some questions:
* after Process.waitFor() returns, what could cause 
Process.getInputStream().read() and Process.getErrorStream().read() to block 
indefinitely?  shouldn't they return -1 or receive IOExceptions almost 
immediately?
* any thoughts on what this command line tool might be doing (perhaps not 
closing its out/err streams gracefully when it terminates?) that would cause 
these reads to block?

thanks.

-mike

import java.io.IOException;

public class cmdstart {
  public static void main(String [] args) throws IOException, 
InterruptedException {
    String cmd = "cmd -start";
    Process process = Runtime.getRuntime().exec(cmd);
    MyReader outReader = new MyReader(process.getInputStream());
    Thread outThread = new Thread(outReader);
    outThread.start();
    MyReader errReader = new MyReader(process.getErrorStream());
    Thread errThread = new Thread(errReader);
    errThread.start();
    System.out.println("exit code is " + process.waitFor());
//process.destroy();
//    process.getInputStream().close();
//    process.getErrorStream().close();
        outThread.join();
        errThread.join();
    System.out.println("joins completed");
  }
}

import java.io.*;

public class MyReader implements Runnable {
  private final InputStream is;

  public MyReader(InputStream is) {
    this.is = is;
  }

  public void run() {
    // normally would read in while here, but these threads never get any 
data -
    // they are blocked on the first read() call forever.
    int length;
    try {
       length = is.read();
       if (length == -1) {
         System.out.println("-1");
         return;
       }
       if (length == 0) {
         System.out.println("0");
         return;
       }
       System.out.println("read something");
    } catch (Throwable t) {
      System.out.println(t);
    }
  }
}



0
snowymike (5)
8/18/2005 11:14:01 PM
comp.lang.java.programmer 52714 articles. 1 followers. Post Follow

6 Replies
629 Views

Similar Articles

[PageSpeed] 20

Michael Grove wrote:
 >
> * after Process.waitFor() returns, what could cause 
> Process.getInputStream().read() and Process.getErrorStream().read() to block 
> indefinitely?  shouldn't they return -1 or receive IOExceptions almost 
> immediately?
> * any thoughts on what this command line tool might be doing (perhaps not 
> closing its out/err streams gracefully when it terminates?) that would cause 
> these reads to block?

As a random guess forking (for instance executing another command) 
and/or reopening stdout/stderr.

Apparently strace is the Linux command to use to see what a process is 
doing. Never tried it myself.

Tom Hawtin
-- 
Unemployed English Java programmer
http://jroller.com/page/tackline/
0
usenet120 (1728)
8/18/2005 11:36:49 PM
thanks for the response.

you're right that something like fork occurs.  i haven't fully tracked it 
down via strace, but i remembered that the first time "cmd -start" is run it 
actually bootstraps the other process it normally connects to if it finds it 
not running.  that process ends up started with init (pid 1) as it's parent 
process.  i didn't see exactly how this happened via strace, but it seemed 
to occur right around a clone() call, which i see is pretty similar to fork. 
i think the cloned process might actually be the one that bootstraps the 
other process before it terminates.  the cloned process vanishes before i 
can get a chance to see what it's doing.

that should at least help me duplicate this problem with a "cmd" program of 
my own creation.  however i still wonder - is it correct behavior in java 
that after the process has terminated the streams are not closed?  even if 
that answer is yes, i'm still confused why a polling InputStream.available() 
would fail with an IOException once i closed the Process streams manually 
yet InputStream.read() would remain blocked.

i know Runtime.exec() is a bit of a crapshoot and the javadocs indicate it 
may not work well in a variety of cases.  wondering though if this behavior 
is 1) expected/correct or 2) a Runtime.exec() limitation or 3) a JVM bug.

thanks.

-mike

"Thomas Hawtin" <usenet@tackline.plus.com> wrote in message 
news:43051c2c$0$97138$ed2619ec@ptn-nntp-reader03.plus.net...
> Michael Grove wrote:
> >
>> * after Process.waitFor() returns, what could cause 
>> Process.getInputStream().read() and Process.getErrorStream().read() to 
>> block indefinitely?  shouldn't they return -1 or receive IOExceptions 
>> almost immediately?
>> * any thoughts on what this command line tool might be doing (perhaps not 
>> closing its out/err streams gracefully when it terminates?) that would 
>> cause these reads to block?
>
> As a random guess forking (for instance executing another command) and/or 
> reopening stdout/stderr.
>
> Apparently strace is the Linux command to use to see what a process is 
> doing. Never tried it myself.
>
> Tom Hawtin
> -- 
> Unemployed English Java programmer
> http://jroller.com/page/tackline/ 


0
snowymike (5)
8/19/2005 3:27:04 PM
Michael Grove wrote:
> you're right that something like fork occurs.  i haven't fully tracked it 
> down via strace, but i remembered that the first time "cmd -start" is run it 
> actually bootstraps the other process it normally connects to if it finds it 
> not running.  that process ends up started with init (pid 1) as it's parent 
> process.  i didn't see exactly how this happened via strace, but it seemed 
> to occur right around a clone() call, which i see is pretty similar to fork. 
> i think the cloned process might actually be the one that bootstraps the 
> other process before it terminates.  the cloned process vanishes before i 
> can get a chance to see what it's doing.
> 
> that should at least help me duplicate this problem with a "cmd" program of 
> my own creation.  however i still wonder - is it correct behavior in java 
> that after the process has terminated the streams are not closed?  even if 
> that answer is yes, i'm still confused why a polling InputStream.available() 
> would fail with an IOException once i closed the Process streams manually 
> yet InputStream.read() would remain blocked.
> 
> i know Runtime.exec() is a bit of a crapshoot and the javadocs indicate it 
> may not work well in a variety of cases.  wondering though if this behavior 
> is 1) expected/correct or 2) a Runtime.exec() limitation or 3) a JVM bug.

My guess would be that the observed behavior is more-or-less correct, in 
that the OS-level streams are not being closed, with the result that the 
Java-level streams are not closed either.  This could be the case if the 
Process' stdin and stdout are inherited by the chain of other processes 
that are forked off, and not closed by those processes  I suspect that's 
what is happening.  If so, it's a minor bug in the bootstrapper, 
possibly duplicated in the other process, and it can probably be 
addressed in one or both of those places.

-- 
John Bollinger
jobollin@indiana.edu
0
jobollin (1557)
8/20/2005 3:44:36 AM
On Fri, 19 Aug 2005 15:27:04 GMT, "Michael Grove"
<snowymike@comcast.net> wrote or quoted :

>that should at least help me duplicate this problem with a "cmd" program of 
>my own creation.  however i still wonder - is it correct behavior in java 
>that after the process has terminated the streams are not closed?  even if 
>that answer is yes, i'm still confused why a polling InputStream.available() 
>would fail with an IOException once i closed the Process streams manually 
>yet InputStream.read() would remain blocked.

try looking at java.util.prefs instead.

0
look-on (4215)
8/20/2005 10:44:46 PM
On Sat, 20 Aug 2005 22:44:46 GMT, Roedy Green
<look-on@mindprod.com.invalid> wrote or quoted :

>
>>that should at least help me duplicate this problem with a "cmd" program of 
>>my own creation.  however i still wonder - is it correct behavior in java 
>>that after the process has terminated the streams are not closed?  even if 
>>that answer is yes, i'm still confused why a polling InputStream.available() 
>>would fail with an IOException once i closed the Process streams manually 
>>yet InputStream.read() would remain blocked.
>
>try looking at java.util.prefs instead.

I meant java.lang.ProcessBuilder.  I was thinking of somenone else's
question, and came back to this.


0
look-on (4215)
8/22/2005 9:47:00 PM
thanks for clarifying - i couldn't imagine what preferences you had in mind 
;-).  program has to run on 1.4, so ProcessBuilder is not available.

i've asked the authors of the "cmd" program to make certain streams are 
properly being closed, and will post any solution to the newsgroup.

"Roedy Green" <look-on@mindprod.com.invalid> wrote in message 
news:vohkg1dqfk2uqter3c79b6bbve7otjegd9@4ax.com...
> On Sat, 20 Aug 2005 22:44:46 GMT, Roedy Green
> <look-on@mindprod.com.invalid> wrote or quoted :
>
>>
>>>that should at least help me duplicate this problem with a "cmd" program 
>>>of
>>>my own creation.  however i still wonder - is it correct behavior in java
>>>that after the process has terminated the streams are not closed?  even 
>>>if
>>>that answer is yes, i'm still confused why a polling 
>>>InputStream.available()
>>>would fail with an IOException once i closed the Process streams manually
>>>yet InputStream.read() would remain blocked.
>>
>>try looking at java.util.prefs instead.
>
> I meant java.lang.ProcessBuilder.  I was thinking of somenone else's
> question, and came back to this.
>
> 


0
snowymike (5)
8/23/2005 2:24:00 PM
Reply: