f



stdout funniness from os.system() calls when redirecting output

I have a python script that uses the print function throughout, and as
well uses calls to os.system() to spawn DOS commandline executables.

My issue is that I redirect all of the output from this script to a
file (including that which is printed by the spawned programs) via the
redirect (">") function on a Win2K Command Prompt.  In the captured
output however, the output from the os.system() calls ALWAYS comes
before the output from the print calls in the python script.

This does NOT happen if I run the python script without redirecting
the output to a file.  (everything prints out properly in the Command
Prompt window)

Does anyone have any experience with this, and/or know the fix?

Much Appreciated!
0
garyb1 (2)
10/14/2003 11:11:37 PM
comp.lang.python 77058 articles. 6 followers. Post Follow

3 Replies
413 Views

Similar Articles

[PageSpeed] 21

Birch fed this fish to the penguins on Tuesday 14 October 2003 16:11 pm:

> My issue is that I redirect all of the output from this script to a
> file (including that which is printed by the spawned programs) via the
> redirect (">") function on a Win2K Command Prompt.  In the captured
> output however, the output from the os.system() calls ALWAYS comes
> before the output from the print calls in the python script.
> 
> This does NOT happen if I run the python script without redirecting
> the output to a file.  (everything prints out properly in the Command
> Prompt window)
> 
> Does anyone have any experience with this, and/or know the fix?
>

        Hypothesis:

        Once redirected, the OS no longer sees an "interactive" file (terminal 
window -- I/O is flushed on EOL), but sees a disk file.

        Disk file I/O is buffered.

        The main program uses a buffer separate from that used by each 
os.system() process.

        The buffer is flushed when the process finishes. 

        Hence, os.system() process begins shutdown processing, flushing 
redirected buffer, THEN main process concludes and flushes its buffer.

-- 
 > ============================================================== <
 >   wlfraed@ix.netcom.com  | Wulfraed  Dennis Lee Bieber  KD6MOG <
 >      wulfraed@dm.net     |       Bestiaria Support Staff       <
 > ============================================================== <
 >        Bestiaria Home Page: http://www.beastie.dm.net/         <
 >            Home Page: http://www.dm.net/~wulfraed/             <

0
wlfraed (4596)
10/15/2003 4:29:02 PM
"Birch" <garyb@ea.com> wrote in message
news:fb6e26e.0310141511.3bd42877@posting.google.com...
> I have a python script that uses the print function throughout, and as
> well uses calls to os.system() to spawn DOS commandline executables.
>
> My issue is that I redirect all of the output from this script to a
> file (including that which is printed by the spawned programs) via the
> redirect (">") function on a Win2K Command Prompt.  In the captured
> output however, the output from the os.system() calls ALWAYS comes
> before the output from the print calls in the python script.
>
> This does NOT happen if I run the python script without redirecting
> the output to a file.  (everything prints out properly in the Command
> Prompt window)

You probably need to flush what is in the buffers of the parent process
before calling os.system().  Then, do not attempt redirection on the
os.system() command.  The stdout file handle should be passed to the child.
This worked on AIX under Python 2.1.

#! /usr/bin/env python
import os
import sys
print "python is started"
sys.stdout.flush()
os.system('ls -al')
print "python is done"
sys.stdout.flush()

However, it appears that Win32 Python 2.3 may not be passing the the file
descriptor or something else is awry.  Is this a bug?

C:\src\t>type pt.py
#! /usr/bin/env python
import os
print "python is started"
os.system('dir')
print "python is done"

C:\src\t>pt.py
python is started
 Volume in drive C is dir
 Volume Serial Number is 146D-04D8

 Directory of C:\src\t

2003-10-19  20:24       <DIR>          .
2003-10-19  20:24       <DIR>          ..
2003-10-19  20:25                   35 jj
2003-10-19  20:20                  104 pt.py
               2 File(s)            139 bytes
               2 Dir(s)   4,800,147,456 bytes free
python is done

C:\src\t>pt.py >jj
There is not enough space on the disk.


0
pwatson1 (12)
10/20/2003 1:34:18 AM
"Paul Watson" <pwatson@redlinec.com> wrote in message news:<3f933ba4$1_1@themost.net>...
> "Birch" <garyb@ea.com> wrote in message
> news:fb6e26e.0310141511.3bd42877@posting.google.com...
> > I have a python script that uses the print function throughout, and as
> > well uses calls to os.system() to spawn DOS commandline executables.
> >
> > My issue is that I redirect all of the output from this script to a
> > file (including that which is printed by the spawned programs) via the
> > redirect (">") function on a Win2K Command Prompt.  In the captured
> > output however, the output from the os.system() calls ALWAYS comes
> > before the output from the print calls in the python script.
> >
> > This does NOT happen if I run the python script without redirecting
> > the output to a file.  (everything prints out properly in the Command
> > Prompt window)
> 
> You probably need to flush what is in the buffers of the parent process
> before calling os.system().  Then, do not attempt redirection on the
> os.system() command.  The stdout file handle should be passed to the child.
> This worked on AIX under Python 2.1.
> 
> #! /usr/bin/env python
> import os
> import sys
> print "python is started"
> sys.stdout.flush()
> os.system('ls -al')
> print "python is done"
> sys.stdout.flush()
> 
> However, it appears that Win32 Python 2.3 may not be passing the the file
> descriptor or something else is awry.  Is this a bug?
> 
> C:\src\t>type pt.py
> #! /usr/bin/env python
> import os
> print "python is started"
> os.system('dir')
> print "python is done"
> 
> C:\src\t>pt.py
> python is started
>  Volume in drive C is dir
>  Volume Serial Number is 146D-04D8
> 
>  Directory of C:\src\t
> 
> 2003-10-19  20:24       <DIR>          .
> 2003-10-19  20:24       <DIR>          ..
> 2003-10-19  20:25                   35 jj
> 2003-10-19  20:20                  104 pt.py
>                2 File(s)            139 bytes
>                2 Dir(s)   4,800,147,456 bytes free
> python is done
> 
> C:\src\t>pt.py >jj
> There is not enough space on the disk.

sys.stdout.flush() seems to do it... now I can organize my output log
files to mean something to people other than myself!

Thanks so much!
0
garyb1 (2)
10/21/2003 4:16:30 PM
Reply: