Windows, subprocess.Popen & encodage

  • Permalink
  • submit to reddit
  • Email
  • Follow


Hi!

From long time, I have problems with strings return, in Windows, by 
subprocess.Popen / stdout.read()

Last night, I found, by hazard, than if the second byte equal 0, it's, 
perhaps, the solution.
With a code like this:

     p=subprocess.Popen(u850("cmd /u/c ....
     tdata=p.stdout.read()
     if ord(tdata[1])==0:
         data=tdata.decode('utf-16')
     else:
         data=tdata.decode('cp850')

Diffrents scripts seem run OK. I had try with:
   - common dir
   - dir on unicode-named-files
   - ping
   - various commands


But, I don't found anything, in any documentations, on this.



Sombody can confirm?   Am I misled?  Am I right?



* sorry for my bad english*


@-salutations

Michel Claveau

0
Reply enleverlesX.XmcX (152) 5/8/2007 6:43:04 AM

See related articles to this posting


> But, I don't found anything, in any documentations, on this.
> 
> Sombody can confirm?   Am I misled?  Am I right?

You are right, and you are misled. The encoding of the data
that you get from Popen.read is not under the control of Python:
i.e. not only you don't know, but Python doesn't know, either.
The operating system simply has no mechanism of indicating
what encoding is used on a pipe.

So different processes may chose different encodings. Some
may produce UTF-16, others may produce CP-850, yet others
UTF-8, and so on. There really is no way to tell other than
reading the documentation *of the program you run*, and,
failing that, reading the source code of the program you
run.

On Windows, many programs will indeed use one of the
two system code pages, or UTF-16. It's true that
UTF-16 can be quite reliably detected by looking at the
first two bytes. However, the two system code pages
(OEM CP and ANSI CP) are not so easy to tell apart.

Regards,
Martin
0
Reply ISO 5/8/2007 7:15:49 AM

Thank you.






-- 
@-salutations

Michel Claveau


0
Reply XX.XmcX (47) 5/8/2007 7:34:17 AM
comp.lang.python 72564 articles. 12 followers. Post

2 Replies
66 Views

Similar Articles

[PageSpeed] 10


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Windows, subprocess.Popen & encodage
Hi! From long time, I have problems with strings return, in Windows, by subprocess.Popen / stdout.read() Last night, I found, by hazard, than if the second byte equal 0, it's, perhaps, the solution. With a code like this: p=subprocess.Popen(u850("cmd /u/c .... tdata=p.stdout.read() if ord(tdata[1])==0: data=tdata.decode('utf-16') else: data=tdata.decode('cp850') Diffrents scripts seem run OK. I had try with: - common dir - dir on unicode-named-files - ping - various commands But, I don't found anything, in a...

using subprocess.Popen does not suppress terminal window on Windows
I am calling a ruby program from a python gui and using subprocess.Popen in Windows XP using python 2.6. Unfortunately, whenever the ruby program is called a blank command window appears on screen, annoying my users. Is there a way to suppress this behaviour? Below is a minimal program that demonstrates the problem. The problem does not manifest if the python program is launched via the command line. To duplicate launch from Windows Explorer by double-clicking on the python file. --- call_double.pyw --- from subprocess import * import time time.sleep(3) # to show that command window is ...

subprocess.Popen on Windows
playing with subprocess.Popen on Windows I stumbled into the following problem: Python 2.4.3 (#69, Mar 29 2006, 17:35:34) IDLE 1.1.3 >>> import subprocess >>> p1=subprocess.Popen("c:\\asd.bat") #works OK >>> p2=subprocess.Popen("c:\\asd.bat",stdout=subprocess.PIPE) Traceback (most recent call last): File "<pyshell#2>", line 1, in -toplevel- p2=subprocess.Popen("c:\\asd.bat",stdout=subprocess.PIPE) File "C:\Python24\lib\subprocess.py", line 533, in __init__ (p2cread, p2cwrite, File "C:\Pytho...

subprocess.Popen under windows 7
Hello, i have something like this under windows 7: print("try command...") arglist = [PATH_TO_7ZIP,"a", "-sfx", archive_name, "*", "-r", "-p",PASSWORD] p = subprocess.Popen(args=arglist, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=srcdir) output, error = p.communicate() if output: print output print ("Eyerthing is good") press_any_key_to_continue() The script works, but there is a little problem. When I double-click the python file, then the command line will open and the s...

subprocess.Popen on Windows #2
I am trying to use subprocess - it basically works but. command = 'ping ' + '-n '+ str(count) + ' -l ' + str(size) + ' ' + str(node) print command p = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE) pout = p.stdout.read() This works for me but I see the Windows command window, is there a way to call subprocess without a command window showing? I am trying to replace: fdout, fdin = popen2.popen4('ping...

Using subprocess.Popen() in a Windows service
Hello, I am attempting to use Popen() in a Windows service. I have a small Win32 .exe that I normally run through the os.popen2() function. I've written a class to work with the input and output parameters that are passed and captured from this exe. When I use the class outside of a service using either subprocess.Popen or os.popen2 work just fine. When I use this class inside a Windows service it doesn't work. It doesn't crash the service or anything but there are no values returned from the Popen. Here's how I'm calling Popen: p = subprocess.Popen( cmd, shell=True, buf...

how to use subprocess.Popen execute "find" in windows
In cmd, I can use find like this. C:\>netstat -an | find "445" TCP 0.0.0.0:445 0.0.0.0:0 LISTENING UDP 0.0.0.0:445 *:* C:\> And os.system is OK. >>> import os >>> os.system('netstat -an | find "445"') TCP 0.0.0.0:445 0.0.0.0:0 LISTENING UDP 0.0.0.0:445 *:* 0 >>> But I don't know how to use subprocess.Popen to do this. from subprocess import Popen, PIPE p1 =3D Popen(['netstat', '-an'], stdout =3D PIPE) p2 =3D Popen(['...

[Subprocess/Windows] subprocess module under Windows 98
--==========D6E0F63B57E1D4FCCE02========== Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi, I am using subprocess.Popen() to start Java processes (converters) from=20 within Zope under Windows 98 (sorry, we have to support the platform). The=20 call looks like this: P =3D Popen('java.exe arguments...', stdout=3DPIPE, stderr=3DPIPE,=20 stdin=3Dopen('nul:')). stdin=3Dopen('nul:') is required to make Popen() actually running under=20 Windows 98 (stdin=3DNone works fine on XP). Howe...

subprocess on Windows
Whenever my Windows rubyw script launches a subprocess, a command window pops up and immediately disappears. How can I keep this from happening? Details: I keep a daily log of what I do at work. At the command line, I type "log" to edit the current day's log file. A Ruby script generates an appropriate name for the file automatically, then launches vim. Now I'm trying to use the Windows GUI, and I want to have an icon I can click to launch gvim. Here's the code: # log.rbw # Launch gvim on today's logfile, e.g. on # C:\Logs\2006\2006.01\2006.01.26....

popen on Windows
I'm trying to get popen to work on Windows. Here's a simplified example of what I'm trying to get working: I have a hw.c program as follows: #include <stdio.h> main() { printf ("Hello World!\n"); } And here's the 'popentest.php' code: popen test... <?php $fp = popen('C:\\home\\bin\\hw.exe', 'r'); $data = fread($fp, 1024); pclose($fp); echo $data; ?> It works when I run this from a command prompt: C:\Inetpub\wwwroot\php>php popentest.php popen test... Hello World! All I get in a web browser is: popen test......

popen on windows
I am trying to set off commands on Windows 2003 from python. Specifically, I am trying to use diskpart with a script file (pointed to with path). cmd = ["diskpart", "/s", path] p = Popen(cmd, shell=True) The script is meant to loop through twice. It will do so if I comment out the Popen call and print cmd instead. But when Popen is called, one disk will be formated, but not the next. On 27 Dec 2006 09:16:53 -0800, "hubritic" <colinlandrum@gmail.com> wrote: >I am trying to set off commands on Windows 2003 from python. >Spec...

subprocess.Popen
I was looking at Python 2.4 subprocess.Popen. Quite nice and handy, but I wonder why a "kill" method is missing. I am just adding it via subclassing, class Popen(subprocess.Popen): def kill(self, signal = SIGTERM): os.kill(self.pid, signal) but I would prefer to have it in the standard Popen class. I am surprised it is not there. Any comments? Michele Simionato Michele Simionato wrote: > I was looking at Python 2.4 subprocess.Popen. Quite nice and handy, but I > wonder why a "kill" method is missing. I am just adding it via subclassing, > > cl...

Trouble with subprocess.Popen()
Hi, I'm having a very strange issue with subprocess.Popen. I'm using it to call several times an external exe and keep the output in a list. Every time you call this external exe, it will return a different string. However, if I call it several times using Popen, it will always return the SAME string. =:-O It looks like Popen is returning always the same value from stdout, without recalling the exe. This is my code: ------------------------------------------------------------------------------------------------------------------------------------ def get_key(): from subproces...

Popen in main and subprocess
the following code (in the main thread) works well, I grep some files and the search until the first 100 results are found (writing the results to a file), then exit: command = 'grep -F "%s" %s*.txt' % (search_string, DATA_PATH) p = Popen(['/bin/bash', '-c', command], stdout = PIPE) f = open(output_file, 'w+') num_lines = MAX_RESULTS while True: line = p.stdout.readline() print num_lines if line != '': f.write(line) num_lines = num_lines - 1 if num_lines == 0: ...

subprocess.Popen deadlocks
Hi From time to time I observe a deadlock in subprocess.Popen. I see the childprocess beeing <defunct> and if I attach gdb to the stuck python script, I can see it waiting on line 1128 in subprocess.py ( data = _eintr_retry_call(os.read, errpipe_read, 1048576) ). Any ideas of whats going on? I'm on linux using python 2.6.5. I already found http://bugs.python.org/issue1731717 and stubbed out subprocess._cleanup, but this didn't really help and doesn't seem to be the problem anyway... A bit of information about what I'm trying to do: I try to control several...

Output from subprocess.Popen()
Output from the shell: [cpn@s0 teste]$ set | grep IFS IFS=$' \t\n' Output from subprocess.Popen(): >>> import subprocess as sub >>> p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE) >>> p.stdout.readlines()[1] "IFS=' \t\n" Both outputs for comparison: IFS=$' \t\n' "IFS=' \t\n" The subprocess.Popen() output is missing the $ and the last ' How to get the raw shell output from subprocess.Popen()? Regards, Clodoaldo Pinto Neto Clodoaldo Pinto Neto wrote: > Output from the shell: > > [c...

subprocess.Popen inheriting
Hi, I have a file handle I want to inherit in a child process. I am looking at '_make_inheritable' in 'Popen', but it needs an instance, and by the time I have one, the subprocess is already running. Can't I call 'Popen._make_inheritable( None, handle )'? The method does not use 'self'. En Tue, 16 Dec 2008 07:29:19 -0200, Aaron Brady <castironpi@gmail.com> escribi�: > I have a file handle I want to inherit in a child process. I am > looking at '_make_inheritable' in 'Popen', but it needs an instance, > and by the time ...

popen in windows application
Hi, Is there any way to get a HANDLE to a new spawned process as the CRT _open() works for console programs? My requirement is _one_ handle for reading and writing. Thank you. -- ------------------------------------ Gernot Frisch http://www.glbasic.com Hi Gernot, use CreateToolhelp32Snapshot(...) or use EnumProcesses(...) to get a list of process id's and the use OpenProcess(...), then you have your handle! If you use CreateProcess(...), then you can use the OUT parameter to a PROCESS_INFORMATION structure to get the PID and handle. Regards Kerem -...

subprocess escaping POpen?!
Hi All, I have a script that does the following: from subprocess import Popen,PIPE,STDOUT def execute(command,cwd): return Popen( command, stderr=STDOUT, stdout=PIPE, universal_newlines=True, cwd=cwd, shell=True, ).communicate()[0] captured = execute('svn up .') Now, if the subversion update requires authentication credentials, it manages to write to the console running the above script, *and* read input from it too. This is a bit baffling to me, I thought Popen.communicate() was happily ...

subprocess.popen question
I am trying to modify a programming example and I am coming up with two problems... first is that I can't seem to pass along the arguments to the external command (I have been able to do that with the old module and cmd is the command I wish to try) all the output seems to be returned as one line (at least when I run the program in spe). import subprocess from os import system cmd = """gawk -f altertime.awk -v time_offset=4 -v outfile="testdat.sco" "i1.sco" """ #subprocess.Popen. last_line = subprocess.Popen(['gawk.exe'], stdout=s...

subprocess popen trouble
I am having trouble using subprocess popen and stdin/stdout I have the following simple test application written in C++, that just echoes back the stdin. It works fine running from the command line: #include <iostream> #include <time.h> int main (int argc, char * const argv[]) { int currenttemp = 0; int settemp = 0; char* str; while(currenttemp < 500){ gets(str); sscanf(str, ">%d", &settemp); currenttemp = settemp; printf("<%d\n", currenttemp++); } return 0; } ###################################################### Then I have ...

Problem with subprocess.Popen()
Hi all, I'm trying to run a windows batch file from a python script using subprocess.popen(). The issue that I'm facing is that, if i give the batch file as parameter to the popen function, the script runs, but if i provide a parameter, it is not working. Can someone help me with the command. The actual windows command to be executed is: test.bat -t ABC_DE_FG_HI_001 "C:\ABCDEFGHIJKLMNOP.TXT" What is the actual parameter required for subprocess.Popen(parameter chain) in this case?? Kindly help me out... Regards and Thanks in advance... Ashwin Murali. > I'm try...

Problem with popen on windows
Hi all, i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000. When starting my GUI app with rubyw (because i don't like the empty window) there is another window poping up whenever i call popen. (or use backquotes to start another process) Using just ruby.exe all seems fine (except the initial nasty empty window) Someone knows a way around? cheers Simon Kroeger Simon (ext) wrote: > Hi all, > > i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000. > > When starting my GUI app with rubyw (because i don't like the empty > window) there is ...

confused with subprocess.Popen
Hello, for a certain app, i used to use a command: os.system("soundwrapper espeak -f line.txt") now, if i wanted to kill espeak, i would have to run: os.system("killall espeak") since the subprocess module allows sending SIGKILL to the process, i decided to switch to using it. However i cant pass the spaced out arguments to it. For example: a=subprocess.Popen("soundwrapper espeak -f line.txt") results in a OSError, no such file or directory. How do i use the Popen function? Regards Soumen ...

Subprocessing with popen or popen3?
Currently I have a class MyServer that extends IO, and then I call: server = MyServer.popen(COMMAND, 'r+) The only problem is I need to separate stderr and stdout. I contemplated popen3, but I don't see how I can make it associated with an object, which I need in the program. And in addition, it seems to lack a way to get the pid. I could theoretically create the stderr, stdout and stdin within the MyServer class at initialization for popen3, but then I can't use the pid to kill the process when I call the shutdown method. Any ideas? -- Posted via http://www.ruby-forum.com/. ...