blob: 9856e6a3330684d290f7de2d40742e0fc1b50359 [file] [log] [blame]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001# subprocess - Subprocesses with accessible I/O streams
2#
Tim Peterse718f612004-10-12 21:51:32 +00003# For more information about this module, see PEP 324.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00004#
Peter Astrand3a708df2005-09-23 17:37:29 +00005# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00006#
Peter Astrand69bf13f2005-02-14 08:56:32 +00007# Licensed to PSF under a Contributor Agreement.
Peter Astrand3a708df2005-09-23 17:37:29 +00008# See http://www.python.org/2.4/license for licensing details.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00009
Raymond Hettinger837dd932004-10-17 16:36:53 +000010r"""subprocess - Subprocesses with accessible I/O streams
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000011
Fredrik Lundh15aaacc2004-10-17 14:47:05 +000012This module allows you to spawn processes, connect to their
13input/output/error pipes, and obtain their return codes. This module
14intends to replace several other, older modules and functions, like:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000015
16os.system
17os.spawn*
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000018
19Information about how the subprocess module can be used to replace these
20modules and functions can be found below.
21
22
23
24Using the subprocess module
25===========================
26This module defines one class called Popen:
27
28class Popen(args, bufsize=0, executable=None,
29 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000030 preexec_fn=None, close_fds=True, shell=False,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000031 cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000032 startupinfo=None, creationflags=0,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000033 restore_signals=True, start_new_session=False, pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000034
35
36Arguments are:
37
38args should be a string, or a sequence of program arguments. The
39program to execute is normally the first item in the args sequence or
40string, but can be explicitly set by using the executable argument.
41
Gregory P. Smithf5604852010-12-13 06:45:02 +000042On POSIX, with shell=False (default): In this case, the Popen class
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000043uses os.execvp() to execute the child program. args should normally
44be a sequence. A string will be treated as a sequence with the string
45as the only item (the program to execute).
46
Gregory P. Smithf5604852010-12-13 06:45:02 +000047On POSIX, with shell=True: If args is a string, it specifies the
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000048command string to execute through the shell. If args is a sequence,
49the first item specifies the command string, and any additional items
50will be treated as additional shell arguments.
51
52On Windows: the Popen class uses CreateProcess() to execute the child
53program, which operates on strings. If args is a sequence, it will be
54converted to a string using the list2cmdline method. Please note that
55not all MS Windows applications interpret the command line the same
56way: The list2cmdline is designed for applications using the same
57rules as the MS C runtime.
58
59bufsize, if given, has the same meaning as the corresponding argument
60to the built-in open() function: 0 means unbuffered, 1 means line
61buffered, any other positive value means use a buffer of
62(approximately) that size. A negative bufsize means to use the system
63default, which usually means fully buffered. The default value for
64bufsize is 0 (unbuffered).
65
66stdin, stdout and stderr specify the executed programs' standard
67input, standard output and standard error file handles, respectively.
68Valid values are PIPE, an existing file descriptor (a positive
69integer), an existing file object, and None. PIPE indicates that a
70new pipe to the child should be created. With None, no redirection
71will occur; the child's file handles will be inherited from the
72parent. Additionally, stderr can be STDOUT, which indicates that the
73stderr data from the applications should be captured into the same
74file handle as for stdout.
75
Gregory P. Smithf5604852010-12-13 06:45:02 +000076On POSIX, if preexec_fn is set to a callable object, this object will be
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000077called in the child process just before the child is executed. The use
78of preexec_fn is not thread safe, using it in the presence of threads
79could lead to a deadlock in the child process before the new executable
80is executed.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000081
82If close_fds is true, all file descriptors except 0, 1 and 2 will be
Gregory P. Smithf5604852010-12-13 06:45:02 +000083closed before the child process is executed. The default for close_fds
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000084varies by platform: Always true on POSIX. True when stdin/stdout/stderr
85are None on Windows, false otherwise.
86
87pass_fds is an optional sequence of file descriptors to keep open between the
88parent and child. Providing any pass_fds implicitly sets close_fds to true.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000089
90if shell is true, the specified command will be executed through the
91shell.
92
93If cwd is not None, the current directory will be changed to cwd
94before the child is executed.
95
Gregory P. Smithf5604852010-12-13 06:45:02 +000096On POSIX, if restore_signals is True all signals that Python sets to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000097SIG_IGN are restored to SIG_DFL in the child process before the exec.
98Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99parameter does nothing on Windows.
100
Gregory P. Smithf5604852010-12-13 06:45:02 +0000101On POSIX, if start_new_session is True, the setsid() system call will be made
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000102in the child process prior to executing the command.
103
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000104If env is not None, it defines the environment variables for the new
105process.
106
107If universal_newlines is true, the file objects stdout and stderr are
108opened as a text files, but lines may be terminated by any of '\n',
Gregory P. Smithf5604852010-12-13 06:45:02 +0000109the Unix end-of-line convention, '\r', the old Macintosh convention or
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000110'\r\n', the Windows convention. All of these external representations
111are seen as '\n' by the Python program. Note: This feature is only
112available if Python is built with universal newline support (the
113default). Also, the newlines attribute of the file objects stdout,
114stdin and stderr are not updated by the communicate() method.
115
116The startupinfo and creationflags, if given, will be passed to the
117underlying CreateProcess() function. They can specify things such as
118appearance of the main window and priority for the new process.
119(Windows only)
120
121
Georg Brandlf9734072008-12-07 15:30:06 +0000122This module also defines some shortcut functions:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000123
Peter Astrand5f5e1412004-12-05 20:15:36 +0000124call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000125 Run command with arguments. Wait for command to complete, then
126 return the returncode attribute.
127
128 The arguments are the same as for the Popen constructor. Example:
129
Florent Xicluna4886d242010-03-08 13:27:26 +0000130 >>> retcode = subprocess.call(["ls", "-l"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000131
Peter Astrand454f7672005-01-01 09:36:35 +0000132check_call(*popenargs, **kwargs):
133 Run command with arguments. Wait for command to complete. If the
134 exit code was zero then return, otherwise raise
135 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000136 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000137
138 The arguments are the same as for the Popen constructor. Example:
139
Florent Xicluna4886d242010-03-08 13:27:26 +0000140 >>> subprocess.check_call(["ls", "-l"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000141 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000142
Brett Cannona23810f2008-05-26 19:04:21 +0000143getstatusoutput(cmd):
144 Return (status, output) of executing cmd in a shell.
145
146 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
147 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
148 returned output will contain output or error messages. A trailing newline
149 is stripped from the output. The exit status for the command can be
150 interpreted according to the rules for the C function wait(). Example:
151
Brett Cannona23810f2008-05-26 19:04:21 +0000152 >>> subprocess.getstatusoutput('ls /bin/ls')
153 (0, '/bin/ls')
154 >>> subprocess.getstatusoutput('cat /bin/junk')
155 (256, 'cat: /bin/junk: No such file or directory')
156 >>> subprocess.getstatusoutput('/bin/junk')
157 (256, 'sh: /bin/junk: not found')
158
159getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell.
161
162 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example:
164
Brett Cannona23810f2008-05-26 19:04:21 +0000165 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls'
167
Georg Brandlf9734072008-12-07 15:30:06 +0000168check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000169 Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000170
Georg Brandl2708f3a2009-12-20 14:38:23 +0000171 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000174
Georg Brandl2708f3a2009-12-20 14:38:23 +0000175 The arguments are the same as for the Popen constructor. Example:
Georg Brandlf9734072008-12-07 15:30:06 +0000176
Georg Brandl2708f3a2009-12-20 14:38:23 +0000177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandlf9734072008-12-07 15:30:06 +0000178
Brett Cannona23810f2008-05-26 19:04:21 +0000179
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000180Exceptions
181----------
182Exceptions raised in the child process, before the new program has
183started to execute, will be re-raised in the parent. Additionally,
184the exception object will have one extra attribute called
185'child_traceback', which is a string containing traceback information
186from the childs point of view.
187
188The most common exception raised is OSError. This occurs, for
189example, when trying to execute a non-existent file. Applications
190should prepare for OSErrors.
191
192A ValueError will be raised if Popen is called with invalid arguments.
193
Georg Brandlf9734072008-12-07 15:30:06 +0000194check_call() and check_output() will raise CalledProcessError, if the
195called process returns a non-zero return code.
Peter Astrand454f7672005-01-01 09:36:35 +0000196
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000197
198Security
199--------
200Unlike some other popen functions, this implementation will never call
201/bin/sh implicitly. This means that all characters, including shell
202metacharacters, can safely be passed to child processes.
203
204
205Popen objects
206=============
207Instances of the Popen class have the following methods:
208
209poll()
210 Check if child process has terminated. Returns returncode
211 attribute.
212
213wait()
214 Wait for child process to terminate. Returns returncode attribute.
215
216communicate(input=None)
217 Interact with process: Send data to stdin. Read data from stdout
218 and stderr, until end-of-file is reached. Wait for process to
Thomas Wouters902d6eb2007-01-09 23:18:33 +0000219 terminate. The optional input argument should be a string to be
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000220 sent to the child process, or None, if no data should be sent to
221 the child.
Tim Peterse718f612004-10-12 21:51:32 +0000222
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000223 communicate() returns a tuple (stdout, stderr).
224
225 Note: The data read is buffered in memory, so do not use this
226 method if the data size is large or unlimited.
227
228The following attributes are also available:
229
230stdin
231 If the stdin argument is PIPE, this attribute is a file object
232 that provides input to the child process. Otherwise, it is None.
233
234stdout
235 If the stdout argument is PIPE, this attribute is a file object
236 that provides output from the child process. Otherwise, it is
237 None.
238
239stderr
240 If the stderr argument is PIPE, this attribute is file object that
241 provides error output from the child process. Otherwise, it is
242 None.
243
244pid
245 The process ID of the child process.
246
247returncode
248 The child return code. A None value indicates that the process
249 hasn't terminated yet. A negative value -N indicates that the
Gregory P. Smithf5604852010-12-13 06:45:02 +0000250 child was terminated by signal N (POSIX only).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000251
252
253Replacing older functions with the subprocess module
254====================================================
255In this section, "a ==> b" means that b can be used as a replacement
256for a.
257
258Note: All functions in this section fail (more or less) silently if
259the executed program cannot be found; this module raises an OSError
260exception.
261
262In the following examples, we assume that the subprocess module is
263imported with "from subprocess import *".
264
265
266Replacing /bin/sh shell backquote
267---------------------------------
268output=`mycmd myarg`
269==>
270output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
271
272
273Replacing shell pipe line
274-------------------------
275output=`dmesg | grep hda`
276==>
277p1 = Popen(["dmesg"], stdout=PIPE)
Peter Astrand6fdf3cb2004-11-30 18:06:42 +0000278p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000279output = p2.communicate()[0]
280
281
282Replacing os.system()
283---------------------
284sts = os.system("mycmd" + " myarg")
285==>
286p = Popen("mycmd" + " myarg", shell=True)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000287pid, sts = os.waitpid(p.pid, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000288
289Note:
290
291* Calling the program through the shell is usually not required.
292
293* It's easier to look at the returncode attribute than the
294 exitstatus.
295
296A more real-world example would look like this:
297
298try:
299 retcode = call("mycmd" + " myarg", shell=True)
300 if retcode < 0:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000301 print("Child was terminated by signal", -retcode, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000302 else:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000303 print("Child returned", retcode, file=sys.stderr)
304except OSError as e:
305 print("Execution failed:", e, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000306
307
308Replacing os.spawn*
309-------------------
Tim Peterse718f612004-10-12 21:51:32 +0000310P_NOWAIT example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000311
312pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
313==>
314pid = Popen(["/bin/mycmd", "myarg"]).pid
315
316
317P_WAIT example:
318
319retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
320==>
321retcode = call(["/bin/mycmd", "myarg"])
322
323
Tim Peterse718f612004-10-12 21:51:32 +0000324Vector example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000325
326os.spawnvp(os.P_NOWAIT, path, args)
327==>
328Popen([path] + args[1:])
329
330
Tim Peterse718f612004-10-12 21:51:32 +0000331Environment example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000332
333os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
334==>
335Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000336"""
337
338import sys
339mswindows = (sys.platform == "win32")
340
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000341import io
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000342import os
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000343import traceback
Christian Heimesfdab48e2008-01-20 09:06:41 +0000344import gc
Christian Heimesa342c012008-04-20 21:01:16 +0000345import signal
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000346import builtins
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000347import warnings
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200348import errno
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000349
Peter Astrand454f7672005-01-01 09:36:35 +0000350# Exception classes used by this module.
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000351class CalledProcessError(Exception):
Georg Brandlf9734072008-12-07 15:30:06 +0000352 """This exception is raised when a process run by check_call() or
353 check_output() returns a non-zero exit status.
354 The exit status will be stored in the returncode attribute;
355 check_output() will also store the output in the output attribute.
356 """
357 def __init__(self, returncode, cmd, output=None):
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000358 self.returncode = returncode
359 self.cmd = cmd
Georg Brandlf9734072008-12-07 15:30:06 +0000360 self.output = output
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000361 def __str__(self):
362 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
363
Peter Astrand454f7672005-01-01 09:36:35 +0000364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000365if mswindows:
366 import threading
367 import msvcrt
Brian Curtin1ce6b582010-04-24 16:19:22 +0000368 import _subprocess
369 class STARTUPINFO:
370 dwFlags = 0
371 hStdInput = None
372 hStdOutput = None
373 hStdError = None
374 wShowWindow = 0
375 class pywintypes:
376 error = IOError
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000377else:
378 import select
Gregory P. Smithd06fa472009-07-04 02:46:54 +0000379 _has_poll = hasattr(select, 'poll')
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000380 import fcntl
381 import pickle
382
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000383 try:
384 import _posixsubprocess
385 except ImportError:
386 _posixsubprocess = None
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000387 warnings.warn("The _posixsubprocess module is not being used. "
388 "Child process reliability may suffer if your "
389 "program uses threads.", RuntimeWarning)
390
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000391 # When select or poll has indicated that the file is writable,
392 # we can write up to _PIPE_BUF bytes without risk of blocking.
393 # POSIX defines PIPE_BUF as >= 512.
394 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
395
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000396 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
397
398 def _set_cloexec(fd, cloexec):
399 old = fcntl.fcntl(fd, fcntl.F_GETFD)
400 if cloexec:
401 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
402 else:
403 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
404
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000405 if _posixsubprocess:
406 _create_pipe = _posixsubprocess.cloexec_pipe
407 else:
408 def _create_pipe():
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000409 fds = os.pipe()
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000410 _set_cloexec(fds[0], True)
411 _set_cloexec(fds[1], True)
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000412 return fds
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000413
Brett Cannona23810f2008-05-26 19:04:21 +0000414__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
Georg Brandlf9734072008-12-07 15:30:06 +0000415 "getoutput", "check_output", "CalledProcessError"]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000416
Brian Curtin1ce6b582010-04-24 16:19:22 +0000417if mswindows:
Brian Curtin08fd8d92011-04-29 16:11:30 -0500418 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500419 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
420 STD_ERROR_HANDLE, SW_HIDE,
421 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
422
Brian Curtin08fd8d92011-04-29 16:11:30 -0500423 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500424 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
425 "STD_ERROR_HANDLE", "SW_HIDE",
426 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000427try:
428 MAXFD = os.sysconf("SC_OPEN_MAX")
429except:
430 MAXFD = 256
431
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000432_active = []
433
434def _cleanup():
435 for inst in _active[:]:
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000436 res = inst._internal_poll(_deadstate=sys.maxsize)
Guido van Rossumb5d47ef2006-08-24 02:27:45 +0000437 if res is not None and res >= 0:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000438 try:
439 _active.remove(inst)
440 except ValueError:
441 # This can happen if two threads create a new Popen instance.
442 # It's harmless that it was already removed, so ignore.
443 pass
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000444
445PIPE = -1
446STDOUT = -2
447
448
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000449def _eintr_retry_call(func, *args):
450 while True:
451 try:
452 return func(*args)
453 except OSError as e:
454 if e.errno == errno.EINTR:
455 continue
456 raise
457
458
Peter Astrand5f5e1412004-12-05 20:15:36 +0000459def call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000460 """Run command with arguments. Wait for command to complete, then
461 return the returncode attribute.
462
463 The arguments are the same as for the Popen constructor. Example:
464
465 retcode = call(["ls", "-l"])
466 """
Peter Astrand5f5e1412004-12-05 20:15:36 +0000467 return Popen(*popenargs, **kwargs).wait()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000468
469
Peter Astrand454f7672005-01-01 09:36:35 +0000470def check_call(*popenargs, **kwargs):
471 """Run command with arguments. Wait for command to complete. If
472 the exit code was zero then return, otherwise raise
473 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000474 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000475
476 The arguments are the same as for the Popen constructor. Example:
477
478 check_call(["ls", "-l"])
479 """
480 retcode = call(*popenargs, **kwargs)
Peter Astrand454f7672005-01-01 09:36:35 +0000481 if retcode:
Georg Brandlf9734072008-12-07 15:30:06 +0000482 cmd = kwargs.get("args")
483 if cmd is None:
484 cmd = popenargs[0]
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000485 raise CalledProcessError(retcode, cmd)
Georg Brandlf9734072008-12-07 15:30:06 +0000486 return 0
487
488
489def check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000490 r"""Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000491
492 If the exit code was non-zero it raises a CalledProcessError. The
493 CalledProcessError object will have the return code in the returncode
494 attribute and output in the output attribute.
495
496 The arguments are the same as for the Popen constructor. Example:
497
498 >>> check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000499 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000500
501 The stdout argument is not allowed as it is used internally.
Georg Brandl127d4702009-12-28 08:10:38 +0000502 To capture standard error in the result, use stderr=STDOUT.
Georg Brandlf9734072008-12-07 15:30:06 +0000503
504 >>> check_output(["/bin/sh", "-c",
Georg Brandl2708f3a2009-12-20 14:38:23 +0000505 ... "ls -l non_existent_file ; exit 0"],
Georg Brandl127d4702009-12-28 08:10:38 +0000506 ... stderr=STDOUT)
Georg Brandl2708f3a2009-12-20 14:38:23 +0000507 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000508 """
509 if 'stdout' in kwargs:
510 raise ValueError('stdout argument not allowed, it will be overridden.')
511 process = Popen(*popenargs, stdout=PIPE, **kwargs)
512 output, unused_err = process.communicate()
513 retcode = process.poll()
514 if retcode:
515 cmd = kwargs.get("args")
516 if cmd is None:
517 cmd = popenargs[0]
518 raise CalledProcessError(retcode, cmd, output=output)
519 return output
Peter Astrand454f7672005-01-01 09:36:35 +0000520
521
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000522def list2cmdline(seq):
523 """
524 Translate a sequence of arguments into a command line
525 string, using the same rules as the MS C runtime:
526
527 1) Arguments are delimited by white space, which is either a
528 space or a tab.
529
530 2) A string surrounded by double quotation marks is
531 interpreted as a single argument, regardless of white space
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000532 contained within. A quoted string can be embedded in an
533 argument.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000534
535 3) A double quotation mark preceded by a backslash is
536 interpreted as a literal double quotation mark.
537
538 4) Backslashes are interpreted literally, unless they
539 immediately precede a double quotation mark.
540
541 5) If backslashes immediately precede a double quotation mark,
542 every pair of backslashes is interpreted as a literal
543 backslash. If the number of backslashes is odd, the last
544 backslash escapes the next double quotation mark as
545 described in rule 3.
546 """
547
548 # See
Eric Smith3c573af2009-11-09 15:23:15 +0000549 # http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
550 # or search http://msdn.microsoft.com for
551 # "Parsing C++ Command-Line Arguments"
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000552 result = []
553 needquote = False
554 for arg in seq:
555 bs_buf = []
556
557 # Add a space to separate this argument from the others
558 if result:
559 result.append(' ')
560
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000561 needquote = (" " in arg) or ("\t" in arg) or not arg
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000562 if needquote:
563 result.append('"')
564
565 for c in arg:
566 if c == '\\':
Tim Peterse718f612004-10-12 21:51:32 +0000567 # Don't know if we need to double yet.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000568 bs_buf.append(c)
569 elif c == '"':
Christian Heimesfdab48e2008-01-20 09:06:41 +0000570 # Double backslashes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000571 result.append('\\' * len(bs_buf)*2)
572 bs_buf = []
573 result.append('\\"')
574 else:
575 # Normal char
576 if bs_buf:
577 result.extend(bs_buf)
578 bs_buf = []
579 result.append(c)
580
Christian Heimesfdab48e2008-01-20 09:06:41 +0000581 # Add remaining backslashes, if any.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000582 if bs_buf:
583 result.extend(bs_buf)
584
585 if needquote:
Peter Astrand7e78ade2005-03-03 21:10:23 +0000586 result.extend(bs_buf)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000587 result.append('"')
588
589 return ''.join(result)
590
591
Brett Cannona23810f2008-05-26 19:04:21 +0000592# Various tools for executing commands and looking at their output and status.
593#
Gregory P. Smithf5604852010-12-13 06:45:02 +0000594# NB This only works (and is only relevant) for POSIX.
Brett Cannona23810f2008-05-26 19:04:21 +0000595
596def getstatusoutput(cmd):
597 """Return (status, output) of executing cmd in a shell.
598
599 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
600 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
601 returned output will contain output or error messages. A trailing newline
602 is stripped from the output. The exit status for the command can be
603 interpreted according to the rules for the C function wait(). Example:
604
605 >>> import subprocess
606 >>> subprocess.getstatusoutput('ls /bin/ls')
607 (0, '/bin/ls')
608 >>> subprocess.getstatusoutput('cat /bin/junk')
609 (256, 'cat: /bin/junk: No such file or directory')
610 >>> subprocess.getstatusoutput('/bin/junk')
611 (256, 'sh: /bin/junk: not found')
612 """
613 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
614 text = pipe.read()
615 sts = pipe.close()
616 if sts is None: sts = 0
617 if text[-1:] == '\n': text = text[:-1]
618 return sts, text
619
620
621def getoutput(cmd):
622 """Return output (stdout or stderr) of executing cmd in a shell.
623
624 Like getstatusoutput(), except the exit status is ignored and the return
625 value is a string containing the command's output. Example:
626
627 >>> import subprocess
628 >>> subprocess.getoutput('ls /bin/ls')
629 '/bin/ls'
630 """
631 return getstatusoutput(cmd)[1]
632
633
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000634_PLATFORM_DEFAULT_CLOSE_FDS = object()
Gregory P. Smithf5604852010-12-13 06:45:02 +0000635
636
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000637class Popen(object):
638 def __init__(self, args, bufsize=0, executable=None,
639 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000640 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
641 shell=False, cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000642 startupinfo=None, creationflags=0,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000643 restore_signals=True, start_new_session=False,
644 pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000645 """Create new Popen instance."""
646 _cleanup()
647
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000648 self._child_created = False
Guido van Rossum46a05a72007-06-07 21:56:45 +0000649 if bufsize is None:
650 bufsize = 0 # Restore default
Walter Dörwaldaa97f042007-05-03 21:05:51 +0000651 if not isinstance(bufsize, int):
Peter Astrand738131d2004-11-30 21:04:45 +0000652 raise TypeError("bufsize must be an integer")
653
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000654 if mswindows:
Tim Peterse8374a52004-10-13 03:15:00 +0000655 if preexec_fn is not None:
656 raise ValueError("preexec_fn is not supported on Windows "
657 "platforms")
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000658 any_stdio_set = (stdin is not None or stdout is not None or
659 stderr is not None)
660 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
661 if any_stdio_set:
662 close_fds = False
663 else:
664 close_fds = True
665 elif close_fds and any_stdio_set:
666 raise ValueError(
667 "close_fds is not supported on Windows platforms"
668 " if you redirect stdin/stdout/stderr")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000669 else:
670 # POSIX
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000671 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
672 close_fds = True
673 if pass_fds and not close_fds:
674 warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
675 close_fds = True
Tim Peterse8374a52004-10-13 03:15:00 +0000676 if startupinfo is not None:
677 raise ValueError("startupinfo is only supported on Windows "
678 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000679 if creationflags != 0:
Tim Peterse8374a52004-10-13 03:15:00 +0000680 raise ValueError("creationflags is only supported on Windows "
681 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000682
Tim Peterse718f612004-10-12 21:51:32 +0000683 self.stdin = None
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000684 self.stdout = None
685 self.stderr = None
686 self.pid = None
687 self.returncode = None
688 self.universal_newlines = universal_newlines
689
690 # Input and output objects. The general principle is like
691 # this:
692 #
693 # Parent Child
694 # ------ -----
695 # p2cwrite ---stdin---> p2cread
696 # c2pread <--stdout--- c2pwrite
697 # errread <--stderr--- errwrite
698 #
699 # On POSIX, the child objects are file descriptors. On
700 # Windows, these are Windows file handles. The parent objects
701 # are file descriptors on both platforms. The parent objects
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000702 # are -1 when not using PIPEs. The child objects are -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000703 # when not redirecting.
Tim Peterse718f612004-10-12 21:51:32 +0000704
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000705 (p2cread, p2cwrite,
706 c2pread, c2pwrite,
707 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
708
Antoine Pitrouc9982322011-01-04 19:07:07 +0000709 # We wrap OS handles *before* launching the child, otherwise a
710 # quickly terminating child could make our fds unwrappable
711 # (see #8458).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000712
Thomas Wouterscf297e42007-02-23 15:07:44 +0000713 if mswindows:
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000714 if p2cwrite != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000715 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000716 if c2pread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000717 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000718 if errread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000719 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
Thomas Wouterscf297e42007-02-23 15:07:44 +0000720
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000721 if p2cwrite != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000722 self.stdin = io.open(p2cwrite, 'wb', bufsize)
723 if self.universal_newlines:
724 self.stdin = io.TextIOWrapper(self.stdin)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000725 if c2pread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000726 self.stdout = io.open(c2pread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000727 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000728 self.stdout = io.TextIOWrapper(self.stdout)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000729 if errread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000730 self.stderr = io.open(errread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000731 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000732 self.stderr = io.TextIOWrapper(self.stderr)
Tim Peterse718f612004-10-12 21:51:32 +0000733
Antoine Pitrouc9982322011-01-04 19:07:07 +0000734 try:
735 self._execute_child(args, executable, preexec_fn, close_fds,
736 pass_fds, cwd, env, universal_newlines,
737 startupinfo, creationflags, shell,
738 p2cread, p2cwrite,
739 c2pread, c2pwrite,
740 errread, errwrite,
741 restore_signals, start_new_session)
742 except:
743 # Cleanup if the child failed starting
744 for f in filter(None, [self.stdin, self.stdout, self.stderr]):
745 try:
746 f.close()
747 except EnvironmentError:
748 # Ignore EBADF or other errors
749 pass
750 raise
751
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000752
Guido van Rossum98297ee2007-11-06 21:34:58 +0000753 def _translate_newlines(self, data, encoding):
754 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")
755 return data.decode(encoding)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000756
Brian Curtin79cdb662010-12-03 02:46:02 +0000757 def __enter__(self):
758 return self
759
760 def __exit__(self, type, value, traceback):
761 if self.stdout:
762 self.stdout.close()
763 if self.stderr:
764 self.stderr.close()
765 if self.stdin:
766 self.stdin.close()
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000767
Brett Cannon84df1e62010-05-14 00:33:40 +0000768 def __del__(self, _maxsize=sys.maxsize, _active=_active):
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000769 if not self._child_created:
770 # We didn't get to successfully create a child process.
771 return
772 # In case the child hasn't been waited on, check if it's done.
Brett Cannon84df1e62010-05-14 00:33:40 +0000773 self._internal_poll(_deadstate=_maxsize)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000774 if self.returncode is None and _active is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000775 # Child is still running, keep us alive until we can wait on it.
776 _active.append(self)
777
778
Peter Astrand23109f02005-03-03 20:28:59 +0000779 def communicate(self, input=None):
780 """Interact with process: Send data to stdin. Read data from
781 stdout and stderr, until end-of-file is reached. Wait for
782 process to terminate. The optional input argument should be a
783 string to be sent to the child process, or None, if no data
784 should be sent to the child.
Tim Peterseba28be2005-03-28 01:08:02 +0000785
Peter Astrand23109f02005-03-03 20:28:59 +0000786 communicate() returns a tuple (stdout, stderr)."""
787
788 # Optimization: If we are only using one pipe, or no pipe at
789 # all, using select() or threads is unnecessary.
790 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
Tim Peterseba28be2005-03-28 01:08:02 +0000791 stdout = None
792 stderr = None
Peter Astrand23109f02005-03-03 20:28:59 +0000793 if self.stdin:
794 if input:
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200795 try:
796 self.stdin.write(input)
797 except IOError as e:
798 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
799 raise
Peter Astrand23109f02005-03-03 20:28:59 +0000800 self.stdin.close()
801 elif self.stdout:
802 stdout = self.stdout.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000803 self.stdout.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000804 elif self.stderr:
805 stderr = self.stderr.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000806 self.stderr.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000807 self.wait()
808 return (stdout, stderr)
Tim Peterseba28be2005-03-28 01:08:02 +0000809
Peter Astrand23109f02005-03-03 20:28:59 +0000810 return self._communicate(input)
811
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000812
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000813 def poll(self):
814 return self._internal_poll()
815
816
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000817 if mswindows:
818 #
819 # Windows methods
820 #
821 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +0000822 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000823 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
824 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000825 if stdin is None and stdout is None and stderr is None:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000826 return (-1, -1, -1, -1, -1, -1)
Tim Peterse718f612004-10-12 21:51:32 +0000827
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000828 p2cread, p2cwrite = -1, -1
829 c2pread, c2pwrite = -1, -1
830 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000831
Peter Astrandd38ddf42005-02-10 08:32:50 +0000832 if stdin is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000833 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000834 if p2cread is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000835 p2cread, _ = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000836 elif stdin == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000837 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000838 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000839 p2cread = msvcrt.get_osfhandle(stdin)
840 else:
841 # Assuming file-like object
842 p2cread = msvcrt.get_osfhandle(stdin.fileno())
843 p2cread = self._make_inheritable(p2cread)
844
Peter Astrandd38ddf42005-02-10 08:32:50 +0000845 if stdout is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000846 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000847 if c2pwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000848 _, c2pwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000849 elif stdout == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000850 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000851 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000852 c2pwrite = msvcrt.get_osfhandle(stdout)
853 else:
854 # Assuming file-like object
855 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
856 c2pwrite = self._make_inheritable(c2pwrite)
857
Peter Astrandd38ddf42005-02-10 08:32:50 +0000858 if stderr is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000859 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000860 if errwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000861 _, errwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000862 elif stderr == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000863 errread, errwrite = _subprocess.CreatePipe(None, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000864 elif stderr == STDOUT:
865 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +0000866 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000867 errwrite = msvcrt.get_osfhandle(stderr)
868 else:
869 # Assuming file-like object
870 errwrite = msvcrt.get_osfhandle(stderr.fileno())
871 errwrite = self._make_inheritable(errwrite)
872
873 return (p2cread, p2cwrite,
874 c2pread, c2pwrite,
875 errread, errwrite)
876
877
878 def _make_inheritable(self, handle):
879 """Return a duplicate of handle, which is inheritable"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000880 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
881 handle, _subprocess.GetCurrentProcess(), 0, 1,
882 _subprocess.DUPLICATE_SAME_ACCESS)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000883
884
885 def _find_w9xpopen(self):
886 """Find and return absolut path to w9xpopen.exe"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000887 w9xpopen = os.path.join(
888 os.path.dirname(_subprocess.GetModuleFileName(0)),
Tim Peterse8374a52004-10-13 03:15:00 +0000889 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000890 if not os.path.exists(w9xpopen):
891 # Eeek - file-not-found - possibly an embedding
892 # situation - see if we can locate it in sys.exec_prefix
Tim Peterse8374a52004-10-13 03:15:00 +0000893 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
894 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000895 if not os.path.exists(w9xpopen):
Tim Peterse8374a52004-10-13 03:15:00 +0000896 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
897 "needed for Popen to work with your "
898 "shell or platform.")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000899 return w9xpopen
900
Tim Peterse718f612004-10-12 21:51:32 +0000901
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000902 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000903 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000904 startupinfo, creationflags, shell,
905 p2cread, p2cwrite,
906 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000907 errread, errwrite,
908 unused_restore_signals, unused_start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000909 """Execute program (MS Windows version)"""
910
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000911 assert not pass_fds, "pass_fds not supported on Windows."
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000912
Guido van Rossum3172c5d2007-10-16 18:12:55 +0000913 if not isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000914 args = list2cmdline(args)
915
Peter Astrandc1d65362004-11-07 14:30:34 +0000916 # Process startup details
Peter Astrandd38ddf42005-02-10 08:32:50 +0000917 if startupinfo is None:
Thomas Wouters73e5a5b2006-06-08 15:35:45 +0000918 startupinfo = STARTUPINFO()
Victor Stinnerb3693582010-05-21 20:13:12 +0000919 if -1 not in (p2cread, c2pwrite, errwrite):
Brian Curtin1ce6b582010-04-24 16:19:22 +0000920 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
Peter Astrandc1d65362004-11-07 14:30:34 +0000921 startupinfo.hStdInput = p2cread
922 startupinfo.hStdOutput = c2pwrite
923 startupinfo.hStdError = errwrite
924
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000925 if shell:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000926 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
927 startupinfo.wShowWindow = _subprocess.SW_HIDE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000928 comspec = os.environ.get("COMSPEC", "cmd.exe")
Tim Golden126c2962010-08-11 14:20:40 +0000929 args = '{} /c "{}"'.format (comspec, args)
Brian Curtin1ce6b582010-04-24 16:19:22 +0000930 if (_subprocess.GetVersion() >= 0x80000000 or
Tim Peterse8374a52004-10-13 03:15:00 +0000931 os.path.basename(comspec).lower() == "command.com"):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000932 # Win9x, or using command.com on NT. We need to
933 # use the w9xpopen intermediate program. For more
934 # information, see KB Q150956
935 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
936 w9xpopen = self._find_w9xpopen()
937 args = '"%s" %s' % (w9xpopen, args)
938 # Not passing CREATE_NEW_CONSOLE has been known to
939 # cause random failures on win9x. Specifically a
940 # dialog: "Your program accessed mem currently in
941 # use at xxx" and a hopeful warning about the
Mark Dickinson934896d2009-02-21 20:59:32 +0000942 # stability of your system. Cost is Ctrl+C won't
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000943 # kill children.
Brian Curtin1ce6b582010-04-24 16:19:22 +0000944 creationflags |= _subprocess.CREATE_NEW_CONSOLE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000945
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000946 # Start the process
947 try:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000948 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
Tim Peterse8374a52004-10-13 03:15:00 +0000949 # no special security
950 None, None,
Guido van Rossume7ba4952007-06-06 23:52:48 +0000951 int(not close_fds),
Tim Peterse8374a52004-10-13 03:15:00 +0000952 creationflags,
953 env,
954 cwd,
955 startupinfo)
Guido van Rossumb940e112007-01-10 16:19:56 +0000956 except pywintypes.error as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000957 # Translate pywintypes.error to WindowsError, which is
958 # a subclass of OSError. FIXME: We should really
Ezio Melotti13925002011-03-16 11:05:33 +0200959 # translate errno using _sys_errlist (or similar), but
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000960 # how can this be done from Python?
961 raise WindowsError(*e.args)
Tim Goldenad537f22010-08-08 11:18:16 +0000962 finally:
963 # Child is launched. Close the parent's copy of those pipe
964 # handles that only the child should have open. You need
965 # to make sure that no handles to the write end of the
966 # output pipe are maintained in this process or else the
967 # pipe will not close when the child process exits and the
968 # ReadFile will hang.
969 if p2cread != -1:
970 p2cread.Close()
971 if c2pwrite != -1:
972 c2pwrite.Close()
973 if errwrite != -1:
974 errwrite.Close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000975
976 # Retain the process handle, but close the thread handle
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000977 self._child_created = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000978 self._handle = hp
979 self.pid = pid
980 ht.Close()
981
Brett Cannon84df1e62010-05-14 00:33:40 +0000982 def _internal_poll(self, _deadstate=None,
Victor Stinnerc807a612010-05-14 21:53:45 +0000983 _WaitForSingleObject=_subprocess.WaitForSingleObject,
984 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
985 _GetExitCodeProcess=_subprocess.GetExitCodeProcess):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000986 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +0000987 attribute.
988
989 This method is called by __del__, so it can only refer to objects
990 in its local scope.
991
992 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000993 if self.returncode is None:
Brett Cannon84df1e62010-05-14 00:33:40 +0000994 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
995 self.returncode = _GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000996 return self.returncode
997
998
999 def wait(self):
1000 """Wait for child process to terminate. Returns returncode
1001 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001002 if self.returncode is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +00001003 _subprocess.WaitForSingleObject(self._handle,
1004 _subprocess.INFINITE)
1005 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001006 return self.returncode
1007
1008
1009 def _readerthread(self, fh, buffer):
1010 buffer.append(fh.read())
Victor Stinner667d4b52010-12-25 22:40:32 +00001011 fh.close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001012
1013
Peter Astrand23109f02005-03-03 20:28:59 +00001014 def _communicate(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001015 stdout = None # Return
1016 stderr = None # Return
1017
1018 if self.stdout:
1019 stdout = []
Tim Peterse8374a52004-10-13 03:15:00 +00001020 stdout_thread = threading.Thread(target=self._readerthread,
1021 args=(self.stdout, stdout))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001022 stdout_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001023 stdout_thread.start()
1024 if self.stderr:
1025 stderr = []
Tim Peterse8374a52004-10-13 03:15:00 +00001026 stderr_thread = threading.Thread(target=self._readerthread,
1027 args=(self.stderr, stderr))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001028 stderr_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001029 stderr_thread.start()
1030
1031 if self.stdin:
Peter Astrandd38ddf42005-02-10 08:32:50 +00001032 if input is not None:
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001033 try:
1034 self.stdin.write(input)
1035 except IOError as e:
1036 if e.errno != errno.EPIPE:
1037 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001038 self.stdin.close()
1039
1040 if self.stdout:
1041 stdout_thread.join()
1042 if self.stderr:
1043 stderr_thread.join()
1044
1045 # All data exchanged. Translate lists into strings.
Peter Astrandd38ddf42005-02-10 08:32:50 +00001046 if stdout is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001047 stdout = stdout[0]
Peter Astrandd38ddf42005-02-10 08:32:50 +00001048 if stderr is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001049 stderr = stderr[0]
1050
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001051 self.wait()
1052 return (stdout, stderr)
1053
Christian Heimesa342c012008-04-20 21:01:16 +00001054 def send_signal(self, sig):
1055 """Send a signal to the process
1056 """
1057 if sig == signal.SIGTERM:
1058 self.terminate()
Brian Curtineb24d742010-04-12 17:16:38 +00001059 elif sig == signal.CTRL_C_EVENT:
1060 os.kill(self.pid, signal.CTRL_C_EVENT)
1061 elif sig == signal.CTRL_BREAK_EVENT:
1062 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
Christian Heimesa342c012008-04-20 21:01:16 +00001063 else:
Brian Curtin19651362010-09-07 13:24:38 +00001064 raise ValueError("Unsupported signal: {}".format(sig))
Christian Heimesa342c012008-04-20 21:01:16 +00001065
1066 def terminate(self):
1067 """Terminates the process
1068 """
Brian Curtin1ce6b582010-04-24 16:19:22 +00001069 _subprocess.TerminateProcess(self._handle, 1)
Christian Heimesa342c012008-04-20 21:01:16 +00001070
1071 kill = terminate
1072
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001073 else:
1074 #
1075 # POSIX methods
1076 #
1077 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +00001078 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001079 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1080 """
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001081 p2cread, p2cwrite = -1, -1
1082 c2pread, c2pwrite = -1, -1
1083 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001084
Peter Astrandd38ddf42005-02-10 08:32:50 +00001085 if stdin is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001086 pass
1087 elif stdin == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001088 p2cread, p2cwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001089 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001090 p2cread = stdin
1091 else:
1092 # Assuming file-like object
1093 p2cread = stdin.fileno()
1094
Peter Astrandd38ddf42005-02-10 08:32:50 +00001095 if stdout is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001096 pass
1097 elif stdout == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001098 c2pread, c2pwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001099 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001100 c2pwrite = stdout
1101 else:
1102 # Assuming file-like object
Tim Peterse718f612004-10-12 21:51:32 +00001103 c2pwrite = stdout.fileno()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001104
Peter Astrandd38ddf42005-02-10 08:32:50 +00001105 if stderr is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001106 pass
1107 elif stderr == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001108 errread, errwrite = _create_pipe()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001109 elif stderr == STDOUT:
1110 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +00001111 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001112 errwrite = stderr
1113 else:
1114 # Assuming file-like object
1115 errwrite = stderr.fileno()
1116
1117 return (p2cread, p2cwrite,
1118 c2pread, c2pwrite,
1119 errread, errwrite)
1120
1121
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001122 def _close_fds(self, fds_to_keep):
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001123 start_fd = 3
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001124 for fd in sorted(fds_to_keep):
Gregory P. Smith8edd99d2010-12-14 13:43:30 +00001125 if fd >= start_fd:
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001126 os.closerange(start_fd, fd)
1127 start_fd = fd + 1
1128 if start_fd <= MAXFD:
1129 os.closerange(start_fd, MAXFD)
1130
1131
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001132 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001133 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001134 startupinfo, creationflags, shell,
1135 p2cread, p2cwrite,
1136 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001137 errread, errwrite,
1138 restore_signals, start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001139 """Execute program (POSIX version)"""
1140
Guido van Rossum3172c5d2007-10-16 18:12:55 +00001141 if isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001142 args = [args]
Thomas Wouters89f507f2006-12-13 04:49:30 +00001143 else:
1144 args = list(args)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001145
1146 if shell:
1147 args = ["/bin/sh", "-c"] + args
Stefan Krah9542cc62010-07-19 14:20:53 +00001148 if executable:
1149 args[0] = executable
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001150
Peter Astrandd38ddf42005-02-10 08:32:50 +00001151 if executable is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001152 executable = args[0]
1153
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001154 # For transferring possible exec failure from child to parent.
1155 # Data format: "exception name:hex errno:description"
1156 # Pickle is not used; it is complex and involves memory allocation.
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001157 errpipe_read, errpipe_write = _create_pipe()
Christian Heimesfdab48e2008-01-20 09:06:41 +00001158 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001159 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001160
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001161 if _posixsubprocess:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001162 # We must avoid complex work that could involve
1163 # malloc or free in the child process to avoid
1164 # potential deadlocks, thus we do all this here.
1165 # and pass it to fork_exec()
1166
1167 if env:
Victor Stinner449c4662010-05-08 11:10:09 +00001168 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001169 for k, v in env.items()]
1170 else:
1171 env_list = None # Use execv instead of execve.
Victor Stinnerb745a742010-05-18 17:17:23 +00001172 executable = os.fsencode(executable)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001173 if os.path.dirname(executable):
Victor Stinnerb745a742010-05-18 17:17:23 +00001174 executable_list = (executable,)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001175 else:
1176 # This matches the behavior of os._execvpe().
Victor Stinnerb745a742010-05-18 17:17:23 +00001177 executable_list = tuple(
1178 os.path.join(os.fsencode(dir), executable)
1179 for dir in os.get_exec_path(env))
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001180 fds_to_keep = set(pass_fds)
1181 fds_to_keep.add(errpipe_write)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001182 self.pid = _posixsubprocess.fork_exec(
1183 args, executable_list,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001184 close_fds, sorted(fds_to_keep), cwd, env_list,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001185 p2cread, p2cwrite, c2pread, c2pwrite,
1186 errread, errwrite,
1187 errpipe_read, errpipe_write,
1188 restore_signals, start_new_session, preexec_fn)
1189 else:
1190 # Pure Python implementation: It is not thread safe.
1191 # This implementation may deadlock in the child if your
1192 # parent process has any other threads running.
1193
1194 gc_was_enabled = gc.isenabled()
1195 # Disable gc to avoid bug where gc -> file_dealloc ->
1196 # write to stderr -> hang. See issue1336
1197 gc.disable()
1198 try:
1199 self.pid = os.fork()
1200 except:
1201 if gc_was_enabled:
1202 gc.enable()
1203 raise
1204 self._child_created = True
1205 if self.pid == 0:
1206 # Child
1207 try:
1208 # Close parent's pipe ends
1209 if p2cwrite != -1:
1210 os.close(p2cwrite)
1211 if c2pread != -1:
1212 os.close(c2pread)
1213 if errread != -1:
1214 os.close(errread)
1215 os.close(errpipe_read)
1216
1217 # Dup fds for child
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001218 def _dup2(a, b):
1219 # dup2() removes the CLOEXEC flag but
1220 # we must do it ourselves if dup2()
1221 # would be a no-op (issue #10806).
1222 if a == b:
1223 _set_cloexec(a, False)
1224 elif a != -1:
1225 os.dup2(a, b)
1226 _dup2(p2cread, 0)
1227 _dup2(c2pwrite, 1)
1228 _dup2(errwrite, 2)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001229
1230 # Close pipe fds. Make sure we don't close the
1231 # same fd more than once, or standard fds.
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001232 closed = set()
1233 for fd in [p2cread, c2pwrite, errwrite]:
1234 if fd > 2 and fd not in closed:
1235 os.close(fd)
1236 closed.add(fd)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001237
1238 # Close all other fds, if asked for
1239 if close_fds:
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001240 fds_to_keep = set(pass_fds)
1241 fds_to_keep.add(errpipe_write)
1242 self._close_fds(fds_to_keep)
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001243
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001244
1245 if cwd is not None:
1246 os.chdir(cwd)
1247
1248 # This is a copy of Python/pythonrun.c
1249 # _Py_RestoreSignals(). If that were exposed
1250 # as a sys._py_restoresignals func it would be
1251 # better.. but this pure python implementation
1252 # isn't likely to be used much anymore.
1253 if restore_signals:
1254 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1255 for sig in signals:
1256 if hasattr(signal, sig):
1257 signal.signal(getattr(signal, sig),
1258 signal.SIG_DFL)
1259
1260 if start_new_session and hasattr(os, 'setsid'):
1261 os.setsid()
1262
1263 if preexec_fn:
1264 preexec_fn()
1265
1266 if env is None:
1267 os.execvp(executable, args)
1268 else:
1269 os.execvpe(executable, args, env)
1270
1271 except:
1272 try:
1273 exc_type, exc_value = sys.exc_info()[:2]
1274 if isinstance(exc_value, OSError):
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001275 errno_num = exc_value.errno
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001276 else:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001277 errno_num = 0
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001278 message = '%s:%x:%s' % (exc_type.__name__,
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001279 errno_num, exc_value)
Victor Stinner4d078042010-04-23 19:28:32 +00001280 message = message.encode(errors="surrogatepass")
1281 os.write(errpipe_write, message)
1282 except Exception:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001283 # We MUST not allow anything odd happening
1284 # above to prevent us from exiting below.
1285 pass
1286
1287 # This exitcode won't be reported to applications
1288 # so it really doesn't matter what we return.
1289 os._exit(255)
1290
1291 # Parent
Facundo Batista10706e22009-06-19 20:34:30 +00001292 if gc_was_enabled:
1293 gc.enable()
Facundo Batista10706e22009-06-19 20:34:30 +00001294 finally:
1295 # be sure the FD is closed no matter what
1296 os.close(errpipe_write)
1297
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001298 if p2cread != -1 and p2cwrite != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001299 os.close(p2cread)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001300 if c2pwrite != -1 and c2pread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001301 os.close(c2pwrite)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001302 if errwrite != -1 and errread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001303 os.close(errwrite)
1304
1305 # Wait for exec to fail or succeed; possibly raising an
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001306 # exception (limited in size)
1307 data = bytearray()
1308 while True:
1309 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1310 data += part
1311 if not part or len(data) > 50000:
1312 break
Facundo Batista10706e22009-06-19 20:34:30 +00001313 finally:
1314 # be sure the FD is closed no matter what
1315 os.close(errpipe_read)
1316
Guido van Rossumaf2362a2007-05-15 22:32:02 +00001317 if data:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001318 try:
1319 _eintr_retry_call(os.waitpid, self.pid, 0)
1320 except OSError as e:
1321 if e.errno != errno.ECHILD:
1322 raise
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001323 try:
1324 exception_name, hex_errno, err_msg = data.split(b':', 2)
1325 except ValueError:
1326 print('Bad exception data:', repr(data))
1327 exception_name = b'RuntimeError'
1328 hex_errno = b'0'
1329 err_msg = b'Unknown'
1330 child_exception_type = getattr(
1331 builtins, exception_name.decode('ascii'),
1332 RuntimeError)
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001333 for fd in (p2cwrite, c2pread, errread):
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001334 if fd != -1:
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001335 os.close(fd)
Victor Stinner4d078042010-04-23 19:28:32 +00001336 err_msg = err_msg.decode(errors="surrogatepass")
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001337 if issubclass(child_exception_type, OSError) and hex_errno:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001338 errno_num = int(hex_errno, 16)
1339 if errno_num != 0:
1340 err_msg = os.strerror(errno_num)
1341 if errno_num == errno.ENOENT:
Benjamin Peterson5f780402010-11-20 18:07:52 +00001342 err_msg += ': ' + repr(args[0])
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001343 raise child_exception_type(errno_num, err_msg)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001344 raise child_exception_type(err_msg)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001345
1346
Brett Cannon84df1e62010-05-14 00:33:40 +00001347 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1348 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1349 _WEXITSTATUS=os.WEXITSTATUS):
1350 # This method is called (indirectly) by __del__, so it cannot
1351 # refer to anything outside of its local scope."""
1352 if _WIFSIGNALED(sts):
1353 self.returncode = -_WTERMSIG(sts)
1354 elif _WIFEXITED(sts):
1355 self.returncode = _WEXITSTATUS(sts)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001356 else:
1357 # Should never happen
1358 raise RuntimeError("Unknown child exit status!")
1359
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001360
Brett Cannon84df1e62010-05-14 00:33:40 +00001361 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1362 _WNOHANG=os.WNOHANG, _os_error=os.error):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001363 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +00001364 attribute.
1365
1366 This method is called by __del__, so it cannot reference anything
1367 outside of the local scope (nor can any methods it calls).
1368
1369 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001370 if self.returncode is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001371 try:
Brett Cannon84df1e62010-05-14 00:33:40 +00001372 pid, sts = _waitpid(self.pid, _WNOHANG)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001373 if pid == self.pid:
1374 self._handle_exitstatus(sts)
Brett Cannon84df1e62010-05-14 00:33:40 +00001375 except _os_error:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +00001376 if _deadstate is not None:
1377 self.returncode = _deadstate
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001378 return self.returncode
1379
1380
1381 def wait(self):
1382 """Wait for child process to terminate. Returns returncode
1383 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001384 if self.returncode is None:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001385 try:
1386 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1387 except OSError as e:
1388 if e.errno != errno.ECHILD:
1389 raise
1390 # This happens if SIGCLD is set to be ignored or waiting
1391 # for child processes has otherwise been disabled for our
1392 # process. This child is dead, we can't get the status.
1393 sts = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001394 self._handle_exitstatus(sts)
1395 return self.returncode
1396
1397
Peter Astrand23109f02005-03-03 20:28:59 +00001398 def _communicate(self, input):
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001399 if self.stdin:
1400 # Flush stdio buffer. This might block, if the user has
1401 # been writing to .stdin in an uncontrolled fashion.
1402 self.stdin.flush()
1403 if not input:
1404 self.stdin.close()
1405
1406 if _has_poll:
1407 stdout, stderr = self._communicate_with_poll(input)
1408 else:
1409 stdout, stderr = self._communicate_with_select(input)
1410
1411 # All data exchanged. Translate lists into strings.
1412 if stdout is not None:
1413 stdout = b''.join(stdout)
1414 if stderr is not None:
1415 stderr = b''.join(stderr)
1416
1417 # Translate newlines, if requested.
1418 # This also turns bytes into strings.
1419 if self.universal_newlines:
1420 if stdout is not None:
1421 stdout = self._translate_newlines(stdout,
1422 self.stdout.encoding)
1423 if stderr is not None:
1424 stderr = self._translate_newlines(stderr,
1425 self.stderr.encoding)
1426
1427 self.wait()
1428 return (stdout, stderr)
1429
1430
1431 def _communicate_with_poll(self, input):
1432 stdout = None # Return
1433 stderr = None # Return
1434 fd2file = {}
1435 fd2output = {}
1436
1437 poller = select.poll()
1438 def register_and_append(file_obj, eventmask):
1439 poller.register(file_obj.fileno(), eventmask)
1440 fd2file[file_obj.fileno()] = file_obj
1441
1442 def close_unregister_and_remove(fd):
1443 poller.unregister(fd)
1444 fd2file[fd].close()
1445 fd2file.pop(fd)
1446
1447 if self.stdin and input:
1448 register_and_append(self.stdin, select.POLLOUT)
1449
1450 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1451 if self.stdout:
1452 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1453 fd2output[self.stdout.fileno()] = stdout = []
1454 if self.stderr:
1455 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1456 fd2output[self.stderr.fileno()] = stderr = []
1457
1458 input_offset = 0
1459 while fd2file:
1460 try:
1461 ready = poller.poll()
1462 except select.error as e:
1463 if e.args[0] == errno.EINTR:
1464 continue
1465 raise
1466
1467 # XXX Rewrite these to use non-blocking I/O on the
1468 # file objects; they are no longer using C stdio!
1469
1470 for fd, mode in ready:
1471 if mode & select.POLLOUT:
1472 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001473 try:
1474 input_offset += os.write(fd, chunk)
1475 except OSError as e:
1476 if e.errno == errno.EPIPE:
1477 close_unregister_and_remove(fd)
1478 else:
1479 raise
1480 else:
1481 if input_offset >= len(input):
1482 close_unregister_and_remove(fd)
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001483 elif mode & select_POLLIN_POLLPRI:
1484 data = os.read(fd, 4096)
1485 if not data:
1486 close_unregister_and_remove(fd)
1487 fd2output[fd].append(data)
1488 else:
1489 # Ignore hang up or errors.
1490 close_unregister_and_remove(fd)
1491
1492 return (stdout, stderr)
1493
1494
1495 def _communicate_with_select(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001496 read_set = []
1497 write_set = []
1498 stdout = None # Return
1499 stderr = None # Return
1500
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001501 if self.stdin and input:
1502 write_set.append(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001503 if self.stdout:
1504 read_set.append(self.stdout)
Tim Peterse718f612004-10-12 21:51:32 +00001505 stdout = []
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001506 if self.stderr:
1507 read_set.append(self.stderr)
1508 stderr = []
1509
Thomas Wouters902d6eb2007-01-09 23:18:33 +00001510 input_offset = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001511 while read_set or write_set:
Georg Brandl86b2fb92008-07-16 03:43:04 +00001512 try:
1513 rlist, wlist, xlist = select.select(read_set, write_set, [])
1514 except select.error as e:
1515 if e.args[0] == errno.EINTR:
1516 continue
1517 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001518
Guido van Rossum98297ee2007-11-06 21:34:58 +00001519 # XXX Rewrite these to use non-blocking I/O on the
1520 # file objects; they are no longer using C stdio!
1521
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001522 if self.stdin in wlist:
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001523 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001524 try:
1525 bytes_written = os.write(self.stdin.fileno(), chunk)
1526 except OSError as e:
1527 if e.errno == errno.EPIPE:
1528 self.stdin.close()
1529 write_set.remove(self.stdin)
1530 else:
1531 raise
1532 else:
1533 input_offset += bytes_written
1534 if input_offset >= len(input):
1535 self.stdin.close()
1536 write_set.remove(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001537
1538 if self.stdout in rlist:
1539 data = os.read(self.stdout.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001540 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001541 self.stdout.close()
1542 read_set.remove(self.stdout)
1543 stdout.append(data)
1544
1545 if self.stderr in rlist:
1546 data = os.read(self.stderr.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001547 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001548 self.stderr.close()
1549 read_set.remove(self.stderr)
1550 stderr.append(data)
1551
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001552 return (stdout, stderr)
1553
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001554
Christian Heimesa342c012008-04-20 21:01:16 +00001555 def send_signal(self, sig):
1556 """Send a signal to the process
1557 """
1558 os.kill(self.pid, sig)
1559
1560 def terminate(self):
1561 """Terminate the process with SIGTERM
1562 """
1563 self.send_signal(signal.SIGTERM)
1564
1565 def kill(self):
1566 """Kill the process with SIGKILL
1567 """
1568 self.send_signal(signal.SIGKILL)
1569
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001570
1571def _demo_posix():
1572 #
1573 # Example 1: Simple redirection: Get process list
1574 #
1575 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001576 print("Process list:")
1577 print(plist)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001578
1579 #
1580 # Example 2: Change uid before executing child
1581 #
1582 if os.getuid() == 0:
1583 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1584 p.wait()
1585
1586 #
1587 # Example 3: Connecting several subprocesses
1588 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001589 print("Looking for 'hda'...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001590 p1 = Popen(["dmesg"], stdout=PIPE)
1591 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001592 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001593
1594 #
1595 # Example 4: Catch execution error
1596 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001597 print()
1598 print("Trying a weird file...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001599 try:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001600 print(Popen(["/this/path/does/not/exist"]).communicate())
Guido van Rossumb940e112007-01-10 16:19:56 +00001601 except OSError as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001602 if e.errno == errno.ENOENT:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001603 print("The file didn't exist. I thought so...")
1604 print("Child traceback:")
1605 print(e.child_traceback)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001606 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001607 print("Error", e.errno)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001608 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001609 print("Gosh. No error.", file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001610
1611
1612def _demo_windows():
1613 #
1614 # Example 1: Connecting several subprocesses
1615 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001616 print("Looking for 'PROMPT' in set output...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001617 p1 = Popen("set", stdout=PIPE, shell=True)
1618 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001619 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001620
1621 #
1622 # Example 2: Simple execution of program
1623 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001624 print("Executing calc...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001625 p = Popen("calc")
1626 p.wait()
1627
1628
1629if __name__ == "__main__":
1630 if mswindows:
1631 _demo_windows()
1632 else:
1633 _demo_posix()