blob: 28dd691090d7122fe34141640529a613c19717e6 [file] [log] [blame]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001# subprocess - Subprocesses with accessible I/O streams
2#
Tim Peterse718f612004-10-12 21:51:32 +00003# For more information about this module, see PEP 324.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00004#
Peter Astrand3a708df2005-09-23 17:37:29 +00005# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00006#
Peter Astrand69bf13f2005-02-14 08:56:32 +00007# Licensed to PSF under a Contributor Agreement.
Peter Astrand3a708df2005-09-23 17:37:29 +00008# See http://www.python.org/2.4/license for licensing details.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00009
Raymond Hettinger837dd932004-10-17 16:36:53 +000010r"""subprocess - Subprocesses with accessible I/O streams
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000011
Fredrik Lundh15aaacc2004-10-17 14:47:05 +000012This module allows you to spawn processes, connect to their
13input/output/error pipes, and obtain their return codes. This module
14intends to replace several other, older modules and functions, like:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000015
16os.system
17os.spawn*
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000018
19Information about how the subprocess module can be used to replace these
20modules and functions can be found below.
21
22
23
24Using the subprocess module
25===========================
26This module defines one class called Popen:
27
28class Popen(args, bufsize=0, executable=None,
29 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000030 preexec_fn=None, close_fds=True, shell=False,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000031 cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000032 startupinfo=None, creationflags=0,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000033 restore_signals=True, start_new_session=False, pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000034
35
36Arguments are:
37
38args should be a string, or a sequence of program arguments. The
39program to execute is normally the first item in the args sequence or
40string, but can be explicitly set by using the executable argument.
41
Gregory P. Smithf5604852010-12-13 06:45:02 +000042On POSIX, with shell=False (default): In this case, the Popen class
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000043uses os.execvp() to execute the child program. args should normally
44be a sequence. A string will be treated as a sequence with the string
45as the only item (the program to execute).
46
Gregory P. Smithf5604852010-12-13 06:45:02 +000047On POSIX, with shell=True: If args is a string, it specifies the
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000048command string to execute through the shell. If args is a sequence,
49the first item specifies the command string, and any additional items
50will be treated as additional shell arguments.
51
52On Windows: the Popen class uses CreateProcess() to execute the child
53program, which operates on strings. If args is a sequence, it will be
54converted to a string using the list2cmdline method. Please note that
55not all MS Windows applications interpret the command line the same
56way: The list2cmdline is designed for applications using the same
57rules as the MS C runtime.
58
59bufsize, if given, has the same meaning as the corresponding argument
60to the built-in open() function: 0 means unbuffered, 1 means line
61buffered, any other positive value means use a buffer of
62(approximately) that size. A negative bufsize means to use the system
63default, which usually means fully buffered. The default value for
64bufsize is 0 (unbuffered).
65
66stdin, stdout and stderr specify the executed programs' standard
67input, standard output and standard error file handles, respectively.
68Valid values are PIPE, an existing file descriptor (a positive
69integer), an existing file object, and None. PIPE indicates that a
70new pipe to the child should be created. With None, no redirection
71will occur; the child's file handles will be inherited from the
72parent. Additionally, stderr can be STDOUT, which indicates that the
73stderr data from the applications should be captured into the same
74file handle as for stdout.
75
Gregory P. Smithf5604852010-12-13 06:45:02 +000076On POSIX, if preexec_fn is set to a callable object, this object will be
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000077called in the child process just before the child is executed. The use
78of preexec_fn is not thread safe, using it in the presence of threads
79could lead to a deadlock in the child process before the new executable
80is executed.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000081
82If close_fds is true, all file descriptors except 0, 1 and 2 will be
Gregory P. Smithf5604852010-12-13 06:45:02 +000083closed before the child process is executed. The default for close_fds
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000084varies by platform: Always true on POSIX. True when stdin/stdout/stderr
85are None on Windows, false otherwise.
86
87pass_fds is an optional sequence of file descriptors to keep open between the
88parent and child. Providing any pass_fds implicitly sets close_fds to true.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000089
90if shell is true, the specified command will be executed through the
91shell.
92
93If cwd is not None, the current directory will be changed to cwd
94before the child is executed.
95
Gregory P. Smithf5604852010-12-13 06:45:02 +000096On POSIX, if restore_signals is True all signals that Python sets to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000097SIG_IGN are restored to SIG_DFL in the child process before the exec.
98Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99parameter does nothing on Windows.
100
Gregory P. Smithf5604852010-12-13 06:45:02 +0000101On POSIX, if start_new_session is True, the setsid() system call will be made
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000102in the child process prior to executing the command.
103
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000104If env is not None, it defines the environment variables for the new
105process.
106
107If universal_newlines is true, the file objects stdout and stderr are
108opened as a text files, but lines may be terminated by any of '\n',
Gregory P. Smithf5604852010-12-13 06:45:02 +0000109the Unix end-of-line convention, '\r', the old Macintosh convention or
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000110'\r\n', the Windows convention. All of these external representations
111are seen as '\n' by the Python program. Note: This feature is only
112available if Python is built with universal newline support (the
113default). Also, the newlines attribute of the file objects stdout,
114stdin and stderr are not updated by the communicate() method.
115
116The startupinfo and creationflags, if given, will be passed to the
117underlying CreateProcess() function. They can specify things such as
118appearance of the main window and priority for the new process.
119(Windows only)
120
121
Georg Brandlf9734072008-12-07 15:30:06 +0000122This module also defines some shortcut functions:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000123
Peter Astrand5f5e1412004-12-05 20:15:36 +0000124call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000125 Run command with arguments. Wait for command to complete, then
126 return the returncode attribute.
127
128 The arguments are the same as for the Popen constructor. Example:
129
Florent Xicluna4886d242010-03-08 13:27:26 +0000130 >>> retcode = subprocess.call(["ls", "-l"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000131
Peter Astrand454f7672005-01-01 09:36:35 +0000132check_call(*popenargs, **kwargs):
133 Run command with arguments. Wait for command to complete. If the
134 exit code was zero then return, otherwise raise
135 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000136 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000137
138 The arguments are the same as for the Popen constructor. Example:
139
Florent Xicluna4886d242010-03-08 13:27:26 +0000140 >>> subprocess.check_call(["ls", "-l"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000141 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000142
Brett Cannona23810f2008-05-26 19:04:21 +0000143getstatusoutput(cmd):
144 Return (status, output) of executing cmd in a shell.
145
146 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
147 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
148 returned output will contain output or error messages. A trailing newline
149 is stripped from the output. The exit status for the command can be
150 interpreted according to the rules for the C function wait(). Example:
151
Brett Cannona23810f2008-05-26 19:04:21 +0000152 >>> subprocess.getstatusoutput('ls /bin/ls')
153 (0, '/bin/ls')
154 >>> subprocess.getstatusoutput('cat /bin/junk')
155 (256, 'cat: /bin/junk: No such file or directory')
156 >>> subprocess.getstatusoutput('/bin/junk')
157 (256, 'sh: /bin/junk: not found')
158
159getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell.
161
162 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example:
164
Brett Cannona23810f2008-05-26 19:04:21 +0000165 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls'
167
Georg Brandlf9734072008-12-07 15:30:06 +0000168check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000169 Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000170
Georg Brandl2708f3a2009-12-20 14:38:23 +0000171 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000174
Georg Brandl2708f3a2009-12-20 14:38:23 +0000175 The arguments are the same as for the Popen constructor. Example:
Georg Brandlf9734072008-12-07 15:30:06 +0000176
Georg Brandl2708f3a2009-12-20 14:38:23 +0000177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandlf9734072008-12-07 15:30:06 +0000178
Brett Cannona23810f2008-05-26 19:04:21 +0000179
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000180Exceptions
181----------
182Exceptions raised in the child process, before the new program has
183started to execute, will be re-raised in the parent. Additionally,
184the exception object will have one extra attribute called
185'child_traceback', which is a string containing traceback information
186from the childs point of view.
187
188The most common exception raised is OSError. This occurs, for
189example, when trying to execute a non-existent file. Applications
190should prepare for OSErrors.
191
192A ValueError will be raised if Popen is called with invalid arguments.
193
Georg Brandlf9734072008-12-07 15:30:06 +0000194check_call() and check_output() will raise CalledProcessError, if the
195called process returns a non-zero return code.
Peter Astrand454f7672005-01-01 09:36:35 +0000196
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000197
198Security
199--------
200Unlike some other popen functions, this implementation will never call
201/bin/sh implicitly. This means that all characters, including shell
202metacharacters, can safely be passed to child processes.
203
204
205Popen objects
206=============
207Instances of the Popen class have the following methods:
208
209poll()
210 Check if child process has terminated. Returns returncode
211 attribute.
212
213wait()
214 Wait for child process to terminate. Returns returncode attribute.
215
216communicate(input=None)
217 Interact with process: Send data to stdin. Read data from stdout
218 and stderr, until end-of-file is reached. Wait for process to
Thomas Wouters902d6eb2007-01-09 23:18:33 +0000219 terminate. The optional input argument should be a string to be
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000220 sent to the child process, or None, if no data should be sent to
221 the child.
Tim Peterse718f612004-10-12 21:51:32 +0000222
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000223 communicate() returns a tuple (stdout, stderr).
224
225 Note: The data read is buffered in memory, so do not use this
226 method if the data size is large or unlimited.
227
228The following attributes are also available:
229
230stdin
231 If the stdin argument is PIPE, this attribute is a file object
232 that provides input to the child process. Otherwise, it is None.
233
234stdout
235 If the stdout argument is PIPE, this attribute is a file object
236 that provides output from the child process. Otherwise, it is
237 None.
238
239stderr
240 If the stderr argument is PIPE, this attribute is file object that
241 provides error output from the child process. Otherwise, it is
242 None.
243
244pid
245 The process ID of the child process.
246
247returncode
248 The child return code. A None value indicates that the process
249 hasn't terminated yet. A negative value -N indicates that the
Gregory P. Smithf5604852010-12-13 06:45:02 +0000250 child was terminated by signal N (POSIX only).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000251
252
253Replacing older functions with the subprocess module
254====================================================
255In this section, "a ==> b" means that b can be used as a replacement
256for a.
257
258Note: All functions in this section fail (more or less) silently if
259the executed program cannot be found; this module raises an OSError
260exception.
261
262In the following examples, we assume that the subprocess module is
263imported with "from subprocess import *".
264
265
266Replacing /bin/sh shell backquote
267---------------------------------
268output=`mycmd myarg`
269==>
270output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
271
272
273Replacing shell pipe line
274-------------------------
275output=`dmesg | grep hda`
276==>
277p1 = Popen(["dmesg"], stdout=PIPE)
Peter Astrand6fdf3cb2004-11-30 18:06:42 +0000278p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000279output = p2.communicate()[0]
280
281
282Replacing os.system()
283---------------------
284sts = os.system("mycmd" + " myarg")
285==>
286p = Popen("mycmd" + " myarg", shell=True)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000287pid, sts = os.waitpid(p.pid, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000288
289Note:
290
291* Calling the program through the shell is usually not required.
292
293* It's easier to look at the returncode attribute than the
294 exitstatus.
295
296A more real-world example would look like this:
297
298try:
299 retcode = call("mycmd" + " myarg", shell=True)
300 if retcode < 0:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000301 print("Child was terminated by signal", -retcode, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000302 else:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000303 print("Child returned", retcode, file=sys.stderr)
304except OSError as e:
305 print("Execution failed:", e, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000306
307
308Replacing os.spawn*
309-------------------
Tim Peterse718f612004-10-12 21:51:32 +0000310P_NOWAIT example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000311
312pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
313==>
314pid = Popen(["/bin/mycmd", "myarg"]).pid
315
316
317P_WAIT example:
318
319retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
320==>
321retcode = call(["/bin/mycmd", "myarg"])
322
323
Tim Peterse718f612004-10-12 21:51:32 +0000324Vector example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000325
326os.spawnvp(os.P_NOWAIT, path, args)
327==>
328Popen([path] + args[1:])
329
330
Tim Peterse718f612004-10-12 21:51:32 +0000331Environment example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000332
333os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
334==>
335Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000336"""
337
338import sys
339mswindows = (sys.platform == "win32")
340
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000341import io
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000342import os
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000343import traceback
Christian Heimesfdab48e2008-01-20 09:06:41 +0000344import gc
Christian Heimesa342c012008-04-20 21:01:16 +0000345import signal
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000346import builtins
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000347import warnings
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200348import errno
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000349
Peter Astrand454f7672005-01-01 09:36:35 +0000350# Exception classes used by this module.
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000351class CalledProcessError(Exception):
Georg Brandlf9734072008-12-07 15:30:06 +0000352 """This exception is raised when a process run by check_call() or
353 check_output() returns a non-zero exit status.
354 The exit status will be stored in the returncode attribute;
355 check_output() will also store the output in the output attribute.
356 """
357 def __init__(self, returncode, cmd, output=None):
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000358 self.returncode = returncode
359 self.cmd = cmd
Georg Brandlf9734072008-12-07 15:30:06 +0000360 self.output = output
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000361 def __str__(self):
362 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
363
Peter Astrand454f7672005-01-01 09:36:35 +0000364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000365if mswindows:
366 import threading
367 import msvcrt
Brian Curtin1ce6b582010-04-24 16:19:22 +0000368 import _subprocess
369 class STARTUPINFO:
370 dwFlags = 0
371 hStdInput = None
372 hStdOutput = None
373 hStdError = None
374 wShowWindow = 0
375 class pywintypes:
376 error = IOError
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000377else:
378 import select
Gregory P. Smithd06fa472009-07-04 02:46:54 +0000379 _has_poll = hasattr(select, 'poll')
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000380 import fcntl
381 import pickle
382
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000383 try:
384 import _posixsubprocess
385 except ImportError:
386 _posixsubprocess = None
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000387 warnings.warn("The _posixsubprocess module is not being used. "
388 "Child process reliability may suffer if your "
389 "program uses threads.", RuntimeWarning)
390
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000391 # When select or poll has indicated that the file is writable,
392 # we can write up to _PIPE_BUF bytes without risk of blocking.
393 # POSIX defines PIPE_BUF as >= 512.
394 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
395
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000396 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
397
398 def _set_cloexec(fd, cloexec):
399 old = fcntl.fcntl(fd, fcntl.F_GETFD)
400 if cloexec:
401 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
402 else:
403 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
404
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000405 if _posixsubprocess:
406 _create_pipe = _posixsubprocess.cloexec_pipe
407 else:
408 def _create_pipe():
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000409 fds = os.pipe()
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000410 _set_cloexec(fds[0], True)
411 _set_cloexec(fds[1], True)
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000412 return fds
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000413
Brett Cannona23810f2008-05-26 19:04:21 +0000414__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
Georg Brandlf9734072008-12-07 15:30:06 +0000415 "getoutput", "check_output", "CalledProcessError"]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000416
Brian Curtin1ce6b582010-04-24 16:19:22 +0000417if mswindows:
Brian Curtin08fd8d92011-04-29 16:11:30 -0500418 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500419 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
420 STD_ERROR_HANDLE, SW_HIDE,
421 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
Gregory P. Smith0ef3e392011-05-11 22:20:11 -0700422
Brian Curtin08fd8d92011-04-29 16:11:30 -0500423 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500424 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
425 "STD_ERROR_HANDLE", "SW_HIDE",
426 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000427try:
428 MAXFD = os.sysconf("SC_OPEN_MAX")
429except:
430 MAXFD = 256
431
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000432_active = []
433
434def _cleanup():
435 for inst in _active[:]:
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000436 res = inst._internal_poll(_deadstate=sys.maxsize)
Guido van Rossumb5d47ef2006-08-24 02:27:45 +0000437 if res is not None and res >= 0:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000438 try:
439 _active.remove(inst)
440 except ValueError:
441 # This can happen if two threads create a new Popen instance.
442 # It's harmless that it was already removed, so ignore.
443 pass
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000444
445PIPE = -1
446STDOUT = -2
447
448
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000449def _eintr_retry_call(func, *args):
450 while True:
451 try:
452 return func(*args)
453 except OSError as e:
454 if e.errno == errno.EINTR:
455 continue
456 raise
457
458
Peter Astrand5f5e1412004-12-05 20:15:36 +0000459def call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000460 """Run command with arguments. Wait for command to complete, then
461 return the returncode attribute.
462
463 The arguments are the same as for the Popen constructor. Example:
464
465 retcode = call(["ls", "-l"])
466 """
Peter Astrand5f5e1412004-12-05 20:15:36 +0000467 return Popen(*popenargs, **kwargs).wait()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000468
469
Peter Astrand454f7672005-01-01 09:36:35 +0000470def check_call(*popenargs, **kwargs):
471 """Run command with arguments. Wait for command to complete. If
472 the exit code was zero then return, otherwise raise
473 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000474 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000475
476 The arguments are the same as for the Popen constructor. Example:
477
478 check_call(["ls", "-l"])
479 """
480 retcode = call(*popenargs, **kwargs)
Peter Astrand454f7672005-01-01 09:36:35 +0000481 if retcode:
Georg Brandlf9734072008-12-07 15:30:06 +0000482 cmd = kwargs.get("args")
483 if cmd is None:
484 cmd = popenargs[0]
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000485 raise CalledProcessError(retcode, cmd)
Georg Brandlf9734072008-12-07 15:30:06 +0000486 return 0
487
488
489def check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000490 r"""Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000491
492 If the exit code was non-zero it raises a CalledProcessError. The
493 CalledProcessError object will have the return code in the returncode
494 attribute and output in the output attribute.
495
496 The arguments are the same as for the Popen constructor. Example:
497
498 >>> check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000499 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000500
501 The stdout argument is not allowed as it is used internally.
Georg Brandl127d4702009-12-28 08:10:38 +0000502 To capture standard error in the result, use stderr=STDOUT.
Georg Brandlf9734072008-12-07 15:30:06 +0000503
504 >>> check_output(["/bin/sh", "-c",
Georg Brandl2708f3a2009-12-20 14:38:23 +0000505 ... "ls -l non_existent_file ; exit 0"],
Georg Brandl127d4702009-12-28 08:10:38 +0000506 ... stderr=STDOUT)
Georg Brandl2708f3a2009-12-20 14:38:23 +0000507 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000508 """
509 if 'stdout' in kwargs:
510 raise ValueError('stdout argument not allowed, it will be overridden.')
511 process = Popen(*popenargs, stdout=PIPE, **kwargs)
512 output, unused_err = process.communicate()
513 retcode = process.poll()
514 if retcode:
515 cmd = kwargs.get("args")
516 if cmd is None:
517 cmd = popenargs[0]
518 raise CalledProcessError(retcode, cmd, output=output)
519 return output
Peter Astrand454f7672005-01-01 09:36:35 +0000520
521
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000522def list2cmdline(seq):
523 """
524 Translate a sequence of arguments into a command line
525 string, using the same rules as the MS C runtime:
526
527 1) Arguments are delimited by white space, which is either a
528 space or a tab.
529
530 2) A string surrounded by double quotation marks is
531 interpreted as a single argument, regardless of white space
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000532 contained within. A quoted string can be embedded in an
533 argument.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000534
535 3) A double quotation mark preceded by a backslash is
536 interpreted as a literal double quotation mark.
537
538 4) Backslashes are interpreted literally, unless they
539 immediately precede a double quotation mark.
540
541 5) If backslashes immediately precede a double quotation mark,
542 every pair of backslashes is interpreted as a literal
543 backslash. If the number of backslashes is odd, the last
544 backslash escapes the next double quotation mark as
545 described in rule 3.
546 """
547
548 # See
Eric Smith3c573af2009-11-09 15:23:15 +0000549 # http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
550 # or search http://msdn.microsoft.com for
551 # "Parsing C++ Command-Line Arguments"
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000552 result = []
553 needquote = False
554 for arg in seq:
555 bs_buf = []
556
557 # Add a space to separate this argument from the others
558 if result:
559 result.append(' ')
560
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000561 needquote = (" " in arg) or ("\t" in arg) or not arg
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000562 if needquote:
563 result.append('"')
564
565 for c in arg:
566 if c == '\\':
Tim Peterse718f612004-10-12 21:51:32 +0000567 # Don't know if we need to double yet.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000568 bs_buf.append(c)
569 elif c == '"':
Christian Heimesfdab48e2008-01-20 09:06:41 +0000570 # Double backslashes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000571 result.append('\\' * len(bs_buf)*2)
572 bs_buf = []
573 result.append('\\"')
574 else:
575 # Normal char
576 if bs_buf:
577 result.extend(bs_buf)
578 bs_buf = []
579 result.append(c)
580
Christian Heimesfdab48e2008-01-20 09:06:41 +0000581 # Add remaining backslashes, if any.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000582 if bs_buf:
583 result.extend(bs_buf)
584
585 if needquote:
Peter Astrand7e78ade2005-03-03 21:10:23 +0000586 result.extend(bs_buf)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000587 result.append('"')
588
589 return ''.join(result)
590
591
Brett Cannona23810f2008-05-26 19:04:21 +0000592# Various tools for executing commands and looking at their output and status.
593#
Gregory P. Smithf5604852010-12-13 06:45:02 +0000594# NB This only works (and is only relevant) for POSIX.
Brett Cannona23810f2008-05-26 19:04:21 +0000595
596def getstatusoutput(cmd):
597 """Return (status, output) of executing cmd in a shell.
598
599 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
600 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
601 returned output will contain output or error messages. A trailing newline
602 is stripped from the output. The exit status for the command can be
603 interpreted according to the rules for the C function wait(). Example:
604
605 >>> import subprocess
606 >>> subprocess.getstatusoutput('ls /bin/ls')
607 (0, '/bin/ls')
608 >>> subprocess.getstatusoutput('cat /bin/junk')
609 (256, 'cat: /bin/junk: No such file or directory')
610 >>> subprocess.getstatusoutput('/bin/junk')
611 (256, 'sh: /bin/junk: not found')
612 """
613 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
614 text = pipe.read()
615 sts = pipe.close()
616 if sts is None: sts = 0
617 if text[-1:] == '\n': text = text[:-1]
618 return sts, text
619
620
621def getoutput(cmd):
622 """Return output (stdout or stderr) of executing cmd in a shell.
623
624 Like getstatusoutput(), except the exit status is ignored and the return
625 value is a string containing the command's output. Example:
626
627 >>> import subprocess
628 >>> subprocess.getoutput('ls /bin/ls')
629 '/bin/ls'
630 """
631 return getstatusoutput(cmd)[1]
632
633
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000634_PLATFORM_DEFAULT_CLOSE_FDS = object()
Gregory P. Smithf5604852010-12-13 06:45:02 +0000635
636
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000637class Popen(object):
638 def __init__(self, args, bufsize=0, executable=None,
639 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000640 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
641 shell=False, cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000642 startupinfo=None, creationflags=0,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000643 restore_signals=True, start_new_session=False,
644 pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000645 """Create new Popen instance."""
646 _cleanup()
647
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000648 self._child_created = False
Guido van Rossum46a05a72007-06-07 21:56:45 +0000649 if bufsize is None:
650 bufsize = 0 # Restore default
Walter Dörwaldaa97f042007-05-03 21:05:51 +0000651 if not isinstance(bufsize, int):
Peter Astrand738131d2004-11-30 21:04:45 +0000652 raise TypeError("bufsize must be an integer")
653
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000654 if mswindows:
Tim Peterse8374a52004-10-13 03:15:00 +0000655 if preexec_fn is not None:
656 raise ValueError("preexec_fn is not supported on Windows "
657 "platforms")
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000658 any_stdio_set = (stdin is not None or stdout is not None or
659 stderr is not None)
660 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
661 if any_stdio_set:
662 close_fds = False
663 else:
664 close_fds = True
665 elif close_fds and any_stdio_set:
666 raise ValueError(
667 "close_fds is not supported on Windows platforms"
668 " if you redirect stdin/stdout/stderr")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000669 else:
670 # POSIX
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000671 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
672 close_fds = True
673 if pass_fds and not close_fds:
674 warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
675 close_fds = True
Tim Peterse8374a52004-10-13 03:15:00 +0000676 if startupinfo is not None:
677 raise ValueError("startupinfo is only supported on Windows "
678 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000679 if creationflags != 0:
Tim Peterse8374a52004-10-13 03:15:00 +0000680 raise ValueError("creationflags is only supported on Windows "
681 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000682
Tim Peterse718f612004-10-12 21:51:32 +0000683 self.stdin = None
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000684 self.stdout = None
685 self.stderr = None
686 self.pid = None
687 self.returncode = None
688 self.universal_newlines = universal_newlines
689
690 # Input and output objects. The general principle is like
691 # this:
692 #
693 # Parent Child
694 # ------ -----
695 # p2cwrite ---stdin---> p2cread
696 # c2pread <--stdout--- c2pwrite
697 # errread <--stderr--- errwrite
698 #
699 # On POSIX, the child objects are file descriptors. On
700 # Windows, these are Windows file handles. The parent objects
701 # are file descriptors on both platforms. The parent objects
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000702 # are -1 when not using PIPEs. The child objects are -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000703 # when not redirecting.
Tim Peterse718f612004-10-12 21:51:32 +0000704
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000705 (p2cread, p2cwrite,
706 c2pread, c2pwrite,
707 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
708
Antoine Pitrouc9982322011-01-04 19:07:07 +0000709 # We wrap OS handles *before* launching the child, otherwise a
710 # quickly terminating child could make our fds unwrappable
711 # (see #8458).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000712
Thomas Wouterscf297e42007-02-23 15:07:44 +0000713 if mswindows:
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000714 if p2cwrite != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000715 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000716 if c2pread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000717 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000718 if errread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000719 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
Thomas Wouterscf297e42007-02-23 15:07:44 +0000720
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000721 if p2cwrite != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000722 self.stdin = io.open(p2cwrite, 'wb', bufsize)
723 if self.universal_newlines:
724 self.stdin = io.TextIOWrapper(self.stdin)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000725 if c2pread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000726 self.stdout = io.open(c2pread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000727 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000728 self.stdout = io.TextIOWrapper(self.stdout)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000729 if errread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000730 self.stderr = io.open(errread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000731 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000732 self.stderr = io.TextIOWrapper(self.stderr)
Tim Peterse718f612004-10-12 21:51:32 +0000733
Antoine Pitrouc9982322011-01-04 19:07:07 +0000734 try:
735 self._execute_child(args, executable, preexec_fn, close_fds,
736 pass_fds, cwd, env, universal_newlines,
737 startupinfo, creationflags, shell,
738 p2cread, p2cwrite,
739 c2pread, c2pwrite,
740 errread, errwrite,
741 restore_signals, start_new_session)
742 except:
743 # Cleanup if the child failed starting
744 for f in filter(None, [self.stdin, self.stdout, self.stderr]):
745 try:
746 f.close()
747 except EnvironmentError:
748 # Ignore EBADF or other errors
749 pass
750 raise
751
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000752
Guido van Rossum98297ee2007-11-06 21:34:58 +0000753 def _translate_newlines(self, data, encoding):
754 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")
755 return data.decode(encoding)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000756
Brian Curtin79cdb662010-12-03 02:46:02 +0000757 def __enter__(self):
758 return self
759
760 def __exit__(self, type, value, traceback):
761 if self.stdout:
762 self.stdout.close()
763 if self.stderr:
764 self.stderr.close()
765 if self.stdin:
766 self.stdin.close()
Gregory P. Smithc9557af2011-05-11 22:18:23 -0700767 # Wait for the process to terminate, to avoid zombies.
768 self.wait()
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000769
Brett Cannon84df1e62010-05-14 00:33:40 +0000770 def __del__(self, _maxsize=sys.maxsize, _active=_active):
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000771 if not self._child_created:
772 # We didn't get to successfully create a child process.
773 return
774 # In case the child hasn't been waited on, check if it's done.
Brett Cannon84df1e62010-05-14 00:33:40 +0000775 self._internal_poll(_deadstate=_maxsize)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000776 if self.returncode is None and _active is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000777 # Child is still running, keep us alive until we can wait on it.
778 _active.append(self)
779
780
Peter Astrand23109f02005-03-03 20:28:59 +0000781 def communicate(self, input=None):
782 """Interact with process: Send data to stdin. Read data from
783 stdout and stderr, until end-of-file is reached. Wait for
784 process to terminate. The optional input argument should be a
785 string to be sent to the child process, or None, if no data
786 should be sent to the child.
Tim Peterseba28be2005-03-28 01:08:02 +0000787
Peter Astrand23109f02005-03-03 20:28:59 +0000788 communicate() returns a tuple (stdout, stderr)."""
789
790 # Optimization: If we are only using one pipe, or no pipe at
791 # all, using select() or threads is unnecessary.
792 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
Tim Peterseba28be2005-03-28 01:08:02 +0000793 stdout = None
794 stderr = None
Peter Astrand23109f02005-03-03 20:28:59 +0000795 if self.stdin:
796 if input:
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200797 try:
798 self.stdin.write(input)
799 except IOError as e:
800 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
801 raise
Peter Astrand23109f02005-03-03 20:28:59 +0000802 self.stdin.close()
803 elif self.stdout:
804 stdout = self.stdout.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000805 self.stdout.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000806 elif self.stderr:
807 stderr = self.stderr.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000808 self.stderr.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000809 self.wait()
810 return (stdout, stderr)
Tim Peterseba28be2005-03-28 01:08:02 +0000811
Peter Astrand23109f02005-03-03 20:28:59 +0000812 return self._communicate(input)
813
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000814
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000815 def poll(self):
816 return self._internal_poll()
817
818
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000819 if mswindows:
820 #
821 # Windows methods
822 #
823 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +0000824 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000825 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
826 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000827 if stdin is None and stdout is None and stderr is None:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000828 return (-1, -1, -1, -1, -1, -1)
Tim Peterse718f612004-10-12 21:51:32 +0000829
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000830 p2cread, p2cwrite = -1, -1
831 c2pread, c2pwrite = -1, -1
832 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000833
Peter Astrandd38ddf42005-02-10 08:32:50 +0000834 if stdin is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000835 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000836 if p2cread is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000837 p2cread, _ = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000838 elif stdin == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000839 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000840 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000841 p2cread = msvcrt.get_osfhandle(stdin)
842 else:
843 # Assuming file-like object
844 p2cread = msvcrt.get_osfhandle(stdin.fileno())
845 p2cread = self._make_inheritable(p2cread)
846
Peter Astrandd38ddf42005-02-10 08:32:50 +0000847 if stdout is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000848 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000849 if c2pwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000850 _, c2pwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000851 elif stdout == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000852 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000853 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000854 c2pwrite = msvcrt.get_osfhandle(stdout)
855 else:
856 # Assuming file-like object
857 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
858 c2pwrite = self._make_inheritable(c2pwrite)
859
Peter Astrandd38ddf42005-02-10 08:32:50 +0000860 if stderr is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000861 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000862 if errwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000863 _, errwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000864 elif stderr == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000865 errread, errwrite = _subprocess.CreatePipe(None, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000866 elif stderr == STDOUT:
867 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +0000868 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000869 errwrite = msvcrt.get_osfhandle(stderr)
870 else:
871 # Assuming file-like object
872 errwrite = msvcrt.get_osfhandle(stderr.fileno())
873 errwrite = self._make_inheritable(errwrite)
874
875 return (p2cread, p2cwrite,
876 c2pread, c2pwrite,
877 errread, errwrite)
878
879
880 def _make_inheritable(self, handle):
881 """Return a duplicate of handle, which is inheritable"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000882 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
883 handle, _subprocess.GetCurrentProcess(), 0, 1,
884 _subprocess.DUPLICATE_SAME_ACCESS)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000885
886
887 def _find_w9xpopen(self):
888 """Find and return absolut path to w9xpopen.exe"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000889 w9xpopen = os.path.join(
890 os.path.dirname(_subprocess.GetModuleFileName(0)),
Tim Peterse8374a52004-10-13 03:15:00 +0000891 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000892 if not os.path.exists(w9xpopen):
893 # Eeek - file-not-found - possibly an embedding
894 # situation - see if we can locate it in sys.exec_prefix
Tim Peterse8374a52004-10-13 03:15:00 +0000895 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
896 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000897 if not os.path.exists(w9xpopen):
Tim Peterse8374a52004-10-13 03:15:00 +0000898 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
899 "needed for Popen to work with your "
900 "shell or platform.")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000901 return w9xpopen
902
Tim Peterse718f612004-10-12 21:51:32 +0000903
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000904 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000905 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000906 startupinfo, creationflags, shell,
907 p2cread, p2cwrite,
908 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000909 errread, errwrite,
910 unused_restore_signals, unused_start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000911 """Execute program (MS Windows version)"""
912
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000913 assert not pass_fds, "pass_fds not supported on Windows."
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000914
Guido van Rossum3172c5d2007-10-16 18:12:55 +0000915 if not isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000916 args = list2cmdline(args)
917
Peter Astrandc1d65362004-11-07 14:30:34 +0000918 # Process startup details
Peter Astrandd38ddf42005-02-10 08:32:50 +0000919 if startupinfo is None:
Thomas Wouters73e5a5b2006-06-08 15:35:45 +0000920 startupinfo = STARTUPINFO()
Victor Stinnerb3693582010-05-21 20:13:12 +0000921 if -1 not in (p2cread, c2pwrite, errwrite):
Brian Curtin1ce6b582010-04-24 16:19:22 +0000922 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
Peter Astrandc1d65362004-11-07 14:30:34 +0000923 startupinfo.hStdInput = p2cread
924 startupinfo.hStdOutput = c2pwrite
925 startupinfo.hStdError = errwrite
926
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000927 if shell:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000928 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
929 startupinfo.wShowWindow = _subprocess.SW_HIDE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000930 comspec = os.environ.get("COMSPEC", "cmd.exe")
Tim Golden126c2962010-08-11 14:20:40 +0000931 args = '{} /c "{}"'.format (comspec, args)
Brian Curtin1ce6b582010-04-24 16:19:22 +0000932 if (_subprocess.GetVersion() >= 0x80000000 or
Tim Peterse8374a52004-10-13 03:15:00 +0000933 os.path.basename(comspec).lower() == "command.com"):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000934 # Win9x, or using command.com on NT. We need to
935 # use the w9xpopen intermediate program. For more
936 # information, see KB Q150956
937 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
938 w9xpopen = self._find_w9xpopen()
939 args = '"%s" %s' % (w9xpopen, args)
940 # Not passing CREATE_NEW_CONSOLE has been known to
941 # cause random failures on win9x. Specifically a
942 # dialog: "Your program accessed mem currently in
943 # use at xxx" and a hopeful warning about the
Mark Dickinson934896d2009-02-21 20:59:32 +0000944 # stability of your system. Cost is Ctrl+C won't
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000945 # kill children.
Brian Curtin1ce6b582010-04-24 16:19:22 +0000946 creationflags |= _subprocess.CREATE_NEW_CONSOLE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000947
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000948 # Start the process
949 try:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000950 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
Tim Peterse8374a52004-10-13 03:15:00 +0000951 # no special security
952 None, None,
Guido van Rossume7ba4952007-06-06 23:52:48 +0000953 int(not close_fds),
Tim Peterse8374a52004-10-13 03:15:00 +0000954 creationflags,
955 env,
956 cwd,
957 startupinfo)
Guido van Rossumb940e112007-01-10 16:19:56 +0000958 except pywintypes.error as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000959 # Translate pywintypes.error to WindowsError, which is
960 # a subclass of OSError. FIXME: We should really
Ezio Melotti13925002011-03-16 11:05:33 +0200961 # translate errno using _sys_errlist (or similar), but
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000962 # how can this be done from Python?
963 raise WindowsError(*e.args)
Tim Goldenad537f22010-08-08 11:18:16 +0000964 finally:
965 # Child is launched. Close the parent's copy of those pipe
966 # handles that only the child should have open. You need
967 # to make sure that no handles to the write end of the
968 # output pipe are maintained in this process or else the
969 # pipe will not close when the child process exits and the
970 # ReadFile will hang.
971 if p2cread != -1:
972 p2cread.Close()
973 if c2pwrite != -1:
974 c2pwrite.Close()
975 if errwrite != -1:
976 errwrite.Close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000977
978 # Retain the process handle, but close the thread handle
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000979 self._child_created = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000980 self._handle = hp
981 self.pid = pid
982 ht.Close()
983
Brett Cannon84df1e62010-05-14 00:33:40 +0000984 def _internal_poll(self, _deadstate=None,
Victor Stinnerc807a612010-05-14 21:53:45 +0000985 _WaitForSingleObject=_subprocess.WaitForSingleObject,
986 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
987 _GetExitCodeProcess=_subprocess.GetExitCodeProcess):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000988 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +0000989 attribute.
990
991 This method is called by __del__, so it can only refer to objects
992 in its local scope.
993
994 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000995 if self.returncode is None:
Brett Cannon84df1e62010-05-14 00:33:40 +0000996 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
997 self.returncode = _GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000998 return self.returncode
999
1000
1001 def wait(self):
1002 """Wait for child process to terminate. Returns returncode
1003 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001004 if self.returncode is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +00001005 _subprocess.WaitForSingleObject(self._handle,
1006 _subprocess.INFINITE)
1007 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001008 return self.returncode
1009
1010
1011 def _readerthread(self, fh, buffer):
1012 buffer.append(fh.read())
Victor Stinner667d4b52010-12-25 22:40:32 +00001013 fh.close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001014
1015
Peter Astrand23109f02005-03-03 20:28:59 +00001016 def _communicate(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001017 stdout = None # Return
1018 stderr = None # Return
1019
1020 if self.stdout:
1021 stdout = []
Tim Peterse8374a52004-10-13 03:15:00 +00001022 stdout_thread = threading.Thread(target=self._readerthread,
1023 args=(self.stdout, stdout))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001024 stdout_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001025 stdout_thread.start()
1026 if self.stderr:
1027 stderr = []
Tim Peterse8374a52004-10-13 03:15:00 +00001028 stderr_thread = threading.Thread(target=self._readerthread,
1029 args=(self.stderr, stderr))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001030 stderr_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001031 stderr_thread.start()
1032
1033 if self.stdin:
Peter Astrandd38ddf42005-02-10 08:32:50 +00001034 if input is not None:
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001035 try:
1036 self.stdin.write(input)
1037 except IOError as e:
1038 if e.errno != errno.EPIPE:
1039 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001040 self.stdin.close()
1041
1042 if self.stdout:
1043 stdout_thread.join()
1044 if self.stderr:
1045 stderr_thread.join()
1046
1047 # All data exchanged. Translate lists into strings.
Peter Astrandd38ddf42005-02-10 08:32:50 +00001048 if stdout is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001049 stdout = stdout[0]
Peter Astrandd38ddf42005-02-10 08:32:50 +00001050 if stderr is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001051 stderr = stderr[0]
1052
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001053 self.wait()
1054 return (stdout, stderr)
1055
Christian Heimesa342c012008-04-20 21:01:16 +00001056 def send_signal(self, sig):
1057 """Send a signal to the process
1058 """
1059 if sig == signal.SIGTERM:
1060 self.terminate()
Brian Curtineb24d742010-04-12 17:16:38 +00001061 elif sig == signal.CTRL_C_EVENT:
1062 os.kill(self.pid, signal.CTRL_C_EVENT)
1063 elif sig == signal.CTRL_BREAK_EVENT:
1064 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
Christian Heimesa342c012008-04-20 21:01:16 +00001065 else:
Brian Curtin19651362010-09-07 13:24:38 +00001066 raise ValueError("Unsupported signal: {}".format(sig))
Christian Heimesa342c012008-04-20 21:01:16 +00001067
1068 def terminate(self):
1069 """Terminates the process
1070 """
Brian Curtin1ce6b582010-04-24 16:19:22 +00001071 _subprocess.TerminateProcess(self._handle, 1)
Christian Heimesa342c012008-04-20 21:01:16 +00001072
1073 kill = terminate
1074
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001075 else:
1076 #
1077 # POSIX methods
1078 #
1079 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +00001080 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001081 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1082 """
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001083 p2cread, p2cwrite = -1, -1
1084 c2pread, c2pwrite = -1, -1
1085 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001086
Peter Astrandd38ddf42005-02-10 08:32:50 +00001087 if stdin is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001088 pass
1089 elif stdin == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001090 p2cread, p2cwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001091 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001092 p2cread = stdin
1093 else:
1094 # Assuming file-like object
1095 p2cread = stdin.fileno()
1096
Peter Astrandd38ddf42005-02-10 08:32:50 +00001097 if stdout is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001098 pass
1099 elif stdout == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001100 c2pread, c2pwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001101 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001102 c2pwrite = stdout
1103 else:
1104 # Assuming file-like object
Tim Peterse718f612004-10-12 21:51:32 +00001105 c2pwrite = stdout.fileno()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001106
Peter Astrandd38ddf42005-02-10 08:32:50 +00001107 if stderr is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001108 pass
1109 elif stderr == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001110 errread, errwrite = _create_pipe()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001111 elif stderr == STDOUT:
1112 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +00001113 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001114 errwrite = stderr
1115 else:
1116 # Assuming file-like object
1117 errwrite = stderr.fileno()
1118
1119 return (p2cread, p2cwrite,
1120 c2pread, c2pwrite,
1121 errread, errwrite)
1122
1123
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001124 def _close_fds(self, fds_to_keep):
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001125 start_fd = 3
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001126 for fd in sorted(fds_to_keep):
Gregory P. Smith8edd99d2010-12-14 13:43:30 +00001127 if fd >= start_fd:
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001128 os.closerange(start_fd, fd)
1129 start_fd = fd + 1
1130 if start_fd <= MAXFD:
1131 os.closerange(start_fd, MAXFD)
1132
1133
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001134 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001135 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001136 startupinfo, creationflags, shell,
1137 p2cread, p2cwrite,
1138 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001139 errread, errwrite,
1140 restore_signals, start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001141 """Execute program (POSIX version)"""
1142
Guido van Rossum3172c5d2007-10-16 18:12:55 +00001143 if isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001144 args = [args]
Thomas Wouters89f507f2006-12-13 04:49:30 +00001145 else:
1146 args = list(args)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001147
1148 if shell:
1149 args = ["/bin/sh", "-c"] + args
Stefan Krah9542cc62010-07-19 14:20:53 +00001150 if executable:
1151 args[0] = executable
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001152
Peter Astrandd38ddf42005-02-10 08:32:50 +00001153 if executable is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001154 executable = args[0]
1155
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001156 # For transferring possible exec failure from child to parent.
1157 # Data format: "exception name:hex errno:description"
1158 # Pickle is not used; it is complex and involves memory allocation.
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001159 errpipe_read, errpipe_write = _create_pipe()
Christian Heimesfdab48e2008-01-20 09:06:41 +00001160 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001161 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001162
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001163 if _posixsubprocess:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001164 # We must avoid complex work that could involve
1165 # malloc or free in the child process to avoid
1166 # potential deadlocks, thus we do all this here.
1167 # and pass it to fork_exec()
1168
1169 if env:
Victor Stinner449c4662010-05-08 11:10:09 +00001170 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001171 for k, v in env.items()]
1172 else:
1173 env_list = None # Use execv instead of execve.
Victor Stinnerb745a742010-05-18 17:17:23 +00001174 executable = os.fsencode(executable)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001175 if os.path.dirname(executable):
Victor Stinnerb745a742010-05-18 17:17:23 +00001176 executable_list = (executable,)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001177 else:
1178 # This matches the behavior of os._execvpe().
Victor Stinnerb745a742010-05-18 17:17:23 +00001179 executable_list = tuple(
1180 os.path.join(os.fsencode(dir), executable)
1181 for dir in os.get_exec_path(env))
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001182 fds_to_keep = set(pass_fds)
1183 fds_to_keep.add(errpipe_write)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001184 self.pid = _posixsubprocess.fork_exec(
1185 args, executable_list,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001186 close_fds, sorted(fds_to_keep), cwd, env_list,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001187 p2cread, p2cwrite, c2pread, c2pwrite,
1188 errread, errwrite,
1189 errpipe_read, errpipe_write,
1190 restore_signals, start_new_session, preexec_fn)
1191 else:
1192 # Pure Python implementation: It is not thread safe.
1193 # This implementation may deadlock in the child if your
1194 # parent process has any other threads running.
1195
1196 gc_was_enabled = gc.isenabled()
1197 # Disable gc to avoid bug where gc -> file_dealloc ->
1198 # write to stderr -> hang. See issue1336
1199 gc.disable()
1200 try:
1201 self.pid = os.fork()
1202 except:
1203 if gc_was_enabled:
1204 gc.enable()
1205 raise
1206 self._child_created = True
1207 if self.pid == 0:
1208 # Child
1209 try:
1210 # Close parent's pipe ends
1211 if p2cwrite != -1:
1212 os.close(p2cwrite)
1213 if c2pread != -1:
1214 os.close(c2pread)
1215 if errread != -1:
1216 os.close(errread)
1217 os.close(errpipe_read)
1218
1219 # Dup fds for child
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001220 def _dup2(a, b):
1221 # dup2() removes the CLOEXEC flag but
1222 # we must do it ourselves if dup2()
1223 # would be a no-op (issue #10806).
1224 if a == b:
1225 _set_cloexec(a, False)
1226 elif a != -1:
1227 os.dup2(a, b)
1228 _dup2(p2cread, 0)
1229 _dup2(c2pwrite, 1)
1230 _dup2(errwrite, 2)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001231
1232 # Close pipe fds. Make sure we don't close the
1233 # same fd more than once, or standard fds.
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001234 closed = set()
1235 for fd in [p2cread, c2pwrite, errwrite]:
1236 if fd > 2 and fd not in closed:
1237 os.close(fd)
1238 closed.add(fd)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001239
1240 # Close all other fds, if asked for
1241 if close_fds:
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001242 fds_to_keep = set(pass_fds)
1243 fds_to_keep.add(errpipe_write)
1244 self._close_fds(fds_to_keep)
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001245
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001246
1247 if cwd is not None:
1248 os.chdir(cwd)
1249
1250 # This is a copy of Python/pythonrun.c
1251 # _Py_RestoreSignals(). If that were exposed
1252 # as a sys._py_restoresignals func it would be
1253 # better.. but this pure python implementation
1254 # isn't likely to be used much anymore.
1255 if restore_signals:
1256 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1257 for sig in signals:
1258 if hasattr(signal, sig):
1259 signal.signal(getattr(signal, sig),
1260 signal.SIG_DFL)
1261
1262 if start_new_session and hasattr(os, 'setsid'):
1263 os.setsid()
1264
1265 if preexec_fn:
1266 preexec_fn()
1267
1268 if env is None:
1269 os.execvp(executable, args)
1270 else:
1271 os.execvpe(executable, args, env)
1272
1273 except:
1274 try:
1275 exc_type, exc_value = sys.exc_info()[:2]
1276 if isinstance(exc_value, OSError):
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001277 errno_num = exc_value.errno
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001278 else:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001279 errno_num = 0
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001280 message = '%s:%x:%s' % (exc_type.__name__,
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001281 errno_num, exc_value)
Victor Stinner4d078042010-04-23 19:28:32 +00001282 message = message.encode(errors="surrogatepass")
1283 os.write(errpipe_write, message)
1284 except Exception:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001285 # We MUST not allow anything odd happening
1286 # above to prevent us from exiting below.
1287 pass
1288
1289 # This exitcode won't be reported to applications
1290 # so it really doesn't matter what we return.
1291 os._exit(255)
1292
1293 # Parent
Facundo Batista10706e22009-06-19 20:34:30 +00001294 if gc_was_enabled:
1295 gc.enable()
Facundo Batista10706e22009-06-19 20:34:30 +00001296 finally:
1297 # be sure the FD is closed no matter what
1298 os.close(errpipe_write)
1299
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001300 if p2cread != -1 and p2cwrite != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001301 os.close(p2cread)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001302 if c2pwrite != -1 and c2pread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001303 os.close(c2pwrite)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001304 if errwrite != -1 and errread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001305 os.close(errwrite)
1306
1307 # Wait for exec to fail or succeed; possibly raising an
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001308 # exception (limited in size)
1309 data = bytearray()
1310 while True:
1311 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1312 data += part
1313 if not part or len(data) > 50000:
1314 break
Facundo Batista10706e22009-06-19 20:34:30 +00001315 finally:
1316 # be sure the FD is closed no matter what
1317 os.close(errpipe_read)
1318
Guido van Rossumaf2362a2007-05-15 22:32:02 +00001319 if data:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001320 try:
1321 _eintr_retry_call(os.waitpid, self.pid, 0)
1322 except OSError as e:
1323 if e.errno != errno.ECHILD:
1324 raise
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001325 try:
1326 exception_name, hex_errno, err_msg = data.split(b':', 2)
1327 except ValueError:
1328 print('Bad exception data:', repr(data))
1329 exception_name = b'RuntimeError'
1330 hex_errno = b'0'
1331 err_msg = b'Unknown'
1332 child_exception_type = getattr(
1333 builtins, exception_name.decode('ascii'),
1334 RuntimeError)
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001335 for fd in (p2cwrite, c2pread, errread):
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001336 if fd != -1:
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001337 os.close(fd)
Victor Stinner4d078042010-04-23 19:28:32 +00001338 err_msg = err_msg.decode(errors="surrogatepass")
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001339 if issubclass(child_exception_type, OSError) and hex_errno:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001340 errno_num = int(hex_errno, 16)
1341 if errno_num != 0:
1342 err_msg = os.strerror(errno_num)
1343 if errno_num == errno.ENOENT:
Benjamin Peterson5f780402010-11-20 18:07:52 +00001344 err_msg += ': ' + repr(args[0])
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001345 raise child_exception_type(errno_num, err_msg)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001346 raise child_exception_type(err_msg)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001347
1348
Brett Cannon84df1e62010-05-14 00:33:40 +00001349 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1350 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1351 _WEXITSTATUS=os.WEXITSTATUS):
1352 # This method is called (indirectly) by __del__, so it cannot
1353 # refer to anything outside of its local scope."""
1354 if _WIFSIGNALED(sts):
1355 self.returncode = -_WTERMSIG(sts)
1356 elif _WIFEXITED(sts):
1357 self.returncode = _WEXITSTATUS(sts)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001358 else:
1359 # Should never happen
1360 raise RuntimeError("Unknown child exit status!")
1361
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001362
Brett Cannon84df1e62010-05-14 00:33:40 +00001363 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1364 _WNOHANG=os.WNOHANG, _os_error=os.error):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001365 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +00001366 attribute.
1367
1368 This method is called by __del__, so it cannot reference anything
1369 outside of the local scope (nor can any methods it calls).
1370
1371 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001372 if self.returncode is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001373 try:
Brett Cannon84df1e62010-05-14 00:33:40 +00001374 pid, sts = _waitpid(self.pid, _WNOHANG)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001375 if pid == self.pid:
1376 self._handle_exitstatus(sts)
Brett Cannon84df1e62010-05-14 00:33:40 +00001377 except _os_error:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +00001378 if _deadstate is not None:
1379 self.returncode = _deadstate
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001380 return self.returncode
1381
1382
1383 def wait(self):
1384 """Wait for child process to terminate. Returns returncode
1385 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001386 if self.returncode is None:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001387 try:
1388 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1389 except OSError as e:
1390 if e.errno != errno.ECHILD:
1391 raise
1392 # This happens if SIGCLD is set to be ignored or waiting
1393 # for child processes has otherwise been disabled for our
1394 # process. This child is dead, we can't get the status.
1395 sts = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001396 self._handle_exitstatus(sts)
1397 return self.returncode
1398
1399
Peter Astrand23109f02005-03-03 20:28:59 +00001400 def _communicate(self, input):
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001401 if self.stdin:
1402 # Flush stdio buffer. This might block, if the user has
1403 # been writing to .stdin in an uncontrolled fashion.
1404 self.stdin.flush()
1405 if not input:
1406 self.stdin.close()
1407
1408 if _has_poll:
1409 stdout, stderr = self._communicate_with_poll(input)
1410 else:
1411 stdout, stderr = self._communicate_with_select(input)
1412
1413 # All data exchanged. Translate lists into strings.
1414 if stdout is not None:
1415 stdout = b''.join(stdout)
1416 if stderr is not None:
1417 stderr = b''.join(stderr)
1418
1419 # Translate newlines, if requested.
1420 # This also turns bytes into strings.
1421 if self.universal_newlines:
1422 if stdout is not None:
1423 stdout = self._translate_newlines(stdout,
1424 self.stdout.encoding)
1425 if stderr is not None:
1426 stderr = self._translate_newlines(stderr,
1427 self.stderr.encoding)
1428
1429 self.wait()
1430 return (stdout, stderr)
1431
1432
1433 def _communicate_with_poll(self, input):
1434 stdout = None # Return
1435 stderr = None # Return
1436 fd2file = {}
1437 fd2output = {}
1438
1439 poller = select.poll()
1440 def register_and_append(file_obj, eventmask):
1441 poller.register(file_obj.fileno(), eventmask)
1442 fd2file[file_obj.fileno()] = file_obj
1443
1444 def close_unregister_and_remove(fd):
1445 poller.unregister(fd)
1446 fd2file[fd].close()
1447 fd2file.pop(fd)
1448
1449 if self.stdin and input:
1450 register_and_append(self.stdin, select.POLLOUT)
1451
1452 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1453 if self.stdout:
1454 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1455 fd2output[self.stdout.fileno()] = stdout = []
1456 if self.stderr:
1457 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1458 fd2output[self.stderr.fileno()] = stderr = []
1459
1460 input_offset = 0
1461 while fd2file:
1462 try:
1463 ready = poller.poll()
1464 except select.error as e:
1465 if e.args[0] == errno.EINTR:
1466 continue
1467 raise
1468
1469 # XXX Rewrite these to use non-blocking I/O on the
1470 # file objects; they are no longer using C stdio!
1471
1472 for fd, mode in ready:
1473 if mode & select.POLLOUT:
1474 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001475 try:
1476 input_offset += os.write(fd, chunk)
1477 except OSError as e:
1478 if e.errno == errno.EPIPE:
1479 close_unregister_and_remove(fd)
1480 else:
1481 raise
1482 else:
1483 if input_offset >= len(input):
1484 close_unregister_and_remove(fd)
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001485 elif mode & select_POLLIN_POLLPRI:
1486 data = os.read(fd, 4096)
1487 if not data:
1488 close_unregister_and_remove(fd)
1489 fd2output[fd].append(data)
1490 else:
1491 # Ignore hang up or errors.
1492 close_unregister_and_remove(fd)
1493
1494 return (stdout, stderr)
1495
1496
1497 def _communicate_with_select(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001498 read_set = []
1499 write_set = []
1500 stdout = None # Return
1501 stderr = None # Return
1502
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001503 if self.stdin and input:
1504 write_set.append(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001505 if self.stdout:
1506 read_set.append(self.stdout)
Tim Peterse718f612004-10-12 21:51:32 +00001507 stdout = []
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001508 if self.stderr:
1509 read_set.append(self.stderr)
1510 stderr = []
1511
Thomas Wouters902d6eb2007-01-09 23:18:33 +00001512 input_offset = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001513 while read_set or write_set:
Georg Brandl86b2fb92008-07-16 03:43:04 +00001514 try:
1515 rlist, wlist, xlist = select.select(read_set, write_set, [])
1516 except select.error as e:
1517 if e.args[0] == errno.EINTR:
1518 continue
1519 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001520
Guido van Rossum98297ee2007-11-06 21:34:58 +00001521 # XXX Rewrite these to use non-blocking I/O on the
1522 # file objects; they are no longer using C stdio!
1523
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001524 if self.stdin in wlist:
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001525 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001526 try:
1527 bytes_written = os.write(self.stdin.fileno(), chunk)
1528 except OSError as e:
1529 if e.errno == errno.EPIPE:
1530 self.stdin.close()
1531 write_set.remove(self.stdin)
1532 else:
1533 raise
1534 else:
1535 input_offset += bytes_written
1536 if input_offset >= len(input):
1537 self.stdin.close()
1538 write_set.remove(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001539
1540 if self.stdout in rlist:
1541 data = os.read(self.stdout.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001542 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001543 self.stdout.close()
1544 read_set.remove(self.stdout)
1545 stdout.append(data)
1546
1547 if self.stderr in rlist:
1548 data = os.read(self.stderr.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001549 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001550 self.stderr.close()
1551 read_set.remove(self.stderr)
1552 stderr.append(data)
1553
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001554 return (stdout, stderr)
1555
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001556
Christian Heimesa342c012008-04-20 21:01:16 +00001557 def send_signal(self, sig):
1558 """Send a signal to the process
1559 """
1560 os.kill(self.pid, sig)
1561
1562 def terminate(self):
1563 """Terminate the process with SIGTERM
1564 """
1565 self.send_signal(signal.SIGTERM)
1566
1567 def kill(self):
1568 """Kill the process with SIGKILL
1569 """
1570 self.send_signal(signal.SIGKILL)
1571
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001572
1573def _demo_posix():
1574 #
1575 # Example 1: Simple redirection: Get process list
1576 #
1577 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001578 print("Process list:")
1579 print(plist)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001580
1581 #
1582 # Example 2: Change uid before executing child
1583 #
1584 if os.getuid() == 0:
1585 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1586 p.wait()
1587
1588 #
1589 # Example 3: Connecting several subprocesses
1590 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001591 print("Looking for 'hda'...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001592 p1 = Popen(["dmesg"], stdout=PIPE)
1593 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001594 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001595
1596 #
1597 # Example 4: Catch execution error
1598 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001599 print()
1600 print("Trying a weird file...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001601 try:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001602 print(Popen(["/this/path/does/not/exist"]).communicate())
Guido van Rossumb940e112007-01-10 16:19:56 +00001603 except OSError as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001604 if e.errno == errno.ENOENT:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001605 print("The file didn't exist. I thought so...")
1606 print("Child traceback:")
1607 print(e.child_traceback)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001608 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001609 print("Error", e.errno)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001610 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001611 print("Gosh. No error.", file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001612
1613
1614def _demo_windows():
1615 #
1616 # Example 1: Connecting several subprocesses
1617 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001618 print("Looking for 'PROMPT' in set output...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001619 p1 = Popen("set", stdout=PIPE, shell=True)
1620 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001621 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001622
1623 #
1624 # Example 2: Simple execution of program
1625 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001626 print("Executing calc...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001627 p = Popen("calc")
1628 p.wait()
1629
1630
1631if __name__ == "__main__":
1632 if mswindows:
1633 _demo_windows()
1634 else:
1635 _demo_posix()