blob: 06285e9462cb99aee13e1db1fd4cc7e81ca9c71c [file] [log] [blame]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001# subprocess - Subprocesses with accessible I/O streams
2#
Tim Peterse718f612004-10-12 21:51:32 +00003# For more information about this module, see PEP 324.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00004#
Peter Astrand3a708df2005-09-23 17:37:29 +00005# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00006#
Peter Astrand69bf13f2005-02-14 08:56:32 +00007# Licensed to PSF under a Contributor Agreement.
Peter Astrand3a708df2005-09-23 17:37:29 +00008# See http://www.python.org/2.4/license for licensing details.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00009
Raymond Hettinger837dd932004-10-17 16:36:53 +000010r"""subprocess - Subprocesses with accessible I/O streams
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000011
Fredrik Lundh15aaacc2004-10-17 14:47:05 +000012This module allows you to spawn processes, connect to their
13input/output/error pipes, and obtain their return codes. This module
14intends to replace several other, older modules and functions, like:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000015
16os.system
17os.spawn*
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000018
19Information about how the subprocess module can be used to replace these
20modules and functions can be found below.
21
22
23
24Using the subprocess module
25===========================
26This module defines one class called Popen:
27
28class Popen(args, bufsize=0, executable=None,
29 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000030 preexec_fn=None, close_fds=True, shell=False,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000031 cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000032 startupinfo=None, creationflags=0,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000033 restore_signals=True, start_new_session=False, pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000034
35
36Arguments are:
37
38args should be a string, or a sequence of program arguments. The
39program to execute is normally the first item in the args sequence or
40string, but can be explicitly set by using the executable argument.
41
Gregory P. Smithf5604852010-12-13 06:45:02 +000042On POSIX, with shell=False (default): In this case, the Popen class
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000043uses os.execvp() to execute the child program. args should normally
44be a sequence. A string will be treated as a sequence with the string
45as the only item (the program to execute).
46
Gregory P. Smithf5604852010-12-13 06:45:02 +000047On POSIX, with shell=True: If args is a string, it specifies the
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000048command string to execute through the shell. If args is a sequence,
49the first item specifies the command string, and any additional items
50will be treated as additional shell arguments.
51
52On Windows: the Popen class uses CreateProcess() to execute the child
53program, which operates on strings. If args is a sequence, it will be
54converted to a string using the list2cmdline method. Please note that
55not all MS Windows applications interpret the command line the same
56way: The list2cmdline is designed for applications using the same
57rules as the MS C runtime.
58
59bufsize, if given, has the same meaning as the corresponding argument
60to the built-in open() function: 0 means unbuffered, 1 means line
61buffered, any other positive value means use a buffer of
62(approximately) that size. A negative bufsize means to use the system
63default, which usually means fully buffered. The default value for
64bufsize is 0 (unbuffered).
65
66stdin, stdout and stderr specify the executed programs' standard
67input, standard output and standard error file handles, respectively.
68Valid values are PIPE, an existing file descriptor (a positive
69integer), an existing file object, and None. PIPE indicates that a
70new pipe to the child should be created. With None, no redirection
71will occur; the child's file handles will be inherited from the
72parent. Additionally, stderr can be STDOUT, which indicates that the
73stderr data from the applications should be captured into the same
74file handle as for stdout.
75
Gregory P. Smithf5604852010-12-13 06:45:02 +000076On POSIX, if preexec_fn is set to a callable object, this object will be
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000077called in the child process just before the child is executed. The use
78of preexec_fn is not thread safe, using it in the presence of threads
79could lead to a deadlock in the child process before the new executable
80is executed.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000081
82If close_fds is true, all file descriptors except 0, 1 and 2 will be
Gregory P. Smithf5604852010-12-13 06:45:02 +000083closed before the child process is executed. The default for close_fds
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000084varies by platform: Always true on POSIX. True when stdin/stdout/stderr
85are None on Windows, false otherwise.
86
87pass_fds is an optional sequence of file descriptors to keep open between the
88parent and child. Providing any pass_fds implicitly sets close_fds to true.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000089
90if shell is true, the specified command will be executed through the
91shell.
92
93If cwd is not None, the current directory will be changed to cwd
94before the child is executed.
95
Gregory P. Smithf5604852010-12-13 06:45:02 +000096On POSIX, if restore_signals is True all signals that Python sets to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000097SIG_IGN are restored to SIG_DFL in the child process before the exec.
98Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99parameter does nothing on Windows.
100
Gregory P. Smithf5604852010-12-13 06:45:02 +0000101On POSIX, if start_new_session is True, the setsid() system call will be made
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000102in the child process prior to executing the command.
103
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000104If env is not None, it defines the environment variables for the new
105process.
106
107If universal_newlines is true, the file objects stdout and stderr are
108opened as a text files, but lines may be terminated by any of '\n',
Gregory P. Smithf5604852010-12-13 06:45:02 +0000109the Unix end-of-line convention, '\r', the old Macintosh convention or
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000110'\r\n', the Windows convention. All of these external representations
111are seen as '\n' by the Python program. Note: This feature is only
112available if Python is built with universal newline support (the
113default). Also, the newlines attribute of the file objects stdout,
114stdin and stderr are not updated by the communicate() method.
115
116The startupinfo and creationflags, if given, will be passed to the
117underlying CreateProcess() function. They can specify things such as
118appearance of the main window and priority for the new process.
119(Windows only)
120
121
Georg Brandlf9734072008-12-07 15:30:06 +0000122This module also defines some shortcut functions:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000123
Peter Astrand5f5e1412004-12-05 20:15:36 +0000124call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000125 Run command with arguments. Wait for command to complete, then
126 return the returncode attribute.
127
128 The arguments are the same as for the Popen constructor. Example:
129
Florent Xicluna4886d242010-03-08 13:27:26 +0000130 >>> retcode = subprocess.call(["ls", "-l"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000131
Peter Astrand454f7672005-01-01 09:36:35 +0000132check_call(*popenargs, **kwargs):
133 Run command with arguments. Wait for command to complete. If the
134 exit code was zero then return, otherwise raise
135 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000136 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000137
138 The arguments are the same as for the Popen constructor. Example:
139
Florent Xicluna4886d242010-03-08 13:27:26 +0000140 >>> subprocess.check_call(["ls", "-l"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000141 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000142
Brett Cannona23810f2008-05-26 19:04:21 +0000143getstatusoutput(cmd):
144 Return (status, output) of executing cmd in a shell.
145
146 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
147 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
148 returned output will contain output or error messages. A trailing newline
149 is stripped from the output. The exit status for the command can be
150 interpreted according to the rules for the C function wait(). Example:
151
Brett Cannona23810f2008-05-26 19:04:21 +0000152 >>> subprocess.getstatusoutput('ls /bin/ls')
153 (0, '/bin/ls')
154 >>> subprocess.getstatusoutput('cat /bin/junk')
155 (256, 'cat: /bin/junk: No such file or directory')
156 >>> subprocess.getstatusoutput('/bin/junk')
157 (256, 'sh: /bin/junk: not found')
158
159getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell.
161
162 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example:
164
Brett Cannona23810f2008-05-26 19:04:21 +0000165 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls'
167
Georg Brandlf9734072008-12-07 15:30:06 +0000168check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000169 Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000170
Georg Brandl2708f3a2009-12-20 14:38:23 +0000171 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000174
Georg Brandl2708f3a2009-12-20 14:38:23 +0000175 The arguments are the same as for the Popen constructor. Example:
Georg Brandlf9734072008-12-07 15:30:06 +0000176
Georg Brandl2708f3a2009-12-20 14:38:23 +0000177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandlf9734072008-12-07 15:30:06 +0000178
Brett Cannona23810f2008-05-26 19:04:21 +0000179
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000180Exceptions
181----------
182Exceptions raised in the child process, before the new program has
183started to execute, will be re-raised in the parent. Additionally,
184the exception object will have one extra attribute called
185'child_traceback', which is a string containing traceback information
186from the childs point of view.
187
188The most common exception raised is OSError. This occurs, for
189example, when trying to execute a non-existent file. Applications
190should prepare for OSErrors.
191
192A ValueError will be raised if Popen is called with invalid arguments.
193
Georg Brandlf9734072008-12-07 15:30:06 +0000194check_call() and check_output() will raise CalledProcessError, if the
195called process returns a non-zero return code.
Peter Astrand454f7672005-01-01 09:36:35 +0000196
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000197
198Security
199--------
200Unlike some other popen functions, this implementation will never call
201/bin/sh implicitly. This means that all characters, including shell
202metacharacters, can safely be passed to child processes.
203
204
205Popen objects
206=============
207Instances of the Popen class have the following methods:
208
209poll()
210 Check if child process has terminated. Returns returncode
211 attribute.
212
213wait()
214 Wait for child process to terminate. Returns returncode attribute.
215
216communicate(input=None)
217 Interact with process: Send data to stdin. Read data from stdout
218 and stderr, until end-of-file is reached. Wait for process to
Thomas Wouters902d6eb2007-01-09 23:18:33 +0000219 terminate. The optional input argument should be a string to be
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000220 sent to the child process, or None, if no data should be sent to
221 the child.
Tim Peterse718f612004-10-12 21:51:32 +0000222
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000223 communicate() returns a tuple (stdout, stderr).
224
225 Note: The data read is buffered in memory, so do not use this
226 method if the data size is large or unlimited.
227
228The following attributes are also available:
229
230stdin
231 If the stdin argument is PIPE, this attribute is a file object
232 that provides input to the child process. Otherwise, it is None.
233
234stdout
235 If the stdout argument is PIPE, this attribute is a file object
236 that provides output from the child process. Otherwise, it is
237 None.
238
239stderr
240 If the stderr argument is PIPE, this attribute is file object that
241 provides error output from the child process. Otherwise, it is
242 None.
243
244pid
245 The process ID of the child process.
246
247returncode
248 The child return code. A None value indicates that the process
249 hasn't terminated yet. A negative value -N indicates that the
Gregory P. Smithf5604852010-12-13 06:45:02 +0000250 child was terminated by signal N (POSIX only).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000251
252
253Replacing older functions with the subprocess module
254====================================================
255In this section, "a ==> b" means that b can be used as a replacement
256for a.
257
258Note: All functions in this section fail (more or less) silently if
259the executed program cannot be found; this module raises an OSError
260exception.
261
262In the following examples, we assume that the subprocess module is
263imported with "from subprocess import *".
264
265
266Replacing /bin/sh shell backquote
267---------------------------------
268output=`mycmd myarg`
269==>
270output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
271
272
273Replacing shell pipe line
274-------------------------
275output=`dmesg | grep hda`
276==>
277p1 = Popen(["dmesg"], stdout=PIPE)
Peter Astrand6fdf3cb2004-11-30 18:06:42 +0000278p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000279output = p2.communicate()[0]
280
281
282Replacing os.system()
283---------------------
284sts = os.system("mycmd" + " myarg")
285==>
286p = Popen("mycmd" + " myarg", shell=True)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000287pid, sts = os.waitpid(p.pid, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000288
289Note:
290
291* Calling the program through the shell is usually not required.
292
293* It's easier to look at the returncode attribute than the
294 exitstatus.
295
296A more real-world example would look like this:
297
298try:
299 retcode = call("mycmd" + " myarg", shell=True)
300 if retcode < 0:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000301 print("Child was terminated by signal", -retcode, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000302 else:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000303 print("Child returned", retcode, file=sys.stderr)
304except OSError as e:
305 print("Execution failed:", e, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000306
307
308Replacing os.spawn*
309-------------------
Tim Peterse718f612004-10-12 21:51:32 +0000310P_NOWAIT example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000311
312pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
313==>
314pid = Popen(["/bin/mycmd", "myarg"]).pid
315
316
317P_WAIT example:
318
319retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
320==>
321retcode = call(["/bin/mycmd", "myarg"])
322
323
Tim Peterse718f612004-10-12 21:51:32 +0000324Vector example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000325
326os.spawnvp(os.P_NOWAIT, path, args)
327==>
328Popen([path] + args[1:])
329
330
Tim Peterse718f612004-10-12 21:51:32 +0000331Environment example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000332
333os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
334==>
335Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000336"""
337
338import sys
339mswindows = (sys.platform == "win32")
340
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000341import io
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000342import os
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000343import traceback
Christian Heimesfdab48e2008-01-20 09:06:41 +0000344import gc
Christian Heimesa342c012008-04-20 21:01:16 +0000345import signal
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000346import builtins
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000347import warnings
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200348import errno
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000349
Peter Astrand454f7672005-01-01 09:36:35 +0000350# Exception classes used by this module.
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000351class CalledProcessError(Exception):
Georg Brandlf9734072008-12-07 15:30:06 +0000352 """This exception is raised when a process run by check_call() or
353 check_output() returns a non-zero exit status.
354 The exit status will be stored in the returncode attribute;
355 check_output() will also store the output in the output attribute.
356 """
357 def __init__(self, returncode, cmd, output=None):
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000358 self.returncode = returncode
359 self.cmd = cmd
Georg Brandlf9734072008-12-07 15:30:06 +0000360 self.output = output
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000361 def __str__(self):
362 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
363
Peter Astrand454f7672005-01-01 09:36:35 +0000364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000365if mswindows:
366 import threading
367 import msvcrt
Brian Curtin1ce6b582010-04-24 16:19:22 +0000368 import _subprocess
369 class STARTUPINFO:
370 dwFlags = 0
371 hStdInput = None
372 hStdOutput = None
373 hStdError = None
374 wShowWindow = 0
375 class pywintypes:
376 error = IOError
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000377else:
378 import select
Gregory P. Smithd06fa472009-07-04 02:46:54 +0000379 _has_poll = hasattr(select, 'poll')
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000380 import fcntl
381 import pickle
382
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000383 try:
384 import _posixsubprocess
385 except ImportError:
386 _posixsubprocess = None
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000387 warnings.warn("The _posixsubprocess module is not being used. "
388 "Child process reliability may suffer if your "
389 "program uses threads.", RuntimeWarning)
390
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000391 # When select or poll has indicated that the file is writable,
392 # we can write up to _PIPE_BUF bytes without risk of blocking.
393 # POSIX defines PIPE_BUF as >= 512.
394 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
395
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000396 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
397
398 def _set_cloexec(fd, cloexec):
399 old = fcntl.fcntl(fd, fcntl.F_GETFD)
400 if cloexec:
401 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
402 else:
403 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
404
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000405 if _posixsubprocess:
406 _create_pipe = _posixsubprocess.cloexec_pipe
407 else:
408 def _create_pipe():
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000409 fds = os.pipe()
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000410 _set_cloexec(fds[0], True)
411 _set_cloexec(fds[1], True)
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000412 return fds
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000413
Brett Cannona23810f2008-05-26 19:04:21 +0000414__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
Georg Brandlf9734072008-12-07 15:30:06 +0000415 "getoutput", "check_output", "CalledProcessError"]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000416
Brian Curtin1ce6b582010-04-24 16:19:22 +0000417if mswindows:
Brian Curtin08fd8d92011-04-29 16:11:30 -0500418 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500419 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
420 STD_ERROR_HANDLE, SW_HIDE,
421 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
Gregory P. Smith0ef3e392011-05-11 22:20:11 -0700422
Brian Curtin08fd8d92011-04-29 16:11:30 -0500423 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500424 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
425 "STD_ERROR_HANDLE", "SW_HIDE",
426 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000427try:
428 MAXFD = os.sysconf("SC_OPEN_MAX")
429except:
430 MAXFD = 256
431
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000432_active = []
433
434def _cleanup():
435 for inst in _active[:]:
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000436 res = inst._internal_poll(_deadstate=sys.maxsize)
Guido van Rossumb5d47ef2006-08-24 02:27:45 +0000437 if res is not None and res >= 0:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000438 try:
439 _active.remove(inst)
440 except ValueError:
441 # This can happen if two threads create a new Popen instance.
442 # It's harmless that it was already removed, so ignore.
443 pass
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000444
445PIPE = -1
446STDOUT = -2
447
448
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000449def _eintr_retry_call(func, *args):
450 while True:
451 try:
452 return func(*args)
453 except OSError as e:
454 if e.errno == errno.EINTR:
455 continue
456 raise
457
458
Peter Astrand5f5e1412004-12-05 20:15:36 +0000459def call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000460 """Run command with arguments. Wait for command to complete, then
461 return the returncode attribute.
462
463 The arguments are the same as for the Popen constructor. Example:
464
465 retcode = call(["ls", "-l"])
466 """
Peter Astrand5f5e1412004-12-05 20:15:36 +0000467 return Popen(*popenargs, **kwargs).wait()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000468
469
Peter Astrand454f7672005-01-01 09:36:35 +0000470def check_call(*popenargs, **kwargs):
471 """Run command with arguments. Wait for command to complete. If
472 the exit code was zero then return, otherwise raise
473 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000474 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000475
476 The arguments are the same as for the Popen constructor. Example:
477
478 check_call(["ls", "-l"])
479 """
480 retcode = call(*popenargs, **kwargs)
Peter Astrand454f7672005-01-01 09:36:35 +0000481 if retcode:
Georg Brandlf9734072008-12-07 15:30:06 +0000482 cmd = kwargs.get("args")
483 if cmd is None:
484 cmd = popenargs[0]
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000485 raise CalledProcessError(retcode, cmd)
Georg Brandlf9734072008-12-07 15:30:06 +0000486 return 0
487
488
489def check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000490 r"""Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000491
492 If the exit code was non-zero it raises a CalledProcessError. The
493 CalledProcessError object will have the return code in the returncode
494 attribute and output in the output attribute.
495
496 The arguments are the same as for the Popen constructor. Example:
497
498 >>> check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000499 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000500
501 The stdout argument is not allowed as it is used internally.
Georg Brandl127d4702009-12-28 08:10:38 +0000502 To capture standard error in the result, use stderr=STDOUT.
Georg Brandlf9734072008-12-07 15:30:06 +0000503
504 >>> check_output(["/bin/sh", "-c",
Georg Brandl2708f3a2009-12-20 14:38:23 +0000505 ... "ls -l non_existent_file ; exit 0"],
Georg Brandl127d4702009-12-28 08:10:38 +0000506 ... stderr=STDOUT)
Georg Brandl2708f3a2009-12-20 14:38:23 +0000507 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000508 """
509 if 'stdout' in kwargs:
510 raise ValueError('stdout argument not allowed, it will be overridden.')
511 process = Popen(*popenargs, stdout=PIPE, **kwargs)
512 output, unused_err = process.communicate()
513 retcode = process.poll()
514 if retcode:
515 cmd = kwargs.get("args")
516 if cmd is None:
517 cmd = popenargs[0]
518 raise CalledProcessError(retcode, cmd, output=output)
519 return output
Peter Astrand454f7672005-01-01 09:36:35 +0000520
521
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000522def list2cmdline(seq):
523 """
524 Translate a sequence of arguments into a command line
525 string, using the same rules as the MS C runtime:
526
527 1) Arguments are delimited by white space, which is either a
528 space or a tab.
529
530 2) A string surrounded by double quotation marks is
531 interpreted as a single argument, regardless of white space
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000532 contained within. A quoted string can be embedded in an
533 argument.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000534
535 3) A double quotation mark preceded by a backslash is
536 interpreted as a literal double quotation mark.
537
538 4) Backslashes are interpreted literally, unless they
539 immediately precede a double quotation mark.
540
541 5) If backslashes immediately precede a double quotation mark,
542 every pair of backslashes is interpreted as a literal
543 backslash. If the number of backslashes is odd, the last
544 backslash escapes the next double quotation mark as
545 described in rule 3.
546 """
547
548 # See
Eric Smith3c573af2009-11-09 15:23:15 +0000549 # http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
550 # or search http://msdn.microsoft.com for
551 # "Parsing C++ Command-Line Arguments"
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000552 result = []
553 needquote = False
554 for arg in seq:
555 bs_buf = []
556
557 # Add a space to separate this argument from the others
558 if result:
559 result.append(' ')
560
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000561 needquote = (" " in arg) or ("\t" in arg) or not arg
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000562 if needquote:
563 result.append('"')
564
565 for c in arg:
566 if c == '\\':
Tim Peterse718f612004-10-12 21:51:32 +0000567 # Don't know if we need to double yet.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000568 bs_buf.append(c)
569 elif c == '"':
Christian Heimesfdab48e2008-01-20 09:06:41 +0000570 # Double backslashes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000571 result.append('\\' * len(bs_buf)*2)
572 bs_buf = []
573 result.append('\\"')
574 else:
575 # Normal char
576 if bs_buf:
577 result.extend(bs_buf)
578 bs_buf = []
579 result.append(c)
580
Christian Heimesfdab48e2008-01-20 09:06:41 +0000581 # Add remaining backslashes, if any.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000582 if bs_buf:
583 result.extend(bs_buf)
584
585 if needquote:
Peter Astrand7e78ade2005-03-03 21:10:23 +0000586 result.extend(bs_buf)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000587 result.append('"')
588
589 return ''.join(result)
590
591
Brett Cannona23810f2008-05-26 19:04:21 +0000592# Various tools for executing commands and looking at their output and status.
593#
Gregory P. Smithf5604852010-12-13 06:45:02 +0000594# NB This only works (and is only relevant) for POSIX.
Brett Cannona23810f2008-05-26 19:04:21 +0000595
596def getstatusoutput(cmd):
597 """Return (status, output) of executing cmd in a shell.
598
599 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
600 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
601 returned output will contain output or error messages. A trailing newline
602 is stripped from the output. The exit status for the command can be
603 interpreted according to the rules for the C function wait(). Example:
604
605 >>> import subprocess
606 >>> subprocess.getstatusoutput('ls /bin/ls')
607 (0, '/bin/ls')
608 >>> subprocess.getstatusoutput('cat /bin/junk')
609 (256, 'cat: /bin/junk: No such file or directory')
610 >>> subprocess.getstatusoutput('/bin/junk')
611 (256, 'sh: /bin/junk: not found')
612 """
613 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
614 text = pipe.read()
615 sts = pipe.close()
616 if sts is None: sts = 0
617 if text[-1:] == '\n': text = text[:-1]
618 return sts, text
619
620
621def getoutput(cmd):
622 """Return output (stdout or stderr) of executing cmd in a shell.
623
624 Like getstatusoutput(), except the exit status is ignored and the return
625 value is a string containing the command's output. Example:
626
627 >>> import subprocess
628 >>> subprocess.getoutput('ls /bin/ls')
629 '/bin/ls'
630 """
631 return getstatusoutput(cmd)[1]
632
633
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000634_PLATFORM_DEFAULT_CLOSE_FDS = object()
Gregory P. Smithf5604852010-12-13 06:45:02 +0000635
636
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000637class Popen(object):
638 def __init__(self, args, bufsize=0, executable=None,
639 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000640 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
641 shell=False, cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000642 startupinfo=None, creationflags=0,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000643 restore_signals=True, start_new_session=False,
644 pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000645 """Create new Popen instance."""
646 _cleanup()
647
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000648 self._child_created = False
Guido van Rossum46a05a72007-06-07 21:56:45 +0000649 if bufsize is None:
650 bufsize = 0 # Restore default
Walter Dörwaldaa97f042007-05-03 21:05:51 +0000651 if not isinstance(bufsize, int):
Peter Astrand738131d2004-11-30 21:04:45 +0000652 raise TypeError("bufsize must be an integer")
653
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000654 if mswindows:
Tim Peterse8374a52004-10-13 03:15:00 +0000655 if preexec_fn is not None:
656 raise ValueError("preexec_fn is not supported on Windows "
657 "platforms")
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000658 any_stdio_set = (stdin is not None or stdout is not None or
659 stderr is not None)
660 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
661 if any_stdio_set:
662 close_fds = False
663 else:
664 close_fds = True
665 elif close_fds and any_stdio_set:
666 raise ValueError(
667 "close_fds is not supported on Windows platforms"
668 " if you redirect stdin/stdout/stderr")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000669 else:
670 # POSIX
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000671 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
672 close_fds = True
673 if pass_fds and not close_fds:
674 warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
675 close_fds = True
Tim Peterse8374a52004-10-13 03:15:00 +0000676 if startupinfo is not None:
677 raise ValueError("startupinfo is only supported on Windows "
678 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000679 if creationflags != 0:
Tim Peterse8374a52004-10-13 03:15:00 +0000680 raise ValueError("creationflags is only supported on Windows "
681 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000682
Tim Peterse718f612004-10-12 21:51:32 +0000683 self.stdin = None
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000684 self.stdout = None
685 self.stderr = None
686 self.pid = None
687 self.returncode = None
688 self.universal_newlines = universal_newlines
689
690 # Input and output objects. The general principle is like
691 # this:
692 #
693 # Parent Child
694 # ------ -----
695 # p2cwrite ---stdin---> p2cread
696 # c2pread <--stdout--- c2pwrite
697 # errread <--stderr--- errwrite
698 #
699 # On POSIX, the child objects are file descriptors. On
700 # Windows, these are Windows file handles. The parent objects
701 # are file descriptors on both platforms. The parent objects
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000702 # are -1 when not using PIPEs. The child objects are -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000703 # when not redirecting.
Tim Peterse718f612004-10-12 21:51:32 +0000704
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000705 (p2cread, p2cwrite,
706 c2pread, c2pwrite,
707 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
708
Antoine Pitrouc9982322011-01-04 19:07:07 +0000709 # We wrap OS handles *before* launching the child, otherwise a
710 # quickly terminating child could make our fds unwrappable
711 # (see #8458).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000712
Thomas Wouterscf297e42007-02-23 15:07:44 +0000713 if mswindows:
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000714 if p2cwrite != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000715 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000716 if c2pread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000717 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000718 if errread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000719 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
Thomas Wouterscf297e42007-02-23 15:07:44 +0000720
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000721 if p2cwrite != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000722 self.stdin = io.open(p2cwrite, 'wb', bufsize)
723 if self.universal_newlines:
724 self.stdin = io.TextIOWrapper(self.stdin)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000725 if c2pread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000726 self.stdout = io.open(c2pread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000727 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000728 self.stdout = io.TextIOWrapper(self.stdout)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000729 if errread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000730 self.stderr = io.open(errread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000731 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000732 self.stderr = io.TextIOWrapper(self.stderr)
Tim Peterse718f612004-10-12 21:51:32 +0000733
Antoine Pitrouc9982322011-01-04 19:07:07 +0000734 try:
735 self._execute_child(args, executable, preexec_fn, close_fds,
736 pass_fds, cwd, env, universal_newlines,
737 startupinfo, creationflags, shell,
738 p2cread, p2cwrite,
739 c2pread, c2pwrite,
740 errread, errwrite,
741 restore_signals, start_new_session)
742 except:
743 # Cleanup if the child failed starting
744 for f in filter(None, [self.stdin, self.stdout, self.stderr]):
745 try:
746 f.close()
747 except EnvironmentError:
748 # Ignore EBADF or other errors
749 pass
750 raise
751
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000752
Guido van Rossum98297ee2007-11-06 21:34:58 +0000753 def _translate_newlines(self, data, encoding):
754 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")
755 return data.decode(encoding)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000756
Brian Curtin79cdb662010-12-03 02:46:02 +0000757 def __enter__(self):
758 return self
759
760 def __exit__(self, type, value, traceback):
761 if self.stdout:
762 self.stdout.close()
763 if self.stderr:
764 self.stderr.close()
765 if self.stdin:
766 self.stdin.close()
Gregory P. Smithc9557af2011-05-11 22:18:23 -0700767 # Wait for the process to terminate, to avoid zombies.
768 self.wait()
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000769
Brett Cannon84df1e62010-05-14 00:33:40 +0000770 def __del__(self, _maxsize=sys.maxsize, _active=_active):
Victor Stinner87b9bc32011-06-01 00:57:47 +0200771 # If __init__ hasn't had a chance to execute (e.g. if it
772 # was passed an undeclared keyword argument), we don't
773 # have a _child_created attribute at all.
774 if not getattr(self, '_child_created', False):
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000775 # We didn't get to successfully create a child process.
776 return
777 # In case the child hasn't been waited on, check if it's done.
Brett Cannon84df1e62010-05-14 00:33:40 +0000778 self._internal_poll(_deadstate=_maxsize)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000779 if self.returncode is None and _active is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000780 # Child is still running, keep us alive until we can wait on it.
781 _active.append(self)
782
783
Peter Astrand23109f02005-03-03 20:28:59 +0000784 def communicate(self, input=None):
785 """Interact with process: Send data to stdin. Read data from
786 stdout and stderr, until end-of-file is reached. Wait for
787 process to terminate. The optional input argument should be a
788 string to be sent to the child process, or None, if no data
789 should be sent to the child.
Tim Peterseba28be2005-03-28 01:08:02 +0000790
Peter Astrand23109f02005-03-03 20:28:59 +0000791 communicate() returns a tuple (stdout, stderr)."""
792
793 # Optimization: If we are only using one pipe, or no pipe at
794 # all, using select() or threads is unnecessary.
795 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
Tim Peterseba28be2005-03-28 01:08:02 +0000796 stdout = None
797 stderr = None
Peter Astrand23109f02005-03-03 20:28:59 +0000798 if self.stdin:
799 if input:
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200800 try:
801 self.stdin.write(input)
802 except IOError as e:
803 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
804 raise
Peter Astrand23109f02005-03-03 20:28:59 +0000805 self.stdin.close()
806 elif self.stdout:
807 stdout = self.stdout.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000808 self.stdout.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000809 elif self.stderr:
810 stderr = self.stderr.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000811 self.stderr.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000812 self.wait()
813 return (stdout, stderr)
Tim Peterseba28be2005-03-28 01:08:02 +0000814
Peter Astrand23109f02005-03-03 20:28:59 +0000815 return self._communicate(input)
816
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000817
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000818 def poll(self):
819 return self._internal_poll()
820
821
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000822 if mswindows:
823 #
824 # Windows methods
825 #
826 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +0000827 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000828 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
829 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000830 if stdin is None and stdout is None and stderr is None:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000831 return (-1, -1, -1, -1, -1, -1)
Tim Peterse718f612004-10-12 21:51:32 +0000832
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000833 p2cread, p2cwrite = -1, -1
834 c2pread, c2pwrite = -1, -1
835 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000836
Peter Astrandd38ddf42005-02-10 08:32:50 +0000837 if stdin is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000838 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000839 if p2cread is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000840 p2cread, _ = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000841 elif stdin == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000842 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000843 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000844 p2cread = msvcrt.get_osfhandle(stdin)
845 else:
846 # Assuming file-like object
847 p2cread = msvcrt.get_osfhandle(stdin.fileno())
848 p2cread = self._make_inheritable(p2cread)
849
Peter Astrandd38ddf42005-02-10 08:32:50 +0000850 if stdout is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000851 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000852 if c2pwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000853 _, c2pwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000854 elif stdout == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000855 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000856 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000857 c2pwrite = msvcrt.get_osfhandle(stdout)
858 else:
859 # Assuming file-like object
860 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
861 c2pwrite = self._make_inheritable(c2pwrite)
862
Peter Astrandd38ddf42005-02-10 08:32:50 +0000863 if stderr is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000864 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000865 if errwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000866 _, errwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000867 elif stderr == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000868 errread, errwrite = _subprocess.CreatePipe(None, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000869 elif stderr == STDOUT:
870 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +0000871 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000872 errwrite = msvcrt.get_osfhandle(stderr)
873 else:
874 # Assuming file-like object
875 errwrite = msvcrt.get_osfhandle(stderr.fileno())
876 errwrite = self._make_inheritable(errwrite)
877
878 return (p2cread, p2cwrite,
879 c2pread, c2pwrite,
880 errread, errwrite)
881
882
883 def _make_inheritable(self, handle):
884 """Return a duplicate of handle, which is inheritable"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000885 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
886 handle, _subprocess.GetCurrentProcess(), 0, 1,
887 _subprocess.DUPLICATE_SAME_ACCESS)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000888
889
890 def _find_w9xpopen(self):
891 """Find and return absolut path to w9xpopen.exe"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000892 w9xpopen = os.path.join(
893 os.path.dirname(_subprocess.GetModuleFileName(0)),
Tim Peterse8374a52004-10-13 03:15:00 +0000894 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000895 if not os.path.exists(w9xpopen):
896 # Eeek - file-not-found - possibly an embedding
897 # situation - see if we can locate it in sys.exec_prefix
Tim Peterse8374a52004-10-13 03:15:00 +0000898 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
899 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000900 if not os.path.exists(w9xpopen):
Tim Peterse8374a52004-10-13 03:15:00 +0000901 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
902 "needed for Popen to work with your "
903 "shell or platform.")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000904 return w9xpopen
905
Tim Peterse718f612004-10-12 21:51:32 +0000906
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000907 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000908 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000909 startupinfo, creationflags, shell,
910 p2cread, p2cwrite,
911 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000912 errread, errwrite,
913 unused_restore_signals, unused_start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000914 """Execute program (MS Windows version)"""
915
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000916 assert not pass_fds, "pass_fds not supported on Windows."
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000917
Guido van Rossum3172c5d2007-10-16 18:12:55 +0000918 if not isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000919 args = list2cmdline(args)
920
Peter Astrandc1d65362004-11-07 14:30:34 +0000921 # Process startup details
Peter Astrandd38ddf42005-02-10 08:32:50 +0000922 if startupinfo is None:
Thomas Wouters73e5a5b2006-06-08 15:35:45 +0000923 startupinfo = STARTUPINFO()
Victor Stinnerb3693582010-05-21 20:13:12 +0000924 if -1 not in (p2cread, c2pwrite, errwrite):
Brian Curtin1ce6b582010-04-24 16:19:22 +0000925 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
Peter Astrandc1d65362004-11-07 14:30:34 +0000926 startupinfo.hStdInput = p2cread
927 startupinfo.hStdOutput = c2pwrite
928 startupinfo.hStdError = errwrite
929
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000930 if shell:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000931 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
932 startupinfo.wShowWindow = _subprocess.SW_HIDE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000933 comspec = os.environ.get("COMSPEC", "cmd.exe")
Tim Golden126c2962010-08-11 14:20:40 +0000934 args = '{} /c "{}"'.format (comspec, args)
Brian Curtin1ce6b582010-04-24 16:19:22 +0000935 if (_subprocess.GetVersion() >= 0x80000000 or
Tim Peterse8374a52004-10-13 03:15:00 +0000936 os.path.basename(comspec).lower() == "command.com"):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000937 # Win9x, or using command.com on NT. We need to
938 # use the w9xpopen intermediate program. For more
939 # information, see KB Q150956
940 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
941 w9xpopen = self._find_w9xpopen()
942 args = '"%s" %s' % (w9xpopen, args)
943 # Not passing CREATE_NEW_CONSOLE has been known to
944 # cause random failures on win9x. Specifically a
945 # dialog: "Your program accessed mem currently in
946 # use at xxx" and a hopeful warning about the
Mark Dickinson934896d2009-02-21 20:59:32 +0000947 # stability of your system. Cost is Ctrl+C won't
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000948 # kill children.
Brian Curtin1ce6b582010-04-24 16:19:22 +0000949 creationflags |= _subprocess.CREATE_NEW_CONSOLE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000950
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000951 # Start the process
952 try:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000953 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
Tim Peterse8374a52004-10-13 03:15:00 +0000954 # no special security
955 None, None,
Guido van Rossume7ba4952007-06-06 23:52:48 +0000956 int(not close_fds),
Tim Peterse8374a52004-10-13 03:15:00 +0000957 creationflags,
958 env,
959 cwd,
960 startupinfo)
Guido van Rossumb940e112007-01-10 16:19:56 +0000961 except pywintypes.error as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000962 # Translate pywintypes.error to WindowsError, which is
963 # a subclass of OSError. FIXME: We should really
Ezio Melotti13925002011-03-16 11:05:33 +0200964 # translate errno using _sys_errlist (or similar), but
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000965 # how can this be done from Python?
966 raise WindowsError(*e.args)
Tim Goldenad537f22010-08-08 11:18:16 +0000967 finally:
968 # Child is launched. Close the parent's copy of those pipe
969 # handles that only the child should have open. You need
970 # to make sure that no handles to the write end of the
971 # output pipe are maintained in this process or else the
972 # pipe will not close when the child process exits and the
973 # ReadFile will hang.
974 if p2cread != -1:
975 p2cread.Close()
976 if c2pwrite != -1:
977 c2pwrite.Close()
978 if errwrite != -1:
979 errwrite.Close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000980
981 # Retain the process handle, but close the thread handle
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000982 self._child_created = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000983 self._handle = hp
984 self.pid = pid
985 ht.Close()
986
Brett Cannon84df1e62010-05-14 00:33:40 +0000987 def _internal_poll(self, _deadstate=None,
Victor Stinnerc807a612010-05-14 21:53:45 +0000988 _WaitForSingleObject=_subprocess.WaitForSingleObject,
989 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
990 _GetExitCodeProcess=_subprocess.GetExitCodeProcess):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000991 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +0000992 attribute.
993
994 This method is called by __del__, so it can only refer to objects
995 in its local scope.
996
997 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000998 if self.returncode is None:
Brett Cannon84df1e62010-05-14 00:33:40 +0000999 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
1000 self.returncode = _GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001001 return self.returncode
1002
1003
1004 def wait(self):
1005 """Wait for child process to terminate. Returns returncode
1006 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001007 if self.returncode is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +00001008 _subprocess.WaitForSingleObject(self._handle,
1009 _subprocess.INFINITE)
1010 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001011 return self.returncode
1012
1013
1014 def _readerthread(self, fh, buffer):
1015 buffer.append(fh.read())
Victor Stinner667d4b52010-12-25 22:40:32 +00001016 fh.close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001017
1018
Peter Astrand23109f02005-03-03 20:28:59 +00001019 def _communicate(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001020 stdout = None # Return
1021 stderr = None # Return
1022
1023 if self.stdout:
1024 stdout = []
Tim Peterse8374a52004-10-13 03:15:00 +00001025 stdout_thread = threading.Thread(target=self._readerthread,
1026 args=(self.stdout, stdout))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001027 stdout_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001028 stdout_thread.start()
1029 if self.stderr:
1030 stderr = []
Tim Peterse8374a52004-10-13 03:15:00 +00001031 stderr_thread = threading.Thread(target=self._readerthread,
1032 args=(self.stderr, stderr))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001033 stderr_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001034 stderr_thread.start()
1035
1036 if self.stdin:
Peter Astrandd38ddf42005-02-10 08:32:50 +00001037 if input is not None:
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001038 try:
1039 self.stdin.write(input)
1040 except IOError as e:
1041 if e.errno != errno.EPIPE:
1042 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001043 self.stdin.close()
1044
1045 if self.stdout:
1046 stdout_thread.join()
1047 if self.stderr:
1048 stderr_thread.join()
1049
1050 # All data exchanged. Translate lists into strings.
Peter Astrandd38ddf42005-02-10 08:32:50 +00001051 if stdout is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001052 stdout = stdout[0]
Peter Astrandd38ddf42005-02-10 08:32:50 +00001053 if stderr is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001054 stderr = stderr[0]
1055
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001056 self.wait()
1057 return (stdout, stderr)
1058
Christian Heimesa342c012008-04-20 21:01:16 +00001059 def send_signal(self, sig):
1060 """Send a signal to the process
1061 """
1062 if sig == signal.SIGTERM:
1063 self.terminate()
Brian Curtineb24d742010-04-12 17:16:38 +00001064 elif sig == signal.CTRL_C_EVENT:
1065 os.kill(self.pid, signal.CTRL_C_EVENT)
1066 elif sig == signal.CTRL_BREAK_EVENT:
1067 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
Christian Heimesa342c012008-04-20 21:01:16 +00001068 else:
Brian Curtin19651362010-09-07 13:24:38 +00001069 raise ValueError("Unsupported signal: {}".format(sig))
Christian Heimesa342c012008-04-20 21:01:16 +00001070
1071 def terminate(self):
1072 """Terminates the process
1073 """
Brian Curtin1ce6b582010-04-24 16:19:22 +00001074 _subprocess.TerminateProcess(self._handle, 1)
Christian Heimesa342c012008-04-20 21:01:16 +00001075
1076 kill = terminate
1077
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001078 else:
1079 #
1080 # POSIX methods
1081 #
1082 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +00001083 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001084 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1085 """
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001086 p2cread, p2cwrite = -1, -1
1087 c2pread, c2pwrite = -1, -1
1088 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001089
Peter Astrandd38ddf42005-02-10 08:32:50 +00001090 if stdin is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001091 pass
1092 elif stdin == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001093 p2cread, p2cwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001094 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001095 p2cread = stdin
1096 else:
1097 # Assuming file-like object
1098 p2cread = stdin.fileno()
1099
Peter Astrandd38ddf42005-02-10 08:32:50 +00001100 if stdout is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001101 pass
1102 elif stdout == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001103 c2pread, c2pwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001104 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001105 c2pwrite = stdout
1106 else:
1107 # Assuming file-like object
Tim Peterse718f612004-10-12 21:51:32 +00001108 c2pwrite = stdout.fileno()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001109
Peter Astrandd38ddf42005-02-10 08:32:50 +00001110 if stderr is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001111 pass
1112 elif stderr == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001113 errread, errwrite = _create_pipe()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001114 elif stderr == STDOUT:
1115 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +00001116 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001117 errwrite = stderr
1118 else:
1119 # Assuming file-like object
1120 errwrite = stderr.fileno()
1121
1122 return (p2cread, p2cwrite,
1123 c2pread, c2pwrite,
1124 errread, errwrite)
1125
1126
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001127 def _close_fds(self, fds_to_keep):
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001128 start_fd = 3
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001129 for fd in sorted(fds_to_keep):
Gregory P. Smith8edd99d2010-12-14 13:43:30 +00001130 if fd >= start_fd:
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001131 os.closerange(start_fd, fd)
1132 start_fd = fd + 1
1133 if start_fd <= MAXFD:
1134 os.closerange(start_fd, MAXFD)
1135
1136
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001137 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001138 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001139 startupinfo, creationflags, shell,
1140 p2cread, p2cwrite,
1141 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001142 errread, errwrite,
1143 restore_signals, start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001144 """Execute program (POSIX version)"""
1145
Guido van Rossum3172c5d2007-10-16 18:12:55 +00001146 if isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001147 args = [args]
Thomas Wouters89f507f2006-12-13 04:49:30 +00001148 else:
1149 args = list(args)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001150
1151 if shell:
1152 args = ["/bin/sh", "-c"] + args
Stefan Krah9542cc62010-07-19 14:20:53 +00001153 if executable:
1154 args[0] = executable
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001155
Peter Astrandd38ddf42005-02-10 08:32:50 +00001156 if executable is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001157 executable = args[0]
1158
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001159 # For transferring possible exec failure from child to parent.
1160 # Data format: "exception name:hex errno:description"
1161 # Pickle is not used; it is complex and involves memory allocation.
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001162 errpipe_read, errpipe_write = _create_pipe()
Christian Heimesfdab48e2008-01-20 09:06:41 +00001163 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001164 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001165
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001166 if _posixsubprocess:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001167 # We must avoid complex work that could involve
1168 # malloc or free in the child process to avoid
1169 # potential deadlocks, thus we do all this here.
1170 # and pass it to fork_exec()
1171
Victor Stinnerf1512a22011-06-21 17:18:38 +02001172 if env is not None:
Victor Stinner449c4662010-05-08 11:10:09 +00001173 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001174 for k, v in env.items()]
1175 else:
1176 env_list = None # Use execv instead of execve.
Victor Stinnerb745a742010-05-18 17:17:23 +00001177 executable = os.fsencode(executable)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001178 if os.path.dirname(executable):
Victor Stinnerb745a742010-05-18 17:17:23 +00001179 executable_list = (executable,)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001180 else:
1181 # This matches the behavior of os._execvpe().
Victor Stinnerb745a742010-05-18 17:17:23 +00001182 executable_list = tuple(
1183 os.path.join(os.fsencode(dir), executable)
1184 for dir in os.get_exec_path(env))
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001185 fds_to_keep = set(pass_fds)
1186 fds_to_keep.add(errpipe_write)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001187 self.pid = _posixsubprocess.fork_exec(
1188 args, executable_list,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001189 close_fds, sorted(fds_to_keep), cwd, env_list,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001190 p2cread, p2cwrite, c2pread, c2pwrite,
1191 errread, errwrite,
1192 errpipe_read, errpipe_write,
1193 restore_signals, start_new_session, preexec_fn)
1194 else:
1195 # Pure Python implementation: It is not thread safe.
1196 # This implementation may deadlock in the child if your
1197 # parent process has any other threads running.
1198
1199 gc_was_enabled = gc.isenabled()
1200 # Disable gc to avoid bug where gc -> file_dealloc ->
1201 # write to stderr -> hang. See issue1336
1202 gc.disable()
1203 try:
1204 self.pid = os.fork()
1205 except:
1206 if gc_was_enabled:
1207 gc.enable()
1208 raise
1209 self._child_created = True
1210 if self.pid == 0:
1211 # Child
1212 try:
1213 # Close parent's pipe ends
1214 if p2cwrite != -1:
1215 os.close(p2cwrite)
1216 if c2pread != -1:
1217 os.close(c2pread)
1218 if errread != -1:
1219 os.close(errread)
1220 os.close(errpipe_read)
1221
1222 # Dup fds for child
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001223 def _dup2(a, b):
1224 # dup2() removes the CLOEXEC flag but
1225 # we must do it ourselves if dup2()
1226 # would be a no-op (issue #10806).
1227 if a == b:
1228 _set_cloexec(a, False)
1229 elif a != -1:
1230 os.dup2(a, b)
1231 _dup2(p2cread, 0)
1232 _dup2(c2pwrite, 1)
1233 _dup2(errwrite, 2)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001234
1235 # Close pipe fds. Make sure we don't close the
1236 # same fd more than once, or standard fds.
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001237 closed = set()
1238 for fd in [p2cread, c2pwrite, errwrite]:
1239 if fd > 2 and fd not in closed:
1240 os.close(fd)
1241 closed.add(fd)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001242
1243 # Close all other fds, if asked for
1244 if close_fds:
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001245 fds_to_keep = set(pass_fds)
1246 fds_to_keep.add(errpipe_write)
1247 self._close_fds(fds_to_keep)
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001248
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001249
1250 if cwd is not None:
1251 os.chdir(cwd)
1252
1253 # This is a copy of Python/pythonrun.c
1254 # _Py_RestoreSignals(). If that were exposed
1255 # as a sys._py_restoresignals func it would be
1256 # better.. but this pure python implementation
1257 # isn't likely to be used much anymore.
1258 if restore_signals:
1259 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1260 for sig in signals:
1261 if hasattr(signal, sig):
1262 signal.signal(getattr(signal, sig),
1263 signal.SIG_DFL)
1264
1265 if start_new_session and hasattr(os, 'setsid'):
1266 os.setsid()
1267
1268 if preexec_fn:
1269 preexec_fn()
1270
1271 if env is None:
1272 os.execvp(executable, args)
1273 else:
1274 os.execvpe(executable, args, env)
1275
1276 except:
1277 try:
1278 exc_type, exc_value = sys.exc_info()[:2]
1279 if isinstance(exc_value, OSError):
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001280 errno_num = exc_value.errno
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001281 else:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001282 errno_num = 0
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001283 message = '%s:%x:%s' % (exc_type.__name__,
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001284 errno_num, exc_value)
Victor Stinner4d078042010-04-23 19:28:32 +00001285 message = message.encode(errors="surrogatepass")
1286 os.write(errpipe_write, message)
1287 except Exception:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001288 # We MUST not allow anything odd happening
1289 # above to prevent us from exiting below.
1290 pass
1291
1292 # This exitcode won't be reported to applications
1293 # so it really doesn't matter what we return.
1294 os._exit(255)
1295
1296 # Parent
Facundo Batista10706e22009-06-19 20:34:30 +00001297 if gc_was_enabled:
1298 gc.enable()
Facundo Batista10706e22009-06-19 20:34:30 +00001299 finally:
1300 # be sure the FD is closed no matter what
1301 os.close(errpipe_write)
1302
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001303 if p2cread != -1 and p2cwrite != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001304 os.close(p2cread)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001305 if c2pwrite != -1 and c2pread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001306 os.close(c2pwrite)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001307 if errwrite != -1 and errread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001308 os.close(errwrite)
1309
1310 # Wait for exec to fail or succeed; possibly raising an
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001311 # exception (limited in size)
1312 data = bytearray()
1313 while True:
1314 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1315 data += part
1316 if not part or len(data) > 50000:
1317 break
Facundo Batista10706e22009-06-19 20:34:30 +00001318 finally:
1319 # be sure the FD is closed no matter what
1320 os.close(errpipe_read)
1321
Guido van Rossumaf2362a2007-05-15 22:32:02 +00001322 if data:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001323 try:
1324 _eintr_retry_call(os.waitpid, self.pid, 0)
1325 except OSError as e:
1326 if e.errno != errno.ECHILD:
1327 raise
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001328 try:
1329 exception_name, hex_errno, err_msg = data.split(b':', 2)
1330 except ValueError:
1331 print('Bad exception data:', repr(data))
1332 exception_name = b'RuntimeError'
1333 hex_errno = b'0'
1334 err_msg = b'Unknown'
1335 child_exception_type = getattr(
1336 builtins, exception_name.decode('ascii'),
1337 RuntimeError)
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001338 for fd in (p2cwrite, c2pread, errread):
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001339 if fd != -1:
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001340 os.close(fd)
Victor Stinner4d078042010-04-23 19:28:32 +00001341 err_msg = err_msg.decode(errors="surrogatepass")
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001342 if issubclass(child_exception_type, OSError) and hex_errno:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001343 errno_num = int(hex_errno, 16)
1344 if errno_num != 0:
1345 err_msg = os.strerror(errno_num)
1346 if errno_num == errno.ENOENT:
Benjamin Peterson5f780402010-11-20 18:07:52 +00001347 err_msg += ': ' + repr(args[0])
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001348 raise child_exception_type(errno_num, err_msg)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001349 raise child_exception_type(err_msg)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001350
1351
Brett Cannon84df1e62010-05-14 00:33:40 +00001352 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1353 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1354 _WEXITSTATUS=os.WEXITSTATUS):
1355 # This method is called (indirectly) by __del__, so it cannot
1356 # refer to anything outside of its local scope."""
1357 if _WIFSIGNALED(sts):
1358 self.returncode = -_WTERMSIG(sts)
1359 elif _WIFEXITED(sts):
1360 self.returncode = _WEXITSTATUS(sts)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001361 else:
1362 # Should never happen
1363 raise RuntimeError("Unknown child exit status!")
1364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001365
Brett Cannon84df1e62010-05-14 00:33:40 +00001366 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1367 _WNOHANG=os.WNOHANG, _os_error=os.error):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001368 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +00001369 attribute.
1370
1371 This method is called by __del__, so it cannot reference anything
1372 outside of the local scope (nor can any methods it calls).
1373
1374 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001375 if self.returncode is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001376 try:
Brett Cannon84df1e62010-05-14 00:33:40 +00001377 pid, sts = _waitpid(self.pid, _WNOHANG)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001378 if pid == self.pid:
1379 self._handle_exitstatus(sts)
Brett Cannon84df1e62010-05-14 00:33:40 +00001380 except _os_error:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +00001381 if _deadstate is not None:
1382 self.returncode = _deadstate
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001383 return self.returncode
1384
1385
1386 def wait(self):
1387 """Wait for child process to terminate. Returns returncode
1388 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001389 if self.returncode is None:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001390 try:
1391 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1392 except OSError as e:
1393 if e.errno != errno.ECHILD:
1394 raise
1395 # This happens if SIGCLD is set to be ignored or waiting
1396 # for child processes has otherwise been disabled for our
1397 # process. This child is dead, we can't get the status.
1398 sts = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001399 self._handle_exitstatus(sts)
1400 return self.returncode
1401
1402
Peter Astrand23109f02005-03-03 20:28:59 +00001403 def _communicate(self, input):
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001404 if self.stdin:
1405 # Flush stdio buffer. This might block, if the user has
1406 # been writing to .stdin in an uncontrolled fashion.
1407 self.stdin.flush()
1408 if not input:
1409 self.stdin.close()
1410
1411 if _has_poll:
1412 stdout, stderr = self._communicate_with_poll(input)
1413 else:
1414 stdout, stderr = self._communicate_with_select(input)
1415
1416 # All data exchanged. Translate lists into strings.
1417 if stdout is not None:
1418 stdout = b''.join(stdout)
1419 if stderr is not None:
1420 stderr = b''.join(stderr)
1421
1422 # Translate newlines, if requested.
1423 # This also turns bytes into strings.
1424 if self.universal_newlines:
1425 if stdout is not None:
1426 stdout = self._translate_newlines(stdout,
1427 self.stdout.encoding)
1428 if stderr is not None:
1429 stderr = self._translate_newlines(stderr,
1430 self.stderr.encoding)
1431
1432 self.wait()
1433 return (stdout, stderr)
1434
1435
1436 def _communicate_with_poll(self, input):
1437 stdout = None # Return
1438 stderr = None # Return
1439 fd2file = {}
1440 fd2output = {}
1441
1442 poller = select.poll()
1443 def register_and_append(file_obj, eventmask):
1444 poller.register(file_obj.fileno(), eventmask)
1445 fd2file[file_obj.fileno()] = file_obj
1446
1447 def close_unregister_and_remove(fd):
1448 poller.unregister(fd)
1449 fd2file[fd].close()
1450 fd2file.pop(fd)
1451
1452 if self.stdin and input:
1453 register_and_append(self.stdin, select.POLLOUT)
1454
1455 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1456 if self.stdout:
1457 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1458 fd2output[self.stdout.fileno()] = stdout = []
1459 if self.stderr:
1460 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1461 fd2output[self.stderr.fileno()] = stderr = []
1462
1463 input_offset = 0
1464 while fd2file:
1465 try:
1466 ready = poller.poll()
1467 except select.error as e:
1468 if e.args[0] == errno.EINTR:
1469 continue
1470 raise
1471
1472 # XXX Rewrite these to use non-blocking I/O on the
1473 # file objects; they are no longer using C stdio!
1474
1475 for fd, mode in ready:
1476 if mode & select.POLLOUT:
1477 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001478 try:
1479 input_offset += os.write(fd, chunk)
1480 except OSError as e:
1481 if e.errno == errno.EPIPE:
1482 close_unregister_and_remove(fd)
1483 else:
1484 raise
1485 else:
1486 if input_offset >= len(input):
1487 close_unregister_and_remove(fd)
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001488 elif mode & select_POLLIN_POLLPRI:
1489 data = os.read(fd, 4096)
1490 if not data:
1491 close_unregister_and_remove(fd)
1492 fd2output[fd].append(data)
1493 else:
1494 # Ignore hang up or errors.
1495 close_unregister_and_remove(fd)
1496
1497 return (stdout, stderr)
1498
1499
1500 def _communicate_with_select(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001501 read_set = []
1502 write_set = []
1503 stdout = None # Return
1504 stderr = None # Return
1505
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001506 if self.stdin and input:
1507 write_set.append(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001508 if self.stdout:
1509 read_set.append(self.stdout)
Tim Peterse718f612004-10-12 21:51:32 +00001510 stdout = []
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001511 if self.stderr:
1512 read_set.append(self.stderr)
1513 stderr = []
1514
Thomas Wouters902d6eb2007-01-09 23:18:33 +00001515 input_offset = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001516 while read_set or write_set:
Georg Brandl86b2fb92008-07-16 03:43:04 +00001517 try:
1518 rlist, wlist, xlist = select.select(read_set, write_set, [])
1519 except select.error as e:
1520 if e.args[0] == errno.EINTR:
1521 continue
1522 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001523
Guido van Rossum98297ee2007-11-06 21:34:58 +00001524 # XXX Rewrite these to use non-blocking I/O on the
1525 # file objects; they are no longer using C stdio!
1526
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001527 if self.stdin in wlist:
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001528 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001529 try:
1530 bytes_written = os.write(self.stdin.fileno(), chunk)
1531 except OSError as e:
1532 if e.errno == errno.EPIPE:
1533 self.stdin.close()
1534 write_set.remove(self.stdin)
1535 else:
1536 raise
1537 else:
1538 input_offset += bytes_written
1539 if input_offset >= len(input):
1540 self.stdin.close()
1541 write_set.remove(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001542
1543 if self.stdout in rlist:
1544 data = os.read(self.stdout.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001545 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001546 self.stdout.close()
1547 read_set.remove(self.stdout)
1548 stdout.append(data)
1549
1550 if self.stderr in rlist:
1551 data = os.read(self.stderr.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001552 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001553 self.stderr.close()
1554 read_set.remove(self.stderr)
1555 stderr.append(data)
1556
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001557 return (stdout, stderr)
1558
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001559
Christian Heimesa342c012008-04-20 21:01:16 +00001560 def send_signal(self, sig):
1561 """Send a signal to the process
1562 """
1563 os.kill(self.pid, sig)
1564
1565 def terminate(self):
1566 """Terminate the process with SIGTERM
1567 """
1568 self.send_signal(signal.SIGTERM)
1569
1570 def kill(self):
1571 """Kill the process with SIGKILL
1572 """
1573 self.send_signal(signal.SIGKILL)
1574
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001575
1576def _demo_posix():
1577 #
1578 # Example 1: Simple redirection: Get process list
1579 #
1580 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001581 print("Process list:")
1582 print(plist)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001583
1584 #
1585 # Example 2: Change uid before executing child
1586 #
1587 if os.getuid() == 0:
1588 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1589 p.wait()
1590
1591 #
1592 # Example 3: Connecting several subprocesses
1593 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001594 print("Looking for 'hda'...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001595 p1 = Popen(["dmesg"], stdout=PIPE)
1596 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001597 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001598
1599 #
1600 # Example 4: Catch execution error
1601 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001602 print()
1603 print("Trying a weird file...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001604 try:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001605 print(Popen(["/this/path/does/not/exist"]).communicate())
Guido van Rossumb940e112007-01-10 16:19:56 +00001606 except OSError as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001607 if e.errno == errno.ENOENT:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001608 print("The file didn't exist. I thought so...")
1609 print("Child traceback:")
1610 print(e.child_traceback)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001611 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001612 print("Error", e.errno)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001613 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001614 print("Gosh. No error.", file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001615
1616
1617def _demo_windows():
1618 #
1619 # Example 1: Connecting several subprocesses
1620 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001621 print("Looking for 'PROMPT' in set output...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001622 p1 = Popen("set", stdout=PIPE, shell=True)
1623 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001624 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001625
1626 #
1627 # Example 2: Simple execution of program
1628 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001629 print("Executing calc...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001630 p = Popen("calc")
1631 p.wait()
1632
1633
1634if __name__ == "__main__":
1635 if mswindows:
1636 _demo_windows()
1637 else:
1638 _demo_posix()