blob: 179f41a85f56c82def6489d77e07f0dede23c66c [file] [log] [blame]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001# subprocess - Subprocesses with accessible I/O streams
2#
Tim Peterse718f612004-10-12 21:51:32 +00003# For more information about this module, see PEP 324.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00004#
Peter Astrand3a708df2005-09-23 17:37:29 +00005# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00006#
Peter Astrand69bf13f2005-02-14 08:56:32 +00007# Licensed to PSF under a Contributor Agreement.
Peter Astrand3a708df2005-09-23 17:37:29 +00008# See http://www.python.org/2.4/license for licensing details.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00009
Raymond Hettinger837dd932004-10-17 16:36:53 +000010r"""subprocess - Subprocesses with accessible I/O streams
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000011
Fredrik Lundh15aaacc2004-10-17 14:47:05 +000012This module allows you to spawn processes, connect to their
13input/output/error pipes, and obtain their return codes. This module
14intends to replace several other, older modules and functions, like:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000015
16os.system
17os.spawn*
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000018
19Information about how the subprocess module can be used to replace these
20modules and functions can be found below.
21
22
23
24Using the subprocess module
25===========================
26This module defines one class called Popen:
27
28class Popen(args, bufsize=0, executable=None,
29 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000030 preexec_fn=None, close_fds=True, shell=False,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000031 cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000032 startupinfo=None, creationflags=0,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000033 restore_signals=True, start_new_session=False, pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000034
35
36Arguments are:
37
38args should be a string, or a sequence of program arguments. The
39program to execute is normally the first item in the args sequence or
40string, but can be explicitly set by using the executable argument.
41
Gregory P. Smithf5604852010-12-13 06:45:02 +000042On POSIX, with shell=False (default): In this case, the Popen class
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000043uses os.execvp() to execute the child program. args should normally
44be a sequence. A string will be treated as a sequence with the string
45as the only item (the program to execute).
46
Gregory P. Smithf5604852010-12-13 06:45:02 +000047On POSIX, with shell=True: If args is a string, it specifies the
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000048command string to execute through the shell. If args is a sequence,
49the first item specifies the command string, and any additional items
50will be treated as additional shell arguments.
51
52On Windows: the Popen class uses CreateProcess() to execute the child
53program, which operates on strings. If args is a sequence, it will be
54converted to a string using the list2cmdline method. Please note that
55not all MS Windows applications interpret the command line the same
56way: The list2cmdline is designed for applications using the same
57rules as the MS C runtime.
58
59bufsize, if given, has the same meaning as the corresponding argument
60to the built-in open() function: 0 means unbuffered, 1 means line
61buffered, any other positive value means use a buffer of
62(approximately) that size. A negative bufsize means to use the system
63default, which usually means fully buffered. The default value for
64bufsize is 0 (unbuffered).
65
66stdin, stdout and stderr specify the executed programs' standard
67input, standard output and standard error file handles, respectively.
68Valid values are PIPE, an existing file descriptor (a positive
69integer), an existing file object, and None. PIPE indicates that a
70new pipe to the child should be created. With None, no redirection
71will occur; the child's file handles will be inherited from the
72parent. Additionally, stderr can be STDOUT, which indicates that the
73stderr data from the applications should be captured into the same
74file handle as for stdout.
75
Gregory P. Smithf5604852010-12-13 06:45:02 +000076On POSIX, if preexec_fn is set to a callable object, this object will be
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000077called in the child process just before the child is executed. The use
78of preexec_fn is not thread safe, using it in the presence of threads
79could lead to a deadlock in the child process before the new executable
80is executed.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000081
82If close_fds is true, all file descriptors except 0, 1 and 2 will be
Gregory P. Smithf5604852010-12-13 06:45:02 +000083closed before the child process is executed. The default for close_fds
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000084varies by platform: Always true on POSIX. True when stdin/stdout/stderr
85are None on Windows, false otherwise.
86
87pass_fds is an optional sequence of file descriptors to keep open between the
88parent and child. Providing any pass_fds implicitly sets close_fds to true.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000089
90if shell is true, the specified command will be executed through the
91shell.
92
93If cwd is not None, the current directory will be changed to cwd
94before the child is executed.
95
Gregory P. Smithf5604852010-12-13 06:45:02 +000096On POSIX, if restore_signals is True all signals that Python sets to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000097SIG_IGN are restored to SIG_DFL in the child process before the exec.
98Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99parameter does nothing on Windows.
100
Gregory P. Smithf5604852010-12-13 06:45:02 +0000101On POSIX, if start_new_session is True, the setsid() system call will be made
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000102in the child process prior to executing the command.
103
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000104If env is not None, it defines the environment variables for the new
105process.
106
107If universal_newlines is true, the file objects stdout and stderr are
108opened as a text files, but lines may be terminated by any of '\n',
Gregory P. Smithf5604852010-12-13 06:45:02 +0000109the Unix end-of-line convention, '\r', the old Macintosh convention or
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000110'\r\n', the Windows convention. All of these external representations
111are seen as '\n' by the Python program. Note: This feature is only
112available if Python is built with universal newline support (the
113default). Also, the newlines attribute of the file objects stdout,
114stdin and stderr are not updated by the communicate() method.
115
116The startupinfo and creationflags, if given, will be passed to the
117underlying CreateProcess() function. They can specify things such as
118appearance of the main window and priority for the new process.
119(Windows only)
120
121
Georg Brandlf9734072008-12-07 15:30:06 +0000122This module also defines some shortcut functions:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000123
Peter Astrand5f5e1412004-12-05 20:15:36 +0000124call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000125 Run command with arguments. Wait for command to complete, then
126 return the returncode attribute.
127
128 The arguments are the same as for the Popen constructor. Example:
129
Florent Xicluna4886d242010-03-08 13:27:26 +0000130 >>> retcode = subprocess.call(["ls", "-l"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000131
Peter Astrand454f7672005-01-01 09:36:35 +0000132check_call(*popenargs, **kwargs):
133 Run command with arguments. Wait for command to complete. If the
134 exit code was zero then return, otherwise raise
135 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000136 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000137
138 The arguments are the same as for the Popen constructor. Example:
139
Florent Xicluna4886d242010-03-08 13:27:26 +0000140 >>> subprocess.check_call(["ls", "-l"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000141 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000142
Brett Cannona23810f2008-05-26 19:04:21 +0000143getstatusoutput(cmd):
144 Return (status, output) of executing cmd in a shell.
145
146 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
147 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
148 returned output will contain output or error messages. A trailing newline
149 is stripped from the output. The exit status for the command can be
150 interpreted according to the rules for the C function wait(). Example:
151
Brett Cannona23810f2008-05-26 19:04:21 +0000152 >>> subprocess.getstatusoutput('ls /bin/ls')
153 (0, '/bin/ls')
154 >>> subprocess.getstatusoutput('cat /bin/junk')
155 (256, 'cat: /bin/junk: No such file or directory')
156 >>> subprocess.getstatusoutput('/bin/junk')
157 (256, 'sh: /bin/junk: not found')
158
159getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell.
161
162 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example:
164
Brett Cannona23810f2008-05-26 19:04:21 +0000165 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls'
167
Georg Brandlf9734072008-12-07 15:30:06 +0000168check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000169 Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000170
Georg Brandl2708f3a2009-12-20 14:38:23 +0000171 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000174
Georg Brandl2708f3a2009-12-20 14:38:23 +0000175 The arguments are the same as for the Popen constructor. Example:
Georg Brandlf9734072008-12-07 15:30:06 +0000176
Georg Brandl2708f3a2009-12-20 14:38:23 +0000177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandlf9734072008-12-07 15:30:06 +0000178
Brett Cannona23810f2008-05-26 19:04:21 +0000179
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000180Exceptions
181----------
182Exceptions raised in the child process, before the new program has
183started to execute, will be re-raised in the parent. Additionally,
184the exception object will have one extra attribute called
185'child_traceback', which is a string containing traceback information
186from the childs point of view.
187
188The most common exception raised is OSError. This occurs, for
189example, when trying to execute a non-existent file. Applications
190should prepare for OSErrors.
191
192A ValueError will be raised if Popen is called with invalid arguments.
193
Georg Brandlf9734072008-12-07 15:30:06 +0000194check_call() and check_output() will raise CalledProcessError, if the
195called process returns a non-zero return code.
Peter Astrand454f7672005-01-01 09:36:35 +0000196
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000197
198Security
199--------
200Unlike some other popen functions, this implementation will never call
201/bin/sh implicitly. This means that all characters, including shell
202metacharacters, can safely be passed to child processes.
203
204
205Popen objects
206=============
207Instances of the Popen class have the following methods:
208
209poll()
210 Check if child process has terminated. Returns returncode
211 attribute.
212
213wait()
214 Wait for child process to terminate. Returns returncode attribute.
215
216communicate(input=None)
217 Interact with process: Send data to stdin. Read data from stdout
218 and stderr, until end-of-file is reached. Wait for process to
Thomas Wouters902d6eb2007-01-09 23:18:33 +0000219 terminate. The optional input argument should be a string to be
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000220 sent to the child process, or None, if no data should be sent to
221 the child.
Tim Peterse718f612004-10-12 21:51:32 +0000222
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000223 communicate() returns a tuple (stdout, stderr).
224
225 Note: The data read is buffered in memory, so do not use this
226 method if the data size is large or unlimited.
227
228The following attributes are also available:
229
230stdin
231 If the stdin argument is PIPE, this attribute is a file object
232 that provides input to the child process. Otherwise, it is None.
233
234stdout
235 If the stdout argument is PIPE, this attribute is a file object
236 that provides output from the child process. Otherwise, it is
237 None.
238
239stderr
240 If the stderr argument is PIPE, this attribute is file object that
241 provides error output from the child process. Otherwise, it is
242 None.
243
244pid
245 The process ID of the child process.
246
247returncode
248 The child return code. A None value indicates that the process
249 hasn't terminated yet. A negative value -N indicates that the
Gregory P. Smithf5604852010-12-13 06:45:02 +0000250 child was terminated by signal N (POSIX only).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000251
252
253Replacing older functions with the subprocess module
254====================================================
255In this section, "a ==> b" means that b can be used as a replacement
256for a.
257
258Note: All functions in this section fail (more or less) silently if
259the executed program cannot be found; this module raises an OSError
260exception.
261
262In the following examples, we assume that the subprocess module is
263imported with "from subprocess import *".
264
265
266Replacing /bin/sh shell backquote
267---------------------------------
268output=`mycmd myarg`
269==>
270output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
271
272
273Replacing shell pipe line
274-------------------------
275output=`dmesg | grep hda`
276==>
277p1 = Popen(["dmesg"], stdout=PIPE)
Peter Astrand6fdf3cb2004-11-30 18:06:42 +0000278p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000279output = p2.communicate()[0]
280
281
282Replacing os.system()
283---------------------
284sts = os.system("mycmd" + " myarg")
285==>
286p = Popen("mycmd" + " myarg", shell=True)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000287pid, sts = os.waitpid(p.pid, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000288
289Note:
290
291* Calling the program through the shell is usually not required.
292
293* It's easier to look at the returncode attribute than the
294 exitstatus.
295
296A more real-world example would look like this:
297
298try:
299 retcode = call("mycmd" + " myarg", shell=True)
300 if retcode < 0:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000301 print("Child was terminated by signal", -retcode, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000302 else:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000303 print("Child returned", retcode, file=sys.stderr)
304except OSError as e:
305 print("Execution failed:", e, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000306
307
308Replacing os.spawn*
309-------------------
Tim Peterse718f612004-10-12 21:51:32 +0000310P_NOWAIT example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000311
312pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
313==>
314pid = Popen(["/bin/mycmd", "myarg"]).pid
315
316
317P_WAIT example:
318
319retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
320==>
321retcode = call(["/bin/mycmd", "myarg"])
322
323
Tim Peterse718f612004-10-12 21:51:32 +0000324Vector example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000325
326os.spawnvp(os.P_NOWAIT, path, args)
327==>
328Popen([path] + args[1:])
329
330
Tim Peterse718f612004-10-12 21:51:32 +0000331Environment example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000332
333os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
334==>
335Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000336"""
337
338import sys
339mswindows = (sys.platform == "win32")
340
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000341import io
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000342import os
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000343import traceback
Christian Heimesfdab48e2008-01-20 09:06:41 +0000344import gc
Christian Heimesa342c012008-04-20 21:01:16 +0000345import signal
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000346import builtins
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000347import warnings
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200348import errno
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000349
Peter Astrand454f7672005-01-01 09:36:35 +0000350# Exception classes used by this module.
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000351class CalledProcessError(Exception):
Georg Brandlf9734072008-12-07 15:30:06 +0000352 """This exception is raised when a process run by check_call() or
353 check_output() returns a non-zero exit status.
354 The exit status will be stored in the returncode attribute;
355 check_output() will also store the output in the output attribute.
356 """
357 def __init__(self, returncode, cmd, output=None):
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000358 self.returncode = returncode
359 self.cmd = cmd
Georg Brandlf9734072008-12-07 15:30:06 +0000360 self.output = output
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000361 def __str__(self):
362 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
363
Peter Astrand454f7672005-01-01 09:36:35 +0000364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000365if mswindows:
366 import threading
367 import msvcrt
Brian Curtin1ce6b582010-04-24 16:19:22 +0000368 import _subprocess
369 class STARTUPINFO:
370 dwFlags = 0
371 hStdInput = None
372 hStdOutput = None
373 hStdError = None
374 wShowWindow = 0
375 class pywintypes:
376 error = IOError
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000377else:
378 import select
Gregory P. Smithd06fa472009-07-04 02:46:54 +0000379 _has_poll = hasattr(select, 'poll')
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000380 import fcntl
381 import pickle
382
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000383 try:
384 import _posixsubprocess
385 except ImportError:
386 _posixsubprocess = None
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000387 warnings.warn("The _posixsubprocess module is not being used. "
388 "Child process reliability may suffer if your "
389 "program uses threads.", RuntimeWarning)
390
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000391 # When select or poll has indicated that the file is writable,
392 # we can write up to _PIPE_BUF bytes without risk of blocking.
393 # POSIX defines PIPE_BUF as >= 512.
394 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
395
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000396 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
397
398 def _set_cloexec(fd, cloexec):
399 old = fcntl.fcntl(fd, fcntl.F_GETFD)
400 if cloexec:
401 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
402 else:
403 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
404
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000405 if _posixsubprocess:
406 _create_pipe = _posixsubprocess.cloexec_pipe
407 else:
408 def _create_pipe():
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000409 fds = os.pipe()
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000410 _set_cloexec(fds[0], True)
411 _set_cloexec(fds[1], True)
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000412 return fds
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000413
Brett Cannona23810f2008-05-26 19:04:21 +0000414__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
Georg Brandlf9734072008-12-07 15:30:06 +0000415 "getoutput", "check_output", "CalledProcessError"]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000416
Brian Curtin1ce6b582010-04-24 16:19:22 +0000417if mswindows:
Brian Curtin08fd8d92011-04-29 16:11:30 -0500418 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500419 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
420 STD_ERROR_HANDLE, SW_HIDE,
421 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
Gregory P. Smith0ef3e392011-05-11 22:20:11 -0700422
Brian Curtin08fd8d92011-04-29 16:11:30 -0500423 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500424 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
425 "STD_ERROR_HANDLE", "SW_HIDE",
426 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000427try:
428 MAXFD = os.sysconf("SC_OPEN_MAX")
429except:
430 MAXFD = 256
431
Charles-François Natali134a8ba2011-08-18 18:49:39 +0200432# This lists holds Popen instances for which the underlying process had not
433# exited at the time its __del__ method got called: those processes are wait()ed
434# for synchronously from _cleanup() when a new Popen object is created, to avoid
435# zombie processes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000436_active = []
437
438def _cleanup():
439 for inst in _active[:]:
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000440 res = inst._internal_poll(_deadstate=sys.maxsize)
Charles-François Natali134a8ba2011-08-18 18:49:39 +0200441 if res is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000442 try:
443 _active.remove(inst)
444 except ValueError:
445 # This can happen if two threads create a new Popen instance.
446 # It's harmless that it was already removed, so ignore.
447 pass
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000448
449PIPE = -1
450STDOUT = -2
451
452
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000453def _eintr_retry_call(func, *args):
454 while True:
455 try:
456 return func(*args)
Victor Stinner2cfb6f32011-07-05 14:00:56 +0200457 except (OSError, IOError) as e:
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000458 if e.errno == errno.EINTR:
459 continue
460 raise
461
462
Peter Astrand5f5e1412004-12-05 20:15:36 +0000463def call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000464 """Run command with arguments. Wait for command to complete, then
465 return the returncode attribute.
466
467 The arguments are the same as for the Popen constructor. Example:
468
469 retcode = call(["ls", "-l"])
470 """
Peter Astrand5f5e1412004-12-05 20:15:36 +0000471 return Popen(*popenargs, **kwargs).wait()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000472
473
Peter Astrand454f7672005-01-01 09:36:35 +0000474def check_call(*popenargs, **kwargs):
475 """Run command with arguments. Wait for command to complete. If
476 the exit code was zero then return, otherwise raise
477 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000478 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000479
480 The arguments are the same as for the Popen constructor. Example:
481
482 check_call(["ls", "-l"])
483 """
484 retcode = call(*popenargs, **kwargs)
Peter Astrand454f7672005-01-01 09:36:35 +0000485 if retcode:
Georg Brandlf9734072008-12-07 15:30:06 +0000486 cmd = kwargs.get("args")
487 if cmd is None:
488 cmd = popenargs[0]
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000489 raise CalledProcessError(retcode, cmd)
Georg Brandlf9734072008-12-07 15:30:06 +0000490 return 0
491
492
493def check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000494 r"""Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000495
496 If the exit code was non-zero it raises a CalledProcessError. The
497 CalledProcessError object will have the return code in the returncode
498 attribute and output in the output attribute.
499
500 The arguments are the same as for the Popen constructor. Example:
501
502 >>> check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000503 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000504
505 The stdout argument is not allowed as it is used internally.
Georg Brandl127d4702009-12-28 08:10:38 +0000506 To capture standard error in the result, use stderr=STDOUT.
Georg Brandlf9734072008-12-07 15:30:06 +0000507
508 >>> check_output(["/bin/sh", "-c",
Georg Brandl2708f3a2009-12-20 14:38:23 +0000509 ... "ls -l non_existent_file ; exit 0"],
Georg Brandl127d4702009-12-28 08:10:38 +0000510 ... stderr=STDOUT)
Georg Brandl2708f3a2009-12-20 14:38:23 +0000511 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000512 """
513 if 'stdout' in kwargs:
514 raise ValueError('stdout argument not allowed, it will be overridden.')
515 process = Popen(*popenargs, stdout=PIPE, **kwargs)
516 output, unused_err = process.communicate()
517 retcode = process.poll()
518 if retcode:
519 cmd = kwargs.get("args")
520 if cmd is None:
521 cmd = popenargs[0]
522 raise CalledProcessError(retcode, cmd, output=output)
523 return output
Peter Astrand454f7672005-01-01 09:36:35 +0000524
525
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000526def list2cmdline(seq):
527 """
528 Translate a sequence of arguments into a command line
529 string, using the same rules as the MS C runtime:
530
531 1) Arguments are delimited by white space, which is either a
532 space or a tab.
533
534 2) A string surrounded by double quotation marks is
535 interpreted as a single argument, regardless of white space
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000536 contained within. A quoted string can be embedded in an
537 argument.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000538
539 3) A double quotation mark preceded by a backslash is
540 interpreted as a literal double quotation mark.
541
542 4) Backslashes are interpreted literally, unless they
543 immediately precede a double quotation mark.
544
545 5) If backslashes immediately precede a double quotation mark,
546 every pair of backslashes is interpreted as a literal
547 backslash. If the number of backslashes is odd, the last
548 backslash escapes the next double quotation mark as
549 described in rule 3.
550 """
551
552 # See
Eric Smith3c573af2009-11-09 15:23:15 +0000553 # http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
554 # or search http://msdn.microsoft.com for
555 # "Parsing C++ Command-Line Arguments"
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000556 result = []
557 needquote = False
558 for arg in seq:
559 bs_buf = []
560
561 # Add a space to separate this argument from the others
562 if result:
563 result.append(' ')
564
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000565 needquote = (" " in arg) or ("\t" in arg) or not arg
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000566 if needquote:
567 result.append('"')
568
569 for c in arg:
570 if c == '\\':
Tim Peterse718f612004-10-12 21:51:32 +0000571 # Don't know if we need to double yet.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000572 bs_buf.append(c)
573 elif c == '"':
Christian Heimesfdab48e2008-01-20 09:06:41 +0000574 # Double backslashes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000575 result.append('\\' * len(bs_buf)*2)
576 bs_buf = []
577 result.append('\\"')
578 else:
579 # Normal char
580 if bs_buf:
581 result.extend(bs_buf)
582 bs_buf = []
583 result.append(c)
584
Christian Heimesfdab48e2008-01-20 09:06:41 +0000585 # Add remaining backslashes, if any.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000586 if bs_buf:
587 result.extend(bs_buf)
588
589 if needquote:
Peter Astrand7e78ade2005-03-03 21:10:23 +0000590 result.extend(bs_buf)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000591 result.append('"')
592
593 return ''.join(result)
594
595
Brett Cannona23810f2008-05-26 19:04:21 +0000596# Various tools for executing commands and looking at their output and status.
597#
Gregory P. Smithf5604852010-12-13 06:45:02 +0000598# NB This only works (and is only relevant) for POSIX.
Brett Cannona23810f2008-05-26 19:04:21 +0000599
600def getstatusoutput(cmd):
601 """Return (status, output) of executing cmd in a shell.
602
603 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
604 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
605 returned output will contain output or error messages. A trailing newline
606 is stripped from the output. The exit status for the command can be
607 interpreted according to the rules for the C function wait(). Example:
608
609 >>> import subprocess
610 >>> subprocess.getstatusoutput('ls /bin/ls')
611 (0, '/bin/ls')
612 >>> subprocess.getstatusoutput('cat /bin/junk')
613 (256, 'cat: /bin/junk: No such file or directory')
614 >>> subprocess.getstatusoutput('/bin/junk')
615 (256, 'sh: /bin/junk: not found')
616 """
617 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
618 text = pipe.read()
619 sts = pipe.close()
620 if sts is None: sts = 0
621 if text[-1:] == '\n': text = text[:-1]
622 return sts, text
623
624
625def getoutput(cmd):
626 """Return output (stdout or stderr) of executing cmd in a shell.
627
628 Like getstatusoutput(), except the exit status is ignored and the return
629 value is a string containing the command's output. Example:
630
631 >>> import subprocess
632 >>> subprocess.getoutput('ls /bin/ls')
633 '/bin/ls'
634 """
635 return getstatusoutput(cmd)[1]
636
637
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000638_PLATFORM_DEFAULT_CLOSE_FDS = object()
Gregory P. Smithf5604852010-12-13 06:45:02 +0000639
640
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000641class Popen(object):
642 def __init__(self, args, bufsize=0, executable=None,
643 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000644 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
645 shell=False, cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000646 startupinfo=None, creationflags=0,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000647 restore_signals=True, start_new_session=False,
648 pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000649 """Create new Popen instance."""
650 _cleanup()
651
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000652 self._child_created = False
Guido van Rossum46a05a72007-06-07 21:56:45 +0000653 if bufsize is None:
654 bufsize = 0 # Restore default
Walter Dörwaldaa97f042007-05-03 21:05:51 +0000655 if not isinstance(bufsize, int):
Peter Astrand738131d2004-11-30 21:04:45 +0000656 raise TypeError("bufsize must be an integer")
657
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000658 if mswindows:
Tim Peterse8374a52004-10-13 03:15:00 +0000659 if preexec_fn is not None:
660 raise ValueError("preexec_fn is not supported on Windows "
661 "platforms")
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000662 any_stdio_set = (stdin is not None or stdout is not None or
663 stderr is not None)
664 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
665 if any_stdio_set:
666 close_fds = False
667 else:
668 close_fds = True
669 elif close_fds and any_stdio_set:
670 raise ValueError(
671 "close_fds is not supported on Windows platforms"
672 " if you redirect stdin/stdout/stderr")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000673 else:
674 # POSIX
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000675 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
676 close_fds = True
677 if pass_fds and not close_fds:
678 warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
679 close_fds = True
Tim Peterse8374a52004-10-13 03:15:00 +0000680 if startupinfo is not None:
681 raise ValueError("startupinfo is only supported on Windows "
682 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000683 if creationflags != 0:
Tim Peterse8374a52004-10-13 03:15:00 +0000684 raise ValueError("creationflags is only supported on Windows "
685 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000686
Tim Peterse718f612004-10-12 21:51:32 +0000687 self.stdin = None
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000688 self.stdout = None
689 self.stderr = None
690 self.pid = None
691 self.returncode = None
692 self.universal_newlines = universal_newlines
693
694 # Input and output objects. The general principle is like
695 # this:
696 #
697 # Parent Child
698 # ------ -----
699 # p2cwrite ---stdin---> p2cread
700 # c2pread <--stdout--- c2pwrite
701 # errread <--stderr--- errwrite
702 #
703 # On POSIX, the child objects are file descriptors. On
704 # Windows, these are Windows file handles. The parent objects
705 # are file descriptors on both platforms. The parent objects
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000706 # are -1 when not using PIPEs. The child objects are -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000707 # when not redirecting.
Tim Peterse718f612004-10-12 21:51:32 +0000708
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000709 (p2cread, p2cwrite,
710 c2pread, c2pwrite,
711 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
712
Antoine Pitrouc9982322011-01-04 19:07:07 +0000713 # We wrap OS handles *before* launching the child, otherwise a
714 # quickly terminating child could make our fds unwrappable
715 # (see #8458).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000716
Thomas Wouterscf297e42007-02-23 15:07:44 +0000717 if mswindows:
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000718 if p2cwrite != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000719 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000720 if c2pread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000721 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000722 if errread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000723 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
Thomas Wouterscf297e42007-02-23 15:07:44 +0000724
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000725 if p2cwrite != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000726 self.stdin = io.open(p2cwrite, 'wb', bufsize)
727 if self.universal_newlines:
Antoine Pitrouab85ff32011-07-23 22:03:45 +0200728 self.stdin = io.TextIOWrapper(self.stdin, write_through=True)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000729 if c2pread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000730 self.stdout = io.open(c2pread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000731 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000732 self.stdout = io.TextIOWrapper(self.stdout)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000733 if errread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000734 self.stderr = io.open(errread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000735 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000736 self.stderr = io.TextIOWrapper(self.stderr)
Tim Peterse718f612004-10-12 21:51:32 +0000737
Antoine Pitrouc9982322011-01-04 19:07:07 +0000738 try:
739 self._execute_child(args, executable, preexec_fn, close_fds,
740 pass_fds, cwd, env, universal_newlines,
741 startupinfo, creationflags, shell,
742 p2cread, p2cwrite,
743 c2pread, c2pwrite,
744 errread, errwrite,
745 restore_signals, start_new_session)
746 except:
747 # Cleanup if the child failed starting
748 for f in filter(None, [self.stdin, self.stdout, self.stderr]):
749 try:
750 f.close()
751 except EnvironmentError:
752 # Ignore EBADF or other errors
753 pass
754 raise
755
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000756
Guido van Rossum98297ee2007-11-06 21:34:58 +0000757 def _translate_newlines(self, data, encoding):
758 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")
759 return data.decode(encoding)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000760
Brian Curtin79cdb662010-12-03 02:46:02 +0000761 def __enter__(self):
762 return self
763
764 def __exit__(self, type, value, traceback):
765 if self.stdout:
766 self.stdout.close()
767 if self.stderr:
768 self.stderr.close()
769 if self.stdin:
770 self.stdin.close()
Gregory P. Smithc9557af2011-05-11 22:18:23 -0700771 # Wait for the process to terminate, to avoid zombies.
772 self.wait()
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000773
Brett Cannon84df1e62010-05-14 00:33:40 +0000774 def __del__(self, _maxsize=sys.maxsize, _active=_active):
Victor Stinner87b9bc32011-06-01 00:57:47 +0200775 # If __init__ hasn't had a chance to execute (e.g. if it
776 # was passed an undeclared keyword argument), we don't
777 # have a _child_created attribute at all.
778 if not getattr(self, '_child_created', False):
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000779 # We didn't get to successfully create a child process.
780 return
781 # In case the child hasn't been waited on, check if it's done.
Brett Cannon84df1e62010-05-14 00:33:40 +0000782 self._internal_poll(_deadstate=_maxsize)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000783 if self.returncode is None and _active is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000784 # Child is still running, keep us alive until we can wait on it.
785 _active.append(self)
786
787
Peter Astrand23109f02005-03-03 20:28:59 +0000788 def communicate(self, input=None):
789 """Interact with process: Send data to stdin. Read data from
790 stdout and stderr, until end-of-file is reached. Wait for
791 process to terminate. The optional input argument should be a
792 string to be sent to the child process, or None, if no data
793 should be sent to the child.
Tim Peterseba28be2005-03-28 01:08:02 +0000794
Peter Astrand23109f02005-03-03 20:28:59 +0000795 communicate() returns a tuple (stdout, stderr)."""
796
797 # Optimization: If we are only using one pipe, or no pipe at
798 # all, using select() or threads is unnecessary.
799 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
Tim Peterseba28be2005-03-28 01:08:02 +0000800 stdout = None
801 stderr = None
Peter Astrand23109f02005-03-03 20:28:59 +0000802 if self.stdin:
803 if input:
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200804 try:
805 self.stdin.write(input)
806 except IOError as e:
807 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
808 raise
Peter Astrand23109f02005-03-03 20:28:59 +0000809 self.stdin.close()
810 elif self.stdout:
Victor Stinner2cfb6f32011-07-05 14:00:56 +0200811 stdout = _eintr_retry_call(self.stdout.read)
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000812 self.stdout.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000813 elif self.stderr:
Victor Stinner2cfb6f32011-07-05 14:00:56 +0200814 stderr = _eintr_retry_call(self.stderr.read)
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000815 self.stderr.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000816 self.wait()
817 return (stdout, stderr)
Tim Peterseba28be2005-03-28 01:08:02 +0000818
Peter Astrand23109f02005-03-03 20:28:59 +0000819 return self._communicate(input)
820
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000821
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000822 def poll(self):
823 return self._internal_poll()
824
825
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000826 if mswindows:
827 #
828 # Windows methods
829 #
830 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +0000831 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000832 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
833 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000834 if stdin is None and stdout is None and stderr is None:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000835 return (-1, -1, -1, -1, -1, -1)
Tim Peterse718f612004-10-12 21:51:32 +0000836
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000837 p2cread, p2cwrite = -1, -1
838 c2pread, c2pwrite = -1, -1
839 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000840
Peter Astrandd38ddf42005-02-10 08:32:50 +0000841 if stdin is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000842 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000843 if p2cread is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000844 p2cread, _ = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000845 elif stdin == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000846 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000847 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000848 p2cread = msvcrt.get_osfhandle(stdin)
849 else:
850 # Assuming file-like object
851 p2cread = msvcrt.get_osfhandle(stdin.fileno())
852 p2cread = self._make_inheritable(p2cread)
853
Peter Astrandd38ddf42005-02-10 08:32:50 +0000854 if stdout is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000855 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000856 if c2pwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000857 _, c2pwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000858 elif stdout == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000859 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000860 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000861 c2pwrite = msvcrt.get_osfhandle(stdout)
862 else:
863 # Assuming file-like object
864 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
865 c2pwrite = self._make_inheritable(c2pwrite)
866
Peter Astrandd38ddf42005-02-10 08:32:50 +0000867 if stderr is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000868 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000869 if errwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000870 _, errwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000871 elif stderr == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000872 errread, errwrite = _subprocess.CreatePipe(None, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000873 elif stderr == STDOUT:
874 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +0000875 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000876 errwrite = msvcrt.get_osfhandle(stderr)
877 else:
878 # Assuming file-like object
879 errwrite = msvcrt.get_osfhandle(stderr.fileno())
880 errwrite = self._make_inheritable(errwrite)
881
882 return (p2cread, p2cwrite,
883 c2pread, c2pwrite,
884 errread, errwrite)
885
886
887 def _make_inheritable(self, handle):
888 """Return a duplicate of handle, which is inheritable"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000889 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
890 handle, _subprocess.GetCurrentProcess(), 0, 1,
891 _subprocess.DUPLICATE_SAME_ACCESS)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000892
893
894 def _find_w9xpopen(self):
895 """Find and return absolut path to w9xpopen.exe"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000896 w9xpopen = os.path.join(
897 os.path.dirname(_subprocess.GetModuleFileName(0)),
Tim Peterse8374a52004-10-13 03:15:00 +0000898 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000899 if not os.path.exists(w9xpopen):
900 # Eeek - file-not-found - possibly an embedding
901 # situation - see if we can locate it in sys.exec_prefix
Tim Peterse8374a52004-10-13 03:15:00 +0000902 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
903 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000904 if not os.path.exists(w9xpopen):
Tim Peterse8374a52004-10-13 03:15:00 +0000905 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
906 "needed for Popen to work with your "
907 "shell or platform.")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000908 return w9xpopen
909
Tim Peterse718f612004-10-12 21:51:32 +0000910
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000911 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000912 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000913 startupinfo, creationflags, shell,
914 p2cread, p2cwrite,
915 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000916 errread, errwrite,
917 unused_restore_signals, unused_start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000918 """Execute program (MS Windows version)"""
919
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000920 assert not pass_fds, "pass_fds not supported on Windows."
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000921
Guido van Rossum3172c5d2007-10-16 18:12:55 +0000922 if not isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000923 args = list2cmdline(args)
924
Peter Astrandc1d65362004-11-07 14:30:34 +0000925 # Process startup details
Peter Astrandd38ddf42005-02-10 08:32:50 +0000926 if startupinfo is None:
Thomas Wouters73e5a5b2006-06-08 15:35:45 +0000927 startupinfo = STARTUPINFO()
Victor Stinnerb3693582010-05-21 20:13:12 +0000928 if -1 not in (p2cread, c2pwrite, errwrite):
Brian Curtin1ce6b582010-04-24 16:19:22 +0000929 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
Peter Astrandc1d65362004-11-07 14:30:34 +0000930 startupinfo.hStdInput = p2cread
931 startupinfo.hStdOutput = c2pwrite
932 startupinfo.hStdError = errwrite
933
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000934 if shell:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000935 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
936 startupinfo.wShowWindow = _subprocess.SW_HIDE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000937 comspec = os.environ.get("COMSPEC", "cmd.exe")
Tim Golden126c2962010-08-11 14:20:40 +0000938 args = '{} /c "{}"'.format (comspec, args)
Brian Curtin1ce6b582010-04-24 16:19:22 +0000939 if (_subprocess.GetVersion() >= 0x80000000 or
Tim Peterse8374a52004-10-13 03:15:00 +0000940 os.path.basename(comspec).lower() == "command.com"):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000941 # Win9x, or using command.com on NT. We need to
942 # use the w9xpopen intermediate program. For more
943 # information, see KB Q150956
944 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
945 w9xpopen = self._find_w9xpopen()
946 args = '"%s" %s' % (w9xpopen, args)
947 # Not passing CREATE_NEW_CONSOLE has been known to
948 # cause random failures on win9x. Specifically a
949 # dialog: "Your program accessed mem currently in
950 # use at xxx" and a hopeful warning about the
Mark Dickinson934896d2009-02-21 20:59:32 +0000951 # stability of your system. Cost is Ctrl+C won't
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000952 # kill children.
Brian Curtin1ce6b582010-04-24 16:19:22 +0000953 creationflags |= _subprocess.CREATE_NEW_CONSOLE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000954
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000955 # Start the process
956 try:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000957 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
Tim Peterse8374a52004-10-13 03:15:00 +0000958 # no special security
959 None, None,
Guido van Rossume7ba4952007-06-06 23:52:48 +0000960 int(not close_fds),
Tim Peterse8374a52004-10-13 03:15:00 +0000961 creationflags,
962 env,
963 cwd,
964 startupinfo)
Guido van Rossumb940e112007-01-10 16:19:56 +0000965 except pywintypes.error as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000966 # Translate pywintypes.error to WindowsError, which is
967 # a subclass of OSError. FIXME: We should really
Ezio Melotti13925002011-03-16 11:05:33 +0200968 # translate errno using _sys_errlist (or similar), but
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000969 # how can this be done from Python?
970 raise WindowsError(*e.args)
Tim Goldenad537f22010-08-08 11:18:16 +0000971 finally:
972 # Child is launched. Close the parent's copy of those pipe
973 # handles that only the child should have open. You need
974 # to make sure that no handles to the write end of the
975 # output pipe are maintained in this process or else the
976 # pipe will not close when the child process exits and the
977 # ReadFile will hang.
978 if p2cread != -1:
979 p2cread.Close()
980 if c2pwrite != -1:
981 c2pwrite.Close()
982 if errwrite != -1:
983 errwrite.Close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000984
985 # Retain the process handle, but close the thread handle
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000986 self._child_created = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000987 self._handle = hp
988 self.pid = pid
989 ht.Close()
990
Brett Cannon84df1e62010-05-14 00:33:40 +0000991 def _internal_poll(self, _deadstate=None,
Victor Stinnerc807a612010-05-14 21:53:45 +0000992 _WaitForSingleObject=_subprocess.WaitForSingleObject,
993 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
994 _GetExitCodeProcess=_subprocess.GetExitCodeProcess):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000995 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +0000996 attribute.
997
998 This method is called by __del__, so it can only refer to objects
999 in its local scope.
1000
1001 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001002 if self.returncode is None:
Brett Cannon84df1e62010-05-14 00:33:40 +00001003 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
1004 self.returncode = _GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001005 return self.returncode
1006
1007
1008 def wait(self):
1009 """Wait for child process to terminate. Returns returncode
1010 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001011 if self.returncode is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +00001012 _subprocess.WaitForSingleObject(self._handle,
1013 _subprocess.INFINITE)
1014 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001015 return self.returncode
1016
1017
1018 def _readerthread(self, fh, buffer):
1019 buffer.append(fh.read())
Victor Stinner667d4b52010-12-25 22:40:32 +00001020 fh.close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001021
1022
Peter Astrand23109f02005-03-03 20:28:59 +00001023 def _communicate(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001024 stdout = None # Return
1025 stderr = None # Return
1026
1027 if self.stdout:
1028 stdout = []
Tim Peterse8374a52004-10-13 03:15:00 +00001029 stdout_thread = threading.Thread(target=self._readerthread,
1030 args=(self.stdout, stdout))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001031 stdout_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001032 stdout_thread.start()
1033 if self.stderr:
1034 stderr = []
Tim Peterse8374a52004-10-13 03:15:00 +00001035 stderr_thread = threading.Thread(target=self._readerthread,
1036 args=(self.stderr, stderr))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001037 stderr_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001038 stderr_thread.start()
1039
1040 if self.stdin:
Peter Astrandd38ddf42005-02-10 08:32:50 +00001041 if input is not None:
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001042 try:
1043 self.stdin.write(input)
1044 except IOError as e:
1045 if e.errno != errno.EPIPE:
1046 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001047 self.stdin.close()
1048
1049 if self.stdout:
1050 stdout_thread.join()
1051 if self.stderr:
1052 stderr_thread.join()
1053
1054 # All data exchanged. Translate lists into strings.
Peter Astrandd38ddf42005-02-10 08:32:50 +00001055 if stdout is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001056 stdout = stdout[0]
Peter Astrandd38ddf42005-02-10 08:32:50 +00001057 if stderr is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001058 stderr = stderr[0]
1059
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001060 self.wait()
1061 return (stdout, stderr)
1062
Christian Heimesa342c012008-04-20 21:01:16 +00001063 def send_signal(self, sig):
1064 """Send a signal to the process
1065 """
1066 if sig == signal.SIGTERM:
1067 self.terminate()
Brian Curtineb24d742010-04-12 17:16:38 +00001068 elif sig == signal.CTRL_C_EVENT:
1069 os.kill(self.pid, signal.CTRL_C_EVENT)
1070 elif sig == signal.CTRL_BREAK_EVENT:
1071 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
Christian Heimesa342c012008-04-20 21:01:16 +00001072 else:
Brian Curtin19651362010-09-07 13:24:38 +00001073 raise ValueError("Unsupported signal: {}".format(sig))
Christian Heimesa342c012008-04-20 21:01:16 +00001074
1075 def terminate(self):
1076 """Terminates the process
1077 """
Antoine Pitrou1f9a8352012-03-11 19:29:12 +01001078 try:
1079 _subprocess.TerminateProcess(self._handle, 1)
1080 except OSError as e:
1081 # ERROR_ACCESS_DENIED (winerror 5) is received when the
1082 # process already died.
1083 if e.winerror != 5:
1084 raise
1085 rc = _subprocess.GetExitCodeProcess(self._handle)
1086 if rc == _subprocess.STILL_ACTIVE:
1087 raise
1088 self.returncode = rc
Christian Heimesa342c012008-04-20 21:01:16 +00001089
1090 kill = terminate
1091
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001092 else:
1093 #
1094 # POSIX methods
1095 #
1096 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +00001097 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001098 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1099 """
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001100 p2cread, p2cwrite = -1, -1
1101 c2pread, c2pwrite = -1, -1
1102 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001103
Peter Astrandd38ddf42005-02-10 08:32:50 +00001104 if stdin is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001105 pass
1106 elif stdin == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001107 p2cread, p2cwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001108 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001109 p2cread = stdin
1110 else:
1111 # Assuming file-like object
1112 p2cread = stdin.fileno()
1113
Peter Astrandd38ddf42005-02-10 08:32:50 +00001114 if stdout is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001115 pass
1116 elif stdout == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001117 c2pread, c2pwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001118 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001119 c2pwrite = stdout
1120 else:
1121 # Assuming file-like object
Tim Peterse718f612004-10-12 21:51:32 +00001122 c2pwrite = stdout.fileno()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001123
Peter Astrandd38ddf42005-02-10 08:32:50 +00001124 if stderr is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001125 pass
1126 elif stderr == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001127 errread, errwrite = _create_pipe()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001128 elif stderr == STDOUT:
1129 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +00001130 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001131 errwrite = stderr
1132 else:
1133 # Assuming file-like object
1134 errwrite = stderr.fileno()
1135
1136 return (p2cread, p2cwrite,
1137 c2pread, c2pwrite,
1138 errread, errwrite)
1139
1140
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001141 def _close_fds(self, fds_to_keep):
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001142 start_fd = 3
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001143 for fd in sorted(fds_to_keep):
Gregory P. Smith8edd99d2010-12-14 13:43:30 +00001144 if fd >= start_fd:
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001145 os.closerange(start_fd, fd)
1146 start_fd = fd + 1
1147 if start_fd <= MAXFD:
1148 os.closerange(start_fd, MAXFD)
1149
1150
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001151 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001152 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001153 startupinfo, creationflags, shell,
1154 p2cread, p2cwrite,
1155 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001156 errread, errwrite,
1157 restore_signals, start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001158 """Execute program (POSIX version)"""
1159
Guido van Rossum3172c5d2007-10-16 18:12:55 +00001160 if isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001161 args = [args]
Thomas Wouters89f507f2006-12-13 04:49:30 +00001162 else:
1163 args = list(args)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001164
1165 if shell:
1166 args = ["/bin/sh", "-c"] + args
Stefan Krah9542cc62010-07-19 14:20:53 +00001167 if executable:
1168 args[0] = executable
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001169
Peter Astrandd38ddf42005-02-10 08:32:50 +00001170 if executable is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001171 executable = args[0]
1172
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001173 # For transferring possible exec failure from child to parent.
1174 # Data format: "exception name:hex errno:description"
1175 # Pickle is not used; it is complex and involves memory allocation.
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001176 errpipe_read, errpipe_write = _create_pipe()
Christian Heimesfdab48e2008-01-20 09:06:41 +00001177 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001178 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001179
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001180 if _posixsubprocess:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001181 # We must avoid complex work that could involve
1182 # malloc or free in the child process to avoid
1183 # potential deadlocks, thus we do all this here.
1184 # and pass it to fork_exec()
1185
Victor Stinnerf1512a22011-06-21 17:18:38 +02001186 if env is not None:
Victor Stinner449c4662010-05-08 11:10:09 +00001187 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001188 for k, v in env.items()]
1189 else:
1190 env_list = None # Use execv instead of execve.
Victor Stinnerb745a742010-05-18 17:17:23 +00001191 executable = os.fsencode(executable)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001192 if os.path.dirname(executable):
Victor Stinnerb745a742010-05-18 17:17:23 +00001193 executable_list = (executable,)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001194 else:
1195 # This matches the behavior of os._execvpe().
Victor Stinnerb745a742010-05-18 17:17:23 +00001196 executable_list = tuple(
1197 os.path.join(os.fsencode(dir), executable)
1198 for dir in os.get_exec_path(env))
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001199 fds_to_keep = set(pass_fds)
1200 fds_to_keep.add(errpipe_write)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001201 self.pid = _posixsubprocess.fork_exec(
1202 args, executable_list,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001203 close_fds, sorted(fds_to_keep), cwd, env_list,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001204 p2cread, p2cwrite, c2pread, c2pwrite,
1205 errread, errwrite,
1206 errpipe_read, errpipe_write,
1207 restore_signals, start_new_session, preexec_fn)
Charles-François Natali134a8ba2011-08-18 18:49:39 +02001208 self._child_created = True
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001209 else:
1210 # Pure Python implementation: It is not thread safe.
1211 # This implementation may deadlock in the child if your
1212 # parent process has any other threads running.
1213
1214 gc_was_enabled = gc.isenabled()
1215 # Disable gc to avoid bug where gc -> file_dealloc ->
1216 # write to stderr -> hang. See issue1336
1217 gc.disable()
1218 try:
1219 self.pid = os.fork()
1220 except:
1221 if gc_was_enabled:
1222 gc.enable()
1223 raise
1224 self._child_created = True
1225 if self.pid == 0:
1226 # Child
1227 try:
1228 # Close parent's pipe ends
1229 if p2cwrite != -1:
1230 os.close(p2cwrite)
1231 if c2pread != -1:
1232 os.close(c2pread)
1233 if errread != -1:
1234 os.close(errread)
1235 os.close(errpipe_read)
1236
Ross Lagerwalld98646e2011-07-27 07:16:31 +02001237 # When duping fds, if there arises a situation
1238 # where one of the fds is either 0, 1 or 2, it
1239 # is possible that it is overwritten (#12607).
1240 if c2pwrite == 0:
1241 c2pwrite = os.dup(c2pwrite)
1242 if errwrite == 0 or errwrite == 1:
1243 errwrite = os.dup(errwrite)
1244
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001245 # Dup fds for child
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001246 def _dup2(a, b):
1247 # dup2() removes the CLOEXEC flag but
1248 # we must do it ourselves if dup2()
1249 # would be a no-op (issue #10806).
1250 if a == b:
1251 _set_cloexec(a, False)
1252 elif a != -1:
1253 os.dup2(a, b)
1254 _dup2(p2cread, 0)
1255 _dup2(c2pwrite, 1)
1256 _dup2(errwrite, 2)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001257
1258 # Close pipe fds. Make sure we don't close the
1259 # same fd more than once, or standard fds.
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001260 closed = set()
1261 for fd in [p2cread, c2pwrite, errwrite]:
1262 if fd > 2 and fd not in closed:
1263 os.close(fd)
1264 closed.add(fd)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001265
1266 # Close all other fds, if asked for
1267 if close_fds:
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001268 fds_to_keep = set(pass_fds)
1269 fds_to_keep.add(errpipe_write)
1270 self._close_fds(fds_to_keep)
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001271
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001272
1273 if cwd is not None:
1274 os.chdir(cwd)
1275
1276 # This is a copy of Python/pythonrun.c
1277 # _Py_RestoreSignals(). If that were exposed
1278 # as a sys._py_restoresignals func it would be
1279 # better.. but this pure python implementation
1280 # isn't likely to be used much anymore.
1281 if restore_signals:
1282 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1283 for sig in signals:
1284 if hasattr(signal, sig):
1285 signal.signal(getattr(signal, sig),
1286 signal.SIG_DFL)
1287
1288 if start_new_session and hasattr(os, 'setsid'):
1289 os.setsid()
1290
1291 if preexec_fn:
1292 preexec_fn()
1293
1294 if env is None:
1295 os.execvp(executable, args)
1296 else:
1297 os.execvpe(executable, args, env)
1298
1299 except:
1300 try:
1301 exc_type, exc_value = sys.exc_info()[:2]
1302 if isinstance(exc_value, OSError):
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001303 errno_num = exc_value.errno
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001304 else:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001305 errno_num = 0
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001306 message = '%s:%x:%s' % (exc_type.__name__,
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001307 errno_num, exc_value)
Victor Stinner4d078042010-04-23 19:28:32 +00001308 message = message.encode(errors="surrogatepass")
1309 os.write(errpipe_write, message)
1310 except Exception:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001311 # We MUST not allow anything odd happening
1312 # above to prevent us from exiting below.
1313 pass
1314
1315 # This exitcode won't be reported to applications
1316 # so it really doesn't matter what we return.
1317 os._exit(255)
1318
1319 # Parent
Facundo Batista10706e22009-06-19 20:34:30 +00001320 if gc_was_enabled:
1321 gc.enable()
Facundo Batista10706e22009-06-19 20:34:30 +00001322 finally:
1323 # be sure the FD is closed no matter what
1324 os.close(errpipe_write)
1325
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001326 if p2cread != -1 and p2cwrite != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001327 os.close(p2cread)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001328 if c2pwrite != -1 and c2pread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001329 os.close(c2pwrite)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001330 if errwrite != -1 and errread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001331 os.close(errwrite)
1332
1333 # Wait for exec to fail or succeed; possibly raising an
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001334 # exception (limited in size)
1335 data = bytearray()
1336 while True:
1337 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1338 data += part
1339 if not part or len(data) > 50000:
1340 break
Facundo Batista10706e22009-06-19 20:34:30 +00001341 finally:
1342 # be sure the FD is closed no matter what
1343 os.close(errpipe_read)
1344
Guido van Rossumaf2362a2007-05-15 22:32:02 +00001345 if data:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001346 try:
1347 _eintr_retry_call(os.waitpid, self.pid, 0)
1348 except OSError as e:
1349 if e.errno != errno.ECHILD:
1350 raise
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001351 try:
1352 exception_name, hex_errno, err_msg = data.split(b':', 2)
1353 except ValueError:
1354 print('Bad exception data:', repr(data))
1355 exception_name = b'RuntimeError'
1356 hex_errno = b'0'
1357 err_msg = b'Unknown'
1358 child_exception_type = getattr(
1359 builtins, exception_name.decode('ascii'),
1360 RuntimeError)
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001361 for fd in (p2cwrite, c2pread, errread):
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001362 if fd != -1:
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001363 os.close(fd)
Victor Stinner4d078042010-04-23 19:28:32 +00001364 err_msg = err_msg.decode(errors="surrogatepass")
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001365 if issubclass(child_exception_type, OSError) and hex_errno:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001366 errno_num = int(hex_errno, 16)
1367 if errno_num != 0:
1368 err_msg = os.strerror(errno_num)
1369 if errno_num == errno.ENOENT:
Benjamin Peterson5f780402010-11-20 18:07:52 +00001370 err_msg += ': ' + repr(args[0])
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001371 raise child_exception_type(errno_num, err_msg)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001372 raise child_exception_type(err_msg)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001373
1374
Brett Cannon84df1e62010-05-14 00:33:40 +00001375 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1376 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1377 _WEXITSTATUS=os.WEXITSTATUS):
1378 # This method is called (indirectly) by __del__, so it cannot
1379 # refer to anything outside of its local scope."""
1380 if _WIFSIGNALED(sts):
1381 self.returncode = -_WTERMSIG(sts)
1382 elif _WIFEXITED(sts):
1383 self.returncode = _WEXITSTATUS(sts)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001384 else:
1385 # Should never happen
1386 raise RuntimeError("Unknown child exit status!")
1387
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001388
Brett Cannon84df1e62010-05-14 00:33:40 +00001389 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1390 _WNOHANG=os.WNOHANG, _os_error=os.error):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001391 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +00001392 attribute.
1393
1394 This method is called by __del__, so it cannot reference anything
1395 outside of the local scope (nor can any methods it calls).
1396
1397 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001398 if self.returncode is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001399 try:
Brett Cannon84df1e62010-05-14 00:33:40 +00001400 pid, sts = _waitpid(self.pid, _WNOHANG)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001401 if pid == self.pid:
1402 self._handle_exitstatus(sts)
Brett Cannon84df1e62010-05-14 00:33:40 +00001403 except _os_error:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +00001404 if _deadstate is not None:
1405 self.returncode = _deadstate
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001406 return self.returncode
1407
1408
1409 def wait(self):
1410 """Wait for child process to terminate. Returns returncode
1411 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001412 if self.returncode is None:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001413 try:
1414 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1415 except OSError as e:
1416 if e.errno != errno.ECHILD:
1417 raise
1418 # This happens if SIGCLD is set to be ignored or waiting
1419 # for child processes has otherwise been disabled for our
1420 # process. This child is dead, we can't get the status.
1421 sts = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001422 self._handle_exitstatus(sts)
1423 return self.returncode
1424
1425
Peter Astrand23109f02005-03-03 20:28:59 +00001426 def _communicate(self, input):
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001427 if self.stdin:
1428 # Flush stdio buffer. This might block, if the user has
1429 # been writing to .stdin in an uncontrolled fashion.
1430 self.stdin.flush()
1431 if not input:
1432 self.stdin.close()
1433
1434 if _has_poll:
1435 stdout, stderr = self._communicate_with_poll(input)
1436 else:
1437 stdout, stderr = self._communicate_with_select(input)
1438
1439 # All data exchanged. Translate lists into strings.
1440 if stdout is not None:
1441 stdout = b''.join(stdout)
1442 if stderr is not None:
1443 stderr = b''.join(stderr)
1444
1445 # Translate newlines, if requested.
1446 # This also turns bytes into strings.
1447 if self.universal_newlines:
1448 if stdout is not None:
1449 stdout = self._translate_newlines(stdout,
1450 self.stdout.encoding)
1451 if stderr is not None:
1452 stderr = self._translate_newlines(stderr,
1453 self.stderr.encoding)
1454
1455 self.wait()
1456 return (stdout, stderr)
1457
1458
1459 def _communicate_with_poll(self, input):
1460 stdout = None # Return
1461 stderr = None # Return
1462 fd2file = {}
1463 fd2output = {}
1464
1465 poller = select.poll()
1466 def register_and_append(file_obj, eventmask):
1467 poller.register(file_obj.fileno(), eventmask)
1468 fd2file[file_obj.fileno()] = file_obj
1469
1470 def close_unregister_and_remove(fd):
1471 poller.unregister(fd)
1472 fd2file[fd].close()
1473 fd2file.pop(fd)
1474
1475 if self.stdin and input:
1476 register_and_append(self.stdin, select.POLLOUT)
1477
1478 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1479 if self.stdout:
1480 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1481 fd2output[self.stdout.fileno()] = stdout = []
1482 if self.stderr:
1483 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1484 fd2output[self.stderr.fileno()] = stderr = []
1485
1486 input_offset = 0
1487 while fd2file:
1488 try:
1489 ready = poller.poll()
1490 except select.error as e:
1491 if e.args[0] == errno.EINTR:
1492 continue
1493 raise
1494
1495 # XXX Rewrite these to use non-blocking I/O on the
1496 # file objects; they are no longer using C stdio!
1497
1498 for fd, mode in ready:
1499 if mode & select.POLLOUT:
1500 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001501 try:
1502 input_offset += os.write(fd, chunk)
1503 except OSError as e:
1504 if e.errno == errno.EPIPE:
1505 close_unregister_and_remove(fd)
1506 else:
1507 raise
1508 else:
1509 if input_offset >= len(input):
1510 close_unregister_and_remove(fd)
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001511 elif mode & select_POLLIN_POLLPRI:
1512 data = os.read(fd, 4096)
1513 if not data:
1514 close_unregister_and_remove(fd)
1515 fd2output[fd].append(data)
1516 else:
1517 # Ignore hang up or errors.
1518 close_unregister_and_remove(fd)
1519
1520 return (stdout, stderr)
1521
1522
1523 def _communicate_with_select(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001524 read_set = []
1525 write_set = []
1526 stdout = None # Return
1527 stderr = None # Return
1528
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001529 if self.stdin and input:
1530 write_set.append(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001531 if self.stdout:
1532 read_set.append(self.stdout)
Tim Peterse718f612004-10-12 21:51:32 +00001533 stdout = []
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001534 if self.stderr:
1535 read_set.append(self.stderr)
1536 stderr = []
1537
Thomas Wouters902d6eb2007-01-09 23:18:33 +00001538 input_offset = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001539 while read_set or write_set:
Georg Brandl86b2fb92008-07-16 03:43:04 +00001540 try:
1541 rlist, wlist, xlist = select.select(read_set, write_set, [])
1542 except select.error as e:
1543 if e.args[0] == errno.EINTR:
1544 continue
1545 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001546
Guido van Rossum98297ee2007-11-06 21:34:58 +00001547 # XXX Rewrite these to use non-blocking I/O on the
1548 # file objects; they are no longer using C stdio!
1549
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001550 if self.stdin in wlist:
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001551 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001552 try:
1553 bytes_written = os.write(self.stdin.fileno(), chunk)
1554 except OSError as e:
1555 if e.errno == errno.EPIPE:
1556 self.stdin.close()
1557 write_set.remove(self.stdin)
1558 else:
1559 raise
1560 else:
1561 input_offset += bytes_written
1562 if input_offset >= len(input):
1563 self.stdin.close()
1564 write_set.remove(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001565
1566 if self.stdout in rlist:
1567 data = os.read(self.stdout.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001568 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001569 self.stdout.close()
1570 read_set.remove(self.stdout)
1571 stdout.append(data)
1572
1573 if self.stderr in rlist:
1574 data = os.read(self.stderr.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001575 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001576 self.stderr.close()
1577 read_set.remove(self.stderr)
1578 stderr.append(data)
1579
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001580 return (stdout, stderr)
1581
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001582
Christian Heimesa342c012008-04-20 21:01:16 +00001583 def send_signal(self, sig):
1584 """Send a signal to the process
1585 """
1586 os.kill(self.pid, sig)
1587
1588 def terminate(self):
1589 """Terminate the process with SIGTERM
1590 """
1591 self.send_signal(signal.SIGTERM)
1592
1593 def kill(self):
1594 """Kill the process with SIGKILL
1595 """
1596 self.send_signal(signal.SIGKILL)
1597
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001598
1599def _demo_posix():
1600 #
1601 # Example 1: Simple redirection: Get process list
1602 #
1603 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001604 print("Process list:")
1605 print(plist)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001606
1607 #
1608 # Example 2: Change uid before executing child
1609 #
1610 if os.getuid() == 0:
1611 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1612 p.wait()
1613
1614 #
1615 # Example 3: Connecting several subprocesses
1616 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001617 print("Looking for 'hda'...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001618 p1 = Popen(["dmesg"], stdout=PIPE)
1619 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001620 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001621
1622 #
1623 # Example 4: Catch execution error
1624 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001625 print()
1626 print("Trying a weird file...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001627 try:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001628 print(Popen(["/this/path/does/not/exist"]).communicate())
Guido van Rossumb940e112007-01-10 16:19:56 +00001629 except OSError as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001630 if e.errno == errno.ENOENT:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001631 print("The file didn't exist. I thought so...")
1632 print("Child traceback:")
1633 print(e.child_traceback)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001634 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001635 print("Error", e.errno)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001636 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001637 print("Gosh. No error.", file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001638
1639
1640def _demo_windows():
1641 #
1642 # Example 1: Connecting several subprocesses
1643 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001644 print("Looking for 'PROMPT' in set output...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001645 p1 = Popen("set", stdout=PIPE, shell=True)
1646 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001647 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001648
1649 #
1650 # Example 2: Simple execution of program
1651 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001652 print("Executing calc...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001653 p = Popen("calc")
1654 p.wait()
1655
1656
1657if __name__ == "__main__":
1658 if mswindows:
1659 _demo_windows()
1660 else:
1661 _demo_posix()