blob: 2d85b507103fceea0b3baa85f22cbf849a346754 [file] [log] [blame]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001# subprocess - Subprocesses with accessible I/O streams
2#
Tim Peterse718f612004-10-12 21:51:32 +00003# For more information about this module, see PEP 324.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00004#
Peter Astrand3a708df2005-09-23 17:37:29 +00005# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00006#
Peter Astrand69bf13f2005-02-14 08:56:32 +00007# Licensed to PSF under a Contributor Agreement.
Peter Astrand3a708df2005-09-23 17:37:29 +00008# See http://www.python.org/2.4/license for licensing details.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00009
Raymond Hettinger837dd932004-10-17 16:36:53 +000010r"""subprocess - Subprocesses with accessible I/O streams
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000011
Fredrik Lundh15aaacc2004-10-17 14:47:05 +000012This module allows you to spawn processes, connect to their
13input/output/error pipes, and obtain their return codes. This module
14intends to replace several other, older modules and functions, like:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000015
16os.system
17os.spawn*
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000018
19Information about how the subprocess module can be used to replace these
20modules and functions can be found below.
21
22
23
24Using the subprocess module
25===========================
26This module defines one class called Popen:
27
28class Popen(args, bufsize=0, executable=None,
29 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000030 preexec_fn=None, close_fds=True, shell=False,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000031 cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000032 startupinfo=None, creationflags=0,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000033 restore_signals=True, start_new_session=False, pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000034
35
36Arguments are:
37
38args should be a string, or a sequence of program arguments. The
39program to execute is normally the first item in the args sequence or
40string, but can be explicitly set by using the executable argument.
41
Gregory P. Smithf5604852010-12-13 06:45:02 +000042On POSIX, with shell=False (default): In this case, the Popen class
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000043uses os.execvp() to execute the child program. args should normally
44be a sequence. A string will be treated as a sequence with the string
45as the only item (the program to execute).
46
Gregory P. Smithf5604852010-12-13 06:45:02 +000047On POSIX, with shell=True: If args is a string, it specifies the
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000048command string to execute through the shell. If args is a sequence,
49the first item specifies the command string, and any additional items
50will be treated as additional shell arguments.
51
52On Windows: the Popen class uses CreateProcess() to execute the child
53program, which operates on strings. If args is a sequence, it will be
54converted to a string using the list2cmdline method. Please note that
55not all MS Windows applications interpret the command line the same
56way: The list2cmdline is designed for applications using the same
57rules as the MS C runtime.
58
59bufsize, if given, has the same meaning as the corresponding argument
60to the built-in open() function: 0 means unbuffered, 1 means line
61buffered, any other positive value means use a buffer of
62(approximately) that size. A negative bufsize means to use the system
63default, which usually means fully buffered. The default value for
64bufsize is 0 (unbuffered).
65
66stdin, stdout and stderr specify the executed programs' standard
67input, standard output and standard error file handles, respectively.
68Valid values are PIPE, an existing file descriptor (a positive
69integer), an existing file object, and None. PIPE indicates that a
70new pipe to the child should be created. With None, no redirection
71will occur; the child's file handles will be inherited from the
72parent. Additionally, stderr can be STDOUT, which indicates that the
73stderr data from the applications should be captured into the same
74file handle as for stdout.
75
Gregory P. Smithf5604852010-12-13 06:45:02 +000076On POSIX, if preexec_fn is set to a callable object, this object will be
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000077called in the child process just before the child is executed. The use
78of preexec_fn is not thread safe, using it in the presence of threads
79could lead to a deadlock in the child process before the new executable
80is executed.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000081
82If close_fds is true, all file descriptors except 0, 1 and 2 will be
Gregory P. Smithf5604852010-12-13 06:45:02 +000083closed before the child process is executed. The default for close_fds
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000084varies by platform: Always true on POSIX. True when stdin/stdout/stderr
85are None on Windows, false otherwise.
86
87pass_fds is an optional sequence of file descriptors to keep open between the
88parent and child. Providing any pass_fds implicitly sets close_fds to true.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000089
90if shell is true, the specified command will be executed through the
91shell.
92
93If cwd is not None, the current directory will be changed to cwd
94before the child is executed.
95
Gregory P. Smithf5604852010-12-13 06:45:02 +000096On POSIX, if restore_signals is True all signals that Python sets to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000097SIG_IGN are restored to SIG_DFL in the child process before the exec.
98Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99parameter does nothing on Windows.
100
Gregory P. Smithf5604852010-12-13 06:45:02 +0000101On POSIX, if start_new_session is True, the setsid() system call will be made
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000102in the child process prior to executing the command.
103
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000104If env is not None, it defines the environment variables for the new
105process.
106
107If universal_newlines is true, the file objects stdout and stderr are
108opened as a text files, but lines may be terminated by any of '\n',
Gregory P. Smithf5604852010-12-13 06:45:02 +0000109the Unix end-of-line convention, '\r', the old Macintosh convention or
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000110'\r\n', the Windows convention. All of these external representations
111are seen as '\n' by the Python program. Note: This feature is only
112available if Python is built with universal newline support (the
113default). Also, the newlines attribute of the file objects stdout,
114stdin and stderr are not updated by the communicate() method.
115
116The startupinfo and creationflags, if given, will be passed to the
117underlying CreateProcess() function. They can specify things such as
118appearance of the main window and priority for the new process.
119(Windows only)
120
121
Georg Brandlf9734072008-12-07 15:30:06 +0000122This module also defines some shortcut functions:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000123
Peter Astrand5f5e1412004-12-05 20:15:36 +0000124call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000125 Run command with arguments. Wait for command to complete, then
126 return the returncode attribute.
127
128 The arguments are the same as for the Popen constructor. Example:
129
Florent Xicluna4886d242010-03-08 13:27:26 +0000130 >>> retcode = subprocess.call(["ls", "-l"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000131
Peter Astrand454f7672005-01-01 09:36:35 +0000132check_call(*popenargs, **kwargs):
133 Run command with arguments. Wait for command to complete. If the
134 exit code was zero then return, otherwise raise
135 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000136 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000137
138 The arguments are the same as for the Popen constructor. Example:
139
Florent Xicluna4886d242010-03-08 13:27:26 +0000140 >>> subprocess.check_call(["ls", "-l"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000141 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000142
Brett Cannona23810f2008-05-26 19:04:21 +0000143getstatusoutput(cmd):
144 Return (status, output) of executing cmd in a shell.
145
146 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
147 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
148 returned output will contain output or error messages. A trailing newline
149 is stripped from the output. The exit status for the command can be
150 interpreted according to the rules for the C function wait(). Example:
151
Brett Cannona23810f2008-05-26 19:04:21 +0000152 >>> subprocess.getstatusoutput('ls /bin/ls')
153 (0, '/bin/ls')
154 >>> subprocess.getstatusoutput('cat /bin/junk')
155 (256, 'cat: /bin/junk: No such file or directory')
156 >>> subprocess.getstatusoutput('/bin/junk')
157 (256, 'sh: /bin/junk: not found')
158
159getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell.
161
162 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example:
164
Brett Cannona23810f2008-05-26 19:04:21 +0000165 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls'
167
Georg Brandlf9734072008-12-07 15:30:06 +0000168check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000169 Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000170
Georg Brandl2708f3a2009-12-20 14:38:23 +0000171 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000174
Georg Brandl2708f3a2009-12-20 14:38:23 +0000175 The arguments are the same as for the Popen constructor. Example:
Georg Brandlf9734072008-12-07 15:30:06 +0000176
Georg Brandl2708f3a2009-12-20 14:38:23 +0000177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandlf9734072008-12-07 15:30:06 +0000178
Brett Cannona23810f2008-05-26 19:04:21 +0000179
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000180Exceptions
181----------
182Exceptions raised in the child process, before the new program has
183started to execute, will be re-raised in the parent. Additionally,
184the exception object will have one extra attribute called
185'child_traceback', which is a string containing traceback information
186from the childs point of view.
187
188The most common exception raised is OSError. This occurs, for
189example, when trying to execute a non-existent file. Applications
190should prepare for OSErrors.
191
192A ValueError will be raised if Popen is called with invalid arguments.
193
Georg Brandlf9734072008-12-07 15:30:06 +0000194check_call() and check_output() will raise CalledProcessError, if the
195called process returns a non-zero return code.
Peter Astrand454f7672005-01-01 09:36:35 +0000196
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000197
198Security
199--------
200Unlike some other popen functions, this implementation will never call
201/bin/sh implicitly. This means that all characters, including shell
202metacharacters, can safely be passed to child processes.
203
204
205Popen objects
206=============
207Instances of the Popen class have the following methods:
208
209poll()
210 Check if child process has terminated. Returns returncode
211 attribute.
212
213wait()
214 Wait for child process to terminate. Returns returncode attribute.
215
216communicate(input=None)
217 Interact with process: Send data to stdin. Read data from stdout
218 and stderr, until end-of-file is reached. Wait for process to
Thomas Wouters902d6eb2007-01-09 23:18:33 +0000219 terminate. The optional input argument should be a string to be
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000220 sent to the child process, or None, if no data should be sent to
221 the child.
Tim Peterse718f612004-10-12 21:51:32 +0000222
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000223 communicate() returns a tuple (stdout, stderr).
224
225 Note: The data read is buffered in memory, so do not use this
226 method if the data size is large or unlimited.
227
228The following attributes are also available:
229
230stdin
231 If the stdin argument is PIPE, this attribute is a file object
232 that provides input to the child process. Otherwise, it is None.
233
234stdout
235 If the stdout argument is PIPE, this attribute is a file object
236 that provides output from the child process. Otherwise, it is
237 None.
238
239stderr
240 If the stderr argument is PIPE, this attribute is file object that
241 provides error output from the child process. Otherwise, it is
242 None.
243
244pid
245 The process ID of the child process.
246
247returncode
248 The child return code. A None value indicates that the process
249 hasn't terminated yet. A negative value -N indicates that the
Gregory P. Smithf5604852010-12-13 06:45:02 +0000250 child was terminated by signal N (POSIX only).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000251
252
253Replacing older functions with the subprocess module
254====================================================
255In this section, "a ==> b" means that b can be used as a replacement
256for a.
257
258Note: All functions in this section fail (more or less) silently if
259the executed program cannot be found; this module raises an OSError
260exception.
261
262In the following examples, we assume that the subprocess module is
263imported with "from subprocess import *".
264
265
266Replacing /bin/sh shell backquote
267---------------------------------
268output=`mycmd myarg`
269==>
270output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
271
272
273Replacing shell pipe line
274-------------------------
275output=`dmesg | grep hda`
276==>
277p1 = Popen(["dmesg"], stdout=PIPE)
Peter Astrand6fdf3cb2004-11-30 18:06:42 +0000278p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000279output = p2.communicate()[0]
280
281
282Replacing os.system()
283---------------------
284sts = os.system("mycmd" + " myarg")
285==>
286p = Popen("mycmd" + " myarg", shell=True)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000287pid, sts = os.waitpid(p.pid, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000288
289Note:
290
291* Calling the program through the shell is usually not required.
292
293* It's easier to look at the returncode attribute than the
294 exitstatus.
295
296A more real-world example would look like this:
297
298try:
299 retcode = call("mycmd" + " myarg", shell=True)
300 if retcode < 0:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000301 print("Child was terminated by signal", -retcode, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000302 else:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000303 print("Child returned", retcode, file=sys.stderr)
304except OSError as e:
305 print("Execution failed:", e, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000306
307
308Replacing os.spawn*
309-------------------
Tim Peterse718f612004-10-12 21:51:32 +0000310P_NOWAIT example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000311
312pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
313==>
314pid = Popen(["/bin/mycmd", "myarg"]).pid
315
316
317P_WAIT example:
318
319retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
320==>
321retcode = call(["/bin/mycmd", "myarg"])
322
323
Tim Peterse718f612004-10-12 21:51:32 +0000324Vector example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000325
326os.spawnvp(os.P_NOWAIT, path, args)
327==>
328Popen([path] + args[1:])
329
330
Tim Peterse718f612004-10-12 21:51:32 +0000331Environment example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000332
333os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
334==>
335Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000336"""
337
338import sys
339mswindows = (sys.platform == "win32")
340
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000341import io
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000342import os
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000343import traceback
Christian Heimesfdab48e2008-01-20 09:06:41 +0000344import gc
Christian Heimesa342c012008-04-20 21:01:16 +0000345import signal
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000346import builtins
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000347import warnings
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200348import errno
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000349
Peter Astrand454f7672005-01-01 09:36:35 +0000350# Exception classes used by this module.
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000351class CalledProcessError(Exception):
Georg Brandlf9734072008-12-07 15:30:06 +0000352 """This exception is raised when a process run by check_call() or
353 check_output() returns a non-zero exit status.
354 The exit status will be stored in the returncode attribute;
355 check_output() will also store the output in the output attribute.
356 """
357 def __init__(self, returncode, cmd, output=None):
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000358 self.returncode = returncode
359 self.cmd = cmd
Georg Brandlf9734072008-12-07 15:30:06 +0000360 self.output = output
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000361 def __str__(self):
362 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
363
Peter Astrand454f7672005-01-01 09:36:35 +0000364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000365if mswindows:
366 import threading
367 import msvcrt
Brian Curtin1ce6b582010-04-24 16:19:22 +0000368 import _subprocess
369 class STARTUPINFO:
370 dwFlags = 0
371 hStdInput = None
372 hStdOutput = None
373 hStdError = None
374 wShowWindow = 0
375 class pywintypes:
376 error = IOError
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000377else:
378 import select
Gregory P. Smithd06fa472009-07-04 02:46:54 +0000379 _has_poll = hasattr(select, 'poll')
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000380 import fcntl
381 import pickle
382
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000383 try:
384 import _posixsubprocess
385 except ImportError:
386 _posixsubprocess = None
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000387 warnings.warn("The _posixsubprocess module is not being used. "
388 "Child process reliability may suffer if your "
389 "program uses threads.", RuntimeWarning)
390
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000391 # When select or poll has indicated that the file is writable,
392 # we can write up to _PIPE_BUF bytes without risk of blocking.
393 # POSIX defines PIPE_BUF as >= 512.
394 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
395
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000396 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
397
398 def _set_cloexec(fd, cloexec):
399 old = fcntl.fcntl(fd, fcntl.F_GETFD)
400 if cloexec:
401 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
402 else:
403 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
404
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000405 if _posixsubprocess:
406 _create_pipe = _posixsubprocess.cloexec_pipe
407 else:
408 def _create_pipe():
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000409 fds = os.pipe()
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000410 _set_cloexec(fds[0], True)
411 _set_cloexec(fds[1], True)
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000412 return fds
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000413
Brett Cannona23810f2008-05-26 19:04:21 +0000414__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
Georg Brandlf9734072008-12-07 15:30:06 +0000415 "getoutput", "check_output", "CalledProcessError"]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000416
Brian Curtin1ce6b582010-04-24 16:19:22 +0000417if mswindows:
Brian Curtin08fd8d92011-04-29 16:11:30 -0500418 from _subprocess import (CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP,
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500419 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
420 STD_ERROR_HANDLE, SW_HIDE,
421 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW)
Gregory P. Smith0ef3e392011-05-11 22:20:11 -0700422
Brian Curtin08fd8d92011-04-29 16:11:30 -0500423 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP",
Brian Curtin8b8e7f42011-04-29 15:48:13 -0500424 "STD_INPUT_HANDLE", "STD_OUTPUT_HANDLE",
425 "STD_ERROR_HANDLE", "SW_HIDE",
426 "STARTF_USESTDHANDLES", "STARTF_USESHOWWINDOW"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000427try:
428 MAXFD = os.sysconf("SC_OPEN_MAX")
429except:
430 MAXFD = 256
431
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000432_active = []
433
434def _cleanup():
435 for inst in _active[:]:
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000436 res = inst._internal_poll(_deadstate=sys.maxsize)
Guido van Rossumb5d47ef2006-08-24 02:27:45 +0000437 if res is not None and res >= 0:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000438 try:
439 _active.remove(inst)
440 except ValueError:
441 # This can happen if two threads create a new Popen instance.
442 # It's harmless that it was already removed, so ignore.
443 pass
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000444
445PIPE = -1
446STDOUT = -2
447
448
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000449def _eintr_retry_call(func, *args):
450 while True:
451 try:
452 return func(*args)
Victor Stinner2cfb6f32011-07-05 14:00:56 +0200453 except (OSError, IOError) as e:
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000454 if e.errno == errno.EINTR:
455 continue
456 raise
457
458
Peter Astrand5f5e1412004-12-05 20:15:36 +0000459def call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000460 """Run command with arguments. Wait for command to complete, then
461 return the returncode attribute.
462
463 The arguments are the same as for the Popen constructor. Example:
464
465 retcode = call(["ls", "-l"])
466 """
Peter Astrand5f5e1412004-12-05 20:15:36 +0000467 return Popen(*popenargs, **kwargs).wait()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000468
469
Peter Astrand454f7672005-01-01 09:36:35 +0000470def check_call(*popenargs, **kwargs):
471 """Run command with arguments. Wait for command to complete. If
472 the exit code was zero then return, otherwise raise
473 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000474 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000475
476 The arguments are the same as for the Popen constructor. Example:
477
478 check_call(["ls", "-l"])
479 """
480 retcode = call(*popenargs, **kwargs)
Peter Astrand454f7672005-01-01 09:36:35 +0000481 if retcode:
Georg Brandlf9734072008-12-07 15:30:06 +0000482 cmd = kwargs.get("args")
483 if cmd is None:
484 cmd = popenargs[0]
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000485 raise CalledProcessError(retcode, cmd)
Georg Brandlf9734072008-12-07 15:30:06 +0000486 return 0
487
488
489def check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000490 r"""Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000491
492 If the exit code was non-zero it raises a CalledProcessError. The
493 CalledProcessError object will have the return code in the returncode
494 attribute and output in the output attribute.
495
496 The arguments are the same as for the Popen constructor. Example:
497
498 >>> check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000499 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000500
501 The stdout argument is not allowed as it is used internally.
Georg Brandl127d4702009-12-28 08:10:38 +0000502 To capture standard error in the result, use stderr=STDOUT.
Georg Brandlf9734072008-12-07 15:30:06 +0000503
504 >>> check_output(["/bin/sh", "-c",
Georg Brandl2708f3a2009-12-20 14:38:23 +0000505 ... "ls -l non_existent_file ; exit 0"],
Georg Brandl127d4702009-12-28 08:10:38 +0000506 ... stderr=STDOUT)
Georg Brandl2708f3a2009-12-20 14:38:23 +0000507 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000508 """
509 if 'stdout' in kwargs:
510 raise ValueError('stdout argument not allowed, it will be overridden.')
511 process = Popen(*popenargs, stdout=PIPE, **kwargs)
512 output, unused_err = process.communicate()
513 retcode = process.poll()
514 if retcode:
515 cmd = kwargs.get("args")
516 if cmd is None:
517 cmd = popenargs[0]
518 raise CalledProcessError(retcode, cmd, output=output)
519 return output
Peter Astrand454f7672005-01-01 09:36:35 +0000520
521
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000522def list2cmdline(seq):
523 """
524 Translate a sequence of arguments into a command line
525 string, using the same rules as the MS C runtime:
526
527 1) Arguments are delimited by white space, which is either a
528 space or a tab.
529
530 2) A string surrounded by double quotation marks is
531 interpreted as a single argument, regardless of white space
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000532 contained within. A quoted string can be embedded in an
533 argument.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000534
535 3) A double quotation mark preceded by a backslash is
536 interpreted as a literal double quotation mark.
537
538 4) Backslashes are interpreted literally, unless they
539 immediately precede a double quotation mark.
540
541 5) If backslashes immediately precede a double quotation mark,
542 every pair of backslashes is interpreted as a literal
543 backslash. If the number of backslashes is odd, the last
544 backslash escapes the next double quotation mark as
545 described in rule 3.
546 """
547
548 # See
Eric Smith3c573af2009-11-09 15:23:15 +0000549 # http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
550 # or search http://msdn.microsoft.com for
551 # "Parsing C++ Command-Line Arguments"
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000552 result = []
553 needquote = False
554 for arg in seq:
555 bs_buf = []
556
557 # Add a space to separate this argument from the others
558 if result:
559 result.append(' ')
560
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000561 needquote = (" " in arg) or ("\t" in arg) or not arg
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000562 if needquote:
563 result.append('"')
564
565 for c in arg:
566 if c == '\\':
Tim Peterse718f612004-10-12 21:51:32 +0000567 # Don't know if we need to double yet.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000568 bs_buf.append(c)
569 elif c == '"':
Christian Heimesfdab48e2008-01-20 09:06:41 +0000570 # Double backslashes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000571 result.append('\\' * len(bs_buf)*2)
572 bs_buf = []
573 result.append('\\"')
574 else:
575 # Normal char
576 if bs_buf:
577 result.extend(bs_buf)
578 bs_buf = []
579 result.append(c)
580
Christian Heimesfdab48e2008-01-20 09:06:41 +0000581 # Add remaining backslashes, if any.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000582 if bs_buf:
583 result.extend(bs_buf)
584
585 if needquote:
Peter Astrand7e78ade2005-03-03 21:10:23 +0000586 result.extend(bs_buf)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000587 result.append('"')
588
589 return ''.join(result)
590
591
Brett Cannona23810f2008-05-26 19:04:21 +0000592# Various tools for executing commands and looking at their output and status.
593#
Gregory P. Smithf5604852010-12-13 06:45:02 +0000594# NB This only works (and is only relevant) for POSIX.
Brett Cannona23810f2008-05-26 19:04:21 +0000595
596def getstatusoutput(cmd):
597 """Return (status, output) of executing cmd in a shell.
598
599 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
600 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
601 returned output will contain output or error messages. A trailing newline
602 is stripped from the output. The exit status for the command can be
603 interpreted according to the rules for the C function wait(). Example:
604
605 >>> import subprocess
606 >>> subprocess.getstatusoutput('ls /bin/ls')
607 (0, '/bin/ls')
608 >>> subprocess.getstatusoutput('cat /bin/junk')
609 (256, 'cat: /bin/junk: No such file or directory')
610 >>> subprocess.getstatusoutput('/bin/junk')
611 (256, 'sh: /bin/junk: not found')
612 """
613 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
614 text = pipe.read()
615 sts = pipe.close()
616 if sts is None: sts = 0
617 if text[-1:] == '\n': text = text[:-1]
618 return sts, text
619
620
621def getoutput(cmd):
622 """Return output (stdout or stderr) of executing cmd in a shell.
623
624 Like getstatusoutput(), except the exit status is ignored and the return
625 value is a string containing the command's output. Example:
626
627 >>> import subprocess
628 >>> subprocess.getoutput('ls /bin/ls')
629 '/bin/ls'
630 """
631 return getstatusoutput(cmd)[1]
632
633
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000634_PLATFORM_DEFAULT_CLOSE_FDS = object()
Gregory P. Smithf5604852010-12-13 06:45:02 +0000635
636
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000637class Popen(object):
638 def __init__(self, args, bufsize=0, executable=None,
639 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000640 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
641 shell=False, cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000642 startupinfo=None, creationflags=0,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000643 restore_signals=True, start_new_session=False,
644 pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000645 """Create new Popen instance."""
646 _cleanup()
647
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000648 self._child_created = False
Guido van Rossum46a05a72007-06-07 21:56:45 +0000649 if bufsize is None:
650 bufsize = 0 # Restore default
Walter Dörwaldaa97f042007-05-03 21:05:51 +0000651 if not isinstance(bufsize, int):
Peter Astrand738131d2004-11-30 21:04:45 +0000652 raise TypeError("bufsize must be an integer")
653
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000654 if mswindows:
Tim Peterse8374a52004-10-13 03:15:00 +0000655 if preexec_fn is not None:
656 raise ValueError("preexec_fn is not supported on Windows "
657 "platforms")
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000658 any_stdio_set = (stdin is not None or stdout is not None or
659 stderr is not None)
660 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
661 if any_stdio_set:
662 close_fds = False
663 else:
664 close_fds = True
665 elif close_fds and any_stdio_set:
666 raise ValueError(
667 "close_fds is not supported on Windows platforms"
668 " if you redirect stdin/stdout/stderr")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000669 else:
670 # POSIX
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000671 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
672 close_fds = True
673 if pass_fds and not close_fds:
674 warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
675 close_fds = True
Tim Peterse8374a52004-10-13 03:15:00 +0000676 if startupinfo is not None:
677 raise ValueError("startupinfo is only supported on Windows "
678 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000679 if creationflags != 0:
Tim Peterse8374a52004-10-13 03:15:00 +0000680 raise ValueError("creationflags is only supported on Windows "
681 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000682
Tim Peterse718f612004-10-12 21:51:32 +0000683 self.stdin = None
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000684 self.stdout = None
685 self.stderr = None
686 self.pid = None
687 self.returncode = None
688 self.universal_newlines = universal_newlines
689
690 # Input and output objects. The general principle is like
691 # this:
692 #
693 # Parent Child
694 # ------ -----
695 # p2cwrite ---stdin---> p2cread
696 # c2pread <--stdout--- c2pwrite
697 # errread <--stderr--- errwrite
698 #
699 # On POSIX, the child objects are file descriptors. On
700 # Windows, these are Windows file handles. The parent objects
701 # are file descriptors on both platforms. The parent objects
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000702 # are -1 when not using PIPEs. The child objects are -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000703 # when not redirecting.
Tim Peterse718f612004-10-12 21:51:32 +0000704
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000705 (p2cread, p2cwrite,
706 c2pread, c2pwrite,
707 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
708
Antoine Pitrouc9982322011-01-04 19:07:07 +0000709 # We wrap OS handles *before* launching the child, otherwise a
710 # quickly terminating child could make our fds unwrappable
711 # (see #8458).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000712
Thomas Wouterscf297e42007-02-23 15:07:44 +0000713 if mswindows:
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000714 if p2cwrite != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000715 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000716 if c2pread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000717 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000718 if errread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000719 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
Thomas Wouterscf297e42007-02-23 15:07:44 +0000720
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000721 if p2cwrite != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000722 self.stdin = io.open(p2cwrite, 'wb', bufsize)
723 if self.universal_newlines:
Antoine Pitrouab85ff32011-07-23 22:03:45 +0200724 self.stdin = io.TextIOWrapper(self.stdin, write_through=True)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000725 if c2pread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000726 self.stdout = io.open(c2pread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000727 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000728 self.stdout = io.TextIOWrapper(self.stdout)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000729 if errread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000730 self.stderr = io.open(errread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000731 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000732 self.stderr = io.TextIOWrapper(self.stderr)
Tim Peterse718f612004-10-12 21:51:32 +0000733
Antoine Pitrouc9982322011-01-04 19:07:07 +0000734 try:
735 self._execute_child(args, executable, preexec_fn, close_fds,
736 pass_fds, cwd, env, universal_newlines,
737 startupinfo, creationflags, shell,
738 p2cread, p2cwrite,
739 c2pread, c2pwrite,
740 errread, errwrite,
741 restore_signals, start_new_session)
742 except:
743 # Cleanup if the child failed starting
744 for f in filter(None, [self.stdin, self.stdout, self.stderr]):
745 try:
746 f.close()
747 except EnvironmentError:
748 # Ignore EBADF or other errors
749 pass
750 raise
751
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000752
Guido van Rossum98297ee2007-11-06 21:34:58 +0000753 def _translate_newlines(self, data, encoding):
754 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")
755 return data.decode(encoding)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000756
Brian Curtin79cdb662010-12-03 02:46:02 +0000757 def __enter__(self):
758 return self
759
760 def __exit__(self, type, value, traceback):
761 if self.stdout:
762 self.stdout.close()
763 if self.stderr:
764 self.stderr.close()
765 if self.stdin:
766 self.stdin.close()
Gregory P. Smithc9557af2011-05-11 22:18:23 -0700767 # Wait for the process to terminate, to avoid zombies.
768 self.wait()
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000769
Brett Cannon84df1e62010-05-14 00:33:40 +0000770 def __del__(self, _maxsize=sys.maxsize, _active=_active):
Victor Stinner87b9bc32011-06-01 00:57:47 +0200771 # If __init__ hasn't had a chance to execute (e.g. if it
772 # was passed an undeclared keyword argument), we don't
773 # have a _child_created attribute at all.
774 if not getattr(self, '_child_created', False):
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000775 # We didn't get to successfully create a child process.
776 return
777 # In case the child hasn't been waited on, check if it's done.
Brett Cannon84df1e62010-05-14 00:33:40 +0000778 self._internal_poll(_deadstate=_maxsize)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000779 if self.returncode is None and _active is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000780 # Child is still running, keep us alive until we can wait on it.
781 _active.append(self)
782
783
Peter Astrand23109f02005-03-03 20:28:59 +0000784 def communicate(self, input=None):
785 """Interact with process: Send data to stdin. Read data from
786 stdout and stderr, until end-of-file is reached. Wait for
787 process to terminate. The optional input argument should be a
788 string to be sent to the child process, or None, if no data
789 should be sent to the child.
Tim Peterseba28be2005-03-28 01:08:02 +0000790
Peter Astrand23109f02005-03-03 20:28:59 +0000791 communicate() returns a tuple (stdout, stderr)."""
792
793 # Optimization: If we are only using one pipe, or no pipe at
794 # all, using select() or threads is unnecessary.
795 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
Tim Peterseba28be2005-03-28 01:08:02 +0000796 stdout = None
797 stderr = None
Peter Astrand23109f02005-03-03 20:28:59 +0000798 if self.stdin:
799 if input:
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200800 try:
801 self.stdin.write(input)
802 except IOError as e:
803 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
804 raise
Peter Astrand23109f02005-03-03 20:28:59 +0000805 self.stdin.close()
806 elif self.stdout:
Victor Stinner2cfb6f32011-07-05 14:00:56 +0200807 stdout = _eintr_retry_call(self.stdout.read)
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000808 self.stdout.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000809 elif self.stderr:
Victor Stinner2cfb6f32011-07-05 14:00:56 +0200810 stderr = _eintr_retry_call(self.stderr.read)
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000811 self.stderr.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000812 self.wait()
813 return (stdout, stderr)
Tim Peterseba28be2005-03-28 01:08:02 +0000814
Peter Astrand23109f02005-03-03 20:28:59 +0000815 return self._communicate(input)
816
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000817
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000818 def poll(self):
819 return self._internal_poll()
820
821
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000822 if mswindows:
823 #
824 # Windows methods
825 #
826 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +0000827 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000828 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
829 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000830 if stdin is None and stdout is None and stderr is None:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000831 return (-1, -1, -1, -1, -1, -1)
Tim Peterse718f612004-10-12 21:51:32 +0000832
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000833 p2cread, p2cwrite = -1, -1
834 c2pread, c2pwrite = -1, -1
835 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000836
Peter Astrandd38ddf42005-02-10 08:32:50 +0000837 if stdin is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000838 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000839 if p2cread is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000840 p2cread, _ = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000841 elif stdin == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000842 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000843 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000844 p2cread = msvcrt.get_osfhandle(stdin)
845 else:
846 # Assuming file-like object
847 p2cread = msvcrt.get_osfhandle(stdin.fileno())
848 p2cread = self._make_inheritable(p2cread)
849
Peter Astrandd38ddf42005-02-10 08:32:50 +0000850 if stdout is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000851 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000852 if c2pwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000853 _, c2pwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000854 elif stdout == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000855 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000856 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000857 c2pwrite = msvcrt.get_osfhandle(stdout)
858 else:
859 # Assuming file-like object
860 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
861 c2pwrite = self._make_inheritable(c2pwrite)
862
Peter Astrandd38ddf42005-02-10 08:32:50 +0000863 if stderr is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000864 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000865 if errwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000866 _, errwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000867 elif stderr == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000868 errread, errwrite = _subprocess.CreatePipe(None, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000869 elif stderr == STDOUT:
870 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +0000871 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000872 errwrite = msvcrt.get_osfhandle(stderr)
873 else:
874 # Assuming file-like object
875 errwrite = msvcrt.get_osfhandle(stderr.fileno())
876 errwrite = self._make_inheritable(errwrite)
877
878 return (p2cread, p2cwrite,
879 c2pread, c2pwrite,
880 errread, errwrite)
881
882
883 def _make_inheritable(self, handle):
884 """Return a duplicate of handle, which is inheritable"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000885 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
886 handle, _subprocess.GetCurrentProcess(), 0, 1,
887 _subprocess.DUPLICATE_SAME_ACCESS)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000888
889
890 def _find_w9xpopen(self):
891 """Find and return absolut path to w9xpopen.exe"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000892 w9xpopen = os.path.join(
893 os.path.dirname(_subprocess.GetModuleFileName(0)),
Tim Peterse8374a52004-10-13 03:15:00 +0000894 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000895 if not os.path.exists(w9xpopen):
896 # Eeek - file-not-found - possibly an embedding
897 # situation - see if we can locate it in sys.exec_prefix
Tim Peterse8374a52004-10-13 03:15:00 +0000898 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
899 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000900 if not os.path.exists(w9xpopen):
Tim Peterse8374a52004-10-13 03:15:00 +0000901 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
902 "needed for Popen to work with your "
903 "shell or platform.")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000904 return w9xpopen
905
Tim Peterse718f612004-10-12 21:51:32 +0000906
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000907 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000908 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000909 startupinfo, creationflags, shell,
910 p2cread, p2cwrite,
911 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000912 errread, errwrite,
913 unused_restore_signals, unused_start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000914 """Execute program (MS Windows version)"""
915
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000916 assert not pass_fds, "pass_fds not supported on Windows."
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000917
Guido van Rossum3172c5d2007-10-16 18:12:55 +0000918 if not isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000919 args = list2cmdline(args)
920
Peter Astrandc1d65362004-11-07 14:30:34 +0000921 # Process startup details
Peter Astrandd38ddf42005-02-10 08:32:50 +0000922 if startupinfo is None:
Thomas Wouters73e5a5b2006-06-08 15:35:45 +0000923 startupinfo = STARTUPINFO()
Victor Stinnerb3693582010-05-21 20:13:12 +0000924 if -1 not in (p2cread, c2pwrite, errwrite):
Brian Curtin1ce6b582010-04-24 16:19:22 +0000925 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
Peter Astrandc1d65362004-11-07 14:30:34 +0000926 startupinfo.hStdInput = p2cread
927 startupinfo.hStdOutput = c2pwrite
928 startupinfo.hStdError = errwrite
929
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000930 if shell:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000931 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
932 startupinfo.wShowWindow = _subprocess.SW_HIDE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000933 comspec = os.environ.get("COMSPEC", "cmd.exe")
Tim Golden126c2962010-08-11 14:20:40 +0000934 args = '{} /c "{}"'.format (comspec, args)
Brian Curtin1ce6b582010-04-24 16:19:22 +0000935 if (_subprocess.GetVersion() >= 0x80000000 or
Tim Peterse8374a52004-10-13 03:15:00 +0000936 os.path.basename(comspec).lower() == "command.com"):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000937 # Win9x, or using command.com on NT. We need to
938 # use the w9xpopen intermediate program. For more
939 # information, see KB Q150956
940 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
941 w9xpopen = self._find_w9xpopen()
942 args = '"%s" %s' % (w9xpopen, args)
943 # Not passing CREATE_NEW_CONSOLE has been known to
944 # cause random failures on win9x. Specifically a
945 # dialog: "Your program accessed mem currently in
946 # use at xxx" and a hopeful warning about the
Mark Dickinson934896d2009-02-21 20:59:32 +0000947 # stability of your system. Cost is Ctrl+C won't
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000948 # kill children.
Brian Curtin1ce6b582010-04-24 16:19:22 +0000949 creationflags |= _subprocess.CREATE_NEW_CONSOLE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000950
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000951 # Start the process
952 try:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000953 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
Tim Peterse8374a52004-10-13 03:15:00 +0000954 # no special security
955 None, None,
Guido van Rossume7ba4952007-06-06 23:52:48 +0000956 int(not close_fds),
Tim Peterse8374a52004-10-13 03:15:00 +0000957 creationflags,
958 env,
959 cwd,
960 startupinfo)
Guido van Rossumb940e112007-01-10 16:19:56 +0000961 except pywintypes.error as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000962 # Translate pywintypes.error to WindowsError, which is
963 # a subclass of OSError. FIXME: We should really
Ezio Melotti13925002011-03-16 11:05:33 +0200964 # translate errno using _sys_errlist (or similar), but
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000965 # how can this be done from Python?
966 raise WindowsError(*e.args)
Tim Goldenad537f22010-08-08 11:18:16 +0000967 finally:
968 # Child is launched. Close the parent's copy of those pipe
969 # handles that only the child should have open. You need
970 # to make sure that no handles to the write end of the
971 # output pipe are maintained in this process or else the
972 # pipe will not close when the child process exits and the
973 # ReadFile will hang.
974 if p2cread != -1:
975 p2cread.Close()
976 if c2pwrite != -1:
977 c2pwrite.Close()
978 if errwrite != -1:
979 errwrite.Close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000980
981 # Retain the process handle, but close the thread handle
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000982 self._child_created = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000983 self._handle = hp
984 self.pid = pid
985 ht.Close()
986
Brett Cannon84df1e62010-05-14 00:33:40 +0000987 def _internal_poll(self, _deadstate=None,
Victor Stinnerc807a612010-05-14 21:53:45 +0000988 _WaitForSingleObject=_subprocess.WaitForSingleObject,
989 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
990 _GetExitCodeProcess=_subprocess.GetExitCodeProcess):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000991 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +0000992 attribute.
993
994 This method is called by __del__, so it can only refer to objects
995 in its local scope.
996
997 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000998 if self.returncode is None:
Brett Cannon84df1e62010-05-14 00:33:40 +0000999 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
1000 self.returncode = _GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001001 return self.returncode
1002
1003
1004 def wait(self):
1005 """Wait for child process to terminate. Returns returncode
1006 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001007 if self.returncode is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +00001008 _subprocess.WaitForSingleObject(self._handle,
1009 _subprocess.INFINITE)
1010 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001011 return self.returncode
1012
1013
1014 def _readerthread(self, fh, buffer):
1015 buffer.append(fh.read())
Victor Stinner667d4b52010-12-25 22:40:32 +00001016 fh.close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001017
1018
Peter Astrand23109f02005-03-03 20:28:59 +00001019 def _communicate(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001020 stdout = None # Return
1021 stderr = None # Return
1022
1023 if self.stdout:
1024 stdout = []
Tim Peterse8374a52004-10-13 03:15:00 +00001025 stdout_thread = threading.Thread(target=self._readerthread,
1026 args=(self.stdout, stdout))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001027 stdout_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001028 stdout_thread.start()
1029 if self.stderr:
1030 stderr = []
Tim Peterse8374a52004-10-13 03:15:00 +00001031 stderr_thread = threading.Thread(target=self._readerthread,
1032 args=(self.stderr, stderr))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001033 stderr_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001034 stderr_thread.start()
1035
1036 if self.stdin:
Peter Astrandd38ddf42005-02-10 08:32:50 +00001037 if input is not None:
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001038 try:
1039 self.stdin.write(input)
1040 except IOError as e:
1041 if e.errno != errno.EPIPE:
1042 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001043 self.stdin.close()
1044
1045 if self.stdout:
1046 stdout_thread.join()
1047 if self.stderr:
1048 stderr_thread.join()
1049
1050 # All data exchanged. Translate lists into strings.
Peter Astrandd38ddf42005-02-10 08:32:50 +00001051 if stdout is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001052 stdout = stdout[0]
Peter Astrandd38ddf42005-02-10 08:32:50 +00001053 if stderr is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001054 stderr = stderr[0]
1055
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001056 self.wait()
1057 return (stdout, stderr)
1058
Christian Heimesa342c012008-04-20 21:01:16 +00001059 def send_signal(self, sig):
1060 """Send a signal to the process
1061 """
1062 if sig == signal.SIGTERM:
1063 self.terminate()
Brian Curtineb24d742010-04-12 17:16:38 +00001064 elif sig == signal.CTRL_C_EVENT:
1065 os.kill(self.pid, signal.CTRL_C_EVENT)
1066 elif sig == signal.CTRL_BREAK_EVENT:
1067 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
Christian Heimesa342c012008-04-20 21:01:16 +00001068 else:
Brian Curtin19651362010-09-07 13:24:38 +00001069 raise ValueError("Unsupported signal: {}".format(sig))
Christian Heimesa342c012008-04-20 21:01:16 +00001070
1071 def terminate(self):
1072 """Terminates the process
1073 """
Brian Curtin1ce6b582010-04-24 16:19:22 +00001074 _subprocess.TerminateProcess(self._handle, 1)
Christian Heimesa342c012008-04-20 21:01:16 +00001075
1076 kill = terminate
1077
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001078 else:
1079 #
1080 # POSIX methods
1081 #
1082 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +00001083 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001084 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1085 """
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001086 p2cread, p2cwrite = -1, -1
1087 c2pread, c2pwrite = -1, -1
1088 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001089
Peter Astrandd38ddf42005-02-10 08:32:50 +00001090 if stdin is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001091 pass
1092 elif stdin == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001093 p2cread, p2cwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001094 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001095 p2cread = stdin
1096 else:
1097 # Assuming file-like object
1098 p2cread = stdin.fileno()
1099
Peter Astrandd38ddf42005-02-10 08:32:50 +00001100 if stdout is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001101 pass
1102 elif stdout == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001103 c2pread, c2pwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001104 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001105 c2pwrite = stdout
1106 else:
1107 # Assuming file-like object
Tim Peterse718f612004-10-12 21:51:32 +00001108 c2pwrite = stdout.fileno()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001109
Peter Astrandd38ddf42005-02-10 08:32:50 +00001110 if stderr is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001111 pass
1112 elif stderr == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001113 errread, errwrite = _create_pipe()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001114 elif stderr == STDOUT:
1115 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +00001116 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001117 errwrite = stderr
1118 else:
1119 # Assuming file-like object
1120 errwrite = stderr.fileno()
1121
1122 return (p2cread, p2cwrite,
1123 c2pread, c2pwrite,
1124 errread, errwrite)
1125
1126
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001127 def _close_fds(self, fds_to_keep):
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001128 start_fd = 3
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001129 for fd in sorted(fds_to_keep):
Gregory P. Smith8edd99d2010-12-14 13:43:30 +00001130 if fd >= start_fd:
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001131 os.closerange(start_fd, fd)
1132 start_fd = fd + 1
1133 if start_fd <= MAXFD:
1134 os.closerange(start_fd, MAXFD)
1135
1136
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001137 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001138 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001139 startupinfo, creationflags, shell,
1140 p2cread, p2cwrite,
1141 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001142 errread, errwrite,
1143 restore_signals, start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001144 """Execute program (POSIX version)"""
1145
Guido van Rossum3172c5d2007-10-16 18:12:55 +00001146 if isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001147 args = [args]
Thomas Wouters89f507f2006-12-13 04:49:30 +00001148 else:
1149 args = list(args)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001150
1151 if shell:
1152 args = ["/bin/sh", "-c"] + args
Stefan Krah9542cc62010-07-19 14:20:53 +00001153 if executable:
1154 args[0] = executable
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001155
Peter Astrandd38ddf42005-02-10 08:32:50 +00001156 if executable is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001157 executable = args[0]
1158
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001159 # For transferring possible exec failure from child to parent.
1160 # Data format: "exception name:hex errno:description"
1161 # Pickle is not used; it is complex and involves memory allocation.
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001162 errpipe_read, errpipe_write = _create_pipe()
Christian Heimesfdab48e2008-01-20 09:06:41 +00001163 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001164 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001165
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001166 if _posixsubprocess:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001167 # We must avoid complex work that could involve
1168 # malloc or free in the child process to avoid
1169 # potential deadlocks, thus we do all this here.
1170 # and pass it to fork_exec()
1171
Victor Stinnerf1512a22011-06-21 17:18:38 +02001172 if env is not None:
Victor Stinner449c4662010-05-08 11:10:09 +00001173 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001174 for k, v in env.items()]
1175 else:
1176 env_list = None # Use execv instead of execve.
Victor Stinnerb745a742010-05-18 17:17:23 +00001177 executable = os.fsencode(executable)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001178 if os.path.dirname(executable):
Victor Stinnerb745a742010-05-18 17:17:23 +00001179 executable_list = (executable,)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001180 else:
1181 # This matches the behavior of os._execvpe().
Victor Stinnerb745a742010-05-18 17:17:23 +00001182 executable_list = tuple(
1183 os.path.join(os.fsencode(dir), executable)
1184 for dir in os.get_exec_path(env))
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001185 fds_to_keep = set(pass_fds)
1186 fds_to_keep.add(errpipe_write)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001187 self.pid = _posixsubprocess.fork_exec(
1188 args, executable_list,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001189 close_fds, sorted(fds_to_keep), cwd, env_list,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001190 p2cread, p2cwrite, c2pread, c2pwrite,
1191 errread, errwrite,
1192 errpipe_read, errpipe_write,
1193 restore_signals, start_new_session, preexec_fn)
1194 else:
1195 # Pure Python implementation: It is not thread safe.
1196 # This implementation may deadlock in the child if your
1197 # parent process has any other threads running.
1198
1199 gc_was_enabled = gc.isenabled()
1200 # Disable gc to avoid bug where gc -> file_dealloc ->
1201 # write to stderr -> hang. See issue1336
1202 gc.disable()
1203 try:
1204 self.pid = os.fork()
1205 except:
1206 if gc_was_enabled:
1207 gc.enable()
1208 raise
1209 self._child_created = True
1210 if self.pid == 0:
1211 # Child
1212 try:
1213 # Close parent's pipe ends
1214 if p2cwrite != -1:
1215 os.close(p2cwrite)
1216 if c2pread != -1:
1217 os.close(c2pread)
1218 if errread != -1:
1219 os.close(errread)
1220 os.close(errpipe_read)
1221
Ross Lagerwalld98646e2011-07-27 07:16:31 +02001222 # When duping fds, if there arises a situation
1223 # where one of the fds is either 0, 1 or 2, it
1224 # is possible that it is overwritten (#12607).
1225 if c2pwrite == 0:
1226 c2pwrite = os.dup(c2pwrite)
1227 if errwrite == 0 or errwrite == 1:
1228 errwrite = os.dup(errwrite)
1229
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001230 # Dup fds for child
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001231 def _dup2(a, b):
1232 # dup2() removes the CLOEXEC flag but
1233 # we must do it ourselves if dup2()
1234 # would be a no-op (issue #10806).
1235 if a == b:
1236 _set_cloexec(a, False)
1237 elif a != -1:
1238 os.dup2(a, b)
1239 _dup2(p2cread, 0)
1240 _dup2(c2pwrite, 1)
1241 _dup2(errwrite, 2)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001242
1243 # Close pipe fds. Make sure we don't close the
1244 # same fd more than once, or standard fds.
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001245 closed = set()
1246 for fd in [p2cread, c2pwrite, errwrite]:
1247 if fd > 2 and fd not in closed:
1248 os.close(fd)
1249 closed.add(fd)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001250
1251 # Close all other fds, if asked for
1252 if close_fds:
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001253 fds_to_keep = set(pass_fds)
1254 fds_to_keep.add(errpipe_write)
1255 self._close_fds(fds_to_keep)
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001256
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001257
1258 if cwd is not None:
1259 os.chdir(cwd)
1260
1261 # This is a copy of Python/pythonrun.c
1262 # _Py_RestoreSignals(). If that were exposed
1263 # as a sys._py_restoresignals func it would be
1264 # better.. but this pure python implementation
1265 # isn't likely to be used much anymore.
1266 if restore_signals:
1267 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1268 for sig in signals:
1269 if hasattr(signal, sig):
1270 signal.signal(getattr(signal, sig),
1271 signal.SIG_DFL)
1272
1273 if start_new_session and hasattr(os, 'setsid'):
1274 os.setsid()
1275
1276 if preexec_fn:
1277 preexec_fn()
1278
1279 if env is None:
1280 os.execvp(executable, args)
1281 else:
1282 os.execvpe(executable, args, env)
1283
1284 except:
1285 try:
1286 exc_type, exc_value = sys.exc_info()[:2]
1287 if isinstance(exc_value, OSError):
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001288 errno_num = exc_value.errno
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001289 else:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001290 errno_num = 0
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001291 message = '%s:%x:%s' % (exc_type.__name__,
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001292 errno_num, exc_value)
Victor Stinner4d078042010-04-23 19:28:32 +00001293 message = message.encode(errors="surrogatepass")
1294 os.write(errpipe_write, message)
1295 except Exception:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001296 # We MUST not allow anything odd happening
1297 # above to prevent us from exiting below.
1298 pass
1299
1300 # This exitcode won't be reported to applications
1301 # so it really doesn't matter what we return.
1302 os._exit(255)
1303
1304 # Parent
Facundo Batista10706e22009-06-19 20:34:30 +00001305 if gc_was_enabled:
1306 gc.enable()
Facundo Batista10706e22009-06-19 20:34:30 +00001307 finally:
1308 # be sure the FD is closed no matter what
1309 os.close(errpipe_write)
1310
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001311 if p2cread != -1 and p2cwrite != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001312 os.close(p2cread)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001313 if c2pwrite != -1 and c2pread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001314 os.close(c2pwrite)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001315 if errwrite != -1 and errread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001316 os.close(errwrite)
1317
1318 # Wait for exec to fail or succeed; possibly raising an
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001319 # exception (limited in size)
1320 data = bytearray()
1321 while True:
1322 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1323 data += part
1324 if not part or len(data) > 50000:
1325 break
Facundo Batista10706e22009-06-19 20:34:30 +00001326 finally:
1327 # be sure the FD is closed no matter what
1328 os.close(errpipe_read)
1329
Guido van Rossumaf2362a2007-05-15 22:32:02 +00001330 if data:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001331 try:
1332 _eintr_retry_call(os.waitpid, self.pid, 0)
1333 except OSError as e:
1334 if e.errno != errno.ECHILD:
1335 raise
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001336 try:
1337 exception_name, hex_errno, err_msg = data.split(b':', 2)
1338 except ValueError:
1339 print('Bad exception data:', repr(data))
1340 exception_name = b'RuntimeError'
1341 hex_errno = b'0'
1342 err_msg = b'Unknown'
1343 child_exception_type = getattr(
1344 builtins, exception_name.decode('ascii'),
1345 RuntimeError)
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001346 for fd in (p2cwrite, c2pread, errread):
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001347 if fd != -1:
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001348 os.close(fd)
Victor Stinner4d078042010-04-23 19:28:32 +00001349 err_msg = err_msg.decode(errors="surrogatepass")
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001350 if issubclass(child_exception_type, OSError) and hex_errno:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001351 errno_num = int(hex_errno, 16)
1352 if errno_num != 0:
1353 err_msg = os.strerror(errno_num)
1354 if errno_num == errno.ENOENT:
Benjamin Peterson5f780402010-11-20 18:07:52 +00001355 err_msg += ': ' + repr(args[0])
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001356 raise child_exception_type(errno_num, err_msg)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001357 raise child_exception_type(err_msg)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001358
1359
Brett Cannon84df1e62010-05-14 00:33:40 +00001360 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1361 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1362 _WEXITSTATUS=os.WEXITSTATUS):
1363 # This method is called (indirectly) by __del__, so it cannot
1364 # refer to anything outside of its local scope."""
1365 if _WIFSIGNALED(sts):
1366 self.returncode = -_WTERMSIG(sts)
1367 elif _WIFEXITED(sts):
1368 self.returncode = _WEXITSTATUS(sts)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001369 else:
1370 # Should never happen
1371 raise RuntimeError("Unknown child exit status!")
1372
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001373
Brett Cannon84df1e62010-05-14 00:33:40 +00001374 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1375 _WNOHANG=os.WNOHANG, _os_error=os.error):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001376 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +00001377 attribute.
1378
1379 This method is called by __del__, so it cannot reference anything
1380 outside of the local scope (nor can any methods it calls).
1381
1382 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001383 if self.returncode is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001384 try:
Brett Cannon84df1e62010-05-14 00:33:40 +00001385 pid, sts = _waitpid(self.pid, _WNOHANG)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001386 if pid == self.pid:
1387 self._handle_exitstatus(sts)
Brett Cannon84df1e62010-05-14 00:33:40 +00001388 except _os_error:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +00001389 if _deadstate is not None:
1390 self.returncode = _deadstate
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001391 return self.returncode
1392
1393
1394 def wait(self):
1395 """Wait for child process to terminate. Returns returncode
1396 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001397 if self.returncode is None:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001398 try:
1399 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1400 except OSError as e:
1401 if e.errno != errno.ECHILD:
1402 raise
1403 # This happens if SIGCLD is set to be ignored or waiting
1404 # for child processes has otherwise been disabled for our
1405 # process. This child is dead, we can't get the status.
1406 sts = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001407 self._handle_exitstatus(sts)
1408 return self.returncode
1409
1410
Peter Astrand23109f02005-03-03 20:28:59 +00001411 def _communicate(self, input):
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001412 if self.stdin:
1413 # Flush stdio buffer. This might block, if the user has
1414 # been writing to .stdin in an uncontrolled fashion.
1415 self.stdin.flush()
1416 if not input:
1417 self.stdin.close()
1418
1419 if _has_poll:
1420 stdout, stderr = self._communicate_with_poll(input)
1421 else:
1422 stdout, stderr = self._communicate_with_select(input)
1423
1424 # All data exchanged. Translate lists into strings.
1425 if stdout is not None:
1426 stdout = b''.join(stdout)
1427 if stderr is not None:
1428 stderr = b''.join(stderr)
1429
1430 # Translate newlines, if requested.
1431 # This also turns bytes into strings.
1432 if self.universal_newlines:
1433 if stdout is not None:
1434 stdout = self._translate_newlines(stdout,
1435 self.stdout.encoding)
1436 if stderr is not None:
1437 stderr = self._translate_newlines(stderr,
1438 self.stderr.encoding)
1439
1440 self.wait()
1441 return (stdout, stderr)
1442
1443
1444 def _communicate_with_poll(self, input):
1445 stdout = None # Return
1446 stderr = None # Return
1447 fd2file = {}
1448 fd2output = {}
1449
1450 poller = select.poll()
1451 def register_and_append(file_obj, eventmask):
1452 poller.register(file_obj.fileno(), eventmask)
1453 fd2file[file_obj.fileno()] = file_obj
1454
1455 def close_unregister_and_remove(fd):
1456 poller.unregister(fd)
1457 fd2file[fd].close()
1458 fd2file.pop(fd)
1459
1460 if self.stdin and input:
1461 register_and_append(self.stdin, select.POLLOUT)
1462
1463 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1464 if self.stdout:
1465 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1466 fd2output[self.stdout.fileno()] = stdout = []
1467 if self.stderr:
1468 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1469 fd2output[self.stderr.fileno()] = stderr = []
1470
1471 input_offset = 0
1472 while fd2file:
1473 try:
1474 ready = poller.poll()
1475 except select.error as e:
1476 if e.args[0] == errno.EINTR:
1477 continue
1478 raise
1479
1480 # XXX Rewrite these to use non-blocking I/O on the
1481 # file objects; they are no longer using C stdio!
1482
1483 for fd, mode in ready:
1484 if mode & select.POLLOUT:
1485 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001486 try:
1487 input_offset += os.write(fd, chunk)
1488 except OSError as e:
1489 if e.errno == errno.EPIPE:
1490 close_unregister_and_remove(fd)
1491 else:
1492 raise
1493 else:
1494 if input_offset >= len(input):
1495 close_unregister_and_remove(fd)
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001496 elif mode & select_POLLIN_POLLPRI:
1497 data = os.read(fd, 4096)
1498 if not data:
1499 close_unregister_and_remove(fd)
1500 fd2output[fd].append(data)
1501 else:
1502 # Ignore hang up or errors.
1503 close_unregister_and_remove(fd)
1504
1505 return (stdout, stderr)
1506
1507
1508 def _communicate_with_select(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001509 read_set = []
1510 write_set = []
1511 stdout = None # Return
1512 stderr = None # Return
1513
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001514 if self.stdin and input:
1515 write_set.append(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001516 if self.stdout:
1517 read_set.append(self.stdout)
Tim Peterse718f612004-10-12 21:51:32 +00001518 stdout = []
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001519 if self.stderr:
1520 read_set.append(self.stderr)
1521 stderr = []
1522
Thomas Wouters902d6eb2007-01-09 23:18:33 +00001523 input_offset = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001524 while read_set or write_set:
Georg Brandl86b2fb92008-07-16 03:43:04 +00001525 try:
1526 rlist, wlist, xlist = select.select(read_set, write_set, [])
1527 except select.error as e:
1528 if e.args[0] == errno.EINTR:
1529 continue
1530 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001531
Guido van Rossum98297ee2007-11-06 21:34:58 +00001532 # XXX Rewrite these to use non-blocking I/O on the
1533 # file objects; they are no longer using C stdio!
1534
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001535 if self.stdin in wlist:
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001536 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001537 try:
1538 bytes_written = os.write(self.stdin.fileno(), chunk)
1539 except OSError as e:
1540 if e.errno == errno.EPIPE:
1541 self.stdin.close()
1542 write_set.remove(self.stdin)
1543 else:
1544 raise
1545 else:
1546 input_offset += bytes_written
1547 if input_offset >= len(input):
1548 self.stdin.close()
1549 write_set.remove(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001550
1551 if self.stdout in rlist:
1552 data = os.read(self.stdout.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001553 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001554 self.stdout.close()
1555 read_set.remove(self.stdout)
1556 stdout.append(data)
1557
1558 if self.stderr in rlist:
1559 data = os.read(self.stderr.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001560 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001561 self.stderr.close()
1562 read_set.remove(self.stderr)
1563 stderr.append(data)
1564
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001565 return (stdout, stderr)
1566
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001567
Christian Heimesa342c012008-04-20 21:01:16 +00001568 def send_signal(self, sig):
1569 """Send a signal to the process
1570 """
1571 os.kill(self.pid, sig)
1572
1573 def terminate(self):
1574 """Terminate the process with SIGTERM
1575 """
1576 self.send_signal(signal.SIGTERM)
1577
1578 def kill(self):
1579 """Kill the process with SIGKILL
1580 """
1581 self.send_signal(signal.SIGKILL)
1582
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001583
1584def _demo_posix():
1585 #
1586 # Example 1: Simple redirection: Get process list
1587 #
1588 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001589 print("Process list:")
1590 print(plist)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001591
1592 #
1593 # Example 2: Change uid before executing child
1594 #
1595 if os.getuid() == 0:
1596 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1597 p.wait()
1598
1599 #
1600 # Example 3: Connecting several subprocesses
1601 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001602 print("Looking for 'hda'...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001603 p1 = Popen(["dmesg"], stdout=PIPE)
1604 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001605 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001606
1607 #
1608 # Example 4: Catch execution error
1609 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001610 print()
1611 print("Trying a weird file...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001612 try:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001613 print(Popen(["/this/path/does/not/exist"]).communicate())
Guido van Rossumb940e112007-01-10 16:19:56 +00001614 except OSError as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001615 if e.errno == errno.ENOENT:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001616 print("The file didn't exist. I thought so...")
1617 print("Child traceback:")
1618 print(e.child_traceback)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001619 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001620 print("Error", e.errno)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001621 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001622 print("Gosh. No error.", file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001623
1624
1625def _demo_windows():
1626 #
1627 # Example 1: Connecting several subprocesses
1628 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001629 print("Looking for 'PROMPT' in set output...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001630 p1 = Popen("set", stdout=PIPE, shell=True)
1631 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001632 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001633
1634 #
1635 # Example 2: Simple execution of program
1636 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001637 print("Executing calc...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001638 p = Popen("calc")
1639 p.wait()
1640
1641
1642if __name__ == "__main__":
1643 if mswindows:
1644 _demo_windows()
1645 else:
1646 _demo_posix()