blob: b3fb9355c9bd3ea032ed009ab44f18ac074b6127 [file] [log] [blame]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001# subprocess - Subprocesses with accessible I/O streams
2#
Tim Peterse718f612004-10-12 21:51:32 +00003# For more information about this module, see PEP 324.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00004#
Peter Astrand3a708df2005-09-23 17:37:29 +00005# Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00006#
Peter Astrand69bf13f2005-02-14 08:56:32 +00007# Licensed to PSF under a Contributor Agreement.
Peter Astrand3a708df2005-09-23 17:37:29 +00008# See http://www.python.org/2.4/license for licensing details.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00009
Raymond Hettinger837dd932004-10-17 16:36:53 +000010r"""subprocess - Subprocesses with accessible I/O streams
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000011
Fredrik Lundh15aaacc2004-10-17 14:47:05 +000012This module allows you to spawn processes, connect to their
13input/output/error pipes, and obtain their return codes. This module
14intends to replace several other, older modules and functions, like:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000015
16os.system
17os.spawn*
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000018
19Information about how the subprocess module can be used to replace these
20modules and functions can be found below.
21
22
23
24Using the subprocess module
25===========================
26This module defines one class called Popen:
27
28class Popen(args, bufsize=0, executable=None,
29 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000030 preexec_fn=None, close_fds=True, shell=False,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000031 cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000032 startupinfo=None, creationflags=0,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000033 restore_signals=True, start_new_session=False, pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000034
35
36Arguments are:
37
38args should be a string, or a sequence of program arguments. The
39program to execute is normally the first item in the args sequence or
40string, but can be explicitly set by using the executable argument.
41
Gregory P. Smithf5604852010-12-13 06:45:02 +000042On POSIX, with shell=False (default): In this case, the Popen class
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000043uses os.execvp() to execute the child program. args should normally
44be a sequence. A string will be treated as a sequence with the string
45as the only item (the program to execute).
46
Gregory P. Smithf5604852010-12-13 06:45:02 +000047On POSIX, with shell=True: If args is a string, it specifies the
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000048command string to execute through the shell. If args is a sequence,
49the first item specifies the command string, and any additional items
50will be treated as additional shell arguments.
51
52On Windows: the Popen class uses CreateProcess() to execute the child
53program, which operates on strings. If args is a sequence, it will be
54converted to a string using the list2cmdline method. Please note that
55not all MS Windows applications interpret the command line the same
56way: The list2cmdline is designed for applications using the same
57rules as the MS C runtime.
58
59bufsize, if given, has the same meaning as the corresponding argument
60to the built-in open() function: 0 means unbuffered, 1 means line
61buffered, any other positive value means use a buffer of
62(approximately) that size. A negative bufsize means to use the system
63default, which usually means fully buffered. The default value for
64bufsize is 0 (unbuffered).
65
66stdin, stdout and stderr specify the executed programs' standard
67input, standard output and standard error file handles, respectively.
68Valid values are PIPE, an existing file descriptor (a positive
69integer), an existing file object, and None. PIPE indicates that a
70new pipe to the child should be created. With None, no redirection
71will occur; the child's file handles will be inherited from the
72parent. Additionally, stderr can be STDOUT, which indicates that the
73stderr data from the applications should be captured into the same
74file handle as for stdout.
75
Gregory P. Smithf5604852010-12-13 06:45:02 +000076On POSIX, if preexec_fn is set to a callable object, this object will be
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000077called in the child process just before the child is executed. The use
78of preexec_fn is not thread safe, using it in the presence of threads
79could lead to a deadlock in the child process before the new executable
80is executed.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000081
82If close_fds is true, all file descriptors except 0, 1 and 2 will be
Gregory P. Smithf5604852010-12-13 06:45:02 +000083closed before the child process is executed. The default for close_fds
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000084varies by platform: Always true on POSIX. True when stdin/stdout/stderr
85are None on Windows, false otherwise.
86
87pass_fds is an optional sequence of file descriptors to keep open between the
88parent and child. Providing any pass_fds implicitly sets close_fds to true.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +000089
90if shell is true, the specified command will be executed through the
91shell.
92
93If cwd is not None, the current directory will be changed to cwd
94before the child is executed.
95
Gregory P. Smithf5604852010-12-13 06:45:02 +000096On POSIX, if restore_signals is True all signals that Python sets to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000097SIG_IGN are restored to SIG_DFL in the child process before the exec.
98Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. This
99parameter does nothing on Windows.
100
Gregory P. Smithf5604852010-12-13 06:45:02 +0000101On POSIX, if start_new_session is True, the setsid() system call will be made
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000102in the child process prior to executing the command.
103
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000104If env is not None, it defines the environment variables for the new
105process.
106
107If universal_newlines is true, the file objects stdout and stderr are
108opened as a text files, but lines may be terminated by any of '\n',
Gregory P. Smithf5604852010-12-13 06:45:02 +0000109the Unix end-of-line convention, '\r', the old Macintosh convention or
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000110'\r\n', the Windows convention. All of these external representations
111are seen as '\n' by the Python program. Note: This feature is only
112available if Python is built with universal newline support (the
113default). Also, the newlines attribute of the file objects stdout,
114stdin and stderr are not updated by the communicate() method.
115
116The startupinfo and creationflags, if given, will be passed to the
117underlying CreateProcess() function. They can specify things such as
118appearance of the main window and priority for the new process.
119(Windows only)
120
121
Georg Brandlf9734072008-12-07 15:30:06 +0000122This module also defines some shortcut functions:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000123
Peter Astrand5f5e1412004-12-05 20:15:36 +0000124call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000125 Run command with arguments. Wait for command to complete, then
126 return the returncode attribute.
127
128 The arguments are the same as for the Popen constructor. Example:
129
Florent Xicluna4886d242010-03-08 13:27:26 +0000130 >>> retcode = subprocess.call(["ls", "-l"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000131
Peter Astrand454f7672005-01-01 09:36:35 +0000132check_call(*popenargs, **kwargs):
133 Run command with arguments. Wait for command to complete. If the
134 exit code was zero then return, otherwise raise
135 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000136 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000137
138 The arguments are the same as for the Popen constructor. Example:
139
Florent Xicluna4886d242010-03-08 13:27:26 +0000140 >>> subprocess.check_call(["ls", "-l"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000141 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000142
Brett Cannona23810f2008-05-26 19:04:21 +0000143getstatusoutput(cmd):
144 Return (status, output) of executing cmd in a shell.
145
146 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
147 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
148 returned output will contain output or error messages. A trailing newline
149 is stripped from the output. The exit status for the command can be
150 interpreted according to the rules for the C function wait(). Example:
151
Brett Cannona23810f2008-05-26 19:04:21 +0000152 >>> subprocess.getstatusoutput('ls /bin/ls')
153 (0, '/bin/ls')
154 >>> subprocess.getstatusoutput('cat /bin/junk')
155 (256, 'cat: /bin/junk: No such file or directory')
156 >>> subprocess.getstatusoutput('/bin/junk')
157 (256, 'sh: /bin/junk: not found')
158
159getoutput(cmd):
160 Return output (stdout or stderr) of executing cmd in a shell.
161
162 Like getstatusoutput(), except the exit status is ignored and the return
163 value is a string containing the command's output. Example:
164
Brett Cannona23810f2008-05-26 19:04:21 +0000165 >>> subprocess.getoutput('ls /bin/ls')
166 '/bin/ls'
167
Georg Brandlf9734072008-12-07 15:30:06 +0000168check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000169 Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000170
Georg Brandl2708f3a2009-12-20 14:38:23 +0000171 If the exit code was non-zero it raises a CalledProcessError. The
172 CalledProcessError object will have the return code in the returncode
173 attribute and output in the output attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000174
Georg Brandl2708f3a2009-12-20 14:38:23 +0000175 The arguments are the same as for the Popen constructor. Example:
Georg Brandlf9734072008-12-07 15:30:06 +0000176
Georg Brandl2708f3a2009-12-20 14:38:23 +0000177 >>> output = subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandlf9734072008-12-07 15:30:06 +0000178
Brett Cannona23810f2008-05-26 19:04:21 +0000179
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000180Exceptions
181----------
182Exceptions raised in the child process, before the new program has
183started to execute, will be re-raised in the parent. Additionally,
184the exception object will have one extra attribute called
185'child_traceback', which is a string containing traceback information
186from the childs point of view.
187
188The most common exception raised is OSError. This occurs, for
189example, when trying to execute a non-existent file. Applications
190should prepare for OSErrors.
191
192A ValueError will be raised if Popen is called with invalid arguments.
193
Georg Brandlf9734072008-12-07 15:30:06 +0000194check_call() and check_output() will raise CalledProcessError, if the
195called process returns a non-zero return code.
Peter Astrand454f7672005-01-01 09:36:35 +0000196
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000197
198Security
199--------
200Unlike some other popen functions, this implementation will never call
201/bin/sh implicitly. This means that all characters, including shell
202metacharacters, can safely be passed to child processes.
203
204
205Popen objects
206=============
207Instances of the Popen class have the following methods:
208
209poll()
210 Check if child process has terminated. Returns returncode
211 attribute.
212
213wait()
214 Wait for child process to terminate. Returns returncode attribute.
215
216communicate(input=None)
217 Interact with process: Send data to stdin. Read data from stdout
218 and stderr, until end-of-file is reached. Wait for process to
Thomas Wouters902d6eb2007-01-09 23:18:33 +0000219 terminate. The optional input argument should be a string to be
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000220 sent to the child process, or None, if no data should be sent to
221 the child.
Tim Peterse718f612004-10-12 21:51:32 +0000222
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000223 communicate() returns a tuple (stdout, stderr).
224
225 Note: The data read is buffered in memory, so do not use this
226 method if the data size is large or unlimited.
227
228The following attributes are also available:
229
230stdin
231 If the stdin argument is PIPE, this attribute is a file object
232 that provides input to the child process. Otherwise, it is None.
233
234stdout
235 If the stdout argument is PIPE, this attribute is a file object
236 that provides output from the child process. Otherwise, it is
237 None.
238
239stderr
240 If the stderr argument is PIPE, this attribute is file object that
241 provides error output from the child process. Otherwise, it is
242 None.
243
244pid
245 The process ID of the child process.
246
247returncode
248 The child return code. A None value indicates that the process
249 hasn't terminated yet. A negative value -N indicates that the
Gregory P. Smithf5604852010-12-13 06:45:02 +0000250 child was terminated by signal N (POSIX only).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000251
252
253Replacing older functions with the subprocess module
254====================================================
255In this section, "a ==> b" means that b can be used as a replacement
256for a.
257
258Note: All functions in this section fail (more or less) silently if
259the executed program cannot be found; this module raises an OSError
260exception.
261
262In the following examples, we assume that the subprocess module is
263imported with "from subprocess import *".
264
265
266Replacing /bin/sh shell backquote
267---------------------------------
268output=`mycmd myarg`
269==>
270output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
271
272
273Replacing shell pipe line
274-------------------------
275output=`dmesg | grep hda`
276==>
277p1 = Popen(["dmesg"], stdout=PIPE)
Peter Astrand6fdf3cb2004-11-30 18:06:42 +0000278p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000279output = p2.communicate()[0]
280
281
282Replacing os.system()
283---------------------
284sts = os.system("mycmd" + " myarg")
285==>
286p = Popen("mycmd" + " myarg", shell=True)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000287pid, sts = os.waitpid(p.pid, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000288
289Note:
290
291* Calling the program through the shell is usually not required.
292
293* It's easier to look at the returncode attribute than the
294 exitstatus.
295
296A more real-world example would look like this:
297
298try:
299 retcode = call("mycmd" + " myarg", shell=True)
300 if retcode < 0:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000301 print("Child was terminated by signal", -retcode, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000302 else:
Guido van Rossumc2f93dc2007-05-24 00:50:02 +0000303 print("Child returned", retcode, file=sys.stderr)
304except OSError as e:
305 print("Execution failed:", e, file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000306
307
308Replacing os.spawn*
309-------------------
Tim Peterse718f612004-10-12 21:51:32 +0000310P_NOWAIT example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000311
312pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
313==>
314pid = Popen(["/bin/mycmd", "myarg"]).pid
315
316
317P_WAIT example:
318
319retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
320==>
321retcode = call(["/bin/mycmd", "myarg"])
322
323
Tim Peterse718f612004-10-12 21:51:32 +0000324Vector example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000325
326os.spawnvp(os.P_NOWAIT, path, args)
327==>
328Popen([path] + args[1:])
329
330
Tim Peterse718f612004-10-12 21:51:32 +0000331Environment example:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000332
333os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
334==>
335Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000336"""
337
338import sys
339mswindows = (sys.platform == "win32")
340
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000341import io
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000342import os
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000343import traceback
Christian Heimesfdab48e2008-01-20 09:06:41 +0000344import gc
Christian Heimesa342c012008-04-20 21:01:16 +0000345import signal
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000346import builtins
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000347import warnings
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200348import errno
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000349
Peter Astrand454f7672005-01-01 09:36:35 +0000350# Exception classes used by this module.
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000351class CalledProcessError(Exception):
Georg Brandlf9734072008-12-07 15:30:06 +0000352 """This exception is raised when a process run by check_call() or
353 check_output() returns a non-zero exit status.
354 The exit status will be stored in the returncode attribute;
355 check_output() will also store the output in the output attribute.
356 """
357 def __init__(self, returncode, cmd, output=None):
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000358 self.returncode = returncode
359 self.cmd = cmd
Georg Brandlf9734072008-12-07 15:30:06 +0000360 self.output = output
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000361 def __str__(self):
362 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
363
Peter Astrand454f7672005-01-01 09:36:35 +0000364
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000365if mswindows:
366 import threading
367 import msvcrt
Brian Curtin1ce6b582010-04-24 16:19:22 +0000368 import _subprocess
369 class STARTUPINFO:
370 dwFlags = 0
371 hStdInput = None
372 hStdOutput = None
373 hStdError = None
374 wShowWindow = 0
375 class pywintypes:
376 error = IOError
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000377else:
378 import select
Gregory P. Smithd06fa472009-07-04 02:46:54 +0000379 _has_poll = hasattr(select, 'poll')
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000380 import fcntl
381 import pickle
382
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000383 try:
384 import _posixsubprocess
385 except ImportError:
386 _posixsubprocess = None
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000387 warnings.warn("The _posixsubprocess module is not being used. "
388 "Child process reliability may suffer if your "
389 "program uses threads.", RuntimeWarning)
390
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000391 # When select or poll has indicated that the file is writable,
392 # we can write up to _PIPE_BUF bytes without risk of blocking.
393 # POSIX defines PIPE_BUF as >= 512.
394 _PIPE_BUF = getattr(select, 'PIPE_BUF', 512)
395
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000396 _FD_CLOEXEC = getattr(fcntl, 'FD_CLOEXEC', 1)
397
398 def _set_cloexec(fd, cloexec):
399 old = fcntl.fcntl(fd, fcntl.F_GETFD)
400 if cloexec:
401 fcntl.fcntl(fd, fcntl.F_SETFD, old | _FD_CLOEXEC)
402 else:
403 fcntl.fcntl(fd, fcntl.F_SETFD, old & ~_FD_CLOEXEC)
404
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000405 if _posixsubprocess:
406 _create_pipe = _posixsubprocess.cloexec_pipe
407 else:
408 def _create_pipe():
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000409 fds = os.pipe()
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +0000410 _set_cloexec(fds[0], True)
411 _set_cloexec(fds[1], True)
Gregory P. Smith51ee2702010-12-13 07:59:39 +0000412 return fds
Amaury Forgeot d'Arcace31022009-07-09 22:44:11 +0000413
Brett Cannona23810f2008-05-26 19:04:21 +0000414__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput",
Georg Brandlf9734072008-12-07 15:30:06 +0000415 "getoutput", "check_output", "CalledProcessError"]
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000416
Brian Curtin1ce6b582010-04-24 16:19:22 +0000417if mswindows:
Brett Cannon84df1e62010-05-14 00:33:40 +0000418 from _subprocess import CREATE_NEW_CONSOLE, CREATE_NEW_PROCESS_GROUP
Brian Curtin1ce6b582010-04-24 16:19:22 +0000419 __all__.extend(["CREATE_NEW_CONSOLE", "CREATE_NEW_PROCESS_GROUP"])
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000420try:
421 MAXFD = os.sysconf("SC_OPEN_MAX")
422except:
423 MAXFD = 256
424
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000425_active = []
426
427def _cleanup():
428 for inst in _active[:]:
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000429 res = inst._internal_poll(_deadstate=sys.maxsize)
Guido van Rossumb5d47ef2006-08-24 02:27:45 +0000430 if res is not None and res >= 0:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000431 try:
432 _active.remove(inst)
433 except ValueError:
434 # This can happen if two threads create a new Popen instance.
435 # It's harmless that it was already removed, so ignore.
436 pass
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000437
438PIPE = -1
439STDOUT = -2
440
441
Gregory P. Smitha59c59f2010-03-01 00:17:40 +0000442def _eintr_retry_call(func, *args):
443 while True:
444 try:
445 return func(*args)
446 except OSError as e:
447 if e.errno == errno.EINTR:
448 continue
449 raise
450
451
Peter Astrand5f5e1412004-12-05 20:15:36 +0000452def call(*popenargs, **kwargs):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000453 """Run command with arguments. Wait for command to complete, then
454 return the returncode attribute.
455
456 The arguments are the same as for the Popen constructor. Example:
457
458 retcode = call(["ls", "-l"])
459 """
Peter Astrand5f5e1412004-12-05 20:15:36 +0000460 return Popen(*popenargs, **kwargs).wait()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000461
462
Peter Astrand454f7672005-01-01 09:36:35 +0000463def check_call(*popenargs, **kwargs):
464 """Run command with arguments. Wait for command to complete. If
465 the exit code was zero then return, otherwise raise
466 CalledProcessError. The CalledProcessError object will have the
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000467 return code in the returncode attribute.
Peter Astrand454f7672005-01-01 09:36:35 +0000468
469 The arguments are the same as for the Popen constructor. Example:
470
471 check_call(["ls", "-l"])
472 """
473 retcode = call(*popenargs, **kwargs)
Peter Astrand454f7672005-01-01 09:36:35 +0000474 if retcode:
Georg Brandlf9734072008-12-07 15:30:06 +0000475 cmd = kwargs.get("args")
476 if cmd is None:
477 cmd = popenargs[0]
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000478 raise CalledProcessError(retcode, cmd)
Georg Brandlf9734072008-12-07 15:30:06 +0000479 return 0
480
481
482def check_output(*popenargs, **kwargs):
Georg Brandl2708f3a2009-12-20 14:38:23 +0000483 r"""Run command with arguments and return its output as a byte string.
Georg Brandlf9734072008-12-07 15:30:06 +0000484
485 If the exit code was non-zero it raises a CalledProcessError. The
486 CalledProcessError object will have the return code in the returncode
487 attribute and output in the output attribute.
488
489 The arguments are the same as for the Popen constructor. Example:
490
491 >>> check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000492 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000493
494 The stdout argument is not allowed as it is used internally.
Georg Brandl127d4702009-12-28 08:10:38 +0000495 To capture standard error in the result, use stderr=STDOUT.
Georg Brandlf9734072008-12-07 15:30:06 +0000496
497 >>> check_output(["/bin/sh", "-c",
Georg Brandl2708f3a2009-12-20 14:38:23 +0000498 ... "ls -l non_existent_file ; exit 0"],
Georg Brandl127d4702009-12-28 08:10:38 +0000499 ... stderr=STDOUT)
Georg Brandl2708f3a2009-12-20 14:38:23 +0000500 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000501 """
502 if 'stdout' in kwargs:
503 raise ValueError('stdout argument not allowed, it will be overridden.')
504 process = Popen(*popenargs, stdout=PIPE, **kwargs)
505 output, unused_err = process.communicate()
506 retcode = process.poll()
507 if retcode:
508 cmd = kwargs.get("args")
509 if cmd is None:
510 cmd = popenargs[0]
511 raise CalledProcessError(retcode, cmd, output=output)
512 return output
Peter Astrand454f7672005-01-01 09:36:35 +0000513
514
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000515def list2cmdline(seq):
516 """
517 Translate a sequence of arguments into a command line
518 string, using the same rules as the MS C runtime:
519
520 1) Arguments are delimited by white space, which is either a
521 space or a tab.
522
523 2) A string surrounded by double quotation marks is
524 interpreted as a single argument, regardless of white space
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000525 contained within. A quoted string can be embedded in an
526 argument.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000527
528 3) A double quotation mark preceded by a backslash is
529 interpreted as a literal double quotation mark.
530
531 4) Backslashes are interpreted literally, unless they
532 immediately precede a double quotation mark.
533
534 5) If backslashes immediately precede a double quotation mark,
535 every pair of backslashes is interpreted as a literal
536 backslash. If the number of backslashes is odd, the last
537 backslash escapes the next double quotation mark as
538 described in rule 3.
539 """
540
541 # See
Eric Smith3c573af2009-11-09 15:23:15 +0000542 # http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
543 # or search http://msdn.microsoft.com for
544 # "Parsing C++ Command-Line Arguments"
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000545 result = []
546 needquote = False
547 for arg in seq:
548 bs_buf = []
549
550 # Add a space to separate this argument from the others
551 if result:
552 result.append(' ')
553
Jean-Paul Calderone1ddd4072010-06-18 20:03:54 +0000554 needquote = (" " in arg) or ("\t" in arg) or not arg
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000555 if needquote:
556 result.append('"')
557
558 for c in arg:
559 if c == '\\':
Tim Peterse718f612004-10-12 21:51:32 +0000560 # Don't know if we need to double yet.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000561 bs_buf.append(c)
562 elif c == '"':
Christian Heimesfdab48e2008-01-20 09:06:41 +0000563 # Double backslashes.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000564 result.append('\\' * len(bs_buf)*2)
565 bs_buf = []
566 result.append('\\"')
567 else:
568 # Normal char
569 if bs_buf:
570 result.extend(bs_buf)
571 bs_buf = []
572 result.append(c)
573
Christian Heimesfdab48e2008-01-20 09:06:41 +0000574 # Add remaining backslashes, if any.
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000575 if bs_buf:
576 result.extend(bs_buf)
577
578 if needquote:
Peter Astrand7e78ade2005-03-03 21:10:23 +0000579 result.extend(bs_buf)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000580 result.append('"')
581
582 return ''.join(result)
583
584
Brett Cannona23810f2008-05-26 19:04:21 +0000585# Various tools for executing commands and looking at their output and status.
586#
Gregory P. Smithf5604852010-12-13 06:45:02 +0000587# NB This only works (and is only relevant) for POSIX.
Brett Cannona23810f2008-05-26 19:04:21 +0000588
589def getstatusoutput(cmd):
590 """Return (status, output) of executing cmd in a shell.
591
592 Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple
593 (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the
594 returned output will contain output or error messages. A trailing newline
595 is stripped from the output. The exit status for the command can be
596 interpreted according to the rules for the C function wait(). Example:
597
598 >>> import subprocess
599 >>> subprocess.getstatusoutput('ls /bin/ls')
600 (0, '/bin/ls')
601 >>> subprocess.getstatusoutput('cat /bin/junk')
602 (256, 'cat: /bin/junk: No such file or directory')
603 >>> subprocess.getstatusoutput('/bin/junk')
604 (256, 'sh: /bin/junk: not found')
605 """
606 pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r')
607 text = pipe.read()
608 sts = pipe.close()
609 if sts is None: sts = 0
610 if text[-1:] == '\n': text = text[:-1]
611 return sts, text
612
613
614def getoutput(cmd):
615 """Return output (stdout or stderr) of executing cmd in a shell.
616
617 Like getstatusoutput(), except the exit status is ignored and the return
618 value is a string containing the command's output. Example:
619
620 >>> import subprocess
621 >>> subprocess.getoutput('ls /bin/ls')
622 '/bin/ls'
623 """
624 return getstatusoutput(cmd)[1]
625
626
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000627_PLATFORM_DEFAULT_CLOSE_FDS = object()
Gregory P. Smithf5604852010-12-13 06:45:02 +0000628
629
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000630class Popen(object):
631 def __init__(self, args, bufsize=0, executable=None,
632 stdin=None, stdout=None, stderr=None,
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000633 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
634 shell=False, cwd=None, env=None, universal_newlines=False,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000635 startupinfo=None, creationflags=0,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000636 restore_signals=True, start_new_session=False,
637 pass_fds=()):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000638 """Create new Popen instance."""
639 _cleanup()
640
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000641 self._child_created = False
Guido van Rossum46a05a72007-06-07 21:56:45 +0000642 if bufsize is None:
643 bufsize = 0 # Restore default
Walter Dörwaldaa97f042007-05-03 21:05:51 +0000644 if not isinstance(bufsize, int):
Peter Astrand738131d2004-11-30 21:04:45 +0000645 raise TypeError("bufsize must be an integer")
646
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000647 if mswindows:
Tim Peterse8374a52004-10-13 03:15:00 +0000648 if preexec_fn is not None:
649 raise ValueError("preexec_fn is not supported on Windows "
650 "platforms")
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000651 any_stdio_set = (stdin is not None or stdout is not None or
652 stderr is not None)
653 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
654 if any_stdio_set:
655 close_fds = False
656 else:
657 close_fds = True
658 elif close_fds and any_stdio_set:
659 raise ValueError(
660 "close_fds is not supported on Windows platforms"
661 " if you redirect stdin/stdout/stderr")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000662 else:
663 # POSIX
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000664 if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS:
665 close_fds = True
666 if pass_fds and not close_fds:
667 warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
668 close_fds = True
Tim Peterse8374a52004-10-13 03:15:00 +0000669 if startupinfo is not None:
670 raise ValueError("startupinfo is only supported on Windows "
671 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000672 if creationflags != 0:
Tim Peterse8374a52004-10-13 03:15:00 +0000673 raise ValueError("creationflags is only supported on Windows "
674 "platforms")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000675
Tim Peterse718f612004-10-12 21:51:32 +0000676 self.stdin = None
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000677 self.stdout = None
678 self.stderr = None
679 self.pid = None
680 self.returncode = None
681 self.universal_newlines = universal_newlines
682
683 # Input and output objects. The general principle is like
684 # this:
685 #
686 # Parent Child
687 # ------ -----
688 # p2cwrite ---stdin---> p2cread
689 # c2pread <--stdout--- c2pwrite
690 # errread <--stderr--- errwrite
691 #
692 # On POSIX, the child objects are file descriptors. On
693 # Windows, these are Windows file handles. The parent objects
694 # are file descriptors on both platforms. The parent objects
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000695 # are -1 when not using PIPEs. The child objects are -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000696 # when not redirecting.
Tim Peterse718f612004-10-12 21:51:32 +0000697
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000698 (p2cread, p2cwrite,
699 c2pread, c2pwrite,
700 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
701
Antoine Pitrouc9982322011-01-04 19:07:07 +0000702 # We wrap OS handles *before* launching the child, otherwise a
703 # quickly terminating child could make our fds unwrappable
704 # (see #8458).
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000705
Thomas Wouterscf297e42007-02-23 15:07:44 +0000706 if mswindows:
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000707 if p2cwrite != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000708 p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000709 if c2pread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000710 c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
Florent Xicluna3b8bfef2010-03-14 12:31:06 +0000711 if errread != -1:
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000712 errread = msvcrt.open_osfhandle(errread.Detach(), 0)
Thomas Wouterscf297e42007-02-23 15:07:44 +0000713
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000714 if p2cwrite != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000715 self.stdin = io.open(p2cwrite, 'wb', bufsize)
716 if self.universal_newlines:
717 self.stdin = io.TextIOWrapper(self.stdin)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000718 if c2pread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000719 self.stdout = io.open(c2pread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000720 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000721 self.stdout = io.TextIOWrapper(self.stdout)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000722 if errread != -1:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000723 self.stderr = io.open(errread, 'rb', bufsize)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000724 if universal_newlines:
Guido van Rossumfa0054a2007-05-24 04:05:35 +0000725 self.stderr = io.TextIOWrapper(self.stderr)
Tim Peterse718f612004-10-12 21:51:32 +0000726
Antoine Pitrouc9982322011-01-04 19:07:07 +0000727 try:
728 self._execute_child(args, executable, preexec_fn, close_fds,
729 pass_fds, cwd, env, universal_newlines,
730 startupinfo, creationflags, shell,
731 p2cread, p2cwrite,
732 c2pread, c2pwrite,
733 errread, errwrite,
734 restore_signals, start_new_session)
735 except:
736 # Cleanup if the child failed starting
737 for f in filter(None, [self.stdin, self.stdout, self.stderr]):
738 try:
739 f.close()
740 except EnvironmentError:
741 # Ignore EBADF or other errors
742 pass
743 raise
744
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000745
Guido van Rossum98297ee2007-11-06 21:34:58 +0000746 def _translate_newlines(self, data, encoding):
747 data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")
748 return data.decode(encoding)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000749
Brian Curtin79cdb662010-12-03 02:46:02 +0000750 def __enter__(self):
751 return self
752
753 def __exit__(self, type, value, traceback):
754 if self.stdout:
755 self.stdout.close()
756 if self.stderr:
757 self.stderr.close()
758 if self.stdin:
759 self.stdin.close()
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000760
Brett Cannon84df1e62010-05-14 00:33:40 +0000761 def __del__(self, _maxsize=sys.maxsize, _active=_active):
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000762 if not self._child_created:
763 # We didn't get to successfully create a child process.
764 return
765 # In case the child hasn't been waited on, check if it's done.
Brett Cannon84df1e62010-05-14 00:33:40 +0000766 self._internal_poll(_deadstate=_maxsize)
Thomas Wouters0e3f5912006-08-11 14:57:12 +0000767 if self.returncode is None and _active is not None:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000768 # Child is still running, keep us alive until we can wait on it.
769 _active.append(self)
770
771
Peter Astrand23109f02005-03-03 20:28:59 +0000772 def communicate(self, input=None):
773 """Interact with process: Send data to stdin. Read data from
774 stdout and stderr, until end-of-file is reached. Wait for
775 process to terminate. The optional input argument should be a
776 string to be sent to the child process, or None, if no data
777 should be sent to the child.
Tim Peterseba28be2005-03-28 01:08:02 +0000778
Peter Astrand23109f02005-03-03 20:28:59 +0000779 communicate() returns a tuple (stdout, stderr)."""
780
781 # Optimization: If we are only using one pipe, or no pipe at
782 # all, using select() or threads is unnecessary.
783 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
Tim Peterseba28be2005-03-28 01:08:02 +0000784 stdout = None
785 stderr = None
Peter Astrand23109f02005-03-03 20:28:59 +0000786 if self.stdin:
787 if input:
Ross Lagerwall4f61b022011-04-05 15:34:00 +0200788 try:
789 self.stdin.write(input)
790 except IOError as e:
791 if e.errno != errno.EPIPE and e.errno != errno.EINVAL:
792 raise
Peter Astrand23109f02005-03-03 20:28:59 +0000793 self.stdin.close()
794 elif self.stdout:
795 stdout = self.stdout.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000796 self.stdout.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000797 elif self.stderr:
798 stderr = self.stderr.read()
Georg Brandlf08a9dd2008-06-10 16:57:31 +0000799 self.stderr.close()
Peter Astrand23109f02005-03-03 20:28:59 +0000800 self.wait()
801 return (stdout, stderr)
Tim Peterseba28be2005-03-28 01:08:02 +0000802
Peter Astrand23109f02005-03-03 20:28:59 +0000803 return self._communicate(input)
804
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000805
Georg Brandl6aa2d1f2008-08-12 08:35:52 +0000806 def poll(self):
807 return self._internal_poll()
808
809
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000810 if mswindows:
811 #
812 # Windows methods
813 #
814 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +0000815 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000816 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
817 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000818 if stdin is None and stdout is None and stderr is None:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000819 return (-1, -1, -1, -1, -1, -1)
Tim Peterse718f612004-10-12 21:51:32 +0000820
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000821 p2cread, p2cwrite = -1, -1
822 c2pread, c2pwrite = -1, -1
823 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000824
Peter Astrandd38ddf42005-02-10 08:32:50 +0000825 if stdin is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000826 p2cread = _subprocess.GetStdHandle(_subprocess.STD_INPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000827 if p2cread is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000828 p2cread, _ = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000829 elif stdin == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000830 p2cread, p2cwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000831 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000832 p2cread = msvcrt.get_osfhandle(stdin)
833 else:
834 # Assuming file-like object
835 p2cread = msvcrt.get_osfhandle(stdin.fileno())
836 p2cread = self._make_inheritable(p2cread)
837
Peter Astrandd38ddf42005-02-10 08:32:50 +0000838 if stdout is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000839 c2pwrite = _subprocess.GetStdHandle(_subprocess.STD_OUTPUT_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000840 if c2pwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000841 _, c2pwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000842 elif stdout == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000843 c2pread, c2pwrite = _subprocess.CreatePipe(None, 0)
Peter Astrandd38ddf42005-02-10 08:32:50 +0000844 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000845 c2pwrite = msvcrt.get_osfhandle(stdout)
846 else:
847 # Assuming file-like object
848 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
849 c2pwrite = self._make_inheritable(c2pwrite)
850
Peter Astrandd38ddf42005-02-10 08:32:50 +0000851 if stderr is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000852 errwrite = _subprocess.GetStdHandle(_subprocess.STD_ERROR_HANDLE)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000853 if errwrite is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000854 _, errwrite = _subprocess.CreatePipe(None, 0)
Hirokazu Yamamoto0c988172009-03-03 22:41:26 +0000855 elif stderr == PIPE:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000856 errread, errwrite = _subprocess.CreatePipe(None, 0)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000857 elif stderr == STDOUT:
858 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +0000859 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000860 errwrite = msvcrt.get_osfhandle(stderr)
861 else:
862 # Assuming file-like object
863 errwrite = msvcrt.get_osfhandle(stderr.fileno())
864 errwrite = self._make_inheritable(errwrite)
865
866 return (p2cread, p2cwrite,
867 c2pread, c2pwrite,
868 errread, errwrite)
869
870
871 def _make_inheritable(self, handle):
872 """Return a duplicate of handle, which is inheritable"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000873 return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
874 handle, _subprocess.GetCurrentProcess(), 0, 1,
875 _subprocess.DUPLICATE_SAME_ACCESS)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000876
877
878 def _find_w9xpopen(self):
879 """Find and return absolut path to w9xpopen.exe"""
Brian Curtin1ce6b582010-04-24 16:19:22 +0000880 w9xpopen = os.path.join(
881 os.path.dirname(_subprocess.GetModuleFileName(0)),
Tim Peterse8374a52004-10-13 03:15:00 +0000882 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000883 if not os.path.exists(w9xpopen):
884 # Eeek - file-not-found - possibly an embedding
885 # situation - see if we can locate it in sys.exec_prefix
Tim Peterse8374a52004-10-13 03:15:00 +0000886 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
887 "w9xpopen.exe")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000888 if not os.path.exists(w9xpopen):
Tim Peterse8374a52004-10-13 03:15:00 +0000889 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
890 "needed for Popen to work with your "
891 "shell or platform.")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000892 return w9xpopen
893
Tim Peterse718f612004-10-12 21:51:32 +0000894
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000895 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000896 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000897 startupinfo, creationflags, shell,
898 p2cread, p2cwrite,
899 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000900 errread, errwrite,
901 unused_restore_signals, unused_start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000902 """Execute program (MS Windows version)"""
903
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000904 assert not pass_fds, "pass_fds not supported on Windows."
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +0000905
Guido van Rossum3172c5d2007-10-16 18:12:55 +0000906 if not isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000907 args = list2cmdline(args)
908
Peter Astrandc1d65362004-11-07 14:30:34 +0000909 # Process startup details
Peter Astrandd38ddf42005-02-10 08:32:50 +0000910 if startupinfo is None:
Thomas Wouters73e5a5b2006-06-08 15:35:45 +0000911 startupinfo = STARTUPINFO()
Victor Stinnerb3693582010-05-21 20:13:12 +0000912 if -1 not in (p2cread, c2pwrite, errwrite):
Brian Curtin1ce6b582010-04-24 16:19:22 +0000913 startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
Peter Astrandc1d65362004-11-07 14:30:34 +0000914 startupinfo.hStdInput = p2cread
915 startupinfo.hStdOutput = c2pwrite
916 startupinfo.hStdError = errwrite
917
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000918 if shell:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000919 startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
920 startupinfo.wShowWindow = _subprocess.SW_HIDE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000921 comspec = os.environ.get("COMSPEC", "cmd.exe")
Tim Golden126c2962010-08-11 14:20:40 +0000922 args = '{} /c "{}"'.format (comspec, args)
Brian Curtin1ce6b582010-04-24 16:19:22 +0000923 if (_subprocess.GetVersion() >= 0x80000000 or
Tim Peterse8374a52004-10-13 03:15:00 +0000924 os.path.basename(comspec).lower() == "command.com"):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000925 # Win9x, or using command.com on NT. We need to
926 # use the w9xpopen intermediate program. For more
927 # information, see KB Q150956
928 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
929 w9xpopen = self._find_w9xpopen()
930 args = '"%s" %s' % (w9xpopen, args)
931 # Not passing CREATE_NEW_CONSOLE has been known to
932 # cause random failures on win9x. Specifically a
933 # dialog: "Your program accessed mem currently in
934 # use at xxx" and a hopeful warning about the
Mark Dickinson934896d2009-02-21 20:59:32 +0000935 # stability of your system. Cost is Ctrl+C won't
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000936 # kill children.
Brian Curtin1ce6b582010-04-24 16:19:22 +0000937 creationflags |= _subprocess.CREATE_NEW_CONSOLE
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000938
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000939 # Start the process
940 try:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000941 hp, ht, pid, tid = _subprocess.CreateProcess(executable, args,
Tim Peterse8374a52004-10-13 03:15:00 +0000942 # no special security
943 None, None,
Guido van Rossume7ba4952007-06-06 23:52:48 +0000944 int(not close_fds),
Tim Peterse8374a52004-10-13 03:15:00 +0000945 creationflags,
946 env,
947 cwd,
948 startupinfo)
Guido van Rossumb940e112007-01-10 16:19:56 +0000949 except pywintypes.error as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000950 # Translate pywintypes.error to WindowsError, which is
951 # a subclass of OSError. FIXME: We should really
Ezio Melotti13925002011-03-16 11:05:33 +0200952 # translate errno using _sys_errlist (or similar), but
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000953 # how can this be done from Python?
954 raise WindowsError(*e.args)
Tim Goldenad537f22010-08-08 11:18:16 +0000955 finally:
956 # Child is launched. Close the parent's copy of those pipe
957 # handles that only the child should have open. You need
958 # to make sure that no handles to the write end of the
959 # output pipe are maintained in this process or else the
960 # pipe will not close when the child process exits and the
961 # ReadFile will hang.
962 if p2cread != -1:
963 p2cread.Close()
964 if c2pwrite != -1:
965 c2pwrite.Close()
966 if errwrite != -1:
967 errwrite.Close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000968
969 # Retain the process handle, but close the thread handle
Thomas Wouters49fd7fa2006-04-21 10:40:58 +0000970 self._child_created = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000971 self._handle = hp
972 self.pid = pid
973 ht.Close()
974
Brett Cannon84df1e62010-05-14 00:33:40 +0000975 def _internal_poll(self, _deadstate=None,
Victor Stinnerc807a612010-05-14 21:53:45 +0000976 _WaitForSingleObject=_subprocess.WaitForSingleObject,
977 _WAIT_OBJECT_0=_subprocess.WAIT_OBJECT_0,
978 _GetExitCodeProcess=_subprocess.GetExitCodeProcess):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000979 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +0000980 attribute.
981
982 This method is called by __del__, so it can only refer to objects
983 in its local scope.
984
985 """
Peter Astrandd38ddf42005-02-10 08:32:50 +0000986 if self.returncode is None:
Brett Cannon84df1e62010-05-14 00:33:40 +0000987 if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
988 self.returncode = _GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000989 return self.returncode
990
991
992 def wait(self):
993 """Wait for child process to terminate. Returns returncode
994 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +0000995 if self.returncode is None:
Brian Curtin1ce6b582010-04-24 16:19:22 +0000996 _subprocess.WaitForSingleObject(self._handle,
997 _subprocess.INFINITE)
998 self.returncode = _subprocess.GetExitCodeProcess(self._handle)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +0000999 return self.returncode
1000
1001
1002 def _readerthread(self, fh, buffer):
1003 buffer.append(fh.read())
Victor Stinner667d4b52010-12-25 22:40:32 +00001004 fh.close()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001005
1006
Peter Astrand23109f02005-03-03 20:28:59 +00001007 def _communicate(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001008 stdout = None # Return
1009 stderr = None # Return
1010
1011 if self.stdout:
1012 stdout = []
Tim Peterse8374a52004-10-13 03:15:00 +00001013 stdout_thread = threading.Thread(target=self._readerthread,
1014 args=(self.stdout, stdout))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001015 stdout_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001016 stdout_thread.start()
1017 if self.stderr:
1018 stderr = []
Tim Peterse8374a52004-10-13 03:15:00 +00001019 stderr_thread = threading.Thread(target=self._readerthread,
1020 args=(self.stderr, stderr))
Benjamin Peterson632e0362008-08-18 19:08:51 +00001021 stderr_thread.daemon = True
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001022 stderr_thread.start()
1023
1024 if self.stdin:
Peter Astrandd38ddf42005-02-10 08:32:50 +00001025 if input is not None:
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001026 try:
1027 self.stdin.write(input)
1028 except IOError as e:
1029 if e.errno != errno.EPIPE:
1030 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001031 self.stdin.close()
1032
1033 if self.stdout:
1034 stdout_thread.join()
1035 if self.stderr:
1036 stderr_thread.join()
1037
1038 # All data exchanged. Translate lists into strings.
Peter Astrandd38ddf42005-02-10 08:32:50 +00001039 if stdout is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001040 stdout = stdout[0]
Peter Astrandd38ddf42005-02-10 08:32:50 +00001041 if stderr is not None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001042 stderr = stderr[0]
1043
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001044 self.wait()
1045 return (stdout, stderr)
1046
Christian Heimesa342c012008-04-20 21:01:16 +00001047 def send_signal(self, sig):
1048 """Send a signal to the process
1049 """
1050 if sig == signal.SIGTERM:
1051 self.terminate()
Brian Curtineb24d742010-04-12 17:16:38 +00001052 elif sig == signal.CTRL_C_EVENT:
1053 os.kill(self.pid, signal.CTRL_C_EVENT)
1054 elif sig == signal.CTRL_BREAK_EVENT:
1055 os.kill(self.pid, signal.CTRL_BREAK_EVENT)
Christian Heimesa342c012008-04-20 21:01:16 +00001056 else:
Brian Curtin19651362010-09-07 13:24:38 +00001057 raise ValueError("Unsupported signal: {}".format(sig))
Christian Heimesa342c012008-04-20 21:01:16 +00001058
1059 def terminate(self):
1060 """Terminates the process
1061 """
Brian Curtin1ce6b582010-04-24 16:19:22 +00001062 _subprocess.TerminateProcess(self._handle, 1)
Christian Heimesa342c012008-04-20 21:01:16 +00001063
1064 kill = terminate
1065
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001066 else:
1067 #
1068 # POSIX methods
1069 #
1070 def _get_handles(self, stdin, stdout, stderr):
Alexandre Vassalotti711ed4a2009-07-17 10:42:05 +00001071 """Construct and return tuple with IO objects:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001072 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
1073 """
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001074 p2cread, p2cwrite = -1, -1
1075 c2pread, c2pwrite = -1, -1
1076 errread, errwrite = -1, -1
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001077
Peter Astrandd38ddf42005-02-10 08:32:50 +00001078 if stdin is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001079 pass
1080 elif stdin == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001081 p2cread, p2cwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001082 elif isinstance(stdin, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001083 p2cread = stdin
1084 else:
1085 # Assuming file-like object
1086 p2cread = stdin.fileno()
1087
Peter Astrandd38ddf42005-02-10 08:32:50 +00001088 if stdout is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001089 pass
1090 elif stdout == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001091 c2pread, c2pwrite = _create_pipe()
Peter Astrandd38ddf42005-02-10 08:32:50 +00001092 elif isinstance(stdout, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001093 c2pwrite = stdout
1094 else:
1095 # Assuming file-like object
Tim Peterse718f612004-10-12 21:51:32 +00001096 c2pwrite = stdout.fileno()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001097
Peter Astrandd38ddf42005-02-10 08:32:50 +00001098 if stderr is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001099 pass
1100 elif stderr == PIPE:
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001101 errread, errwrite = _create_pipe()
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001102 elif stderr == STDOUT:
1103 errwrite = c2pwrite
Peter Astrandd38ddf42005-02-10 08:32:50 +00001104 elif isinstance(stderr, int):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001105 errwrite = stderr
1106 else:
1107 # Assuming file-like object
1108 errwrite = stderr.fileno()
1109
1110 return (p2cread, p2cwrite,
1111 c2pread, c2pwrite,
1112 errread, errwrite)
1113
1114
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001115 def _close_fds(self, fds_to_keep):
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001116 start_fd = 3
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001117 for fd in sorted(fds_to_keep):
Gregory P. Smith8edd99d2010-12-14 13:43:30 +00001118 if fd >= start_fd:
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001119 os.closerange(start_fd, fd)
1120 start_fd = fd + 1
1121 if start_fd <= MAXFD:
1122 os.closerange(start_fd, MAXFD)
1123
1124
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001125 def _execute_child(self, args, executable, preexec_fn, close_fds,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001126 pass_fds, cwd, env, universal_newlines,
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001127 startupinfo, creationflags, shell,
1128 p2cread, p2cwrite,
1129 c2pread, c2pwrite,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001130 errread, errwrite,
1131 restore_signals, start_new_session):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001132 """Execute program (POSIX version)"""
1133
Guido van Rossum3172c5d2007-10-16 18:12:55 +00001134 if isinstance(args, str):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001135 args = [args]
Thomas Wouters89f507f2006-12-13 04:49:30 +00001136 else:
1137 args = list(args)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001138
1139 if shell:
1140 args = ["/bin/sh", "-c"] + args
Stefan Krah9542cc62010-07-19 14:20:53 +00001141 if executable:
1142 args[0] = executable
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001143
Peter Astrandd38ddf42005-02-10 08:32:50 +00001144 if executable is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001145 executable = args[0]
1146
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001147 # For transferring possible exec failure from child to parent.
1148 # Data format: "exception name:hex errno:description"
1149 # Pickle is not used; it is complex and involves memory allocation.
Gregory P. Smith51ee2702010-12-13 07:59:39 +00001150 errpipe_read, errpipe_write = _create_pipe()
Christian Heimesfdab48e2008-01-20 09:06:41 +00001151 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001152 try:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001153
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001154 if _posixsubprocess:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001155 # We must avoid complex work that could involve
1156 # malloc or free in the child process to avoid
1157 # potential deadlocks, thus we do all this here.
1158 # and pass it to fork_exec()
1159
1160 if env:
Victor Stinner449c4662010-05-08 11:10:09 +00001161 env_list = [os.fsencode(k) + b'=' + os.fsencode(v)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001162 for k, v in env.items()]
1163 else:
1164 env_list = None # Use execv instead of execve.
Victor Stinnerb745a742010-05-18 17:17:23 +00001165 executable = os.fsencode(executable)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001166 if os.path.dirname(executable):
Victor Stinnerb745a742010-05-18 17:17:23 +00001167 executable_list = (executable,)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001168 else:
1169 # This matches the behavior of os._execvpe().
Victor Stinnerb745a742010-05-18 17:17:23 +00001170 executable_list = tuple(
1171 os.path.join(os.fsencode(dir), executable)
1172 for dir in os.get_exec_path(env))
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001173 fds_to_keep = set(pass_fds)
1174 fds_to_keep.add(errpipe_write)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001175 self.pid = _posixsubprocess.fork_exec(
1176 args, executable_list,
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001177 close_fds, sorted(fds_to_keep), cwd, env_list,
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001178 p2cread, p2cwrite, c2pread, c2pwrite,
1179 errread, errwrite,
1180 errpipe_read, errpipe_write,
1181 restore_signals, start_new_session, preexec_fn)
1182 else:
1183 # Pure Python implementation: It is not thread safe.
1184 # This implementation may deadlock in the child if your
1185 # parent process has any other threads running.
1186
1187 gc_was_enabled = gc.isenabled()
1188 # Disable gc to avoid bug where gc -> file_dealloc ->
1189 # write to stderr -> hang. See issue1336
1190 gc.disable()
1191 try:
1192 self.pid = os.fork()
1193 except:
1194 if gc_was_enabled:
1195 gc.enable()
1196 raise
1197 self._child_created = True
1198 if self.pid == 0:
1199 # Child
1200 try:
1201 # Close parent's pipe ends
1202 if p2cwrite != -1:
1203 os.close(p2cwrite)
1204 if c2pread != -1:
1205 os.close(c2pread)
1206 if errread != -1:
1207 os.close(errread)
1208 os.close(errpipe_read)
1209
1210 # Dup fds for child
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001211 def _dup2(a, b):
1212 # dup2() removes the CLOEXEC flag but
1213 # we must do it ourselves if dup2()
1214 # would be a no-op (issue #10806).
1215 if a == b:
1216 _set_cloexec(a, False)
1217 elif a != -1:
1218 os.dup2(a, b)
1219 _dup2(p2cread, 0)
1220 _dup2(c2pwrite, 1)
1221 _dup2(errwrite, 2)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001222
1223 # Close pipe fds. Make sure we don't close the
1224 # same fd more than once, or standard fds.
Antoine Pitrouc9c83ba2011-01-03 18:23:55 +00001225 closed = set()
1226 for fd in [p2cread, c2pwrite, errwrite]:
1227 if fd > 2 and fd not in closed:
1228 os.close(fd)
1229 closed.add(fd)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001230
1231 # Close all other fds, if asked for
1232 if close_fds:
Antoine Pitrou47f14ba2011-01-03 23:42:01 +00001233 fds_to_keep = set(pass_fds)
1234 fds_to_keep.add(errpipe_write)
1235 self._close_fds(fds_to_keep)
Gregory P. Smithd4cc7bf2010-12-04 11:22:11 +00001236
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001237
1238 if cwd is not None:
1239 os.chdir(cwd)
1240
1241 # This is a copy of Python/pythonrun.c
1242 # _Py_RestoreSignals(). If that were exposed
1243 # as a sys._py_restoresignals func it would be
1244 # better.. but this pure python implementation
1245 # isn't likely to be used much anymore.
1246 if restore_signals:
1247 signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
1248 for sig in signals:
1249 if hasattr(signal, sig):
1250 signal.signal(getattr(signal, sig),
1251 signal.SIG_DFL)
1252
1253 if start_new_session and hasattr(os, 'setsid'):
1254 os.setsid()
1255
1256 if preexec_fn:
1257 preexec_fn()
1258
1259 if env is None:
1260 os.execvp(executable, args)
1261 else:
1262 os.execvpe(executable, args, env)
1263
1264 except:
1265 try:
1266 exc_type, exc_value = sys.exc_info()[:2]
1267 if isinstance(exc_value, OSError):
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001268 errno_num = exc_value.errno
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001269 else:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001270 errno_num = 0
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001271 message = '%s:%x:%s' % (exc_type.__name__,
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001272 errno_num, exc_value)
Victor Stinner4d078042010-04-23 19:28:32 +00001273 message = message.encode(errors="surrogatepass")
1274 os.write(errpipe_write, message)
1275 except Exception:
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001276 # We MUST not allow anything odd happening
1277 # above to prevent us from exiting below.
1278 pass
1279
1280 # This exitcode won't be reported to applications
1281 # so it really doesn't matter what we return.
1282 os._exit(255)
1283
1284 # Parent
Facundo Batista10706e22009-06-19 20:34:30 +00001285 if gc_was_enabled:
1286 gc.enable()
Facundo Batista10706e22009-06-19 20:34:30 +00001287 finally:
1288 # be sure the FD is closed no matter what
1289 os.close(errpipe_write)
1290
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001291 if p2cread != -1 and p2cwrite != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001292 os.close(p2cread)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001293 if c2pwrite != -1 and c2pread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001294 os.close(c2pwrite)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001295 if errwrite != -1 and errread != -1:
Facundo Batista10706e22009-06-19 20:34:30 +00001296 os.close(errwrite)
1297
1298 # Wait for exec to fail or succeed; possibly raising an
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001299 # exception (limited in size)
1300 data = bytearray()
1301 while True:
1302 part = _eintr_retry_call(os.read, errpipe_read, 50000)
1303 data += part
1304 if not part or len(data) > 50000:
1305 break
Facundo Batista10706e22009-06-19 20:34:30 +00001306 finally:
1307 # be sure the FD is closed no matter what
1308 os.close(errpipe_read)
1309
Guido van Rossumaf2362a2007-05-15 22:32:02 +00001310 if data:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001311 try:
1312 _eintr_retry_call(os.waitpid, self.pid, 0)
1313 except OSError as e:
1314 if e.errno != errno.ECHILD:
1315 raise
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001316 try:
1317 exception_name, hex_errno, err_msg = data.split(b':', 2)
1318 except ValueError:
1319 print('Bad exception data:', repr(data))
1320 exception_name = b'RuntimeError'
1321 hex_errno = b'0'
1322 err_msg = b'Unknown'
1323 child_exception_type = getattr(
1324 builtins, exception_name.decode('ascii'),
1325 RuntimeError)
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001326 for fd in (p2cwrite, c2pread, errread):
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001327 if fd != -1:
Benjamin Petersond75fcb42009-02-19 04:22:03 +00001328 os.close(fd)
Victor Stinner4d078042010-04-23 19:28:32 +00001329 err_msg = err_msg.decode(errors="surrogatepass")
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001330 if issubclass(child_exception_type, OSError) and hex_errno:
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001331 errno_num = int(hex_errno, 16)
1332 if errno_num != 0:
1333 err_msg = os.strerror(errno_num)
1334 if errno_num == errno.ENOENT:
Benjamin Peterson5f780402010-11-20 18:07:52 +00001335 err_msg += ': ' + repr(args[0])
Benjamin Petersonb8bc4392010-11-20 18:24:54 +00001336 raise child_exception_type(errno_num, err_msg)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +00001337 raise child_exception_type(err_msg)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001338
1339
Brett Cannon84df1e62010-05-14 00:33:40 +00001340 def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
1341 _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED,
1342 _WEXITSTATUS=os.WEXITSTATUS):
1343 # This method is called (indirectly) by __del__, so it cannot
1344 # refer to anything outside of its local scope."""
1345 if _WIFSIGNALED(sts):
1346 self.returncode = -_WTERMSIG(sts)
1347 elif _WIFEXITED(sts):
1348 self.returncode = _WEXITSTATUS(sts)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001349 else:
1350 # Should never happen
1351 raise RuntimeError("Unknown child exit status!")
1352
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001353
Brett Cannon84df1e62010-05-14 00:33:40 +00001354 def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid,
1355 _WNOHANG=os.WNOHANG, _os_error=os.error):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001356 """Check if child process has terminated. Returns returncode
Brett Cannon84df1e62010-05-14 00:33:40 +00001357 attribute.
1358
1359 This method is called by __del__, so it cannot reference anything
1360 outside of the local scope (nor can any methods it calls).
1361
1362 """
Peter Astrandd38ddf42005-02-10 08:32:50 +00001363 if self.returncode is None:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001364 try:
Brett Cannon84df1e62010-05-14 00:33:40 +00001365 pid, sts = _waitpid(self.pid, _WNOHANG)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001366 if pid == self.pid:
1367 self._handle_exitstatus(sts)
Brett Cannon84df1e62010-05-14 00:33:40 +00001368 except _os_error:
Thomas Wouters49fd7fa2006-04-21 10:40:58 +00001369 if _deadstate is not None:
1370 self.returncode = _deadstate
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001371 return self.returncode
1372
1373
1374 def wait(self):
1375 """Wait for child process to terminate. Returns returncode
1376 attribute."""
Peter Astrandd38ddf42005-02-10 08:32:50 +00001377 if self.returncode is None:
Gregory P. Smithe85db2b2010-12-14 14:38:00 +00001378 try:
1379 pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
1380 except OSError as e:
1381 if e.errno != errno.ECHILD:
1382 raise
1383 # This happens if SIGCLD is set to be ignored or waiting
1384 # for child processes has otherwise been disabled for our
1385 # process. This child is dead, we can't get the status.
1386 sts = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001387 self._handle_exitstatus(sts)
1388 return self.returncode
1389
1390
Peter Astrand23109f02005-03-03 20:28:59 +00001391 def _communicate(self, input):
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001392 if self.stdin:
1393 # Flush stdio buffer. This might block, if the user has
1394 # been writing to .stdin in an uncontrolled fashion.
1395 self.stdin.flush()
1396 if not input:
1397 self.stdin.close()
1398
1399 if _has_poll:
1400 stdout, stderr = self._communicate_with_poll(input)
1401 else:
1402 stdout, stderr = self._communicate_with_select(input)
1403
1404 # All data exchanged. Translate lists into strings.
1405 if stdout is not None:
1406 stdout = b''.join(stdout)
1407 if stderr is not None:
1408 stderr = b''.join(stderr)
1409
1410 # Translate newlines, if requested.
1411 # This also turns bytes into strings.
1412 if self.universal_newlines:
1413 if stdout is not None:
1414 stdout = self._translate_newlines(stdout,
1415 self.stdout.encoding)
1416 if stderr is not None:
1417 stderr = self._translate_newlines(stderr,
1418 self.stderr.encoding)
1419
1420 self.wait()
1421 return (stdout, stderr)
1422
1423
1424 def _communicate_with_poll(self, input):
1425 stdout = None # Return
1426 stderr = None # Return
1427 fd2file = {}
1428 fd2output = {}
1429
1430 poller = select.poll()
1431 def register_and_append(file_obj, eventmask):
1432 poller.register(file_obj.fileno(), eventmask)
1433 fd2file[file_obj.fileno()] = file_obj
1434
1435 def close_unregister_and_remove(fd):
1436 poller.unregister(fd)
1437 fd2file[fd].close()
1438 fd2file.pop(fd)
1439
1440 if self.stdin and input:
1441 register_and_append(self.stdin, select.POLLOUT)
1442
1443 select_POLLIN_POLLPRI = select.POLLIN | select.POLLPRI
1444 if self.stdout:
1445 register_and_append(self.stdout, select_POLLIN_POLLPRI)
1446 fd2output[self.stdout.fileno()] = stdout = []
1447 if self.stderr:
1448 register_and_append(self.stderr, select_POLLIN_POLLPRI)
1449 fd2output[self.stderr.fileno()] = stderr = []
1450
1451 input_offset = 0
1452 while fd2file:
1453 try:
1454 ready = poller.poll()
1455 except select.error as e:
1456 if e.args[0] == errno.EINTR:
1457 continue
1458 raise
1459
1460 # XXX Rewrite these to use non-blocking I/O on the
1461 # file objects; they are no longer using C stdio!
1462
1463 for fd, mode in ready:
1464 if mode & select.POLLOUT:
1465 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001466 try:
1467 input_offset += os.write(fd, chunk)
1468 except OSError as e:
1469 if e.errno == errno.EPIPE:
1470 close_unregister_and_remove(fd)
1471 else:
1472 raise
1473 else:
1474 if input_offset >= len(input):
1475 close_unregister_and_remove(fd)
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001476 elif mode & select_POLLIN_POLLPRI:
1477 data = os.read(fd, 4096)
1478 if not data:
1479 close_unregister_and_remove(fd)
1480 fd2output[fd].append(data)
1481 else:
1482 # Ignore hang up or errors.
1483 close_unregister_and_remove(fd)
1484
1485 return (stdout, stderr)
1486
1487
1488 def _communicate_with_select(self, input):
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001489 read_set = []
1490 write_set = []
1491 stdout = None # Return
1492 stderr = None # Return
1493
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001494 if self.stdin and input:
1495 write_set.append(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001496 if self.stdout:
1497 read_set.append(self.stdout)
Tim Peterse718f612004-10-12 21:51:32 +00001498 stdout = []
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001499 if self.stderr:
1500 read_set.append(self.stderr)
1501 stderr = []
1502
Thomas Wouters902d6eb2007-01-09 23:18:33 +00001503 input_offset = 0
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001504 while read_set or write_set:
Georg Brandl86b2fb92008-07-16 03:43:04 +00001505 try:
1506 rlist, wlist, xlist = select.select(read_set, write_set, [])
1507 except select.error as e:
1508 if e.args[0] == errno.EINTR:
1509 continue
1510 raise
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001511
Guido van Rossum98297ee2007-11-06 21:34:58 +00001512 # XXX Rewrite these to use non-blocking I/O on the
1513 # file objects; they are no longer using C stdio!
1514
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001515 if self.stdin in wlist:
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001516 chunk = input[input_offset : input_offset + _PIPE_BUF]
Ross Lagerwall4f61b022011-04-05 15:34:00 +02001517 try:
1518 bytes_written = os.write(self.stdin.fileno(), chunk)
1519 except OSError as e:
1520 if e.errno == errno.EPIPE:
1521 self.stdin.close()
1522 write_set.remove(self.stdin)
1523 else:
1524 raise
1525 else:
1526 input_offset += bytes_written
1527 if input_offset >= len(input):
1528 self.stdin.close()
1529 write_set.remove(self.stdin)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001530
1531 if self.stdout in rlist:
1532 data = os.read(self.stdout.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001533 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001534 self.stdout.close()
1535 read_set.remove(self.stdout)
1536 stdout.append(data)
1537
1538 if self.stderr in rlist:
1539 data = os.read(self.stderr.fileno(), 1024)
Guido van Rossumc9e363c2007-05-15 23:18:55 +00001540 if not data:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001541 self.stderr.close()
1542 read_set.remove(self.stderr)
1543 stderr.append(data)
1544
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001545 return (stdout, stderr)
1546
Gregory P. Smithd06fa472009-07-04 02:46:54 +00001547
Christian Heimesa342c012008-04-20 21:01:16 +00001548 def send_signal(self, sig):
1549 """Send a signal to the process
1550 """
1551 os.kill(self.pid, sig)
1552
1553 def terminate(self):
1554 """Terminate the process with SIGTERM
1555 """
1556 self.send_signal(signal.SIGTERM)
1557
1558 def kill(self):
1559 """Kill the process with SIGKILL
1560 """
1561 self.send_signal(signal.SIGKILL)
1562
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001563
1564def _demo_posix():
1565 #
1566 # Example 1: Simple redirection: Get process list
1567 #
1568 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001569 print("Process list:")
1570 print(plist)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001571
1572 #
1573 # Example 2: Change uid before executing child
1574 #
1575 if os.getuid() == 0:
1576 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1577 p.wait()
1578
1579 #
1580 # Example 3: Connecting several subprocesses
1581 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001582 print("Looking for 'hda'...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001583 p1 = Popen(["dmesg"], stdout=PIPE)
1584 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001585 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001586
1587 #
1588 # Example 4: Catch execution error
1589 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001590 print()
1591 print("Trying a weird file...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001592 try:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001593 print(Popen(["/this/path/does/not/exist"]).communicate())
Guido van Rossumb940e112007-01-10 16:19:56 +00001594 except OSError as e:
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001595 if e.errno == errno.ENOENT:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001596 print("The file didn't exist. I thought so...")
1597 print("Child traceback:")
1598 print(e.child_traceback)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001599 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001600 print("Error", e.errno)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001601 else:
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001602 print("Gosh. No error.", file=sys.stderr)
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001603
1604
1605def _demo_windows():
1606 #
1607 # Example 1: Connecting several subprocesses
1608 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001609 print("Looking for 'PROMPT' in set output...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001610 p1 = Popen("set", stdout=PIPE, shell=True)
1611 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001612 print(repr(p2.communicate()[0]))
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001613
1614 #
1615 # Example 2: Simple execution of program
1616 #
Guido van Rossumbe19ed72007-02-09 05:37:30 +00001617 print("Executing calc...")
Fredrik Lundh5b3687d2004-10-12 15:26:28 +00001618 p = Popen("calc")
1619 p.wait()
1620
1621
1622if __name__ == "__main__":
1623 if mswindows:
1624 _demo_windows()
1625 else:
1626 _demo_posix()