blob: 9f823235b483e66a46b3b914f091ead79b47d759 [file] [log] [blame]
Georg Brandl8ec7f652007-08-15 14:28:01 +00001
2:mod:`subprocess` --- Subprocess management
3===========================================
4
5.. module:: subprocess
6 :synopsis: Subprocess management.
7.. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se>
8.. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se>
9
10
11.. versionadded:: 2.4
12
13The :mod:`subprocess` module allows you to spawn new processes, connect to their
14input/output/error pipes, and obtain their return codes. This module intends to
15replace several other, older modules and functions, such as::
16
17 os.system
18 os.spawn*
19 os.popen*
20 popen2.*
21 commands.*
22
23Information about how the :mod:`subprocess` module can be used to replace these
24modules and functions can be found in the following sections.
25
Georg Brandl68b4e742008-07-01 19:59:00 +000026.. seealso::
27
28 :pep:`324` -- PEP proposing the subprocess module
29
Georg Brandl8ec7f652007-08-15 14:28:01 +000030
31Using the subprocess Module
32---------------------------
33
Nick Coghlan2ed203a2011-10-26 21:05:56 +100034The recommended approach to invoking subprocesses is to use the following
35convenience functions for all use cases they can handle. For more advanced
36use cases, the underlying :class:`Popen` interface can be used directly.
Nick Coghlan86711572011-10-24 22:19:40 +100037
38
Nick Coghlan2ed203a2011-10-26 21:05:56 +100039.. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Nick Coghlan86711572011-10-24 22:19:40 +100040
41 Run the command described by *args*. Wait for command to complete, then
42 return the :attr:`returncode` attribute.
43
44 The arguments shown above are merely the most common ones, described below
Nick Coghlan87ba6422011-10-27 17:55:13 +100045 in :ref:`frequently-used-arguments` (hence the slightly odd notation in
46 the abbreviated signature). The full function signature is the same as
47 that of the :class:`Popen` constructor - this functions passes all
48 supplied arguments directly through to that interface.
Nick Coghlan86711572011-10-24 22:19:40 +100049
50 Examples::
51
52 >>> subprocess.call(["ls", "-l"])
53 0
54
Nick Coghlan2ed203a2011-10-26 21:05:56 +100055 >>> subprocess.call("exit 1", shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +100056 1
57
58 .. warning::
59
Nick Coghlan87ba6422011-10-27 17:55:13 +100060 Invoking the system shell with ``shell=True`` can be a security hazard
61 if combined with untrusted input. See the warning under
62 :ref:`frequently-used-arguments` for details.
63
64 .. note::
65
Nick Coghlan2ed203a2011-10-26 21:05:56 +100066 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
67 the pipes are not being read in the current process, the child
68 process may block if it generates enough output to a pipe to fill up
69 the OS pipe buffer.
Nick Coghlan86711572011-10-24 22:19:40 +100070
71
Nick Coghlan87ba6422011-10-27 17:55:13 +100072.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Nick Coghlan86711572011-10-24 22:19:40 +100073
74 Run command with arguments. Wait for command to complete. If the return
75 code was zero then return, otherwise raise :exc:`CalledProcessError`. The
76 :exc:`CalledProcessError` object will have the return code in the
77 :attr:`returncode` attribute.
78
Nick Coghlan87ba6422011-10-27 17:55:13 +100079 The arguments shown above are merely the most common ones, described below
80 in :ref:`frequently-used-arguments` (hence the slightly odd notation in
81 the abbreviated signature). The full function signature is the same as
82 that of the :class:`Popen` constructor - this functions passes all
83 supplied arguments directly through to that interface.
84
85 Examples::
Nick Coghlan86711572011-10-24 22:19:40 +100086
87 >>> subprocess.check_call(["ls", "-l"])
88 0
89
Nick Coghlan2ed203a2011-10-26 21:05:56 +100090 >>> subprocess.check_call("exit 1", shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +100091 Traceback (most recent call last):
92 ...
Nick Coghlan2ed203a2011-10-26 21:05:56 +100093 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
Nick Coghlan86711572011-10-24 22:19:40 +100094
95 .. versionadded:: 2.5
96
97 .. warning::
98
Nick Coghlan87ba6422011-10-27 17:55:13 +100099 Invoking the system shell with ``shell=True`` can be a security hazard
100 if combined with untrusted input. See the warning under
101 :ref:`frequently-used-arguments` for details.
102
103 .. note::
104
105 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
106 the pipes are not being read in the current process, the child
107 process may block if it generates enough output to a pipe to fill up
108 the OS pipe buffer.
Nick Coghlan86711572011-10-24 22:19:40 +1000109
110
Nick Coghlan87ba6422011-10-27 17:55:13 +1000111.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
Nick Coghlan86711572011-10-24 22:19:40 +1000112
113 Run command with arguments and return its output as a byte string.
114
115 If the return code was non-zero it raises a :exc:`CalledProcessError`. The
116 :exc:`CalledProcessError` object will have the return code in the
117 :attr:`returncode` attribute and any output in the :attr:`output`
118 attribute.
119
Nick Coghlan87ba6422011-10-27 17:55:13 +1000120 The arguments shown above are merely the most common ones, described below
121 in :ref:`frequently-used-arguments` (hence the slightly odd notation in
122 the abbreviated signature). The full function signature is largely the
123 same as that of the :class:`Popen` constructor, except that *stdout* is
124 not permitted as it is used internally. All other supplied arguments are
125 passed directly through to the :class:`Popen` constructor.
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000126
Nick Coghlan86711572011-10-24 22:19:40 +1000127 Examples::
128
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000129 >>> subprocess.check_output(["echo", "Hello World!"])
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000130 'Hello World!\n'
131
132 >>> subprocess.check_output("exit 1", shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +1000133 Traceback (most recent call last):
134 ...
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000135 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
Nick Coghlan86711572011-10-24 22:19:40 +1000136
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000137 To also capture standard error in the result, use
138 ``stderr=subprocess.STDOUT``::
Nick Coghlan86711572011-10-24 22:19:40 +1000139
140 >>> subprocess.check_output(
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000141 ... "ls non_existent_file; exit 0",
142 ... stderr=subprocess.STDOUT,
143 ... shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +1000144 'ls: non_existent_file: No such file or directory\n'
145
146 .. versionadded:: 2.7
147
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000148 .. warning::
149
Nick Coghlan87ba6422011-10-27 17:55:13 +1000150 Invoking the system shell with ``shell=True`` can be a security hazard
151 if combined with untrusted input. See the warning under
152 :ref:`frequently-used-arguments` for details.
153
154 .. note::
155
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000156 Do not use ``stderr=PIPE`` with this function. As the pipe is not being
157 read in the current process, the child process may block if it
158 generates enough output to the pipe to fill up the OS pipe buffer.
159
Nick Coghlan86711572011-10-24 22:19:40 +1000160
161.. data:: PIPE
162
163 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
164 to :class:`Popen` and indicates that a pipe to the standard stream should be
165 opened.
166
167
168.. data:: STDOUT
169
170 Special value that can be used as the *stderr* argument to :class:`Popen` and
171 indicates that standard error should go into the same handle as standard
172 output.
173
174
Andrew Svetlov8afcec42012-08-09 15:23:49 +0300175.. exception:: CalledProcessError
176
177 Exception raised when a process run by :func:`check_call` or
178 :func:`check_output` returns a non-zero exit status.
179
180 .. attribute:: returncode
181
182 Exit status of the child process.
183
184 .. attribute:: cmd
185
186 Command that was used to spawn the child process.
187
188 .. attribute:: output
189
190 Output of the child process if this exception is raised by
191 :func:`check_output`. Otherwise, ``None``.
192
193
194
Nick Coghlan86711572011-10-24 22:19:40 +1000195.. _frequently-used-arguments:
196
197Frequently Used Arguments
198^^^^^^^^^^^^^^^^^^^^^^^^^
199
200To support a wide variety of use cases, the :class:`Popen` constructor (and
201the convenience functions) accept a large number of optional arguments. For
202most typical use cases, many of these arguments can be safely left at their
203default values. The arguments that are most commonly needed are:
204
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000205 *args* is required for all calls and should be a string, or a sequence of
206 program arguments. Providing a sequence of arguments is generally
207 preferred, as it allows the module to take care of any required escaping
208 and quoting of arguments (e.g. to permit spaces in file names). If passing
209 a single string, either *shell* must be :const:`True` (see below) or else
210 the string must simply name the program to be executed without specifying
211 any arguments.
Nick Coghlan86711572011-10-24 22:19:40 +1000212
213 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
214 standard output and standard error file handles, respectively. Valid values
215 are :data:`PIPE`, an existing file descriptor (a positive integer), an
216 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
217 to the child should be created. With the default settings of ``None``, no
218 redirection will occur; the child's file handles will be inherited from the
219 parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that
220 the stderr data from the child process should be captured into the same file
221 handle as for stdout.
222
R David Murray5618aaa2012-08-15 11:15:39 -0400223 .. index::
224 single: universal newlines; subprocess module
225
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000226 When *stdout* or *stderr* are pipes and *universal_newlines* is
R David Murray5618aaa2012-08-15 11:15:39 -0400227 ``True`` then all line endings will be converted to ``'\n'`` as described
228 for the :term:`universal newlines` `'U'`` mode argument to :func:`open`.
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000229
230 If *shell* is :const:`True`, the specified command will be executed through
231 the shell. This can be useful if you are using Python primarily for the
232 enhanced control flow it offers over most system shells and still want
233 access to other shell features such as filename wildcards, shell pipes and
234 environment variable expansion.
235
236 .. warning::
237
238 Executing shell commands that incorporate unsanitized input from an
239 untrusted source makes a program vulnerable to `shell injection
240 <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_,
241 a serious security flaw which can result in arbitrary command execution.
242 For this reason, the use of *shell=True* is **strongly discouraged** in cases
243 where the command string is constructed from external input::
244
245 >>> from subprocess import call
246 >>> filename = input("What file would you like to display?\n")
247 What file would you like to display?
248 non_existent; rm -rf / #
249 >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
250
251 ``shell=False`` disables all shell based features, but does not suffer
252 from this vulnerability; see the Note in the :class:`Popen` constructor
253 documentation for helpful hints in getting ``shell=False`` to work.
254
Nick Coghlan86711572011-10-24 22:19:40 +1000255These options, along with all of the other options, are described in more
256detail in the :class:`Popen` constructor documentation.
257
258
Sandro Tosidbcbd102011-12-25 11:27:22 +0100259Popen Constructor
Sandro Tosi44585bd2011-12-25 17:13:10 +0100260^^^^^^^^^^^^^^^^^
Nick Coghlan86711572011-10-24 22:19:40 +1000261
262The underlying process creation and management in this module is handled by
263the :class:`Popen` class. It offers a lot of flexibility so that developers
264are able to handle the less common cases not covered by the convenience
265functions.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000266
267
268.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)
269
270 Arguments are:
271
Benjamin Petersonfff5cf62008-07-27 15:22:14 +0000272 *args* should be a string, or a sequence of program arguments. The program
R. David Murrayfe6e7842009-05-29 19:30:27 +0000273 to execute is normally the first item in the args sequence or the string if
274 a string is given, but can be explicitly set by using the *executable*
275 argument. When *executable* is given, the first item in the args sequence
276 is still treated by most programs as the command name, which can then be
277 different from the actual executable name. On Unix, it becomes the display
278 name for the executing program in utilities such as :program:`ps`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000279
280 On Unix, with *shell=False* (default): In this case, the Popen class uses
281 :meth:`os.execvp` to execute the child program. *args* should normally be a
Nick Coghlan7dfc9e12010-02-04 12:43:58 +0000282 sequence. If a string is specified for *args*, it will be used as the name
283 or path of the program to execute; this will only work if the program is
284 being given no arguments.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000285
Nick Coghlan7dfc9e12010-02-04 12:43:58 +0000286 .. note::
287
288 :meth:`shlex.split` can be useful when determining the correct
289 tokenization for *args*, especially in complex cases::
290
291 >>> import shlex, subprocess
292 >>> command_line = raw_input()
293 /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
294 >>> args = shlex.split(command_line)
295 >>> print args
296 ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
297 >>> p = subprocess.Popen(args) # Success!
298
299 Note in particular that options (such as *-input*) and arguments (such
300 as *eggs.txt*) that are separated by whitespace in the shell go in separate
301 list elements, while arguments that need quoting or backslash escaping when
302 used in the shell (such as filenames containing spaces or the *echo* command
303 shown above) are single list elements.
304
305 On Unix, with *shell=True*: If args is a string, it specifies the command
306 string to execute through the shell. This means that the string must be
307 formatted exactly as it would be when typed at the shell prompt. This
308 includes, for example, quoting or backslash escaping filenames with spaces in
309 them. If *args* is a sequence, the first item specifies the command string, and
310 any additional items will be treated as additional arguments to the shell
311 itself. That is to say, *Popen* does the equivalent of::
312
313 Popen(['/bin/sh', '-c', args[0], args[1], ...])
Georg Brandl8ec7f652007-08-15 14:28:01 +0000314
R David Murrayfc095202012-07-16 22:31:32 -0400315 On Windows: the :class:`Popen` class uses CreateProcess() to execute the
Eli Bendersky929e2762011-04-15 07:35:06 +0300316 child program, which operates on strings. If *args* is a sequence, it will
317 be converted to a string in a manner described in
318 :ref:`converting-argument-sequence`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000319
320 *bufsize*, if given, has the same meaning as the corresponding argument to the
321 built-in open() function: :const:`0` means unbuffered, :const:`1` means line
322 buffered, any other positive value means use a buffer of (approximately) that
323 size. A negative *bufsize* means to use the system default, which usually means
324 fully buffered. The default value for *bufsize* is :const:`0` (unbuffered).
325
Antoine Pitrouc3955452010-06-02 17:08:47 +0000326 .. note::
327
328 If you experience performance issues, it is recommended that you try to
329 enable buffering by setting *bufsize* to either -1 or a large enough
330 positive value (such as 4096).
331
Georg Brandl8ec7f652007-08-15 14:28:01 +0000332 The *executable* argument specifies the program to execute. It is very seldom
333 needed: Usually, the program to execute is defined by the *args* argument. If
334 ``shell=True``, the *executable* argument specifies which shell to use. On Unix,
335 the default shell is :file:`/bin/sh`. On Windows, the default shell is
Georg Brandl0d8649a2009-06-30 16:17:28 +0000336 specified by the :envvar:`COMSPEC` environment variable. The only reason you
337 would need to specify ``shell=True`` on Windows is where the command you
338 wish to execute is actually built in to the shell, eg ``dir``, ``copy``.
339 You don't need ``shell=True`` to run a batch file, nor to run a console-based
340 executable.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000341
Nick Coghlan86711572011-10-24 22:19:40 +1000342 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
Georg Brandlf5d5a662008-12-06 11:57:12 +0000343 standard output and standard error file handles, respectively. Valid values
344 are :data:`PIPE`, an existing file descriptor (a positive integer), an
345 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
Nick Coghlan86711572011-10-24 22:19:40 +1000346 to the child should be created. With the default settings of ``None``, no
347 redirection will occur; the child's file handles will be inherited from the
348 parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that
349 the stderr data from the child process should be captured into the same file
350 handle as for stdout.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000351
352 If *preexec_fn* is set to a callable object, this object will be called in the
353 child process just before the child is executed. (Unix only)
354
355 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
356 :const:`2` will be closed before the child process is executed. (Unix only).
357 Or, on Windows, if *close_fds* is true then no handles will be inherited by the
358 child process. Note that on Windows, you cannot set *close_fds* to true and
359 also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.
360
361 If *shell* is :const:`True`, the specified command will be executed through the
362 shell.
363
Nick Coghlan87ba6422011-10-27 17:55:13 +1000364 .. warning::
Nick Coghlan65ad31a2011-10-26 21:15:53 +1000365
366 Enabling this option can be a security hazard if combined with untrusted
367 input. See the warning under :ref:`frequently-used-arguments`
368 for details.
369
Georg Brandl8ec7f652007-08-15 14:28:01 +0000370 If *cwd* is not ``None``, the child's current directory will be changed to *cwd*
371 before it is executed. Note that this directory is not considered when
372 searching the executable, so you can't specify the program's path relative to
373 *cwd*.
374
Georg Brandlf801b0f2008-04-19 16:58:49 +0000375 If *env* is not ``None``, it must be a mapping that defines the environment
376 variables for the new process; these are used instead of inheriting the current
377 process' environment, which is the default behavior.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000378
R. David Murray72030812009-04-16 18:12:53 +0000379 .. note::
R. David Murray6076d392009-04-15 22:33:07 +0000380
R. David Murray72030812009-04-16 18:12:53 +0000381 If specified, *env* must provide any variables required
382 for the program to execute. On Windows, in order to run a
383 `side-by-side assembly`_ the specified *env* **must** include a valid
R. David Murray6076d392009-04-15 22:33:07 +0000384 :envvar:`SystemRoot`.
385
R. David Murray72030812009-04-16 18:12:53 +0000386 .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly
387
R David Murray5618aaa2012-08-15 11:15:39 -0400388 If *universal_newlines* is ``True``, the file objects *stdout* and *stderr* are
389 opened as text files in :term:`universal newlines` mode. Lines may be
390 terminated by any of ``'\n'``, the Unix
Georg Brandl9af94982008-09-13 17:41:16 +0000391 end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the
Georg Brandl8ec7f652007-08-15 14:28:01 +0000392 Windows convention. All of these external representations are seen as ``'\n'``
393 by the Python program.
394
395 .. note::
396
Georg Brandl6ab5d082009-12-20 14:33:20 +0000397 This feature is only available if Python is built with universal newline
398 support (the default). Also, the newlines attribute of the file objects
399 :attr:`stdout`, :attr:`stdin` and :attr:`stderr` are not updated by the
400 communicate() method.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000401
Brian Curtinbb23bd62011-04-29 22:23:46 -0500402 If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is
403 passed to the underlying ``CreateProcess`` function.
404 *creationflags*, if given, can be :data:`CREATE_NEW_CONSOLE` or
405 :data:`CREATE_NEW_PROCESS_GROUP`. (Windows only)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000406
407
Georg Brandl8ec7f652007-08-15 14:28:01 +0000408Exceptions
409^^^^^^^^^^
410
411Exceptions raised in the child process, before the new program has started to
412execute, will be re-raised in the parent. Additionally, the exception object
413will have one extra attribute called :attr:`child_traceback`, which is a string
Georg Brandl21946af2010-10-06 09:28:45 +0000414containing traceback information from the child's point of view.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000415
416The most common exception raised is :exc:`OSError`. This occurs, for example,
417when trying to execute a non-existent file. Applications should prepare for
418:exc:`OSError` exceptions.
419
420A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid
421arguments.
422
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000423:func:`check_call` and :func:`check_output` will raise
424:exc:`CalledProcessError` if the called process returns a non-zero return
425code.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000426
427
428Security
429^^^^^^^^
430
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000431Unlike some other popen functions, this implementation will never call a
432system shell implicitly. This means that all characters, including shell
433metacharacters, can safely be passed to child processes. Obviously, if the
434shell is invoked explicitly, then it is the application's responsibility to
Nick Coghlan63c54e82011-10-26 21:34:26 +1000435ensure that all whitespace and metacharacters are quoted appropriately.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000436
437
438Popen Objects
439-------------
440
441Instances of the :class:`Popen` class have the following methods:
442
443
444.. method:: Popen.poll()
445
Georg Brandl2cb103f2008-01-06 16:01:26 +0000446 Check if child process has terminated. Set and return :attr:`returncode`
447 attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000448
449
450.. method:: Popen.wait()
451
Georg Brandl2cb103f2008-01-06 16:01:26 +0000452 Wait for child process to terminate. Set and return :attr:`returncode`
453 attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000454
Georg Brandl143de622008-08-04 06:29:36 +0000455 .. warning::
456
Philip Jenvey26275532009-12-03 02:25:54 +0000457 This will deadlock when using ``stdout=PIPE`` and/or
458 ``stderr=PIPE`` and the child process generates enough output to
459 a pipe such that it blocks waiting for the OS pipe buffer to
460 accept more data. Use :meth:`communicate` to avoid that.
Gregory P. Smith08792502008-08-04 01:03:50 +0000461
Georg Brandl8ec7f652007-08-15 14:28:01 +0000462
463.. method:: Popen.communicate(input=None)
464
465 Interact with process: Send data to stdin. Read data from stdout and stderr,
466 until end-of-file is reached. Wait for process to terminate. The optional
467 *input* argument should be a string to be sent to the child process, or
468 ``None``, if no data should be sent to the child.
469
Georg Brandl17432012008-12-04 21:28:16 +0000470 :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000471
Georg Brandl439f2502007-11-24 11:31:46 +0000472 Note that if you want to send data to the process's stdin, you need to create
473 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than
474 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
475 ``stderr=PIPE`` too.
476
Georg Brandl2cb103f2008-01-06 16:01:26 +0000477 .. note::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000478
Georg Brandl2cb103f2008-01-06 16:01:26 +0000479 The data read is buffered in memory, so do not use this method if the data
480 size is large or unlimited.
481
Georg Brandl8ec7f652007-08-15 14:28:01 +0000482
Christian Heimese74c8f22008-04-19 02:23:57 +0000483.. method:: Popen.send_signal(signal)
484
485 Sends the signal *signal* to the child.
486
487 .. note::
488
Brian Curtine5aa8862010-04-02 23:26:06 +0000489 On Windows, SIGTERM is an alias for :meth:`terminate`. CTRL_C_EVENT and
Ezio Melotti9ccc5812010-04-05 08:16:41 +0000490 CTRL_BREAK_EVENT can be sent to processes started with a *creationflags*
Brian Curtine5aa8862010-04-02 23:26:06 +0000491 parameter which includes `CREATE_NEW_PROCESS_GROUP`.
Georg Brandl734de682008-04-19 08:23:59 +0000492
493 .. versionadded:: 2.6
Christian Heimese74c8f22008-04-19 02:23:57 +0000494
495
496.. method:: Popen.terminate()
497
498 Stop the child. On Posix OSs the method sends SIGTERM to the
Sandro Tosi98ed08f2012-01-14 16:42:02 +0100499 child. On Windows the Win32 API function :c:func:`TerminateProcess` is called
Christian Heimese74c8f22008-04-19 02:23:57 +0000500 to stop the child.
501
Georg Brandl734de682008-04-19 08:23:59 +0000502 .. versionadded:: 2.6
503
Christian Heimese74c8f22008-04-19 02:23:57 +0000504
505.. method:: Popen.kill()
506
507 Kills the child. On Posix OSs the function sends SIGKILL to the child.
Georg Brandl734de682008-04-19 08:23:59 +0000508 On Windows :meth:`kill` is an alias for :meth:`terminate`.
509
510 .. versionadded:: 2.6
Christian Heimese74c8f22008-04-19 02:23:57 +0000511
512
Georg Brandl8ec7f652007-08-15 14:28:01 +0000513The following attributes are also available:
514
Georg Brandl143de622008-08-04 06:29:36 +0000515.. warning::
516
Georg Brandl16a57f62009-04-27 15:29:09 +0000517 Use :meth:`communicate` rather than :attr:`.stdin.write <stdin>`,
518 :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` to avoid
519 deadlocks due to any of the other OS pipe buffers filling up and blocking the
520 child process.
Georg Brandl143de622008-08-04 06:29:36 +0000521
522
Georg Brandl8ec7f652007-08-15 14:28:01 +0000523.. attribute:: Popen.stdin
524
Georg Brandlf5d5a662008-12-06 11:57:12 +0000525 If the *stdin* argument was :data:`PIPE`, this attribute is a file object
526 that provides input to the child process. Otherwise, it is ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000527
528
529.. attribute:: Popen.stdout
530
Georg Brandlf5d5a662008-12-06 11:57:12 +0000531 If the *stdout* argument was :data:`PIPE`, this attribute is a file object
532 that provides output from the child process. Otherwise, it is ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000533
534
535.. attribute:: Popen.stderr
536
Georg Brandlf5d5a662008-12-06 11:57:12 +0000537 If the *stderr* argument was :data:`PIPE`, this attribute is a file object
538 that provides error output from the child process. Otherwise, it is
539 ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000540
541
542.. attribute:: Popen.pid
543
544 The process ID of the child process.
545
Georg Brandl0b56ce02010-03-21 09:28:16 +0000546 Note that if you set the *shell* argument to ``True``, this is the process ID
547 of the spawned shell.
548
Georg Brandl8ec7f652007-08-15 14:28:01 +0000549
550.. attribute:: Popen.returncode
551
Georg Brandl2cb103f2008-01-06 16:01:26 +0000552 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly
553 by :meth:`communicate`). A ``None`` value indicates that the process
554 hasn't terminated yet.
Georg Brandlc62ef8b2009-01-03 20:55:06 +0000555
Georg Brandl2cb103f2008-01-06 16:01:26 +0000556 A negative value ``-N`` indicates that the child was terminated by signal
557 ``N`` (Unix only).
Georg Brandl8ec7f652007-08-15 14:28:01 +0000558
559
Brian Curtinbb23bd62011-04-29 22:23:46 -0500560Windows Popen Helpers
561---------------------
562
563The :class:`STARTUPINFO` class and following constants are only available
564on Windows.
565
566.. class:: STARTUPINFO()
567
568 Partial support of the Windows
569 `STARTUPINFO <http://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx>`__
570 structure is used for :class:`Popen` creation.
571
572 .. attribute:: dwFlags
573
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700574 A bit field that determines whether certain :class:`STARTUPINFO`
575 attributes are used when the process creates a window. ::
Brian Curtinbb23bd62011-04-29 22:23:46 -0500576
577 si = subprocess.STARTUPINFO()
578 si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW
579
580 .. attribute:: hStdInput
581
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700582 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
583 is the standard input handle for the process. If
584 :data:`STARTF_USESTDHANDLES` is not specified, the default for standard
585 input is the keyboard buffer.
Brian Curtinbb23bd62011-04-29 22:23:46 -0500586
587 .. attribute:: hStdOutput
588
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700589 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
590 is the standard output handle for the process. Otherwise, this attribute
591 is ignored and the default for standard output is the console window's
Brian Curtinbb23bd62011-04-29 22:23:46 -0500592 buffer.
593
594 .. attribute:: hStdError
595
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700596 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
597 is the standard error handle for the process. Otherwise, this attribute is
Brian Curtinbb23bd62011-04-29 22:23:46 -0500598 ignored and the default for standard error is the console window's buffer.
599
600 .. attribute:: wShowWindow
601
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700602 If :attr:`dwFlags` specifies :data:`STARTF_USESHOWWINDOW`, this attribute
Brian Curtinbb23bd62011-04-29 22:23:46 -0500603 can be any of the values that can be specified in the ``nCmdShow``
604 parameter for the
605 `ShowWindow <http://msdn.microsoft.com/en-us/library/ms633548(v=vs.85).aspx>`__
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700606 function, except for ``SW_SHOWDEFAULT``. Otherwise, this attribute is
Brian Curtinbb23bd62011-04-29 22:23:46 -0500607 ignored.
608
609 :data:`SW_HIDE` is provided for this attribute. It is used when
610 :class:`Popen` is called with ``shell=True``.
611
612
613Constants
614^^^^^^^^^
615
616The :mod:`subprocess` module exposes the following constants.
617
618.. data:: STD_INPUT_HANDLE
619
620 The standard input device. Initially, this is the console input buffer,
621 ``CONIN$``.
622
623.. data:: STD_OUTPUT_HANDLE
624
625 The standard output device. Initially, this is the active console screen
626 buffer, ``CONOUT$``.
627
628.. data:: STD_ERROR_HANDLE
629
630 The standard error device. Initially, this is the active console screen
631 buffer, ``CONOUT$``.
632
633.. data:: SW_HIDE
634
635 Hides the window. Another window will be activated.
636
637.. data:: STARTF_USESTDHANDLES
638
639 Specifies that the :attr:`STARTUPINFO.hStdInput`,
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700640 :attr:`STARTUPINFO.hStdOutput`, and :attr:`STARTUPINFO.hStdError` attributes
Brian Curtinbb23bd62011-04-29 22:23:46 -0500641 contain additional information.
642
643.. data:: STARTF_USESHOWWINDOW
644
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700645 Specifies that the :attr:`STARTUPINFO.wShowWindow` attribute contains
Brian Curtinbb23bd62011-04-29 22:23:46 -0500646 additional information.
647
648.. data:: CREATE_NEW_CONSOLE
649
650 The new process has a new console, instead of inheriting its parent's
651 console (the default).
652
653 This flag is always set when :class:`Popen` is created with ``shell=True``.
654
655.. data:: CREATE_NEW_PROCESS_GROUP
656
657 A :class:`Popen` ``creationflags`` parameter to specify that a new process
658 group will be created. This flag is necessary for using :func:`os.kill`
659 on the subprocess.
660
661 This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified.
662
663
Georg Brandl0ba92b22008-06-22 09:05:29 +0000664.. _subprocess-replacements:
665
Georg Brandl8ec7f652007-08-15 14:28:01 +0000666Replacing Older Functions with the subprocess Module
667----------------------------------------------------
668
Nick Coghlan86711572011-10-24 22:19:40 +1000669In this section, "a becomes b" means that b can be used as a replacement for a.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000670
671.. note::
672
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000673 All "a" functions in this section fail (more or less) silently if the
674 executed program cannot be found; the "b" replacements raise :exc:`OSError`
675 instead.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000676
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000677 In addition, the replacements using :func:`check_output` will fail with a
678 :exc:`CalledProcessError` if the requested operation produces a non-zero
679 return code. The output is still available as the ``output`` attribute of
680 the raised exception.
681
682In the following examples, we assume that the relevant functions have already
683been imported from the subprocess module.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000684
685
686Replacing /bin/sh shell backquote
687^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
688
689::
690
691 output=`mycmd myarg`
Nick Coghlan86711572011-10-24 22:19:40 +1000692 # becomes
693 output = check_output(["mycmd", "myarg"])
Georg Brandl8ec7f652007-08-15 14:28:01 +0000694
695
Benjamin Petersoncae58482008-10-10 20:38:49 +0000696Replacing shell pipeline
697^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000698
699::
700
701 output=`dmesg | grep hda`
Nick Coghlan86711572011-10-24 22:19:40 +1000702 # becomes
Georg Brandl8ec7f652007-08-15 14:28:01 +0000703 p1 = Popen(["dmesg"], stdout=PIPE)
704 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Gregory P. Smithe3e967f2011-02-05 21:49:56 +0000705 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000706 output = p2.communicate()[0]
707
Gregory P. Smithe3e967f2011-02-05 21:49:56 +0000708The p1.stdout.close() call after starting the p2 is important in order for p1
709to receive a SIGPIPE if p2 exits before p1.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000710
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000711Alternatively, for trusted input, the shell's own pipeline support may still
R David Murray5fc56eb2012-04-03 08:46:05 -0400712be used directly::
Nick Coghlan86711572011-10-24 22:19:40 +1000713
714 output=`dmesg | grep hda`
715 # becomes
716 output=check_output("dmesg | grep hda", shell=True)
717
718
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000719Replacing :func:`os.system`
720^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000721
722::
723
724 sts = os.system("mycmd" + " myarg")
Nick Coghlan86711572011-10-24 22:19:40 +1000725 # becomes
726 sts = call("mycmd" + " myarg", shell=True)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000727
728Notes:
729
730* Calling the program through the shell is usually not required.
731
Georg Brandl8ec7f652007-08-15 14:28:01 +0000732A more realistic example would look like this::
733
734 try:
735 retcode = call("mycmd" + " myarg", shell=True)
736 if retcode < 0:
737 print >>sys.stderr, "Child was terminated by signal", -retcode
738 else:
739 print >>sys.stderr, "Child returned", retcode
740 except OSError, e:
741 print >>sys.stderr, "Execution failed:", e
742
743
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000744Replacing the :func:`os.spawn <os.spawnl>` family
745^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000746
747P_NOWAIT example::
748
749 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
750 ==>
751 pid = Popen(["/bin/mycmd", "myarg"]).pid
752
753P_WAIT example::
754
755 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
756 ==>
757 retcode = call(["/bin/mycmd", "myarg"])
758
759Vector example::
760
761 os.spawnvp(os.P_NOWAIT, path, args)
762 ==>
763 Popen([path] + args[1:])
764
765Environment example::
766
767 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
768 ==>
769 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
770
771
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000772Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
773^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000774
775::
776
Philip Jenvey8b902042009-09-29 19:10:15 +0000777 pipe = os.popen("cmd", 'r', bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000778 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000779 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout
Georg Brandl8ec7f652007-08-15 14:28:01 +0000780
781::
782
Philip Jenvey8b902042009-09-29 19:10:15 +0000783 pipe = os.popen("cmd", 'w', bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000784 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000785 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin
Georg Brandl8ec7f652007-08-15 14:28:01 +0000786
787::
788
Philip Jenvey8b902042009-09-29 19:10:15 +0000789 (child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000790 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000791 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000792 stdin=PIPE, stdout=PIPE, close_fds=True)
793 (child_stdin, child_stdout) = (p.stdin, p.stdout)
794
795::
796
797 (child_stdin,
798 child_stdout,
Philip Jenvey8b902042009-09-29 19:10:15 +0000799 child_stderr) = os.popen3("cmd", mode, bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000800 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000801 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000802 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
803 (child_stdin,
804 child_stdout,
805 child_stderr) = (p.stdin, p.stdout, p.stderr)
806
807::
808
Philip Jenvey8b902042009-09-29 19:10:15 +0000809 (child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
810 bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000811 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000812 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000813 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
814 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
815
Philip Jenvey8b902042009-09-29 19:10:15 +0000816On Unix, os.popen2, os.popen3 and os.popen4 also accept a sequence as
817the command to execute, in which case arguments will be passed
818directly to the program without shell intervention. This usage can be
819replaced as follows::
820
821 (child_stdin, child_stdout) = os.popen2(["/bin/ls", "-l"], mode,
822 bufsize)
823 ==>
824 p = Popen(["/bin/ls", "-l"], bufsize=bufsize, stdin=PIPE, stdout=PIPE)
825 (child_stdin, child_stdout) = (p.stdin, p.stdout)
826
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000827Return code handling translates as follows::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000828
Philip Jenvey8b902042009-09-29 19:10:15 +0000829 pipe = os.popen("cmd", 'w')
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000830 ...
831 rc = pipe.close()
Stefan Kraha253dc12010-07-14 10:06:07 +0000832 if rc is not None and rc >> 8:
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000833 print "There were some errors"
834 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000835 process = Popen("cmd", 'w', shell=True, stdin=PIPE)
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000836 ...
837 process.stdin.close()
838 if process.wait() != 0:
839 print "There were some errors"
840
841
842Replacing functions from the :mod:`popen2` module
843^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000844
Georg Brandl8ec7f652007-08-15 14:28:01 +0000845::
846
847 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
848 ==>
849 p = Popen(["somestring"], shell=True, bufsize=bufsize,
850 stdin=PIPE, stdout=PIPE, close_fds=True)
851 (child_stdout, child_stdin) = (p.stdout, p.stdin)
852
Philip Jenvey8b902042009-09-29 19:10:15 +0000853On Unix, popen2 also accepts a sequence as the command to execute, in
854which case arguments will be passed directly to the program without
855shell intervention. This usage can be replaced as follows::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000856
Philip Jenvey8b902042009-09-29 19:10:15 +0000857 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize,
858 mode)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000859 ==>
860 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
861 stdin=PIPE, stdout=PIPE, close_fds=True)
862 (child_stdout, child_stdin) = (p.stdout, p.stdin)
863
Georg Brandlf5d5a662008-12-06 11:57:12 +0000864:class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
865:class:`subprocess.Popen`, except that:
Georg Brandl8ec7f652007-08-15 14:28:01 +0000866
Georg Brandlf5d5a662008-12-06 11:57:12 +0000867* :class:`Popen` raises an exception if the execution fails.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000868
869* the *capturestderr* argument is replaced with the *stderr* argument.
870
Georg Brandlf5d5a662008-12-06 11:57:12 +0000871* ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000872
873* popen2 closes all file descriptors by default, but you have to specify
Georg Brandlf5d5a662008-12-06 11:57:12 +0000874 ``close_fds=True`` with :class:`Popen`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000875
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000876
Eli Bendersky929e2762011-04-15 07:35:06 +0300877Notes
878-----
879
880.. _converting-argument-sequence:
881
882Converting an argument sequence to a string on Windows
883^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
884
885On Windows, an *args* sequence is converted to a string that can be parsed
886using the following rules (which correspond to the rules used by the MS C
887runtime):
888
8891. Arguments are delimited by white space, which is either a
890 space or a tab.
891
8922. A string surrounded by double quotation marks is
893 interpreted as a single argument, regardless of white space
894 contained within. A quoted string can be embedded in an
895 argument.
896
8973. A double quotation mark preceded by a backslash is
898 interpreted as a literal double quotation mark.
899
9004. Backslashes are interpreted literally, unless they
901 immediately precede a double quotation mark.
902
9035. If backslashes immediately precede a double quotation mark,
904 every pair of backslashes is interpreted as a literal
905 backslash. If the number of backslashes is odd, the last
906 backslash escapes the next double quotation mark as
907 described in rule 3.
908